2024-09-11T00:58:30.968 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-11T00:58:30.985 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T00:58:31.076 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/hyelloji-2024-09-10_17:16:14-rados-wip-hemanth-testing-2024-09-10-0723-distro-default-smithi/7899875 branch: wip-hemanth-testing-2024-09-10-0723 description: rados/singleton/{all/lost-unfound-delete mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/{bluestore-options/write$/{write_v2} bluestore/bluestore-bitmap} rados supported-random-distro$/{ubuntu_latest}} email: hyelloji@redhat.com first_in_suite: false job_id: '7899875' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: hyelloji-2024-09-10_17:16:14-rados-wip-hemanth-testing-2024-09-10-0723-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-hemanth-testing-2024-09-10-0723 ceph: conf: global: mon election default strategy: 3 ms bind msgr2: false ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon scrub interval: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true bluestore write v2: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) mon_bind_msgr2: false sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-hemanth-testing-2024-09-10-0723 sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 owner: scheduled_hyelloji@teuthology priority: 100 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.b - mon.c - mgr.x - osd.0 - osd.1 - osd.2 seed: 9167 sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-hemanth-testing-2024-09-10-0723 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 targets: smithi122.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM+/n2TYwpW+YECWrfF4akfSkyFIEd2DUdQ/HpoGab2ONTv8Lt7O0uDxbYTTpzr15ALmTtGnpIkDLHNHGKUsq5U= tasks: - install: null - ceph: log-ignorelist: - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(OBJECT_ - \(SLOW_OPS\) - slow request - \(POOL_APP_NOT_ENABLED\) pre-mgr-commands: - sudo ceph config set mgr mgr_pool false --force - rep_lost_unfound_delete: null teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_17:16:14 tube: smithi user: hyelloji verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-11T00:58:31.077 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa; will attempt to use it 2024-09-11T00:58:31.077 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa/tasks 2024-09-11T00:58:31.077 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-11T00:58:31.079 INFO:teuthology.task.internal:Checking packages... 2024-09-11T00:58:31.101 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '8f219c476cb4f26fc5719c661c0323719a45cb30' 2024-09-11T00:58:31.101 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-11T00:58:31.101 INFO:teuthology.packaging:ref: None 2024-09-11T00:58:31.101 INFO:teuthology.packaging:tag: None 2024-09-11T00:58:31.101 INFO:teuthology.packaging:branch: wip-hemanth-testing-2024-09-10-0723 2024-09-11T00:58:31.101 INFO:teuthology.packaging:sha1: 8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-11T00:58:31.102 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-hemanth-testing-2024-09-10-0723 2024-09-11T00:58:31.251 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-4854-g8f219c47-1jammy 2024-09-11T00:58:31.252 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-11T00:58:31.261 INFO:teuthology.task.internal:no buildpackages task found 2024-09-11T00:58:31.261 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-11T00:58:31.273 INFO:teuthology.task.internal:Saving configuration 2024-09-11T00:58:31.285 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-11T00:58:31.296 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-11T00:58:31.328 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi122.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/hyelloji-2024-09-10_17:16:14-rados-wip-hemanth-testing-2024-09-10-0723-distro-default-smithi/7899875', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-11 00:52:59.359576', 'locked_by': 'scheduled_hyelloji@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAICVUjd8lMdhHakB4cozh3KWwN2kyM4iKFjHktJn3tkSV'} 2024-09-11T00:58:31.328 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-11T00:58:31.339 INFO:teuthology.task.internal:roles: ubuntu@smithi122.front.sepia.ceph.com - ['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'osd.0', 'osd.1', 'osd.2'] 2024-09-11T00:58:31.339 INFO:teuthology.run_tasks:Running task console_log... 2024-09-11T00:58:31.387 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fec2f2a3be0>, signals=[15]) 2024-09-11T00:58:31.388 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-11T00:58:31.399 INFO:teuthology.task.internal:Opening connections... 2024-09-11T00:58:31.399 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi122.front.sepia.ceph.com 2024-09-11T00:58:31.400 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T00:58:31.488 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-11T00:58:31.496 DEBUG:teuthology.orchestra.run.smithi122:> uname -m 2024-09-11T00:58:31.606 INFO:teuthology.orchestra.run.smithi122.stdout:x86_64 2024-09-11T00:58:31.606 DEBUG:teuthology.orchestra.run.smithi122:> cat /etc/os-release 2024-09-11T00:58:31.652 INFO:teuthology.orchestra.run.smithi122.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-09-11T00:58:31.653 INFO:teuthology.orchestra.run.smithi122.stdout:NAME="Ubuntu" 2024-09-11T00:58:31.653 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_ID="22.04" 2024-09-11T00:58:31.653 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-09-11T00:58:31.653 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_CODENAME=jammy 2024-09-11T00:58:31.653 INFO:teuthology.orchestra.run.smithi122.stdout:ID=ubuntu 2024-09-11T00:58:31.653 INFO:teuthology.orchestra.run.smithi122.stdout:ID_LIKE=debian 2024-09-11T00:58:31.653 INFO:teuthology.orchestra.run.smithi122.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-11T00:58:31.653 INFO:teuthology.orchestra.run.smithi122.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-11T00:58:31.653 INFO:teuthology.orchestra.run.smithi122.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-11T00:58:31.653 INFO:teuthology.orchestra.run.smithi122.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-11T00:58:31.653 INFO:teuthology.orchestra.run.smithi122.stdout:UBUNTU_CODENAME=jammy 2024-09-11T00:58:31.654 INFO:teuthology.lock.ops:Updating smithi122.front.sepia.ceph.com on lock server 2024-09-11T00:58:31.679 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-11T00:58:31.692 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-11T00:58:31.735 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-11T00:58:31.735 DEBUG:teuthology.orchestra.run.smithi122:> test '!' -e /home/ubuntu/cephtest 2024-09-11T00:58:31.740 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-11T00:58:31.777 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-11T00:58:31.777 DEBUG:teuthology.orchestra.run.smithi122:> test -z $(ls -A /var/lib/ceph) 2024-09-11T00:58:31.806 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-11T00:58:31.857 INFO:teuthology.run_tasks:Running task kernel... 2024-09-11T00:58:31.872 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-11T00:58:31.872 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-11T00:58:31.873 DEBUG:teuthology.orchestra.run.smithi122:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-11T00:58:31.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T00:58:31.878 DEBUG:teuthology.orchestra.run.smithi122:> uname -r 2024-09-11T00:58:31.925 INFO:teuthology.orchestra.run.smithi122.stdout:5.15.0-56-generic 2024-09-11T00:58:31.925 INFO:teuthology.task.kernel:Running kernel on smithi122: 5.15.0-56-generic 2024-09-11T00:58:31.925 DEBUG:teuthology.orchestra.run.smithi122:> sudo apt-get clean 2024-09-11T00:58:32.061 DEBUG:teuthology.orchestra.run.smithi122:> sudo apt-get update 2024-09-11T00:58:32.209 INFO:teuthology.orchestra.run.smithi122.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2024-09-11T00:58:32.314 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-11T00:58:32.395 INFO:teuthology.orchestra.run.smithi122.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2024-09-11T00:58:32.418 INFO:teuthology.orchestra.run.smithi122.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,771 kB] 2024-09-11T00:58:32.520 INFO:teuthology.orchestra.run.smithi122.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [533 kB] 2024-09-11T00:58:32.524 INFO:teuthology.orchestra.run.smithi122.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [291 kB] 2024-09-11T00:58:32.526 INFO:teuthology.orchestra.run.smithi122.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [13.3 kB] 2024-09-11T00:58:32.526 INFO:teuthology.orchestra.run.smithi122.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [2,327 kB] 2024-09-11T00:58:32.546 INFO:teuthology.orchestra.run.smithi122.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [37.7 kB] 2024-09-11T00:58:32.546 INFO:teuthology.orchestra.run.smithi122.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [400 kB] 2024-09-11T00:58:32.550 INFO:teuthology.orchestra.run.smithi122.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [584 B] 2024-09-11T00:58:32.550 INFO:teuthology.orchestra.run.smithi122.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [901 kB] 2024-09-11T00:58:32.555 INFO:teuthology.orchestra.run.smithi122.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [626 kB] 2024-09-11T00:58:32.559 INFO:teuthology.orchestra.run.smithi122.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [176 kB] 2024-09-11T00:58:32.560 INFO:teuthology.orchestra.run.smithi122.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [19.2 kB] 2024-09-11T00:58:32.561 INFO:teuthology.orchestra.run.smithi122.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-09-11T00:58:32.561 INFO:teuthology.orchestra.run.smithi122.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-09-11T00:58:32.578 INFO:teuthology.orchestra.run.smithi122.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-09-11T00:58:32.593 INFO:teuthology.orchestra.run.smithi122.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [228 B] 2024-09-11T00:58:32.750 INFO:teuthology.orchestra.run.smithi122.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2024-09-11T00:58:32.889 INFO:teuthology.orchestra.run.smithi122.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [690 kB] 2024-09-11T00:58:33.083 INFO:teuthology.orchestra.run.smithi122.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,988 kB] 2024-09-11T00:58:33.242 INFO:teuthology.orchestra.run.smithi122.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [349 kB] 2024-09-11T00:58:33.256 INFO:teuthology.orchestra.run.smithi122.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [17.8 kB] 2024-09-11T00:58:33.257 INFO:teuthology.orchestra.run.smithi122.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [39.3 kB] 2024-09-11T00:58:33.258 INFO:teuthology.orchestra.run.smithi122.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [2,386 kB] 2024-09-11T00:58:33.327 INFO:teuthology.orchestra.run.smithi122.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [410 kB] 2024-09-11T00:58:33.341 INFO:teuthology.orchestra.run.smithi122.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [616 B] 2024-09-11T00:58:33.342 INFO:teuthology.orchestra.run.smithi122.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,122 kB] 2024-09-11T00:58:33.375 INFO:teuthology.orchestra.run.smithi122.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [731 kB] 2024-09-11T00:58:33.394 INFO:teuthology.orchestra.run.smithi122.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [261 kB] 2024-09-11T00:58:33.400 INFO:teuthology.orchestra.run.smithi122.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [26.2 kB] 2024-09-11T00:58:33.407 INFO:teuthology.orchestra.run.smithi122.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [43.3 kB] 2024-09-11T00:58:33.408 INFO:teuthology.orchestra.run.smithi122.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,752 B] 2024-09-11T00:58:33.408 INFO:teuthology.orchestra.run.smithi122.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.8 kB] 2024-09-11T00:58:33.483 INFO:teuthology.orchestra.run.smithi122.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [444 B] 2024-09-11T00:58:33.483 INFO:teuthology.orchestra.run.smithi122.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.8 kB] 2024-09-11T00:58:33.485 INFO:teuthology.orchestra.run.smithi122.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.9 kB] 2024-09-11T00:58:33.486 INFO:teuthology.orchestra.run.smithi122.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.1 kB] 2024-09-11T00:58:33.486 INFO:teuthology.orchestra.run.smithi122.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-09-11T00:58:33.488 INFO:teuthology.orchestra.run.smithi122.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.8 kB] 2024-09-11T00:58:33.489 INFO:teuthology.orchestra.run.smithi122.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.3 kB] 2024-09-11T00:58:33.570 INFO:teuthology.orchestra.run.smithi122.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-09-11T00:58:33.570 INFO:teuthology.orchestra.run.smithi122.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [672 B] 2024-09-11T00:58:38.978 INFO:teuthology.orchestra.run.smithi122.stdout:Fetched 15.8 MB in 2s (7,480 kB/s) 2024-09-11T00:58:40.539 INFO:teuthology.orchestra.run.smithi122.stdout:Reading package lists... 2024-09-11T00:58:40.555 DEBUG:teuthology.orchestra.run.smithi122:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-11T00:58:40.622 INFO:teuthology.orchestra.run.smithi122.stdout:Reading package lists... 2024-09-11T00:58:40.840 INFO:teuthology.orchestra.run.smithi122.stdout:Building dependency tree... 2024-09-11T00:58:40.840 INFO:teuthology.orchestra.run.smithi122.stdout:Reading state information... 2024-09-11T00:58:41.029 INFO:teuthology.orchestra.run.smithi122.stdout:The following packages were automatically installed and are no longer required: 2024-09-11T00:58:41.029 INFO:teuthology.orchestra.run.smithi122.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-11T00:58:41.030 INFO:teuthology.orchestra.run.smithi122.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-11T00:58:41.030 INFO:teuthology.orchestra.run.smithi122.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-11T00:58:41.030 INFO:teuthology.orchestra.run.smithi122.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-09-11T00:58:41.030 INFO:teuthology.orchestra.run.smithi122.stdout: python2.7-minimal 2024-09-11T00:58:41.030 INFO:teuthology.orchestra.run.smithi122.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-11T00:58:41.031 INFO:teuthology.orchestra.run.smithi122.stdout:The following additional packages will be installed: 2024-09-11T00:58:41.032 INFO:teuthology.orchestra.run.smithi122.stdout: linux-generic linux-headers-5.15.0-119 linux-headers-5.15.0-119-generic 2024-09-11T00:58:41.032 INFO:teuthology.orchestra.run.smithi122.stdout: linux-headers-generic linux-image-5.15.0-119-generic 2024-09-11T00:58:41.032 INFO:teuthology.orchestra.run.smithi122.stdout: linux-modules-5.15.0-119-generic linux-modules-extra-5.15.0-119-generic 2024-09-11T00:58:41.033 INFO:teuthology.orchestra.run.smithi122.stdout:Suggested packages: 2024-09-11T00:58:41.033 INFO:teuthology.orchestra.run.smithi122.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-09-11T00:58:41.033 INFO:teuthology.orchestra.run.smithi122.stdout:Recommended packages: 2024-09-11T00:58:41.033 INFO:teuthology.orchestra.run.smithi122.stdout: thermald 2024-09-11T00:58:41.077 INFO:teuthology.orchestra.run.smithi122.stdout:The following NEW packages will be installed: 2024-09-11T00:58:41.077 INFO:teuthology.orchestra.run.smithi122.stdout: linux-headers-5.15.0-119 linux-headers-5.15.0-119-generic 2024-09-11T00:58:41.078 INFO:teuthology.orchestra.run.smithi122.stdout: linux-image-5.15.0-119-generic linux-modules-5.15.0-119-generic 2024-09-11T00:58:41.078 INFO:teuthology.orchestra.run.smithi122.stdout: linux-modules-extra-5.15.0-119-generic 2024-09-11T00:58:41.079 INFO:teuthology.orchestra.run.smithi122.stdout:The following packages will be upgraded: 2024-09-11T00:58:41.080 INFO:teuthology.orchestra.run.smithi122.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-11T00:58:41.258 INFO:teuthology.orchestra.run.smithi122.stdout:3 upgraded, 5 newly installed, 0 to remove and 344 not upgraded. 2024-09-11T00:58:41.258 INFO:teuthology.orchestra.run.smithi122.stdout:Need to get 113 MB of archives. 2024-09-11T00:58:41.258 INFO:teuthology.orchestra.run.smithi122.stdout:After this operation, 583 MB of additional disk space will be used. 2024-09-11T00:58:41.258 INFO:teuthology.orchestra.run.smithi122.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-119-generic amd64 5.15.0-119.129 [22.6 MB] 2024-09-11T00:58:42.672 INFO:teuthology.orchestra.run.smithi122.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-119-generic amd64 5.15.0-119.129 [11.5 MB] 2024-09-11T00:58:43.021 INFO:teuthology.orchestra.run.smithi122.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-119-generic amd64 5.15.0-119.129 [63.9 MB] 2024-09-11T00:58:45.247 INFO:teuthology.orchestra.run.smithi122.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.119.119 [1,694 B] 2024-09-11T00:58:45.247 INFO:teuthology.orchestra.run.smithi122.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.119.119 [2,502 B] 2024-09-11T00:58:45.248 INFO:teuthology.orchestra.run.smithi122.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-119 all 5.15.0-119.129 [12.3 MB] 2024-09-11T00:58:45.588 INFO:teuthology.orchestra.run.smithi122.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-119-generic amd64 5.15.0-119.129 [2,879 kB] 2024-09-11T00:58:45.668 INFO:teuthology.orchestra.run.smithi122.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.119.119 [2,350 B] 2024-09-11T00:58:46.056 INFO:teuthology.orchestra.run.smithi122.stdout:Fetched 113 MB in 5s (24.7 MB/s) 2024-09-11T00:58:46.246 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package linux-modules-5.15.0-119-generic. 2024-09-11T00:58:48.290 INFO:teuthology.orchestra.run.smithi122.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-09-11T00:58:48.294 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../0-linux-modules-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-11T00:58:48.394 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-modules-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T00:58:50.787 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package linux-image-5.15.0-119-generic. 2024-09-11T00:58:50.808 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../1-linux-image-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-11T00:58:50.905 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-image-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T00:58:51.365 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-119-generic. 2024-09-11T00:58:51.386 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-11T00:58:51.415 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-modules-extra-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T00:58:57.993 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../3-linux-generic_5.15.0.119.119_amd64.deb ... 2024-09-11T00:58:58.118 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-generic (5.15.0.119.119) over (5.15.0.56.54) ... 2024-09-11T00:58:58.384 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.119.119_amd64.deb ... 2024-09-11T00:58:58.505 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-image-generic (5.15.0.119.119) over (5.15.0.56.54) ... 2024-09-11T00:58:58.816 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package linux-headers-5.15.0-119. 2024-09-11T00:58:58.838 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../5-linux-headers-5.15.0-119_5.15.0-119.129_all.deb ... 2024-09-11T00:58:58.882 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-headers-5.15.0-119 (5.15.0-119.129) ... 2024-09-11T00:59:04.888 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package linux-headers-5.15.0-119-generic. 2024-09-11T00:59:04.925 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../6-linux-headers-5.15.0-119-generic_5.15.0-119.129_amd64.deb ... 2024-09-11T00:59:04.963 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-headers-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T00:59:08.367 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.119.119_amd64.deb ... 2024-09-11T00:59:08.701 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking linux-headers-generic (5.15.0.119.119) over (5.15.0.56.54) ... 2024-09-11T00:59:09.051 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-headers-5.15.0-119 (5.15.0-119.129) ... 2024-09-11T00:59:09.180 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-headers-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T00:59:09.323 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-headers-generic (5.15.0.119.119) ... 2024-09-11T00:59:09.444 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-image-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T00:59:10.728 INFO:teuthology.orchestra.run.smithi122.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-09-11T00:59:10.728 INFO:teuthology.orchestra.run.smithi122.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-09-11T00:59:10.728 INFO:teuthology.orchestra.run.smithi122.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-119-generic 2024-09-11T00:59:10.728 INFO:teuthology.orchestra.run.smithi122.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-119-generic 2024-09-11T00:59:10.912 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-modules-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T00:59:12.630 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-modules-extra-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T00:59:14.574 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-image-generic (5.15.0.119.119) ... 2024-09-11T00:59:14.700 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up linux-generic (5.15.0.119.119) ... 2024-09-11T00:59:14.851 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for linux-image-5.15.0-119-generic (5.15.0-119.129) ... 2024-09-11T00:59:14.917 INFO:teuthology.orchestra.run.smithi122.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-11T00:59:14.917 INFO:teuthology.orchestra.run.smithi122.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-119-generic 2024-09-11T00:59:29.106 INFO:teuthology.orchestra.run.smithi122.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-11T00:59:29.106 INFO:teuthology.orchestra.run.smithi122.stdout:Sourcing file `/etc/default/grub' 2024-09-11T00:59:29.127 INFO:teuthology.orchestra.run.smithi122.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-11T00:59:29.146 INFO:teuthology.orchestra.run.smithi122.stdout:Generating grub configuration file ... 2024-09-11T00:59:29.408 INFO:teuthology.orchestra.run.smithi122.stdout:Found linux image: /boot/vmlinuz-5.15.0-119-generic 2024-09-11T00:59:29.438 INFO:teuthology.orchestra.run.smithi122.stdout:Found initrd image: /boot/initrd.img-5.15.0-119-generic 2024-09-11T00:59:29.768 INFO:teuthology.orchestra.run.smithi122.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-11T00:59:29.773 INFO:teuthology.orchestra.run.smithi122.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-11T00:59:29.840 INFO:teuthology.orchestra.run.smithi122.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-11T00:59:29.845 INFO:teuthology.orchestra.run.smithi122.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-11T00:59:30.002 INFO:teuthology.orchestra.run.smithi122.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-11T00:59:30.002 INFO:teuthology.orchestra.run.smithi122.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-09-11T00:59:30.002 INFO:teuthology.orchestra.run.smithi122.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-11T00:59:30.035 INFO:teuthology.orchestra.run.smithi122.stdout:done 2024-09-11T00:59:30.532 DEBUG:teuthology.orchestra.run.smithi122:> dpkg -s linux-image-generic 2024-09-11T00:59:30.551 INFO:teuthology.orchestra.run.smithi122.stdout:Package: linux-image-generic 2024-09-11T00:59:30.551 INFO:teuthology.orchestra.run.smithi122.stdout:Status: install ok installed 2024-09-11T00:59:30.551 INFO:teuthology.orchestra.run.smithi122.stdout:Priority: optional 2024-09-11T00:59:30.551 INFO:teuthology.orchestra.run.smithi122.stdout:Section: kernel 2024-09-11T00:59:30.551 INFO:teuthology.orchestra.run.smithi122.stdout:Installed-Size: 21 2024-09-11T00:59:30.551 INFO:teuthology.orchestra.run.smithi122.stdout:Maintainer: Ubuntu Kernel Team 2024-09-11T00:59:30.551 INFO:teuthology.orchestra.run.smithi122.stdout:Architecture: amd64 2024-09-11T00:59:30.552 INFO:teuthology.orchestra.run.smithi122.stdout:Source: linux-meta 2024-09-11T00:59:30.552 INFO:teuthology.orchestra.run.smithi122.stdout:Version: 5.15.0.119.119 2024-09-11T00:59:30.552 INFO:teuthology.orchestra.run.smithi122.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-119), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-11T00:59:30.552 INFO:teuthology.orchestra.run.smithi122.stdout:Depends: linux-image-5.15.0-119-generic, linux-modules-extra-5.15.0-119-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-11T00:59:30.552 INFO:teuthology.orchestra.run.smithi122.stdout:Recommends: thermald 2024-09-11T00:59:30.552 INFO:teuthology.orchestra.run.smithi122.stdout:Description: Generic Linux kernel image 2024-09-11T00:59:30.552 INFO:teuthology.orchestra.run.smithi122.stdout: This package will always depend on the latest generic kernel image 2024-09-11T00:59:30.552 INFO:teuthology.orchestra.run.smithi122.stdout: available. 2024-09-11T00:59:30.552 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-119-generic 2024-09-11T00:59:30.553 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-11T00:59:30.553 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-11T00:59:30.553 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-11T00:59:30.553 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi122.front.sepia.ceph.com, path=None, version=distro) 2024-09-11T00:59:30.553 DEBUG:teuthology.orchestra.run.smithi122:> sudo apt-get clean 2024-09-11T00:59:30.734 DEBUG:teuthology.orchestra.run.smithi122:> sudo apt-get update 2024-09-11T00:59:30.977 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-11T00:59:30.978 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-11T00:59:31.048 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-11T00:59:31.130 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-11T00:59:32.378 INFO:teuthology.orchestra.run.smithi122.stdout:Reading package lists... 2024-09-11T00:59:32.397 DEBUG:teuthology.orchestra.run.smithi122:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-11T00:59:32.460 INFO:teuthology.orchestra.run.smithi122.stdout:Reading package lists... 2024-09-11T00:59:32.671 INFO:teuthology.orchestra.run.smithi122.stdout:Building dependency tree... 2024-09-11T00:59:32.672 INFO:teuthology.orchestra.run.smithi122.stdout:Reading state information... 2024-09-11T00:59:32.865 INFO:teuthology.orchestra.run.smithi122.stdout:linux-image-generic is already the newest version (5.15.0.119.119). 2024-09-11T00:59:32.866 INFO:teuthology.orchestra.run.smithi122.stdout:The following packages were automatically installed and are no longer required: 2024-09-11T00:59:32.866 INFO:teuthology.orchestra.run.smithi122.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-11T00:59:32.866 INFO:teuthology.orchestra.run.smithi122.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-11T00:59:32.866 INFO:teuthology.orchestra.run.smithi122.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-11T00:59:32.866 INFO:teuthology.orchestra.run.smithi122.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-11T00:59:32.866 INFO:teuthology.orchestra.run.smithi122.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-11T00:59:32.866 INFO:teuthology.orchestra.run.smithi122.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-11T00:59:32.908 INFO:teuthology.orchestra.run.smithi122.stdout:0 upgraded, 0 newly installed, 0 to remove and 344 not upgraded. 2024-09-11T00:59:32.910 DEBUG:teuthology.orchestra.run.smithi122:> dpkg -s linux-image-generic 2024-09-11T00:59:32.925 INFO:teuthology.orchestra.run.smithi122.stdout:Package: linux-image-generic 2024-09-11T00:59:32.925 INFO:teuthology.orchestra.run.smithi122.stdout:Status: install ok installed 2024-09-11T00:59:32.925 INFO:teuthology.orchestra.run.smithi122.stdout:Priority: optional 2024-09-11T00:59:32.925 INFO:teuthology.orchestra.run.smithi122.stdout:Section: kernel 2024-09-11T00:59:32.925 INFO:teuthology.orchestra.run.smithi122.stdout:Installed-Size: 21 2024-09-11T00:59:32.925 INFO:teuthology.orchestra.run.smithi122.stdout:Maintainer: Ubuntu Kernel Team 2024-09-11T00:59:32.925 INFO:teuthology.orchestra.run.smithi122.stdout:Architecture: amd64 2024-09-11T00:59:32.925 INFO:teuthology.orchestra.run.smithi122.stdout:Source: linux-meta 2024-09-11T00:59:32.925 INFO:teuthology.orchestra.run.smithi122.stdout:Version: 5.15.0.119.119 2024-09-11T00:59:32.926 INFO:teuthology.orchestra.run.smithi122.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.4), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-119), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.4) 2024-09-11T00:59:32.926 INFO:teuthology.orchestra.run.smithi122.stdout:Depends: linux-image-5.15.0-119-generic, linux-modules-extra-5.15.0-119-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-11T00:59:32.926 INFO:teuthology.orchestra.run.smithi122.stdout:Recommends: thermald 2024-09-11T00:59:32.926 INFO:teuthology.orchestra.run.smithi122.stdout:Description: Generic Linux kernel image 2024-09-11T00:59:32.926 INFO:teuthology.orchestra.run.smithi122.stdout: This package will always depend on the latest generic kernel image 2024-09-11T00:59:32.926 INFO:teuthology.orchestra.run.smithi122.stdout: available. 2024-09-11T00:59:32.926 DEBUG:teuthology.orchestra.run.smithi122:> mktemp 2024-09-11T00:59:32.975 INFO:teuthology.orchestra.run.smithi122.stdout:/tmp/tmp.Krd0G7yDYH 2024-09-11T00:59:32.975 DEBUG:teuthology.orchestra.run.smithi122:> sudo cp /boot/grub/grub.cfg /tmp/tmp.Krd0G7yDYH 2024-09-11T00:59:33.031 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0666 /tmp/tmp.Krd0G7yDYH 2024-09-11T00:59:33.143 DEBUG:teuthology.orchestra.remote:smithi122:/tmp/tmp.Krd0G7yDYH is 10KB 2024-09-11T00:59:33.156 DEBUG:teuthology.orchestra.run.smithi122:> rm -fr /tmp/tmp.Krd0G7yDYH 2024-09-11T00:59:33.161 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-11T00:59:33.215 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-11T00:59:33.216 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-11T00:59:33.216 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-11T00:59:33.291 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-119-generic 2024-09-11T00:59:33.291 DEBUG:teuthology.orchestra.run.smithi122:> sudo update-grub 2024-09-11T00:59:34.687 INFO:teuthology.orchestra.run.smithi122.stderr:Sourcing file `/etc/default/grub' 2024-09-11T00:59:34.688 INFO:teuthology.orchestra.run.smithi122.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-11T00:59:34.695 INFO:teuthology.orchestra.run.smithi122.stderr:Generating grub configuration file ... 2024-09-11T00:59:34.893 INFO:teuthology.orchestra.run.smithi122.stderr:Found linux image: /boot/vmlinuz-5.15.0-119-generic 2024-09-11T00:59:34.899 INFO:teuthology.orchestra.run.smithi122.stderr:Found initrd image: /boot/initrd.img-5.15.0-119-generic 2024-09-11T00:59:35.139 INFO:teuthology.orchestra.run.smithi122.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-09-11T00:59:35.145 INFO:teuthology.orchestra.run.smithi122.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-09-11T00:59:35.186 INFO:teuthology.orchestra.run.smithi122.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-09-11T00:59:35.191 INFO:teuthology.orchestra.run.smithi122.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-09-11T00:59:35.324 INFO:teuthology.orchestra.run.smithi122.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-09-11T00:59:35.325 INFO:teuthology.orchestra.run.smithi122.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-09-11T00:59:35.325 INFO:teuthology.orchestra.run.smithi122.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-09-11T00:59:35.342 INFO:teuthology.orchestra.run.smithi122.stderr:done 2024-09-11T00:59:35.361 DEBUG:teuthology.orchestra.run.smithi122:> sudo shutdown -r now 2024-09-11T01:00:05.393 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-11T01:00:05.394 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-11T01:00:05.395 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T01:00:23.745 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.122 2024-09-11T01:00:32.753 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-11T01:00:32.754 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T01:00:35.837 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.122 2024-09-11T01:00:47.849 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-11T01:00:47.850 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T01:01:06.237 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.122 2024-09-11T01:01:21.253 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-11T01:01:21.254 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T01:01:21.881 DEBUG:teuthology.orchestra.run.smithi122:> true 2024-09-11T01:01:22.496 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-11T01:01:22.496 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-119-generic"... 2024-09-11T01:01:22.496 DEBUG:teuthology.orchestra.run.smithi122:> uname -r 2024-09-11T01:01:22.541 INFO:teuthology.orchestra.run.smithi122.stdout:5.15.0-119-generic 2024-09-11T01:01:22.541 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-119-generic vs 5.15.0-119-generic 2024-09-11T01:01:22.541 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-11T01:01:22.542 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-11T01:01:23.542 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-11T01:01:23.542 DEBUG:teuthology.orchestra.run.smithi122:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-11T01:01:23.636 INFO:teuthology.orchestra.run.smithi122.stdout:ttyS1 2024-09-11T01:01:23.655 DEBUG:teuthology.parallel:result is None 2024-09-11T01:01:23.655 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-11T01:01:23.666 INFO:teuthology.task.internal:Creating test directory... 2024-09-11T01:01:23.666 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-11T01:01:23.674 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-11T01:01:23.720 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-11T01:01:23.821 INFO:teuthology.task.internal:Creating archive directory... 2024-09-11T01:01:23.822 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-11T01:01:23.848 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-11T01:01:23.942 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-11T01:01:23.942 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-11T01:01:23.968 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T01:01:23.978 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-11T01:01:23.979 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-11T01:01:24.010 INFO:teuthology.task.internal:Configuring sudo... 2024-09-11T01:01:24.011 DEBUG:teuthology.orchestra.run.smithi122:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-11T01:01:24.032 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-11T01:01:24.049 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-11T01:01:24.049 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-11T01:01:24.082 DEBUG:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-11T01:01:24.129 DEBUG:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-11T01:01:24.177 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-11T01:01:24.178 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-11T01:01:24.256 DEBUG:teuthology.orchestra.run.smithi122:> sudo service rsyslog restart 2024-09-11T01:01:24.333 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-11T01:01:24.342 INFO:teuthology.task.internal:Starting timer... 2024-09-11T01:01:24.342 INFO:teuthology.run_tasks:Running task pcp... 2024-09-11T01:01:24.408 INFO:teuthology.run_tasks:Running task selinux... 2024-09-11T01:01:24.515 DEBUG:teuthology.task.selinux:Excluding smithi122: OS 'ubuntu' does not support SELinux 2024-09-11T01:01:24.515 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-11T01:01:24.515 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-11T01:01:24.515 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-11T01:01:24.516 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-11T01:01:24.590 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-11T01:01:24.592 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-11T01:01:24.736 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-11T01:01:24.749 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-11T01:01:24.750 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi122.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-11T01:08:49.646 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi122.front.sepia.ceph.com')] 2024-09-11T01:08:49.648 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-11T01:08:49.649 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-11T01:08:49.747 DEBUG:teuthology.orchestra.run.smithi122:> true 2024-09-11T01:08:49.825 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-11T01:08:49.825 INFO:teuthology.run_tasks:Running task clock... 2024-09-11T01:08:49.836 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-11T01:08:49.836 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-11T01:08:49.837 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T01:08:49.899 INFO:teuthology.orchestra.run.smithi122.stdout:11 Sep 01:08:49 ntpd[18449]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-09-11T01:08:49.899 INFO:teuthology.orchestra.run.smithi122.stdout:11 Sep 01:08:49 ntpd[18449]: Command line: ntpd -gq 2024-09-11T01:08:49.899 INFO:teuthology.orchestra.run.smithi122.stdout:11 Sep 01:08:49 ntpd[18449]: ---------------------------------------------------- 2024-09-11T01:08:49.899 INFO:teuthology.orchestra.run.smithi122.stdout:11 Sep 01:08:49 ntpd[18449]: ntp-4 is maintained by Network Time Foundation, 2024-09-11T01:08:49.899 INFO:teuthology.orchestra.run.smithi122.stdout:11 Sep 01:08:49 ntpd[18449]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-09-11T01:08:49.899 INFO:teuthology.orchestra.run.smithi122.stdout:11 Sep 01:08:49 ntpd[18449]: corporation. Support and training for ntp-4 are 2024-09-11T01:08:49.899 INFO:teuthology.orchestra.run.smithi122.stdout:11 Sep 01:08:49 ntpd[18449]: available at https://www.nwtime.org/support 2024-09-11T01:08:49.899 INFO:teuthology.orchestra.run.smithi122.stdout:11 Sep 01:08:49 ntpd[18449]: ---------------------------------------------------- 2024-09-11T01:08:49.900 INFO:teuthology.orchestra.run.smithi122.stdout:11 Sep 01:08:49 ntpd[18449]: proto: precision = 0.046 usec (-24) 2024-09-11T01:08:49.900 INFO:teuthology.orchestra.run.smithi122.stdout:11 Sep 01:08:49 ntpd[18449]: basedate set to 2022-02-04 2024-09-11T01:08:49.900 INFO:teuthology.orchestra.run.smithi122.stdout:11 Sep 01:08:49 ntpd[18449]: gps base set to 2022-02-06 (week 2196) 2024-09-11T01:08:49.900 INFO:teuthology.orchestra.run.smithi122.stdout:11 Sep 01:08:49 ntpd[18449]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-11T01:08:49.901 INFO:teuthology.orchestra.run.smithi122.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-11T01:08:49.901 INFO:teuthology.orchestra.run.smithi122.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-11T01:08:49.902 INFO:teuthology.orchestra.run.smithi122.stdout:11 Sep 01:08:49 ntpd[18449]: restrict ::: KOD does nothing without LIMITED. 2024-09-11T01:08:49.902 INFO:teuthology.orchestra.run.smithi122.stdout:11 Sep 01:08:49 ntpd[18449]: Listen and drop on 0 v6wildcard [::]:123 2024-09-11T01:08:49.902 INFO:teuthology.orchestra.run.smithi122.stdout:11 Sep 01:08:49 ntpd[18449]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-11T01:08:49.902 INFO:teuthology.orchestra.run.smithi122.stdout:11 Sep 01:08:49 ntpd[18449]: Listen normally on 2 lo 127.0.0.1:123 2024-09-11T01:08:49.902 INFO:teuthology.orchestra.run.smithi122.stdout:11 Sep 01:08:49 ntpd[18449]: Listen normally on 3 enp3s0f1 172.21.15.122:123 2024-09-11T01:08:49.902 INFO:teuthology.orchestra.run.smithi122.stdout:11 Sep 01:08:49 ntpd[18449]: Listen normally on 4 lo [::1]:123 2024-09-11T01:08:49.902 INFO:teuthology.orchestra.run.smithi122.stdout:11 Sep 01:08:49 ntpd[18449]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d515%5]:123 2024-09-11T01:08:49.902 INFO:teuthology.orchestra.run.smithi122.stdout:11 Sep 01:08:49 ntpd[18449]: Listening on routing socket on fd #22 for interface updates 2024-09-11T01:08:50.900 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:50 ntpd[18449]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T01:08:50.900 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:50 ntpd[18449]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T01:08:50.900 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:50 ntpd[18449]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T01:08:50.900 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:50 ntpd[18449]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T01:08:50.900 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:50 ntpd[18449]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T01:08:50.900 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:50 ntpd[18449]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T01:08:50.901 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:50 ntpd[18449]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T01:08:50.901 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:50 ntpd[18449]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T01:08:50.901 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:50 ntpd[18449]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T01:08:50.901 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:50 ntpd[18449]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T01:08:50.901 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:50 ntpd[18449]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T01:08:50.901 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:50 ntpd[18449]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T01:08:52.902 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:52 ntpd[18449]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T01:08:52.902 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:52 ntpd[18449]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T01:08:52.902 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:52 ntpd[18449]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T01:08:52.903 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:52 ntpd[18449]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T01:08:52.903 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:52 ntpd[18449]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T01:08:52.903 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:52 ntpd[18449]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T01:08:52.903 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:52 ntpd[18449]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T01:08:52.903 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:52 ntpd[18449]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T01:08:52.903 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:52 ntpd[18449]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T01:08:52.904 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:52 ntpd[18449]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T01:08:52.904 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:52 ntpd[18449]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T01:08:52.904 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:52 ntpd[18449]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T01:08:52.904 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:52 ntpd[18449]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T01:08:52.904 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:52 ntpd[18449]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T01:08:52.904 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:52 ntpd[18449]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T01:08:52.904 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:52 ntpd[18449]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T01:08:52.905 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:52 ntpd[18449]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T01:08:52.905 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:52 ntpd[18449]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T01:08:52.905 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:52 ntpd[18449]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T01:08:52.905 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:52 ntpd[18449]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T01:08:54.900 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:54 ntpd[18449]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T01:08:54.901 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:54 ntpd[18449]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T01:08:54.901 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:54 ntpd[18449]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T01:08:54.901 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:54 ntpd[18449]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T01:08:54.901 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:54 ntpd[18449]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T01:08:54.901 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:54 ntpd[18449]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T01:08:54.901 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:54 ntpd[18449]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T01:08:54.901 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:54 ntpd[18449]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T01:08:54.901 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:54 ntpd[18449]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T01:08:54.901 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:54 ntpd[18449]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T01:08:54.901 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:54 ntpd[18449]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T01:08:54.901 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:54 ntpd[18449]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T01:08:54.901 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:54 ntpd[18449]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T01:08:54.901 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:54 ntpd[18449]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T01:08:54.902 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:54 ntpd[18449]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T01:08:54.902 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:54 ntpd[18449]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T01:08:54.902 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:54 ntpd[18449]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T01:08:54.902 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:54 ntpd[18449]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T01:08:56.900 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:56 ntpd[18449]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T01:08:56.900 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:56 ntpd[18449]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T01:08:56.900 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:56 ntpd[18449]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T01:08:56.900 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:56 ntpd[18449]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T01:08:56.900 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:56 ntpd[18449]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T01:08:56.900 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:56 ntpd[18449]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T01:08:56.900 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:56 ntpd[18449]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-11T01:08:56.901 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:56 ntpd[18449]: can't open /var/log/ntpstats/rawstats.20240911: Permission denied 2024-09-11T01:08:56.901 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:56 ntpd[18449]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-11T01:08:56.901 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:56 ntpd[18449]: can't open /var/log/ntpstats/peerstats.20240911: Permission denied 2024-09-11T01:08:56.901 INFO:teuthology.orchestra.run.smithi122.stdout:11 Sep 01:08:56 ntpd[18449]: ntpd: time slew -0.001340 s 2024-09-11T01:08:56.901 INFO:teuthology.orchestra.run.smithi122.stdout:ntpd: time slew -0.001340s 2024-09-11T01:08:56.901 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:56 ntpd[18449]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-11T01:08:56.901 INFO:teuthology.orchestra.run.smithi122.stderr:11 Sep 01:08:56 ntpd[18449]: can't open /var/log/ntpstats/loopstats.20240911: Permission denied 2024-09-11T01:08:56.954 INFO:teuthology.orchestra.run.smithi122.stdout: remote refid st t when poll reach delay offset jitter 2024-09-11T01:08:56.954 INFO:teuthology.orchestra.run.smithi122.stdout:============================================================================== 2024-09-11T01:08:56.954 INFO:teuthology.orchestra.run.smithi122.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-11T01:08:56.955 INFO:teuthology.orchestra.run.smithi122.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-11T01:08:56.955 INFO:teuthology.orchestra.run.smithi122.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-11T01:08:56.955 INFO:teuthology.orchestra.run.smithi122.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-09-11T01:08:56.956 INFO:teuthology.run_tasks:Running task install... 2024-09-11T01:08:56.964 DEBUG:teuthology.task.install:project ceph 2024-09-11T01:08:56.964 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}} 2024-09-11T01:08:56.964 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'} 2024-09-11T01:08:56.964 INFO:teuthology.task.install:Using flavor: default 2024-09-11T01:08:56.970 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-11T01:08:56.970 INFO:teuthology.task.install:extra packages: [] 2024-09-11T01:08:56.970 DEBUG:teuthology.orchestra.run.smithi122:> sudo apt-key list | grep Ceph 2024-09-11T01:08:57.059 INFO:teuthology.orchestra.run.smithi122.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-09-11T01:08:57.089 INFO:teuthology.orchestra.run.smithi122.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-11T01:08:57.090 INFO:teuthology.orchestra.run.smithi122.stdout:uid [ unknown] Ceph.com (release key) 2024-09-11T01:08:57.090 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-11T01:08:57.090 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-11T01:08:57.238 INFO:teuthology.task.install.deb:Pulling from https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default/ 2024-09-11T01:08:57.238 INFO:teuthology.task.install.deb:Package version is 19.3.0-4854-g8f219c47-1jammy 2024-09-11T01:08:57.361 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-11T01:08:57.361 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-11T01:08:57.375 DEBUG:teuthology.orchestra.run.smithi122:> sudo apt-get update 2024-09-11T01:08:57.545 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-09-11T01:08:57.553 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-09-11T01:08:57.571 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-09-11T01:08:57.598 INFO:teuthology.orchestra.run.smithi122.stdout:Ign:4 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy InRelease 2024-09-11T01:08:57.619 INFO:teuthology.orchestra.run.smithi122.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy Release [7,689 B] 2024-09-11T01:08:57.641 INFO:teuthology.orchestra.run.smithi122.stdout:Ign:6 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy Release.gpg 2024-09-11T01:08:57.663 INFO:teuthology.orchestra.run.smithi122.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.2 kB] 2024-09-11T01:08:57.668 INFO:teuthology.orchestra.run.smithi122.stdout:Hit:8 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-09-11T01:08:57.690 INFO:teuthology.orchestra.run.smithi122.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,412 B] 2024-09-11T01:08:57.896 INFO:teuthology.orchestra.run.smithi122.stdout:Fetched 29.3 kB in 0s (85.5 kB/s) 2024-09-11T01:08:58.862 INFO:teuthology.orchestra.run.smithi122.stdout:Reading package lists... 2024-09-11T01:08:58.877 DEBUG:teuthology.orchestra.run.smithi122:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.3.0-4854-g8f219c47-1jammy cephadm=19.3.0-4854-g8f219c47-1jammy ceph-mds=19.3.0-4854-g8f219c47-1jammy ceph-mgr=19.3.0-4854-g8f219c47-1jammy ceph-common=19.3.0-4854-g8f219c47-1jammy ceph-fuse=19.3.0-4854-g8f219c47-1jammy ceph-test=19.3.0-4854-g8f219c47-1jammy ceph-volume=19.3.0-4854-g8f219c47-1jammy radosgw=19.3.0-4854-g8f219c47-1jammy python3-rados=19.3.0-4854-g8f219c47-1jammy python3-rgw=19.3.0-4854-g8f219c47-1jammy python3-cephfs=19.3.0-4854-g8f219c47-1jammy python3-rbd=19.3.0-4854-g8f219c47-1jammy libcephfs2=19.3.0-4854-g8f219c47-1jammy libcephfs-dev=19.3.0-4854-g8f219c47-1jammy librados2=19.3.0-4854-g8f219c47-1jammy librbd1=19.3.0-4854-g8f219c47-1jammy rbd-fuse=19.3.0-4854-g8f219c47-1jammy 2024-09-11T01:08:58.931 INFO:teuthology.orchestra.run.smithi122.stdout:Reading package lists... 2024-09-11T01:08:59.115 INFO:teuthology.orchestra.run.smithi122.stdout:Building dependency tree... 2024-09-11T01:08:59.115 INFO:teuthology.orchestra.run.smithi122.stdout:Reading state information... 2024-09-11T01:08:59.297 INFO:teuthology.orchestra.run.smithi122.stdout:The following packages were automatically installed and are no longer required: 2024-09-11T01:08:59.298 INFO:teuthology.orchestra.run.smithi122.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-09-11T01:08:59.298 INFO:teuthology.orchestra.run.smithi122.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-09-11T01:08:59.298 INFO:teuthology.orchestra.run.smithi122.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-09-11T01:08:59.298 INFO:teuthology.orchestra.run.smithi122.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-09-11T01:08:59.299 INFO:teuthology.orchestra.run.smithi122.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-09-11T01:08:59.299 INFO:teuthology.orchestra.run.smithi122.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-11T01:08:59.299 INFO:teuthology.orchestra.run.smithi122.stdout:The following additional packages will be installed: 2024-09-11T01:08:59.299 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-11T01:08:59.299 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-09-11T01:08:59.299 INFO:teuthology.orchestra.run.smithi122.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 liboath0 2024-09-11T01:08:59.299 INFO:teuthology.orchestra.run.smithi122.stdout: libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2024-09-11T01:08:59.299 INFO:teuthology.orchestra.run.smithi122.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2024-09-11T01:08:59.300 INFO:teuthology.orchestra.run.smithi122.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2024-09-11T01:08:59.301 INFO:teuthology.orchestra.run.smithi122.stdout: pkg-config python-asyncssh-doc python-babel-localedata 2024-09-11T01:08:59.301 INFO:teuthology.orchestra.run.smithi122.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2024-09-11T01:08:59.301 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-certifi 2024-09-11T01:08:59.301 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cheroot python3-cherrypy3 python3-google-auth python3-idna 2024-09-11T01:08:59.301 INFO:teuthology.orchestra.run.smithi122.stdout: python3-iniconfig python3-jaraco.classes python3-jaraco.collections 2024-09-11T01:08:59.301 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco.functools python3-jaraco.text python3-jinja2 python3-joblib 2024-09-11T01:08:59.301 INFO:teuthology.orchestra.run.smithi122.stdout: python3-kubernetes python3-logutils python3-mako python3-markupsafe 2024-09-11T01:08:59.301 INFO:teuthology.orchestra.run.smithi122.stdout: python3-natsort python3-openssl python3-paste python3-pastedeploy 2024-09-11T01:08:59.301 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pastescript python3-pecan python3-pluggy python3-portend 2024-09-11T01:08:59.301 INFO:teuthology.orchestra.run.smithi122.stdout: python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-11T01:08:59.301 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-modules python3-pygments python3-pyinotify python3-pytest 2024-09-11T01:08:59.301 INFO:teuthology.orchestra.run.smithi122.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-routes 2024-09-11T01:08:59.302 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rsa python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-11T01:08:59.302 INFO:teuthology.orchestra.run.smithi122.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-09-11T01:08:59.302 INFO:teuthology.orchestra.run.smithi122.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-09-11T01:08:59.302 INFO:teuthology.orchestra.run.smithi122.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-09-11T01:08:59.302 INFO:teuthology.orchestra.run.smithi122.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-09-11T01:08:59.302 INFO:teuthology.orchestra.run.smithi122.stdout: xmlstarlet zip 2024-09-11T01:08:59.303 INFO:teuthology.orchestra.run.smithi122.stdout:Suggested packages: 2024-09-11T01:08:59.303 INFO:teuthology.orchestra.run.smithi122.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-09-11T01:08:59.303 INFO:teuthology.orchestra.run.smithi122.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-09-11T01:08:59.303 INFO:teuthology.orchestra.run.smithi122.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-09-11T01:08:59.303 INFO:teuthology.orchestra.run.smithi122.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-09-11T01:08:59.303 INFO:teuthology.orchestra.run.smithi122.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-09-11T01:08:59.303 INFO:teuthology.orchestra.run.smithi122.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-11T01:08:59.303 INFO:teuthology.orchestra.run.smithi122.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-09-11T01:08:59.303 INFO:teuthology.orchestra.run.smithi122.stdout:Recommended packages: 2024-09-11T01:08:59.303 INFO:teuthology.orchestra.run.smithi122.stdout: btrfs-tools 2024-09-11T01:08:59.405 INFO:teuthology.orchestra.run.smithi122.stdout:The following NEW packages will be installed: 2024-09-11T01:08:59.405 INFO:teuthology.orchestra.run.smithi122.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-11T01:08:59.405 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-11T01:08:59.405 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-09-11T01:08:59.405 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-09-11T01:08:59.406 INFO:teuthology.orchestra.run.smithi122.stdout: liblua5.3-dev libnbd0 liboath0 libonig5 libpcre2-16-0 libqt5core5a 2024-09-11T01:08:59.406 INFO:teuthology.orchestra.run.smithi122.stdout: libqt5dbus5 libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-09-11T01:08:59.406 INFO:teuthology.orchestra.run.smithi122.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-09-11T01:08:59.406 INFO:teuthology.orchestra.run.smithi122.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-09-11T01:08:59.406 INFO:teuthology.orchestra.run.smithi122.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-09-11T01:08:59.406 INFO:teuthology.orchestra.run.smithi122.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-09-11T01:08:59.407 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-09-11T01:08:59.407 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-09-11T01:08:59.407 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-09-11T01:08:59.407 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-09-11T01:08:59.407 INFO:teuthology.orchestra.run.smithi122.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-09-11T01:08:59.408 INFO:teuthology.orchestra.run.smithi122.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-09-11T01:08:59.408 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-09-11T01:08:59.408 INFO:teuthology.orchestra.run.smithi122.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-11T01:08:59.408 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-11T01:08:59.408 INFO:teuthology.orchestra.run.smithi122.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-09-11T01:08:59.408 INFO:teuthology.orchestra.run.smithi122.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-09-11T01:08:59.409 INFO:teuthology.orchestra.run.smithi122.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-09-11T01:08:59.409 INFO:teuthology.orchestra.run.smithi122.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-09-11T01:08:59.409 INFO:teuthology.orchestra.run.smithi122.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-09-11T01:08:59.409 INFO:teuthology.orchestra.run.smithi122.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-09-11T01:08:59.409 INFO:teuthology.orchestra.run.smithi122.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-09-11T01:08:59.546 INFO:teuthology.orchestra.run.smithi122.stdout:0 upgraded, 119 newly installed, 0 to remove and 331 not upgraded. 2024-09-11T01:08:59.546 INFO:teuthology.orchestra.run.smithi122.stdout:Need to get 251 MB of archives. 2024-09-11T01:08:59.546 INFO:teuthology.orchestra.run.smithi122.stdout:After this operation, 934 MB of additional disk space will be used. 2024-09-11T01:08:59.546 INFO:teuthology.orchestra.run.smithi122.stdout:Get:1 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.3.0-4854-g8f219c47-1jammy [3,633 kB] 2024-09-11T01:08:59.594 INFO:teuthology.orchestra.run.smithi122.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-09-11T01:08:59.757 INFO:teuthology.orchestra.run.smithi122.stdout:Get:3 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.3.0-4854-g8f219c47-1jammy [3,330 kB] 2024-09-11T01:08:59.833 INFO:teuthology.orchestra.run.smithi122.stdout:Get:4 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.3.0-4854-g8f219c47-1jammy [721 kB] 2024-09-11T01:08:59.849 INFO:teuthology.orchestra.run.smithi122.stdout:Get:5 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.3.0-4854-g8f219c47-1jammy [356 kB] 2024-09-11T01:08:59.854 INFO:teuthology.orchestra.run.smithi122.stdout:Get:6 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.3.0-4854-g8f219c47-1jammy [32.8 kB] 2024-09-11T01:08:59.855 INFO:teuthology.orchestra.run.smithi122.stdout:Get:7 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.3.0-4854-g8f219c47-1jammy [180 kB] 2024-09-11T01:08:59.858 INFO:teuthology.orchestra.run.smithi122.stdout:Get:8 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.3.0-4854-g8f219c47-1jammy [71.2 kB] 2024-09-11T01:08:59.860 INFO:teuthology.orchestra.run.smithi122.stdout:Get:9 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.3.0-4854-g8f219c47-1jammy [338 kB] 2024-09-11T01:08:59.871 INFO:teuthology.orchestra.run.smithi122.stdout:Get:10 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.3.0-4854-g8f219c47-1jammy [7,988 kB] 2024-09-11T01:09:00.002 INFO:teuthology.orchestra.run.smithi122.stdout:Get:11 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.3.0-4854-g8f219c47-1jammy [112 kB] 2024-09-11T01:09:00.004 INFO:teuthology.orchestra.run.smithi122.stdout:Get:12 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.3.0-4854-g8f219c47-1jammy [473 kB] 2024-09-11T01:09:00.010 INFO:teuthology.orchestra.run.smithi122.stdout:Get:13 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.3.0-4854-g8f219c47-1jammy [28.3 MB] 2024-09-11T01:09:00.016 INFO:teuthology.orchestra.run.smithi122.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-09-11T01:09:00.028 INFO:teuthology.orchestra.run.smithi122.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-09-11T01:09:00.109 INFO:teuthology.orchestra.run.smithi122.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-09-11T01:09:00.338 INFO:teuthology.orchestra.run.smithi122.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-09-11T01:09:00.349 INFO:teuthology.orchestra.run.smithi122.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-09-11T01:09:00.379 INFO:teuthology.orchestra.run.smithi122.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-09-11T01:09:00.387 INFO:teuthology.orchestra.run.smithi122.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2024-09-11T01:09:00.390 INFO:teuthology.orchestra.run.smithi122.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-09-11T01:09:00.391 INFO:teuthology.orchestra.run.smithi122.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-09-11T01:09:00.392 INFO:teuthology.orchestra.run.smithi122.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-09-11T01:09:00.409 INFO:teuthology.orchestra.run.smithi122.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-09-11T01:09:00.414 INFO:teuthology.orchestra.run.smithi122.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-09-11T01:09:00.420 INFO:teuthology.orchestra.run.smithi122.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-09-11T01:09:00.500 INFO:teuthology.orchestra.run.smithi122.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-09-11T01:09:00.535 INFO:teuthology.orchestra.run.smithi122.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-09-11T01:09:00.602 INFO:teuthology.orchestra.run.smithi122.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-09-11T01:09:00.602 INFO:teuthology.orchestra.run.smithi122.stdout:Get:30 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.3.0-4854-g8f219c47-1jammy [5,206 kB] 2024-09-11T01:09:00.606 INFO:teuthology.orchestra.run.smithi122.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-09-11T01:09:00.610 INFO:teuthology.orchestra.run.smithi122.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-09-11T01:09:00.611 INFO:teuthology.orchestra.run.smithi122.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-09-11T01:09:00.612 INFO:teuthology.orchestra.run.smithi122.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-09-11T01:09:00.614 INFO:teuthology.orchestra.run.smithi122.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-09-11T01:09:00.614 INFO:teuthology.orchestra.run.smithi122.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-09-11T01:09:00.615 INFO:teuthology.orchestra.run.smithi122.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-09-11T01:09:00.666 INFO:teuthology.orchestra.run.smithi122.stdout:Get:38 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.3.0-4854-g8f219c47-1jammy [251 kB] 2024-09-11T01:09:00.668 INFO:teuthology.orchestra.run.smithi122.stdout:Get:39 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.3.0-4854-g8f219c47-1jammy [124 kB] 2024-09-11T01:09:00.670 INFO:teuthology.orchestra.run.smithi122.stdout:Get:40 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.3.0-4854-g8f219c47-1jammy [1,562 kB] 2024-09-11T01:09:00.684 INFO:teuthology.orchestra.run.smithi122.stdout:Get:41 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.3.0-4854-g8f219c47-1jammy [6,308 kB] 2024-09-11T01:09:00.685 INFO:teuthology.orchestra.run.smithi122.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-09-11T01:09:00.697 INFO:teuthology.orchestra.run.smithi122.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-09-11T01:09:00.698 INFO:teuthology.orchestra.run.smithi122.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-09-11T01:09:00.698 INFO:teuthology.orchestra.run.smithi122.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-09-11T01:09:00.704 INFO:teuthology.orchestra.run.smithi122.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-09-11T01:09:00.705 INFO:teuthology.orchestra.run.smithi122.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-09-11T01:09:00.767 INFO:teuthology.orchestra.run.smithi122.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-09-11T01:09:00.767 INFO:teuthology.orchestra.run.smithi122.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-09-11T01:09:00.769 INFO:teuthology.orchestra.run.smithi122.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-11T01:09:00.770 INFO:teuthology.orchestra.run.smithi122.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-09-11T01:09:00.776 INFO:teuthology.orchestra.run.smithi122.stdout:Get:52 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.3.0-4854-g8f219c47-1jammy [23.1 MB] 2024-09-11T01:09:00.848 INFO:teuthology.orchestra.run.smithi122.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2024-09-11T01:09:00.850 INFO:teuthology.orchestra.run.smithi122.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-09-11T01:09:00.851 INFO:teuthology.orchestra.run.smithi122.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-09-11T01:09:00.852 INFO:teuthology.orchestra.run.smithi122.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-09-11T01:09:00.863 INFO:teuthology.orchestra.run.smithi122.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-09-11T01:09:00.929 INFO:teuthology.orchestra.run.smithi122.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-09-11T01:09:00.930 INFO:teuthology.orchestra.run.smithi122.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-09-11T01:09:00.931 INFO:teuthology.orchestra.run.smithi122.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-09-11T01:09:00.934 INFO:teuthology.orchestra.run.smithi122.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-09-11T01:09:00.938 INFO:teuthology.orchestra.run.smithi122.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-idna all 3.3-1ubuntu0.1 [52.1 kB] 2024-09-11T01:09:01.011 INFO:teuthology.orchestra.run.smithi122.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-09-11T01:09:01.013 INFO:teuthology.orchestra.run.smithi122.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-09-11T01:09:01.014 INFO:teuthology.orchestra.run.smithi122.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.2 [181 kB] 2024-09-11T01:09:01.019 INFO:teuthology.orchestra.run.smithi122.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-09-11T01:09:01.020 INFO:teuthology.orchestra.run.smithi122.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-09-11T01:09:01.092 INFO:teuthology.orchestra.run.smithi122.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-09-11T01:09:01.097 INFO:teuthology.orchestra.run.smithi122.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-09-11T01:09:01.234 INFO:teuthology.orchestra.run.smithi122.stdout:Get:70 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.3.0-4854-g8f219c47-1jammy [14.0 kB] 2024-09-11T01:09:01.234 INFO:teuthology.orchestra.run.smithi122.stdout:Get:71 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.3.0-4854-g8f219c47-1jammy [880 kB] 2024-09-11T01:09:01.244 INFO:teuthology.orchestra.run.smithi122.stdout:Get:72 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.3.0-4854-g8f219c47-1jammy [2,454 kB] 2024-09-11T01:09:01.289 INFO:teuthology.orchestra.run.smithi122.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-09-11T01:09:01.289 INFO:teuthology.orchestra.run.smithi122.stdout:Get:74 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.3.0-4854-g8f219c47-1jammy [813 kB] 2024-09-11T01:09:01.291 INFO:teuthology.orchestra.run.smithi122.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.2 [108 kB] 2024-09-11T01:09:01.294 INFO:teuthology.orchestra.run.smithi122.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-11T01:09:01.295 INFO:teuthology.orchestra.run.smithi122.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-09-11T01:09:01.297 INFO:teuthology.orchestra.run.smithi122.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-09-11T01:09:01.299 INFO:teuthology.orchestra.run.smithi122.stdout:Get:79 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.3.0-4854-g8f219c47-1jammy [160 kB] 2024-09-11T01:09:01.301 INFO:teuthology.orchestra.run.smithi122.stdout:Get:80 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.3.0-4854-g8f219c47-1jammy [61.4 MB] 2024-09-11T01:09:01.335 INFO:teuthology.orchestra.run.smithi122.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-09-11T01:09:01.339 INFO:teuthology.orchestra.run.smithi122.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-09-11T01:09:01.339 INFO:teuthology.orchestra.run.smithi122.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-09-11T01:09:01.416 INFO:teuthology.orchestra.run.smithi122.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-09-11T01:09:01.416 INFO:teuthology.orchestra.run.smithi122.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-09-11T01:09:01.457 INFO:teuthology.orchestra.run.smithi122.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-09-11T01:09:01.457 INFO:teuthology.orchestra.run.smithi122.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-09-11T01:09:01.458 INFO:teuthology.orchestra.run.smithi122.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-09-11T01:09:01.459 INFO:teuthology.orchestra.run.smithi122.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-09-11T01:09:01.459 INFO:teuthology.orchestra.run.smithi122.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-09-11T01:09:01.460 INFO:teuthology.orchestra.run.smithi122.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-09-11T01:09:01.466 INFO:teuthology.orchestra.run.smithi122.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-09-11T01:09:01.538 INFO:teuthology.orchestra.run.smithi122.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-09-11T01:09:01.541 INFO:teuthology.orchestra.run.smithi122.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-09-11T01:09:01.543 INFO:teuthology.orchestra.run.smithi122.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-09-11T01:09:01.619 INFO:teuthology.orchestra.run.smithi122.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-09-11T01:09:01.626 INFO:teuthology.orchestra.run.smithi122.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-09-11T01:09:01.629 INFO:teuthology.orchestra.run.smithi122.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-09-11T01:09:01.630 INFO:teuthology.orchestra.run.smithi122.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.2 [473 kB] 2024-09-11T01:09:01.641 INFO:teuthology.orchestra.run.smithi122.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-09-11T01:09:01.643 INFO:teuthology.orchestra.run.smithi122.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-09-11T01:09:01.650 INFO:teuthology.orchestra.run.smithi122.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-09-11T01:09:01.701 INFO:teuthology.orchestra.run.smithi122.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-11T01:09:01.702 INFO:teuthology.orchestra.run.smithi122.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-09-11T01:09:01.703 INFO:teuthology.orchestra.run.smithi122.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-09-11T01:09:01.782 INFO:teuthology.orchestra.run.smithi122.stdout:Get:106 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-09-11T01:09:01.784 INFO:teuthology.orchestra.run.smithi122.stdout:Get:107 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-09-11T01:09:01.803 INFO:teuthology.orchestra.run.smithi122.stdout:Get:108 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-09-11T01:09:01.804 INFO:teuthology.orchestra.run.smithi122.stdout:Get:109 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-09-11T01:09:01.805 INFO:teuthology.orchestra.run.smithi122.stdout:Get:110 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-09-11T01:09:01.811 INFO:teuthology.orchestra.run.smithi122.stdout:Get:111 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-09-11T01:09:01.812 INFO:teuthology.orchestra.run.smithi122.stdout:Get:112 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-09-11T01:09:02.540 INFO:teuthology.orchestra.run.smithi122.stdout:Get:113 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.3.0-4854-g8f219c47-1jammy [8,625 kB] 2024-09-11T01:09:02.649 INFO:teuthology.orchestra.run.smithi122.stdout:Get:114 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.3.0-4854-g8f219c47-1jammy [14.2 kB] 2024-09-11T01:09:02.649 INFO:teuthology.orchestra.run.smithi122.stdout:Get:115 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.3.0-4854-g8f219c47-1jammy [55.8 MB] 2024-09-11T01:09:03.840 INFO:teuthology.orchestra.run.smithi122.stdout:Get:116 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.3.0-4854-g8f219c47-1jammy [130 kB] 2024-09-11T01:09:03.842 INFO:teuthology.orchestra.run.smithi122.stdout:Get:117 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.3.0-4854-g8f219c47-1jammy [39.0 kB] 2024-09-11T01:09:03.844 INFO:teuthology.orchestra.run.smithi122.stdout:Get:118 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.3.0-4854-g8f219c47-1jammy [15.9 MB] 2024-09-11T01:09:04.147 INFO:teuthology.orchestra.run.smithi122.stdout:Get:119 https://1.chacra.ceph.com/r/ceph/wip-hemanth-testing-2024-09-10-0723/8f219c476cb4f26fc5719c661c0323719a45cb30/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.3.0-4854-g8f219c47-1jammy [91.2 kB] 2024-09-11T01:09:04.728 INFO:teuthology.orchestra.run.smithi122.stdout:Fetched 251 MB in 5s (53.0 MB/s) 2024-09-11T01:09:04.887 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-09-11T01:09:04.987 INFO:teuthology.orchestra.run.smithi122.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165988 files and directories currently installed.) 2024-09-11T01:09:04.993 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-09-11T01:09:05.085 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-11T01:09:05.439 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-11T01:09:05.466 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-09-11T01:09:05.506 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-11T01:09:05.818 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-11T01:09:05.844 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-09-11T01:09:05.876 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-11T01:09:06.322 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-11T01:09:06.348 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-11T01:09:06.396 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-11T01:09:06.767 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-11T01:09:06.793 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-11T01:09:06.833 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-11T01:09:07.170 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-11T01:09:07.196 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-09-11T01:09:07.236 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-11T01:09:07.549 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-09-11T01:09:07.575 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-09-11T01:09:07.607 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-11T01:09:07.910 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package librados2. 2024-09-11T01:09:07.936 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../007-librados2_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T01:09:07.976 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking librados2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:09:08.589 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libnbd0. 2024-09-11T01:09:08.615 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../008-libnbd0_1.10.5-1_amd64.deb ... 2024-09-11T01:09:08.655 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libnbd0 (1.10.5-1) ... 2024-09-11T01:09:08.917 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package librbd1. 2024-09-11T01:09:08.943 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../009-librbd1_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T01:09:08.974 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking librbd1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:09:09.394 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libcephfs2. 2024-09-11T01:09:09.421 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../010-libcephfs2_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T01:09:09.461 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libcephfs2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:09:09.772 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-rados. 2024-09-11T01:09:09.789 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../011-python3-rados_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T01:09:09.830 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-rados (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:09:10.083 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-11T01:09:10.102 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../012-python3-ceph-argparse_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-11T01:09:10.133 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-ceph-argparse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:09:10.353 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-cephfs. 2024-09-11T01:09:10.368 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../013-python3-cephfs_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T01:09:10.411 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-cephfs (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:09:10.673 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-11T01:09:10.688 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../014-python3-ceph-common_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-11T01:09:10.731 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:09:11.034 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-11T01:09:11.060 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-09-11T01:09:11.101 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-11T01:09:11.421 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-prettytable. 2024-09-11T01:09:11.447 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2024-09-11T01:09:11.487 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-09-11T01:09:11.741 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-rbd. 2024-09-11T01:09:11.773 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../017-python3-rbd_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T01:09:11.807 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-rbd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:09:12.119 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-11T01:09:12.145 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-09-11T01:09:12.185 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-11T01:09:12.438 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-09-11T01:09:12.464 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../019-libreadline-dev_8.1.2-1_amd64.deb ... 2024-09-11T01:09:12.504 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-09-11T01:09:12.800 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-09-11T01:09:12.827 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../020-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-09-11T01:09:12.867 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-11T01:09:13.412 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package lua5.1. 2024-09-11T01:09:13.438 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../021-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-09-11T01:09:13.478 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-09-11T01:09:13.865 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package lua-any. 2024-09-11T01:09:13.888 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../022-lua-any_27ubuntu1_all.deb ... 2024-09-11T01:09:13.932 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking lua-any (27ubuntu1) ... 2024-09-11T01:09:14.201 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package zip. 2024-09-11T01:09:14.227 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../023-zip_3.0-12build2_amd64.deb ... 2024-09-11T01:09:14.268 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking zip (3.0-12build2) ... 2024-09-11T01:09:14.546 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package unzip. 2024-09-11T01:09:14.573 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../024-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-09-11T01:09:14.605 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-09-11T01:09:14.967 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package luarocks. 2024-09-11T01:09:14.994 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../025-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-09-11T01:09:15.034 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-09-11T01:09:15.418 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package librgw2. 2024-09-11T01:09:15.444 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../026-librgw2_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T01:09:15.484 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking librgw2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:09:16.095 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-rgw. 2024-09-11T01:09:16.111 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../027-python3-rgw_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T01:09:16.145 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-rgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:09:16.467 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package liboath0:amd64. 2024-09-11T01:09:16.494 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../028-liboath0_2.6.7-3build1_amd64.deb ... 2024-09-11T01:09:16.534 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-09-11T01:09:16.820 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libradosstriper1. 2024-09-11T01:09:16.836 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../029-libradosstriper1_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T01:09:16.870 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libradosstriper1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:09:17.257 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-common. 2024-09-11T01:09:17.284 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../030-ceph-common_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T01:09:17.324 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:09:18.864 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-base. 2024-09-11T01:09:18.890 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../031-ceph-base_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T01:09:18.940 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-base (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:09:19.477 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-09-11T01:09:19.500 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../032-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-09-11T01:09:19.535 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-09-11T01:09:19.822 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-cheroot. 2024-09-11T01:09:19.848 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../033-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-09-11T01:09:19.888 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-11T01:09:20.217 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-09-11T01:09:20.243 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../034-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-09-11T01:09:20.283 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-09-11T01:09:20.578 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-jaraco.text. 2024-09-11T01:09:20.605 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../035-python3-jaraco.text_3.6.0-2_all.deb ... 2024-09-11T01:09:20.645 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-09-11T01:09:20.898 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-09-11T01:09:20.925 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../036-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-09-11T01:09:20.964 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-09-11T01:09:21.301 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-tempora. 2024-09-11T01:09:21.327 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../037-python3-tempora_4.1.2-1_all.deb ... 2024-09-11T01:09:21.368 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-09-11T01:09:21.671 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-portend. 2024-09-11T01:09:21.698 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../038-python3-portend_3.0.0-1_all.deb ... 2024-09-11T01:09:21.738 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-09-11T01:09:21.991 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-09-11T01:09:22.018 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../039-python3-zc.lockfile_2.0-1_all.deb ... 2024-09-11T01:09:22.057 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-09-11T01:09:22.369 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-11T01:09:22.396 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../040-python3-cherrypy3_18.6.1-4_all.deb ... 2024-09-11T01:09:22.436 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-09-11T01:09:22.757 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-natsort. 2024-09-11T01:09:22.783 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../041-python3-natsort_8.0.2-1_all.deb ... 2024-09-11T01:09:22.831 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-09-11T01:09:23.135 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-logutils. 2024-09-11T01:09:23.151 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../042-python3-logutils_0.3.3-8_all.deb ... 2024-09-11T01:09:23.212 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-09-11T01:09:23.697 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-11T01:09:23.723 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../043-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-09-11T01:09:23.763 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-09-11T01:09:24.041 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-mako. 2024-09-11T01:09:24.068 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../044-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-09-11T01:09:24.108 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-11T01:09:24.387 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-11T01:09:24.413 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../045-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-11T01:09:24.445 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-11T01:09:24.740 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-11T01:09:24.766 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../046-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-09-11T01:09:24.806 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-09-11T01:09:25.118 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-webob. 2024-09-11T01:09:25.144 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../047-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2024-09-11T01:09:25.184 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-11T01:09:25.504 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-waitress. 2024-09-11T01:09:25.519 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../048-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-09-11T01:09:25.557 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-11T01:09:25.833 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-tempita. 2024-09-11T01:09:25.859 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../049-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-09-11T01:09:25.891 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-09-11T01:09:26.194 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-paste. 2024-09-11T01:09:26.221 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../050-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-09-11T01:09:26.261 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-09-11T01:09:26.548 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-11T01:09:26.563 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../051-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-09-11T01:09:26.598 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-09-11T01:09:26.893 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-11T01:09:26.919 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../052-python3-pastedeploy_2.1.1-1_all.deb ... 2024-09-11T01:09:26.959 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-09-11T01:09:27.262 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-webtest. 2024-09-11T01:09:27.289 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../053-python3-webtest_2.0.35-1_all.deb ... 2024-09-11T01:09:27.329 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-09-11T01:09:27.632 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pecan. 2024-09-11T01:09:27.648 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../054-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-09-11T01:09:27.682 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-09-11T01:09:28.029 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-certifi. 2024-09-11T01:09:28.056 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../055-python3-certifi_2020.6.20-1_all.deb ... 2024-09-11T01:09:28.088 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-09-11T01:09:28.383 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-idna. 2024-09-11T01:09:28.410 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../056-python3-idna_3.3-1ubuntu0.1_all.deb ... 2024-09-11T01:09:28.450 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-idna (3.3-1ubuntu0.1) ... 2024-09-11T01:09:28.761 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-urllib3. 2024-09-11T01:09:28.787 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../057-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-09-11T01:09:28.869 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-11T01:09:29.189 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-requests. 2024-09-11T01:09:29.216 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../058-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-09-11T01:09:29.256 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-11T01:09:29.567 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-11T01:09:29.583 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../059-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.2_all.deb ... 2024-09-11T01:09:29.626 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-11T01:09:29.921 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-11T01:09:29.937 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../060-ceph-mgr-modules-core_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-11T01:09:29.979 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mgr-modules-core (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:09:30.310 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-11T01:09:30.337 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../061-libsqlite3-mod-ceph_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T01:09:30.376 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libsqlite3-mod-ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:09:30.696 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-11T01:09:30.723 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../062-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-09-11T01:09:30.763 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-09-11T01:09:31.066 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-openssl. 2024-09-11T01:09:31.092 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../063-python3-openssl_21.0.0-1_all.deb ... 2024-09-11T01:09:31.132 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-09-11T01:09:31.411 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mgr. 2024-09-11T01:09:31.438 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../064-ceph-mgr_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T01:09:31.478 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mgr (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:09:31.873 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mon. 2024-09-11T01:09:31.899 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../065-ceph-mon_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T01:09:31.939 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mon (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:09:32.560 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-osd. 2024-09-11T01:09:32.586 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../066-ceph-osd_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T01:09:32.651 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-osd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:09:34.563 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph. 2024-09-11T01:09:34.590 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../067-ceph_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T01:09:34.630 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:09:34.950 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-fuse. 2024-09-11T01:09:34.976 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../068-ceph-fuse_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T01:09:35.016 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:09:35.337 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mds. 2024-09-11T01:09:35.363 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../069-ceph-mds_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T01:09:35.395 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mds (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:09:35.781 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package cephadm. 2024-09-11T01:09:35.808 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../070-cephadm_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T01:09:35.848 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:09:36.160 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-asyncssh. 2024-09-11T01:09:36.186 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../071-python3-asyncssh_2.5.0-1_all.deb ... 2024-09-11T01:09:36.226 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-09-11T01:09:36.563 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python-babel-localedata. 2024-09-11T01:09:36.578 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../072-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-09-11T01:09:36.621 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-11T01:09:37.602 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-babel. 2024-09-11T01:09:37.629 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../073-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-09-11T01:09:37.668 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-09-11T01:09:37.947 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-jinja2. 2024-09-11T01:09:37.973 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../074-python3-jinja2_3.0.3-1ubuntu0.2_all.deb ... 2024-09-11T01:09:38.005 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-11T01:09:38.308 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-11T01:09:38.335 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../075-ceph-mgr-cephadm_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-11T01:09:38.366 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mgr-cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:09:38.704 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-11T01:09:38.730 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../076-python3-repoze.lru_0.7-2_all.deb ... 2024-09-11T01:09:38.770 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-11T01:09:39.032 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-routes. 2024-09-11T01:09:39.058 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../077-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-09-11T01:09:39.090 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-09-11T01:09:39.427 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-11T01:09:39.454 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../078-ceph-mgr-dashboard_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-11T01:09:39.485 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mgr-dashboard (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:09:42.225 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-09-11T01:09:42.242 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../079-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-09-11T01:09:42.283 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-11T01:09:42.771 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-joblib. 2024-09-11T01:09:42.799 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../080-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-09-11T01:09:42.837 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-09-11T01:09:43.199 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-09-11T01:09:43.227 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../081-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-09-11T01:09:43.266 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-09-11T01:09:43.577 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-sklearn. 2024-09-11T01:09:43.605 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../082-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-09-11T01:09:43.730 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-11T01:09:44.699 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-11T01:09:44.727 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../083-ceph-mgr-diskprediction-local_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-11T01:09:44.765 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mgr-diskprediction-local (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:09:45.685 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-cachetools. 2024-09-11T01:09:45.714 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../084-python3-cachetools_5.0.0-1_all.deb ... 2024-09-11T01:09:45.761 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-09-11T01:09:46.064 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-11T01:09:46.092 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../085-python3-pyasn1_0.4.8-1_all.deb ... 2024-09-11T01:09:46.130 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-09-11T01:09:46.459 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-11T01:09:46.487 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../086-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-09-11T01:09:46.525 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-09-11T01:09:46.846 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-rsa. 2024-09-11T01:09:46.873 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../087-python3-rsa_4.8-1_all.deb ... 2024-09-11T01:09:46.912 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-rsa (4.8-1) ... 2024-09-11T01:09:47.233 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-google-auth. 2024-09-11T01:09:47.260 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../088-python3-google-auth_1.5.1-3_all.deb ... 2024-09-11T01:09:47.299 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-09-11T01:09:47.628 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-11T01:09:47.656 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../089-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-09-11T01:09:47.695 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-11T01:09:47.973 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-websocket. 2024-09-11T01:09:48.001 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../090-python3-websocket_1.2.3-1_all.deb ... 2024-09-11T01:09:48.039 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-09-11T01:09:48.319 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-11T01:09:48.346 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../091-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-09-11T01:09:48.376 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-11T01:09:49.067 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-11T01:09:49.095 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../092-ceph-mgr-k8sevents_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-11T01:09:49.133 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-mgr-k8sevents (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:09:49.411 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-11T01:09:49.439 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../093-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-09-11T01:09:49.478 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-11T01:09:49.806 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-11T01:09:49.834 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../094-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-09-11T01:09:49.873 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-11T01:09:50.084 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package jq. 2024-09-11T01:09:50.099 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../095-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-09-11T01:09:50.133 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-09-11T01:09:50.353 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package socat. 2024-09-11T01:09:50.370 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../096-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-09-11T01:09:50.412 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-09-11T01:09:50.707 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package xmlstarlet. 2024-09-11T01:09:50.724 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../097-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-09-11T01:09:50.765 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-09-11T01:09:51.086 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-test. 2024-09-11T01:09:51.103 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../098-ceph-test_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T01:09:51.153 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-test (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:09:53.675 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package ceph-volume. 2024-09-11T01:09:53.702 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../099-ceph-volume_19.3.0-4854-g8f219c47-1jammy_all.deb ... 2024-09-11T01:09:53.741 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking ceph-volume (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:09:54.048 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-11T01:09:54.062 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../100-libcephfs-dev_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T01:09:54.104 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking libcephfs-dev (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:09:54.499 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package lua-socket:amd64. 2024-09-11T01:09:54.527 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../101-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-09-11T01:09:54.590 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-11T01:09:54.906 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package lua-sec:amd64. 2024-09-11T01:09:54.934 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../102-lua-sec_1.0.2-1_amd64.deb ... 2024-09-11T01:09:54.972 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-09-11T01:09:55.335 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package nvme-cli. 2024-09-11T01:09:55.362 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../103-nvme-cli_1.16-3ubuntu0.2_amd64.deb ... 2024-09-11T01:09:55.401 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking nvme-cli (1.16-3ubuntu0.2) ... 2024-09-11T01:09:55.856 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package pkg-config. 2024-09-11T01:09:55.884 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../104-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-09-11T01:09:55.923 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-09-11T01:09:56.184 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-09-11T01:09:56.212 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../105-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-09-11T01:09:56.250 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-09-11T01:09:56.670 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-iniconfig. 2024-09-11T01:09:56.698 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../106-python3-iniconfig_1.1.1-2_all.deb ... 2024-09-11T01:09:56.736 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-09-11T01:09:57.014 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pastescript. 2024-09-11T01:09:57.042 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-4_all.deb ... 2024-09-11T01:09:57.072 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-11T01:09:57.359 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pluggy. 2024-09-11T01:09:57.375 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../108-python3-pluggy_0.13.0-7.1_all.deb ... 2024-09-11T01:09:57.409 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-09-11T01:09:57.712 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-psutil. 2024-09-11T01:09:57.740 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../109-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-09-11T01:09:57.778 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-09-11T01:09:58.073 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-py. 2024-09-11T01:09:58.101 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../110-python3-py_1.10.0-1_all.deb ... 2024-09-11T01:09:58.139 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-py (1.10.0-1) ... 2024-09-11T01:09:58.426 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pygments. 2024-09-11T01:09:58.454 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../111-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-09-11T01:09:58.492 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-09-11T01:09:58.922 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-11T01:09:58.950 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../112-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-09-11T01:09:58.979 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-09-11T01:09:59.291 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-toml. 2024-09-11T01:09:59.319 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../113-python3-toml_0.10.2-1_all.deb ... 2024-09-11T01:09:59.357 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-09-11T01:09:59.685 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-pytest. 2024-09-11T01:09:59.713 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../114-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-09-11T01:09:59.751 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-09-11T01:10:00.297 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package python3-simplejson. 2024-09-11T01:10:00.325 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../115-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-09-11T01:10:00.363 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-09-11T01:10:00.641 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-11T01:10:00.669 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../116-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-09-11T01:10:00.708 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-09-11T01:10:01.453 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package radosgw. 2024-09-11T01:10:01.481 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../117-radosgw_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T01:10:01.519 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking radosgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:10:02.397 INFO:teuthology.orchestra.run.smithi122.stdout:Selecting previously unselected package rbd-fuse. 2024-09-11T01:10:02.414 INFO:teuthology.orchestra.run.smithi122.stdout:Preparing to unpack .../118-rbd-fuse_19.3.0-4854-g8f219c47-1jammy_amd64.deb ... 2024-09-11T01:10:02.464 INFO:teuthology.orchestra.run.smithi122.stdout:Unpacking rbd-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:10:02.761 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-09-11T01:10:03.027 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-09-11T01:10:03.153 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up nvme-cli (1.16-3ubuntu0.2) ... 2024-09-11T01:10:03.409 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-09-11T01:10:03.727 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-09-11T01:10:04.159 INFO:teuthology.orchestra.run.smithi122.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-09-11T01:10:04.264 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:10:05.139 INFO:teuthology.orchestra.run.smithi122.stdout:Adding system user cephadm....done 2024-09-11T01:10:05.287 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-09-11T01:10:05.639 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-09-11T01:10:05.890 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-09-11T01:10:06.016 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-09-11T01:10:06.259 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-11T01:10:06.518 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-09-11T01:10:06.644 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-py (1.10.0-1) ... 2024-09-11T01:10:06.971 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-09-11T01:10:07.323 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-09-11T01:10:07.574 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-09-11T01:10:07.842 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-09-11T01:10:08.094 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-09-11T01:10:08.345 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-09-11T01:10:08.612 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-09-11T01:10:08.855 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-ceph-argparse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:10:09.123 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-09-11T01:10:09.249 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libnbd0 (1.10.5-1) ... 2024-09-11T01:10:09.391 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-09-11T01:10:09.517 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-09-11T01:10:09.643 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-09-11T01:10:09.894 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-09-11T01:10:10.264 INFO:teuthology.orchestra.run.smithi122.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-09-11T01:10:10.300 INFO:teuthology.orchestra.run.smithi122.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-09-11T01:10:10.356 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-09-11T01:10:10.482 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-09-11T01:10:10.825 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-09-11T01:10:10.951 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-09-11T01:10:11.244 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-09-11T01:10:11.511 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-09-11T01:10:11.821 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up zip (3.0-12build2) ... 2024-09-11T01:10:11.930 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-09-11T01:10:12.574 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-09-11T01:10:12.808 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-09-11T01:10:12.934 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-09-11T01:10:13.060 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-09-11T01:10:13.378 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-09-11T01:10:13.787 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-09-11T01:10:14.021 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-09-11T01:10:14.398 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-09-11T01:10:14.683 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.2) ... 2024-09-11T01:10:15.018 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-09-11T01:10:15.269 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-09-11T01:10:15.395 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:10:15.704 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-idna (3.3-1ubuntu0.1) ... 2024-09-11T01:10:16.022 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-09-11T01:10:17.186 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-09-11T01:10:17.446 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-11T01:10:17.580 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-toml (0.10.2-1) ... 2024-09-11T01:10:17.840 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-09-11T01:10:17.966 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-09-11T01:10:18.092 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-09-11T01:10:18.385 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-09-11T01:10:18.636 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-09-11T01:10:18.912 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-11T01:10:19.038 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-09-11T01:10:19.339 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-09-11T01:10:19.590 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-09-11T01:10:19.850 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-09-11T01:10:20.109 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-11T01:10:20.360 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-09-11T01:10:20.628 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-09-11T01:10:20.746 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-09-11T01:10:21.263 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-09-11T01:10:21.373 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-09-11T01:10:21.666 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2024-09-11T01:10:21.942 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-09-11T01:10:22.176 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-09-11T01:10:22.302 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-09-11T01:10:22.428 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-09-11T01:10:22.525 INFO:teuthology.orchestra.run.smithi122.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-09-11T01:10:22.788 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-09-11T01:10:22.889 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-09-11T01:10:23.274 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-09-11T01:10:23.525 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up lua-any (27ubuntu1) ... 2024-09-11T01:10:23.676 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-portend (3.0.0-1) ... 2024-09-11T01:10:23.935 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-09-11T01:10:24.061 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.2) ... 2024-09-11T01:10:24.379 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-09-11T01:10:24.688 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-09-11T01:10:24.974 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-09-11T01:10:25.091 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-rsa (4.8-1) ... 2024-09-11T01:10:25.350 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-09-11T01:10:25.712 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-09-11T01:10:25.979 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-09-11T01:10:26.340 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-11T01:10:26.658 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-09-11T01:10:27.010 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-09-11T01:10:27.136 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up librados2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:10:27.262 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libsqlite3-mod-ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:10:27.371 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-09-11T01:10:27.683 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libcephfs2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:10:27.809 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libradosstriper1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:10:27.935 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-09-11T01:10:28.211 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up librbd1 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:10:28.337 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mgr-modules-core (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:10:28.438 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:10:28.634 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-11T01:10:28.635 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-11T01:10:29.098 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up libcephfs-dev (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:10:29.224 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-rados (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:10:29.350 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up librgw2 (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:10:29.476 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-09-11T01:10:30.671 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-rbd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:10:30.772 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up rbd-fuse (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:10:30.890 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-rgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:10:31.016 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up python3-cephfs (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:10:31.142 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-common (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:10:31.677 INFO:teuthology.orchestra.run.smithi122.stdout:Adding group ceph....done 2024-09-11T01:10:32.004 INFO:teuthology.orchestra.run.smithi122.stdout:Adding system user ceph....done 2024-09-11T01:10:32.172 INFO:teuthology.orchestra.run.smithi122.stdout:Setting system user ceph properties....done 2024-09-11T01:10:32.187 INFO:teuthology.orchestra.run.smithi122.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-11T01:10:32.329 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-11T01:10:32.643 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-11T01:10:33.093 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-test (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:10:33.219 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up radosgw (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:10:33.660 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-11T01:10:33.661 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-11T01:10:34.113 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-base (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:10:34.430 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-11T01:10:34.906 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mds (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:10:35.131 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-11T01:10:35.131 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-11T01:10:35.599 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mgr (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:10:35.809 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-11T01:10:35.809 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-11T01:10:36.292 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-osd (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:10:36.591 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-11T01:10:36.591 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-11T01:10:37.095 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mgr-k8sevents (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:10:37.221 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mgr-diskprediction-local (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:10:37.355 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mon (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:10:37.548 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-11T01:10:37.549 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-11T01:10:38.023 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mgr-cephadm (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:10:38.174 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:10:38.300 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-mgr-dashboard (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:10:38.451 INFO:teuthology.orchestra.run.smithi122.stdout:Setting up ceph-volume (19.3.0-4854-g8f219c47-1jammy) ... 2024-09-11T01:10:38.804 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-09-11T01:10:38.921 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-09-11T01:10:39.065 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-09-11T01:10:40.191 INFO:teuthology.orchestra.run.smithi122.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-09-11T01:10:40.840 INFO:teuthology.orchestra.run.smithi122.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-11T01:10:40.847 DEBUG:teuthology.parallel:result is None 2024-09-11T01:10:40.847 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=8f219c476cb4f26fc5719c661c0323719a45cb30 2024-09-11T01:10:40.997 DEBUG:teuthology.orchestra.run.smithi122:> dpkg-query -W -f '${Version}' ceph 2024-09-11T01:10:41.016 INFO:teuthology.orchestra.run.smithi122.stdout:19.3.0-4854-g8f219c47-1jammy 2024-09-11T01:10:41.016 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-4854-g8f219c47-1jammy 2024-09-11T01:10:41.016 INFO:teuthology.task.install:The correct ceph version 19.3.0-4854-g8f219c47-1jammy is installed. 2024-09-11T01:10:41.018 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-11T01:10:41.019 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-11T01:10:41.019 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-11T01:10:41.072 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-11T01:10:41.073 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-11T01:10:41.073 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/daemon-helper 2024-09-11T01:10:41.126 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-11T01:10:41.186 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-11T01:10:41.186 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-11T01:10:41.187 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-11T01:10:41.242 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-11T01:10:41.298 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-11T01:10:41.298 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-11T01:10:41.298 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/stdin-killer 2024-09-11T01:10:41.350 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-11T01:10:41.406 INFO:teuthology.run_tasks:Running task ceph... 2024-09-11T01:10:41.485 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-11T01:10:41.485 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 777 /var/log/ceph 2024-09-11T01:10:41.498 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-11T01:10:41.499 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-11T01:10:41.554 INFO:tasks.ceph:Creating extra log directories... 2024-09-11T01:10:41.554 DEBUG:teuthology.orchestra.run.smithi122:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-11T01:10:41.622 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-11T01:10:41.622 INFO:tasks.ceph:config {'conf': {'global': {'mon election default strategy': 3, 'ms bind msgr2': False, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(OBJECT_', '\\(SLOW_OPS\\)', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': False, 'mon_bind_addrvec': True} 2024-09-11T01:10:41.623 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/hyelloji-2024-09-10_17:16:14-rados-wip-hemanth-testing-2024-09-10-0723-distro-default-smithi/7899875', 'branch': 'wip-hemanth-testing-2024-09-10-0723', 'description': 'rados/singleton/{all/lost-unfound-delete mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/{bluestore-options/write$/{write_v2} bluestore/bluestore-bitmap} rados supported-random-distro$/{ubuntu_latest}}', 'email': 'hyelloji@redhat.com', 'first_in_suite': False, 'job_id': '7899875', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'hyelloji-2024-09-10_17:16:14-rados-wip-hemanth-testing-2024-09-10-0723-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 3, 'size': 10}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-hemanth-testing-2024-09-10-0723'}, 'ceph': {'conf': {'global': {'mon election default strategy': 3, 'ms bind msgr2': False, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'mon_bind_msgr2': False, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-hemanth-testing-2024-09-10-0723', 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30'}}, 'owner': 'scheduled_hyelloji@teuthology', 'priority': 100, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.b', 'mon.c', 'mgr.x', 'osd.0', 'osd.1', 'osd.2']], 'seed': 9167, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'wip-hemanth-testing-2024-09-10-0723', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_8f219c476cb4f26fc5719c661c0323719a45cb30/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'targets': {'smithi122.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM+/n2TYwpW+YECWrfF4akfSkyFIEd2DUdQ/HpoGab2ONTv8Lt7O0uDxbYTTpzr15ALmTtGnpIkDLHNHGKUsq5U='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'log-ignorelist': ['objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(OBJECT_', '\\(SLOW_OPS\\)', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'pre-mgr-commands': ['sudo ceph config set mgr mgr_pool false --force'], 'conf': {'global': {'mon election default strategy': 3, 'ms bind msgr2': False, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'fs': 'xfs', 'mon_bind_msgr2': False, 'sha1': '8f219c476cb4f26fc5719c661c0323719a45cb30', 'cluster': 'ceph'}}, {'rep_lost_unfound_delete': None}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-10_17:16:14', 'tube': 'smithi', 'user': 'hyelloji', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-11T01:10:41.623 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-11T01:10:41.663 DEBUG:teuthology.orchestra.run.smithi122:> sudo install -d -m0777 -- /var/run/ceph 2024-09-11T01:10:41.719 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-11T01:10:41.719 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/scratch_devs of=/dev/stdout 2024-09-11T01:10:41.768 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-11T01:10:41.768 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_1 2024-09-11T01:10:41.815 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-11T01:10:41.816 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T01:10:41.816 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 884 Links: 1 2024-09-11T01:10:41.816 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T01:10:41.816 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-09-11 01:05:22.285844498 +0000 2024-09-11T01:10:41.816 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-09-11 01:05:22.045846642 +0000 2024-09-11T01:10:41.816 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-09-11 01:05:22.045846642 +0000 2024-09-11T01:10:41.816 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: - 2024-09-11T01:10:41.816 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-11T01:10:41.869 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-09-11T01:10:41.869 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-09-11T01:10:41.869 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000338616 s, 1.5 MB/s 2024-09-11T01:10:41.871 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-11T01:10:41.920 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_2 2024-09-11T01:10:41.967 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-11T01:10:41.967 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T01:10:41.968 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 906 Links: 1 2024-09-11T01:10:41.968 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T01:10:41.968 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-09-11 01:05:22.769840175 +0000 2024-09-11T01:10:41.968 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-09-11 01:05:22.521842390 +0000 2024-09-11T01:10:41.968 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-09-11 01:05:22.521842390 +0000 2024-09-11T01:10:41.968 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: - 2024-09-11T01:10:41.968 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-11T01:10:42.021 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-09-11T01:10:42.021 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-09-11T01:10:42.021 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000334662 s, 1.5 MB/s 2024-09-11T01:10:42.022 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-11T01:10:42.069 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_3 2024-09-11T01:10:42.115 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-11T01:10:42.115 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T01:10:42.115 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 922 Links: 1 2024-09-11T01:10:42.115 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T01:10:42.115 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-09-11 01:05:23.253835851 +0000 2024-09-11T01:10:42.115 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-09-11 01:05:23.013837996 +0000 2024-09-11T01:10:42.116 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-09-11 01:05:23.013837996 +0000 2024-09-11T01:10:42.116 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: - 2024-09-11T01:10:42.116 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-11T01:10:42.169 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-09-11T01:10:42.170 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-09-11T01:10:42.170 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000304795 s, 1.7 MB/s 2024-09-11T01:10:42.171 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-11T01:10:42.220 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_4 2024-09-11T01:10:42.272 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-11T01:10:42.272 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-11T01:10:42.273 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 937 Links: 1 2024-09-11T01:10:42.273 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-11T01:10:42.273 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-09-11 01:05:23.753831385 +0000 2024-09-11T01:10:42.273 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-09-11 01:05:23.497833673 +0000 2024-09-11T01:10:42.273 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-09-11 01:05:23.497833673 +0000 2024-09-11T01:10:42.273 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: - 2024-09-11T01:10:42.273 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-11T01:10:42.325 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-09-11T01:10:42.325 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-09-11T01:10:42.325 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000301708 s, 1.7 MB/s 2024-09-11T01:10:42.326 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-11T01:10:42.376 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-11T01:10:42.377 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi122.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2024-09-11T01:10:42.377 INFO:tasks.ceph:Generating config... 2024-09-11T01:10:42.378 INFO:tasks.ceph:[global] mon election default strategy = 3 2024-09-11T01:10:42.378 INFO:tasks.ceph:[global] ms bind msgr2 = False 2024-09-11T01:10:42.378 INFO:tasks.ceph:[global] ms type = async 2024-09-11T01:10:42.378 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-11T01:10:42.378 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-11T01:10:42.378 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-11T01:10:42.378 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-11T01:10:42.378 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-11T01:10:42.378 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-09-11T01:10:42.379 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-11T01:10:42.379 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-11T01:10:42.379 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-11T01:10:42.379 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-11T01:10:42.379 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-11T01:10:42.379 INFO:tasks.ceph:[osd] bluestore write v2 = True 2024-09-11T01:10:42.379 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-09-11T01:10:42.379 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-09-11T01:10:42.379 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-09-11T01:10:42.379 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-11T01:10:42.379 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-11T01:10:42.379 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-09-11T01:10:42.379 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-11T01:10:42.379 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-11T01:10:42.380 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-11T01:10:42.380 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-09-11T01:10:42.380 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-09-11T01:10:42.380 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-11T01:10:42.380 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-09-11T01:10:42.380 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-09-11T01:10:42.380 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-11T01:10:42.380 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-09-11T01:10:42.380 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-09-11T01:10:42.380 INFO:tasks.ceph:Setting up mon.a... 2024-09-11T01:10:42.380 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-11T01:10:42.451 INFO:teuthology.orchestra.run.smithi122.stdout:creating /etc/ceph/ceph.keyring 2024-09-11T01:10:42.459 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-11T01:10:42.538 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-11T01:10:42.590 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '[v1:172.21.15.122:6789]'), ('mon.b', '[v1:172.21.15.122:6790]'), ('mon.c', '[v1:172.21.15.122:6791]')] 2024-09-11T01:10:42.591 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '[v1:172.21.15.122:6789],[v1:172.21.15.122:6790],[v1:172.21.15.122:6791]', 'mon election default strategy': 3, 'ms bind msgr2': False, 'ms type': 'async'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore write v2': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon scrub interval': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}} 2024-09-11T01:10:42.591 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-11T01:10:42.591 DEBUG:teuthology.orchestra.run.smithi122:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-11T01:10:42.640 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --addv a '[v1:172.21.15.122:6789]' --addv b '[v1:172.21.15.122:6790]' --addv c '[v1:172.21.15.122:6791]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-11T01:10:42.725 INFO:teuthology.orchestra.run.smithi122.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-11T01:10:42.725 INFO:teuthology.orchestra.run.smithi122.stdout:monmaptool: generated fsid 6d328773-3914-4412-91b7-393379984de3 2024-09-11T01:10:42.725 INFO:teuthology.orchestra.run.smithi122.stdout:setting min_mon_release = quincy 2024-09-11T01:10:42.725 INFO:teuthology.orchestra.run.smithi122.stdout:epoch 0 2024-09-11T01:10:42.725 INFO:teuthology.orchestra.run.smithi122.stdout:fsid 6d328773-3914-4412-91b7-393379984de3 2024-09-11T01:10:42.725 INFO:teuthology.orchestra.run.smithi122.stdout:last_changed 2024-09-11T01:10:42.726440+0000 2024-09-11T01:10:42.725 INFO:teuthology.orchestra.run.smithi122.stdout:created 2024-09-11T01:10:42.726440+0000 2024-09-11T01:10:42.725 INFO:teuthology.orchestra.run.smithi122.stdout:min_mon_release 17 (quincy) 2024-09-11T01:10:42.726 INFO:teuthology.orchestra.run.smithi122.stdout:election_strategy: 3 2024-09-11T01:10:42.726 INFO:teuthology.orchestra.run.smithi122.stdout:0: v1:172.21.15.122:6789/0 mon.a 2024-09-11T01:10:42.726 INFO:teuthology.orchestra.run.smithi122.stdout:1: v1:172.21.15.122:6790/0 mon.b 2024-09-11T01:10:42.726 INFO:teuthology.orchestra.run.smithi122.stdout:2: v1:172.21.15.122:6791/0 mon.c 2024-09-11T01:10:42.726 INFO:teuthology.orchestra.run.smithi122.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-11T01:10:42.727 DEBUG:teuthology.orchestra.run.smithi122:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-11T01:10:42.780 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 6d328773-3914-4412-91b7-393379984de3... 2024-09-11T01:10:42.781 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-11T01:10:42.852 INFO:teuthology.orchestra.run.smithi122.stdout:[global] 2024-09-11T01:10:42.852 INFO:teuthology.orchestra.run.smithi122.stdout: chdir = "" 2024-09-11T01:10:42.852 INFO:teuthology.orchestra.run.smithi122.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-11T01:10:42.852 INFO:teuthology.orchestra.run.smithi122.stdout: auth supported = cephx 2024-09-11T01:10:42.852 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:10:42.852 INFO:teuthology.orchestra.run.smithi122.stdout: filestore xattr use omap = true 2024-09-11T01:10:42.852 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:10:42.852 INFO:teuthology.orchestra.run.smithi122.stdout: mon clock drift allowed = 1.000 2024-09-11T01:10:42.852 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:10:42.852 INFO:teuthology.orchestra.run.smithi122.stdout: osd crush chooseleaf type = 0 2024-09-11T01:10:42.852 INFO:teuthology.orchestra.run.smithi122.stdout: auth debug = true 2024-09-11T01:10:42.853 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:10:42.853 INFO:teuthology.orchestra.run.smithi122.stdout: ms die on old message = true 2024-09-11T01:10:42.853 INFO:teuthology.orchestra.run.smithi122.stdout: ms die on bug = true 2024-09-11T01:10:42.853 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:10:42.853 INFO:teuthology.orchestra.run.smithi122.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-11T01:10:42.853 INFO:teuthology.orchestra.run.smithi122.stdout: mon pg warn max object skew = 0 2024-09-11T01:10:42.853 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:10:42.853 INFO:teuthology.orchestra.run.smithi122.stdout: # disable pg_autoscaler by default for new pools 2024-09-11T01:10:42.853 INFO:teuthology.orchestra.run.smithi122.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-11T01:10:42.853 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:10:42.853 INFO:teuthology.orchestra.run.smithi122.stdout: osd pool default size = 2 2024-09-11T01:10:42.853 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:10:42.853 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd allow primary affinity = true 2024-09-11T01:10:42.853 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd allow pg remap = true 2024-09-11T01:10:42.854 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on legacy crush tunables = false 2024-09-11T01:10:42.854 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on crush straw calc version zero = false 2024-09-11T01:10:42.854 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on no sortbitwise = false 2024-09-11T01:10:42.854 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on osd down out interval zero = false 2024-09-11T01:10:42.854 INFO:teuthology.orchestra.run.smithi122.stdout: mon warn on too few osds = false 2024-09-11T01:10:42.854 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-11T01:10:42.854 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-11T01:10:42.854 INFO:teuthology.orchestra.run.smithi122.stdout: mon_allow_pool_size_one = true 2024-09-11T01:10:42.854 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:10:42.854 INFO:teuthology.orchestra.run.smithi122.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-11T01:10:42.854 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:10:42.854 INFO:teuthology.orchestra.run.smithi122.stdout: osd default data pool replay window = 5 2024-09-11T01:10:42.854 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:10:42.854 INFO:teuthology.orchestra.run.smithi122.stdout: mon allow pool delete = true 2024-09-11T01:10:42.855 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:10:42.855 INFO:teuthology.orchestra.run.smithi122.stdout: mon cluster log file level = debug 2024-09-11T01:10:42.855 INFO:teuthology.orchestra.run.smithi122.stdout: debug asserts on shutdown = true 2024-09-11T01:10:42.855 INFO:teuthology.orchestra.run.smithi122.stdout: mon health detail to clog = false 2024-09-11T01:10:42.855 INFO:teuthology.orchestra.run.smithi122.stdout: mon host = "[v1:172.21.15.122:6789],[v1:172.21.15.122:6790],[v1:172.21.15.122:6791]" 2024-09-11T01:10:42.855 INFO:teuthology.orchestra.run.smithi122.stdout: mon election default strategy = 3 2024-09-11T01:10:42.855 INFO:teuthology.orchestra.run.smithi122.stdout: ms bind msgr2 = False 2024-09-11T01:10:42.855 INFO:teuthology.orchestra.run.smithi122.stdout: ms type = async 2024-09-11T01:10:42.855 INFO:teuthology.orchestra.run.smithi122.stdout: fsid = 6d328773-3914-4412-91b7-393379984de3 2024-09-11T01:10:42.855 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:10:42.855 INFO:teuthology.orchestra.run.smithi122.stdout:[osd] 2024-09-11T01:10:42.855 INFO:teuthology.orchestra.run.smithi122.stdout: osd journal size = 100 2024-09-11T01:10:42.855 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:10:42.856 INFO:teuthology.orchestra.run.smithi122.stdout: osd scrub load threshold = 5.0 2024-09-11T01:10:42.856 INFO:teuthology.orchestra.run.smithi122.stdout: osd scrub max interval = 600 2024-09-11T01:10:42.856 INFO:teuthology.orchestra.run.smithi122.stdout: osd mclock profile = high_recovery_ops 2024-09-11T01:10:42.856 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:10:42.856 INFO:teuthology.orchestra.run.smithi122.stdout: osd recover clone overlap = true 2024-09-11T01:10:42.856 INFO:teuthology.orchestra.run.smithi122.stdout: osd recovery max chunk = 1048576 2024-09-11T01:10:42.856 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:10:42.856 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug shutdown = true 2024-09-11T01:10:42.856 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug op order = true 2024-09-11T01:10:42.856 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug verify stray on activate = true 2024-09-11T01:10:42.856 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug trim objects = true 2024-09-11T01:10:42.856 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:10:42.856 INFO:teuthology.orchestra.run.smithi122.stdout: osd open classes on start = true 2024-09-11T01:10:42.856 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug pg log writeout = true 2024-09-11T01:10:42.857 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:10:42.857 INFO:teuthology.orchestra.run.smithi122.stdout: osd deep scrub update digest min age = 30 2024-09-11T01:10:42.857 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:10:42.857 INFO:teuthology.orchestra.run.smithi122.stdout: osd map max advance = 10 2024-09-11T01:10:42.857 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:10:42.857 INFO:teuthology.orchestra.run.smithi122.stdout: journal zero on create = true 2024-09-11T01:10:42.857 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:10:42.857 INFO:teuthology.orchestra.run.smithi122.stdout: filestore ondisk finisher threads = 3 2024-09-11T01:10:42.857 INFO:teuthology.orchestra.run.smithi122.stdout: filestore apply finisher threads = 3 2024-09-11T01:10:42.857 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:10:42.857 INFO:teuthology.orchestra.run.smithi122.stdout: bdev debug aio = true 2024-09-11T01:10:42.857 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug misdirected ops = true 2024-09-11T01:10:42.857 INFO:teuthology.orchestra.run.smithi122.stdout: bdev async discard = True 2024-09-11T01:10:42.857 INFO:teuthology.orchestra.run.smithi122.stdout: bdev enable discard = True 2024-09-11T01:10:42.858 INFO:teuthology.orchestra.run.smithi122.stdout: bluestore allocator = bitmap 2024-09-11T01:10:42.858 INFO:teuthology.orchestra.run.smithi122.stdout: bluestore block size = 96636764160 2024-09-11T01:10:42.858 INFO:teuthology.orchestra.run.smithi122.stdout: bluestore fsck on mount = True 2024-09-11T01:10:42.858 INFO:teuthology.orchestra.run.smithi122.stdout: bluestore write v2 = True 2024-09-11T01:10:42.858 INFO:teuthology.orchestra.run.smithi122.stdout: bluestore zero block detection = True 2024-09-11T01:10:42.858 INFO:teuthology.orchestra.run.smithi122.stdout: debug bluefs = 20 2024-09-11T01:10:42.858 INFO:teuthology.orchestra.run.smithi122.stdout: debug bluestore = 20 2024-09-11T01:10:42.858 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-09-11T01:10:42.858 INFO:teuthology.orchestra.run.smithi122.stdout: debug osd = 20 2024-09-11T01:10:42.858 INFO:teuthology.orchestra.run.smithi122.stdout: debug rocksdb = 10 2024-09-11T01:10:42.858 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd backfillfull_ratio = 0.85 2024-09-11T01:10:42.858 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd full ratio = 0.9 2024-09-11T01:10:42.858 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd nearfull ratio = 0.8 2024-09-11T01:10:42.858 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug verify cached snaps = True 2024-09-11T01:10:42.859 INFO:teuthology.orchestra.run.smithi122.stdout: osd debug verify missing on start = True 2024-09-11T01:10:42.859 INFO:teuthology.orchestra.run.smithi122.stdout: osd failsafe full ratio = 0.95 2024-09-11T01:10:42.859 INFO:teuthology.orchestra.run.smithi122.stdout: osd mclock override recovery settings = True 2024-09-11T01:10:42.859 INFO:teuthology.orchestra.run.smithi122.stdout: osd objectstore = bluestore 2024-09-11T01:10:42.859 INFO:teuthology.orchestra.run.smithi122.stdout: osd op queue = debug_random 2024-09-11T01:10:42.859 INFO:teuthology.orchestra.run.smithi122.stdout: osd op queue cut off = debug_random 2024-09-11T01:10:42.859 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:10:42.859 INFO:teuthology.orchestra.run.smithi122.stdout:[mgr] 2024-09-11T01:10:42.859 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-09-11T01:10:42.859 INFO:teuthology.orchestra.run.smithi122.stdout: debug mgr = 20 2024-09-11T01:10:42.859 INFO:teuthology.orchestra.run.smithi122.stdout: debug mon = 20 2024-09-11T01:10:42.859 INFO:teuthology.orchestra.run.smithi122.stdout: debug auth = 20 2024-09-11T01:10:42.859 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min pgs per osd = 4 2024-09-11T01:10:42.859 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min bytes per osd = 10 2024-09-11T01:10:42.860 INFO:teuthology.orchestra.run.smithi122.stdout: mgr/telemetry/nag = false 2024-09-11T01:10:42.860 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:10:42.860 INFO:teuthology.orchestra.run.smithi122.stdout:[mon] 2024-09-11T01:10:42.860 INFO:teuthology.orchestra.run.smithi122.stdout: debug ms = 1 2024-09-11T01:10:42.860 INFO:teuthology.orchestra.run.smithi122.stdout: debug mon = 20 2024-09-11T01:10:42.860 INFO:teuthology.orchestra.run.smithi122.stdout: debug paxos = 20 2024-09-11T01:10:42.860 INFO:teuthology.orchestra.run.smithi122.stdout: debug auth = 20 2024-09-11T01:10:42.860 INFO:teuthology.orchestra.run.smithi122.stdout: mon data avail warn = 5 2024-09-11T01:10:42.860 INFO:teuthology.orchestra.run.smithi122.stdout: mon mgr mkfs grace = 240 2024-09-11T01:10:42.860 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min pgs per osd = 4 2024-09-11T01:10:42.860 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd reporter subtree level = osd 2024-09-11T01:10:42.860 INFO:teuthology.orchestra.run.smithi122.stdout: mon osd prime pg temp = true 2024-09-11T01:10:42.860 INFO:teuthology.orchestra.run.smithi122.stdout: mon reweight min bytes per osd = 10 2024-09-11T01:10:42.860 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:10:42.861 INFO:teuthology.orchestra.run.smithi122.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-11T01:10:42.861 INFO:teuthology.orchestra.run.smithi122.stdout: auth mon ticket ttl = 660 # 11m 2024-09-11T01:10:42.861 INFO:teuthology.orchestra.run.smithi122.stdout: auth service ticket ttl = 240 # 4m 2024-09-11T01:10:42.861 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:10:42.861 INFO:teuthology.orchestra.run.smithi122.stdout: # don't complain about insecure global_id in the test suite 2024-09-11T01:10:42.861 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-11T01:10:42.861 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-11T01:10:42.861 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:10:42.861 INFO:teuthology.orchestra.run.smithi122.stdout: # 1m isn't quite enough 2024-09-11T01:10:42.861 INFO:teuthology.orchestra.run.smithi122.stdout: mon_down_mkfs_grace = 2m 2024-09-11T01:10:42.861 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:10:42.861 INFO:teuthology.orchestra.run.smithi122.stdout: mon_warn_on_filestore_osds = false 2024-09-11T01:10:42.861 INFO:teuthology.orchestra.run.smithi122.stdout: mon scrub interval = 300 2024-09-11T01:10:42.861 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:10:42.862 INFO:teuthology.orchestra.run.smithi122.stdout:[client] 2024-09-11T01:10:42.862 INFO:teuthology.orchestra.run.smithi122.stdout: rgw cache enabled = true 2024-09-11T01:10:42.862 INFO:teuthology.orchestra.run.smithi122.stdout: rgw enable ops log = true 2024-09-11T01:10:42.862 INFO:teuthology.orchestra.run.smithi122.stdout: rgw enable usage log = true 2024-09-11T01:10:42.862 INFO:teuthology.orchestra.run.smithi122.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-11T01:10:42.862 INFO:teuthology.orchestra.run.smithi122.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-11T01:10:42.862 INFO:teuthology.orchestra.run.smithi122.stdout:[mon.a] 2024-09-11T01:10:42.862 INFO:teuthology.orchestra.run.smithi122.stdout:[mon.b] 2024-09-11T01:10:42.862 INFO:teuthology.orchestra.run.smithi122.stdout:[mon.c] 2024-09-11T01:10:42.863 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-11T01:10:42.863 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-11T01:10:42.953 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-11T01:10:42.954 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-11T01:10:42.954 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-11T01:10:42.999 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-11T01:10:42.999 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-11T01:10:43.047 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi122.front.sepia.ceph.com 2024-09-11T01:10:43.047 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-11T01:10:43.047 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-11T01:10:43.047 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-11T01:10:43.112 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-11T01:10:43.112 DEBUG:teuthology.orchestra.run.smithi122:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-11T01:10:43.159 INFO:tasks.ceph:Setting up mon nodes... 2024-09-11T01:10:43.160 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-11T01:10:43.160 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-11T01:10:43.244 INFO:teuthology.orchestra.run.smithi122.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-11T01:10:43.247 INFO:tasks.ceph:Setting up mds nodes... 2024-09-11T01:10:43.247 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-11T01:10:43.247 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-11T01:10:43.247 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi122.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2024-09-11T01:10:43.248 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-11T01:10:43.305 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-11T01:10:43.305 INFO:tasks.ceph:role: osd.0 2024-09-11T01:10:43.305 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi122.front.sepia.ceph.com 2024-09-11T01:10:43.305 DEBUG:teuthology.orchestra.run.smithi122:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-11T01:10:43.367 INFO:teuthology.orchestra.run.smithi122.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-11T01:10:43.367 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-11T01:10:43.367 INFO:teuthology.orchestra.run.smithi122.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-11T01:10:43.367 INFO:teuthology.orchestra.run.smithi122.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-11T01:10:43.367 INFO:teuthology.orchestra.run.smithi122.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-11T01:10:43.367 INFO:teuthology.orchestra.run.smithi122.stdout: = sunit=0 swidth=0 blks 2024-09-11T01:10:43.367 INFO:teuthology.orchestra.run.smithi122.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-11T01:10:43.367 INFO:teuthology.orchestra.run.smithi122.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-11T01:10:43.368 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-11T01:10:43.368 INFO:teuthology.orchestra.run.smithi122.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-11T01:10:43.810 INFO:teuthology.orchestra.run.smithi122.stdout:Discarding blocks...Done. 2024-09-11T01:10:43.813 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi122.front.sepia.ceph.com -o noatime 2024-09-11T01:10:43.813 DEBUG:teuthology.orchestra.run.smithi122:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-11T01:10:43.900 DEBUG:teuthology.orchestra.run.smithi122:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-11T01:10:43.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:10:43.909 INFO:teuthology.orchestra.run.smithi122.stderr:sudo: /sbin/restorecon: command not found 2024-09-11T01:10:43.910 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-11T01:10:43.965 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-11T01:10:43.965 INFO:tasks.ceph:role: osd.1 2024-09-11T01:10:43.965 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi122.front.sepia.ceph.com 2024-09-11T01:10:43.965 DEBUG:teuthology.orchestra.run.smithi122:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-11T01:10:44.028 INFO:teuthology.orchestra.run.smithi122.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-11T01:10:44.028 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-11T01:10:44.028 INFO:teuthology.orchestra.run.smithi122.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-11T01:10:44.028 INFO:teuthology.orchestra.run.smithi122.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-11T01:10:44.028 INFO:teuthology.orchestra.run.smithi122.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-11T01:10:44.028 INFO:teuthology.orchestra.run.smithi122.stdout: = sunit=0 swidth=0 blks 2024-09-11T01:10:44.028 INFO:teuthology.orchestra.run.smithi122.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-11T01:10:44.028 INFO:teuthology.orchestra.run.smithi122.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-11T01:10:44.028 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-11T01:10:44.028 INFO:teuthology.orchestra.run.smithi122.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-11T01:10:44.495 INFO:teuthology.orchestra.run.smithi122.stdout:Discarding blocks...Done. 2024-09-11T01:10:44.498 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi122.front.sepia.ceph.com -o noatime 2024-09-11T01:10:44.498 DEBUG:teuthology.orchestra.run.smithi122:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-11T01:10:44.639 DEBUG:teuthology.orchestra.run.smithi122:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-11T01:10:44.650 INFO:teuthology.orchestra.run.smithi122.stderr:sudo: /sbin/restorecon: command not found 2024-09-11T01:10:44.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:10:44.651 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-11T01:10:44.707 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-11T01:10:44.707 INFO:tasks.ceph:role: osd.2 2024-09-11T01:10:44.707 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi122.front.sepia.ceph.com 2024-09-11T01:10:44.707 DEBUG:teuthology.orchestra.run.smithi122:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-11T01:10:44.770 INFO:teuthology.orchestra.run.smithi122.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-11T01:10:44.770 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-11T01:10:44.770 INFO:teuthology.orchestra.run.smithi122.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-11T01:10:44.770 INFO:teuthology.orchestra.run.smithi122.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-09-11T01:10:44.770 INFO:teuthology.orchestra.run.smithi122.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-11T01:10:44.770 INFO:teuthology.orchestra.run.smithi122.stdout: = sunit=0 swidth=0 blks 2024-09-11T01:10:44.770 INFO:teuthology.orchestra.run.smithi122.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-11T01:10:44.770 INFO:teuthology.orchestra.run.smithi122.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-11T01:10:44.770 INFO:teuthology.orchestra.run.smithi122.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-11T01:10:44.770 INFO:teuthology.orchestra.run.smithi122.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-11T01:10:45.236 INFO:teuthology.orchestra.run.smithi122.stdout:Discarding blocks...Done. 2024-09-11T01:10:45.238 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi122.front.sepia.ceph.com -o noatime 2024-09-11T01:10:45.238 DEBUG:teuthology.orchestra.run.smithi122:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-11T01:10:45.363 DEBUG:teuthology.orchestra.run.smithi122:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-11T01:10:45.374 INFO:teuthology.orchestra.run.smithi122.stderr:sudo: /sbin/restorecon: command not found 2024-09-11T01:10:45.375 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:10:45.375 DEBUG:teuthology.orchestra.run.smithi122:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-11T01:10:45.478 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-11T01:10:45.472+0000 7eff341da7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-11T01:10:45.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-11T01:10:45.480+0000 7eff341da7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-11T01:10:45.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-11T01:10:45.480+0000 7eff341da7c0 -1 bdev(0x55aef782f800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-11T01:10:45.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-11T01:10:45.480+0000 7eff341da7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-11T01:10:48.651 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-11T01:10:48.665 DEBUG:teuthology.orchestra.run.smithi122:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-11T01:10:48.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-11T01:10:48.744+0000 7ffb75ebf7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-11T01:10:48.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-11T01:10:48.744+0000 7ffb75ebf7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-11T01:10:48.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-11T01:10:48.744+0000 7ffb75ebf7c0 -1 bdev(0x563fb887d800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-11T01:10:48.750 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-11T01:10:48.744+0000 7ffb75ebf7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-11T01:10:51.934 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-11T01:10:51.992 DEBUG:teuthology.orchestra.run.smithi122:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-11T01:10:52.074 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-11T01:10:52.068+0000 7f6f1c4887c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-11T01:10:52.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-11T01:10:52.072+0000 7f6f1c4887c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-11T01:10:52.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-11T01:10:52.072+0000 7f6f1c4887c0 -1 bdev(0x55dcd26e5800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-11T01:10:52.075 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-11T01:10:52.072+0000 7f6f1c4887c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-11T01:10:55.229 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-11T01:10:55.283 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-11T01:10:55.283 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-11T01:10:55.283 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-11T01:10:55.339 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-11T01:10:55.339 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-11T01:10:55.392 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-11T01:10:55.393 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-11T01:10:55.446 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-11T01:10:55.446 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-11T01:10:55.497 INFO:tasks.ceph:Adding keys to all mons... 2024-09-11T01:10:55.497 DEBUG:teuthology.orchestra.run.smithi122:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-11T01:10:55.547 INFO:teuthology.orchestra.run.smithi122.stdout:[mgr.x] 2024-09-11T01:10:55.548 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQAT7uBmiwmqDhAAp9baLxI38JGyMptzQ7A6nQ== 2024-09-11T01:10:55.548 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.0] 2024-09-11T01:10:55.548 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQAV7uBmDkOeHBAAnCdiLjoMBbHm19Cb1xg80g== 2024-09-11T01:10:55.548 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.1] 2024-09-11T01:10:55.548 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQAY7uBmL4HJLBAAaAyD8VJHdLIZ+hPA8LKM7g== 2024-09-11T01:10:55.548 INFO:teuthology.orchestra.run.smithi122.stdout:[osd.2] 2024-09-11T01:10:55.548 INFO:teuthology.orchestra.run.smithi122.stdout: key = AQAc7uBmMPaEBBAAgalOaxpCcVPhMWFoGAMADA== 2024-09-11T01:10:55.549 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-11T01:10:55.633 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T01:10:55.715 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T01:10:55.799 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-11T01:10:55.886 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-11T01:10:55.886 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-11T01:10:55.938 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-11T01:10:56.027 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-11T01:10:56.038 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-11T01:10:56.093 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-11T01:10:56.901 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-11T01:10:56.913 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-11T01:10:56.965 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-11T01:10:57.059 DEBUG:teuthology.orchestra.run.smithi122:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-11T01:10:57.070 DEBUG:teuthology.orchestra.run.smithi122:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-11T01:10:57.120 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-11T01:10:57.120 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-11T01:10:57.120 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-11T01:10:57.164 INFO:tasks.ceph.mon.a:Started 2024-09-11T01:10:57.164 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-11T01:10:57.165 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-11T01:10:57.167 INFO:tasks.ceph.mon.b:Started 2024-09-11T01:10:57.167 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-11T01:10:57.167 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-11T01:10:57.170 INFO:tasks.ceph.mon.c:Started 2024-09-11T01:10:57.171 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph config set mgr mgr_pool false --force 2024-09-11T01:11:03.633 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-11T01:11:03.634 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-11T01:11:03.634 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-11T01:11:03.636 INFO:tasks.ceph.mgr.x:Started 2024-09-11T01:11:03.636 DEBUG:tasks.ceph:set 0 configs 2024-09-11T01:11:03.636 INFO:tasks.ceph:Setting crush tunables to default 2024-09-11T01:11:03.636 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd crush tunables default 2024-09-11T01:11:03.828 INFO:teuthology.orchestra.run.smithi122.stderr:adjusted tunables profile to default 2024-09-11T01:11:03.845 INFO:tasks.ceph:check_enable_crimson: False 2024-09-11T01:11:03.845 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-11T01:11:03.846 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-11T01:11:03.846 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-11T01:11:03.856 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-11T01:11:03.856 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-11T01:11:03.908 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-11T01:11:03.909 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-11T01:11:03.962 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new e160603b-65bb-420f-9130-6d7c6506f2a5 0 2024-09-11T01:11:04.209 INFO:teuthology.orchestra.run.smithi122.stdout:0 2024-09-11T01:11:04.224 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new 6fbfe9e1-4255-4c44-b619-5fd57486e5fe 1 2024-09-11T01:11:04.415 INFO:teuthology.orchestra.run.smithi122.stdout:1 2024-09-11T01:11:04.431 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd new e27771fd-ac9d-4a64-a01f-465a5acade91 2 2024-09-11T01:11:04.633 INFO:teuthology.orchestra.run.smithi122.stdout:2 2024-09-11T01:11:04.648 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-11T01:11:04.649 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-11T01:11:04.650 INFO:tasks.ceph.osd.0:Started 2024-09-11T01:11:04.650 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-11T01:11:04.651 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-11T01:11:04.652 INFO:tasks.ceph.osd.1:Started 2024-09-11T01:11:04.652 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-11T01:11:04.652 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-11T01:11:04.654 INFO:tasks.ceph.osd.2:Started 2024-09-11T01:11:04.654 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T01:11:04.856 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:11:04.856 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":5,"fsid":"6d328773-3914-4412-91b7-393379984de3","created":"2024-09-11T01:11:03.508628+0000","modified":"2024-09-11T01:11:04.628439+0000","last_up_change":"0.000000","last_in_change":"2024-09-11T01:11:04.628439+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"e160603b-65bb-420f-9130-6d7c6506f2a5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6fbfe9e1-4255-4c44-b619-5fd57486e5fe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e27771fd-ac9d-4a64-a01f-465a5acade91","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T01:11:04.872 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-11T01:11:04.872 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-11T01:11:05.173 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-11T01:11:05.355 INFO:teuthology.misc.health.smithi122.stdout: 2024-09-11T01:11:05.355 INFO:teuthology.misc.health.smithi122.stdout:{"epoch":5,"fsid":"6d328773-3914-4412-91b7-393379984de3","created":"2024-09-11T01:11:03.508628+0000","modified":"2024-09-11T01:11:04.628439+0000","last_up_change":"0.000000","last_in_change":"2024-09-11T01:11:04.628439+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"e160603b-65bb-420f-9130-6d7c6506f2a5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6fbfe9e1-4255-4c44-b619-5fd57486e5fe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e27771fd-ac9d-4a64-a01f-465a5acade91","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T01:11:05.369 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-09-11T01:11:06.279 INFO:tasks.ceph.osd.2.smithi122.stderr:2024-09-11T01:11:06.276+0000 7fe05997b7c0 -1 Falling back to public interface 2024-09-11T01:11:06.499 INFO:tasks.ceph.osd.1.smithi122.stderr:2024-09-11T01:11:06.496+0000 7f03815047c0 -1 Falling back to public interface 2024-09-11T01:11:06.592 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:11:06.588+0000 7f9eb3cca7c0 -1 Falling back to public interface 2024-09-11T01:11:06.644 INFO:tasks.ceph.mgr.x.smithi122.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-09-11T01:11:06.644 INFO:tasks.ceph.mgr.x.smithi122.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-09-11T01:11:06.645 INFO:tasks.ceph.mgr.x.smithi122.stderr: from numpy import show_config as show_numpy_config 2024-09-11T01:11:09.534 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-11T01:11:09.532+0000 7f1fb81e8640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-11T01:11:09.535 INFO:tasks.ceph.mgr.x.smithi122.stderr:2024-09-11T01:11:09.532+0000 7f1fb81e8640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-11T01:11:10.381 INFO:tasks.ceph.osd.2.smithi122.stderr:2024-09-11T01:11:10.376+0000 7fe05997b7c0 -1 osd.2 0 log_to_monitors true 2024-09-11T01:11:10.488 INFO:tasks.ceph.osd.1.smithi122.stderr:2024-09-11T01:11:10.484+0000 7f03815047c0 -1 osd.1 0 log_to_monitors true 2024-09-11T01:11:11.064 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:11:11.060+0000 7f9eb3cca7c0 -1 osd.0 0 log_to_monitors true 2024-09-11T01:11:11.587 INFO:tasks.ceph.osd.1.smithi122.stderr:2024-09-11T01:11:11.584+0000 7f037d4ad640 -1 osd.1 0 waiting for initial osdmap 2024-09-11T01:11:11.610 INFO:tasks.ceph.osd.1.smithi122.stderr:2024-09-11T01:11:11.608+0000 7f03792e7640 -1 osd.1 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T01:11:11.675 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-11T01:11:12.057 INFO:teuthology.misc.health.smithi122.stdout: 2024-09-11T01:11:12.057 INFO:teuthology.misc.health.smithi122.stdout:{"epoch":7,"fsid":"6d328773-3914-4412-91b7-393379984de3","created":"2024-09-11T01:11:03.508628+0000","modified":"2024-09-11T01:11:11.576454+0000","last_up_change":"0.000000","last_in_change":"2024-09-11T01:11:04.628439+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"e160603b-65bb-420f-9130-6d7c6506f2a5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6fbfe9e1-4255-4c44-b619-5fd57486e5fe","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e27771fd-ac9d-4a64-a01f-465a5acade91","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T01:11:12.081 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-09-11T01:11:12.286 INFO:tasks.ceph.osd.2.smithi122.stderr:2024-09-11T01:11:12.280+0000 7fe055924640 -1 osd.2 0 waiting for initial osdmap 2024-09-11T01:11:12.289 INFO:tasks.ceph.osd.2.smithi122.stderr:2024-09-11T01:11:12.284+0000 7fe04d756640 -1 osd.2 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T01:11:12.601 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:11:12.596+0000 7f9eafc73640 -1 osd.0 0 waiting for initial osdmap 2024-09-11T01:11:12.606 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:11:12.600+0000 7f9eabaad640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T01:11:18.384 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-11T01:11:18.696 INFO:teuthology.misc.health.smithi122.stdout: 2024-09-11T01:11:18.697 INFO:teuthology.misc.health.smithi122.stdout:{"epoch":10,"fsid":"6d328773-3914-4412-91b7-393379984de3","created":"2024-09-11T01:11:03.508628+0000","modified":"2024-09-11T01:11:14.609849+0000","last_up_change":"2024-09-11T01:11:13.597327+0000","last_in_change":"2024-09-11T01:11:04.628439+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"e160603b-65bb-420f-9130-6d7c6506f2a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6808","nonce":3528172777}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6809","nonce":3528172777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6811","nonce":3528172777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6810","nonce":3528172777}]},"public_addr":"172.21.15.122:6808/3528172777","cluster_addr":"172.21.15.122:6809/3528172777","heartbeat_back_addr":"172.21.15.122:6811/3528172777","heartbeat_front_addr":"172.21.15.122:6810/3528172777","state":["exists","up"]},{"osd":1,"uuid":"6fbfe9e1-4255-4c44-b619-5fd57486e5fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6804","nonce":2638428424}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6805","nonce":2638428424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6807","nonce":2638428424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6806","nonce":2638428424}]},"public_addr":"172.21.15.122:6804/2638428424","cluster_addr":"172.21.15.122:6805/2638428424","heartbeat_back_addr":"172.21.15.122:6807/2638428424","heartbeat_front_addr":"172.21.15.122:6806/2638428424","state":["exists","up"]},{"osd":2,"uuid":"e27771fd-ac9d-4a64-a01f-465a5acade91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6800","nonce":2913015415}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6801","nonce":2913015415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6803","nonce":2913015415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6802","nonce":2913015415}]},"public_addr":"172.21.15.122:6800/2913015415","cluster_addr":"172.21.15.122:6801/2913015415","heartbeat_back_addr":"172.21.15.122:6803/2913015415","heartbeat_front_addr":"172.21.15.122:6802/2913015415","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:11:12.059865+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:11:11.519118+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:11:11.402743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T01:11:18.713 DEBUG:teuthology.misc:3 of 3 OSDs are up 2024-09-11T01:11:18.713 INFO:tasks.ceph:Creating RBD pool 2024-09-11T01:11:18.713 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-11T01:11:19.632 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'rbd' created 2024-09-11T01:11:19.658 DEBUG:teuthology.orchestra.run.smithi122:> rbd --cluster ceph pool init rbd 2024-09-11T01:11:22.687 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-11T01:11:22.688 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-11T01:11:22.689 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-11T01:11:23.003 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:11:23.021 INFO:teuthology.orchestra.run.smithi122.stdout:[{"version":3,"timestamp":"2024-09-11T01:11:12.289134+0000","name":"","changes":[{"name":"osd.2/osd_mclock_max_capacity_iops_ssd","new_value":"11323.336694"}]}] 2024-09-11T01:11:23.021 INFO:tasks.ceph_manager:config epoch is 3 2024-09-11T01:11:23.021 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-11T01:11:23.021 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-11T01:11:23.022 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-11T01:11:23.366 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:11:23.383 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":4,"flags":0,"active_gid":4109,"active_name":"x","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6812","nonce":3083070649}]},"active_addr":"172.21.15.122:6812/3083070649","active_change":"2024-09-11T01:11:08.510930+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:1.26.1","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.122:0","nonce":1612540053}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.122:0","nonce":2178757453}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.122:0","nonce":984116584}]}]} 2024-09-11T01:11:23.386 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-11T01:11:23.386 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-11T01:11:23.386 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T01:11:23.685 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:11:23.686 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":14,"fsid":"6d328773-3914-4412-91b7-393379984de3","created":"2024-09-11T01:11:03.508628+0000","modified":"2024-09-11T01:11:22.656113+0000","last_up_change":"2024-09-11T01:11:13.597327+0000","last_in_change":"2024-09-11T01:11:04.628439+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-11T01:11:18.999872+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e160603b-65bb-420f-9130-6d7c6506f2a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6808","nonce":3528172777}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6809","nonce":3528172777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6811","nonce":3528172777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6810","nonce":3528172777}]},"public_addr":"172.21.15.122:6808/3528172777","cluster_addr":"172.21.15.122:6809/3528172777","heartbeat_back_addr":"172.21.15.122:6811/3528172777","heartbeat_front_addr":"172.21.15.122:6810/3528172777","state":["exists","up"]},{"osd":1,"uuid":"6fbfe9e1-4255-4c44-b619-5fd57486e5fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6804","nonce":2638428424}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6805","nonce":2638428424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6807","nonce":2638428424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6806","nonce":2638428424}]},"public_addr":"172.21.15.122:6804/2638428424","cluster_addr":"172.21.15.122:6805/2638428424","heartbeat_back_addr":"172.21.15.122:6807/2638428424","heartbeat_front_addr":"172.21.15.122:6806/2638428424","state":["exists","up"]},{"osd":2,"uuid":"e27771fd-ac9d-4a64-a01f-465a5acade91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6800","nonce":2913015415}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6801","nonce":2913015415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6803","nonce":2913015415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6802","nonce":2913015415}]},"public_addr":"172.21.15.122:6800/2913015415","cluster_addr":"172.21.15.122:6801/2913015415","heartbeat_back_addr":"172.21.15.122:6803/2913015415","heartbeat_front_addr":"172.21.15.122:6802/2913015415","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:11:12.059865+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:11:11.519118+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:11:11.402743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T01:11:23.703 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-11T01:11:23.704 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T01:11:24.004 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:11:24.004 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":14,"fsid":"6d328773-3914-4412-91b7-393379984de3","created":"2024-09-11T01:11:03.508628+0000","modified":"2024-09-11T01:11:22.656113+0000","last_up_change":"2024-09-11T01:11:13.597327+0000","last_in_change":"2024-09-11T01:11:04.628439+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-11T01:11:18.999872+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e160603b-65bb-420f-9130-6d7c6506f2a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6808","nonce":3528172777}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6809","nonce":3528172777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6811","nonce":3528172777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6810","nonce":3528172777}]},"public_addr":"172.21.15.122:6808/3528172777","cluster_addr":"172.21.15.122:6809/3528172777","heartbeat_back_addr":"172.21.15.122:6811/3528172777","heartbeat_front_addr":"172.21.15.122:6810/3528172777","state":["exists","up"]},{"osd":1,"uuid":"6fbfe9e1-4255-4c44-b619-5fd57486e5fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6804","nonce":2638428424}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6805","nonce":2638428424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6807","nonce":2638428424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6806","nonce":2638428424}]},"public_addr":"172.21.15.122:6804/2638428424","cluster_addr":"172.21.15.122:6805/2638428424","heartbeat_back_addr":"172.21.15.122:6807/2638428424","heartbeat_front_addr":"172.21.15.122:6806/2638428424","state":["exists","up"]},{"osd":2,"uuid":"e27771fd-ac9d-4a64-a01f-465a5acade91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6800","nonce":2913015415}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6801","nonce":2913015415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6803","nonce":2913015415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6802","nonce":2913015415}]},"public_addr":"172.21.15.122:6800/2913015415","cluster_addr":"172.21.15.122:6801/2913015415","heartbeat_back_addr":"172.21.15.122:6803/2913015415","heartbeat_front_addr":"172.21.15.122:6802/2913015415","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:11:12.059865+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:11:11.519118+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:11:11.402743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T01:11:24.022 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-11T01:11:24.023 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-11T01:11:24.023 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-11T01:11:24.168 INFO:teuthology.orchestra.run.smithi122.stdout:34359738372 2024-09-11T01:11:24.168 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-11T01:11:24.185 INFO:teuthology.orchestra.run.smithi122.stdout:38654705667 2024-09-11T01:11:24.185 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T01:11:24.186 INFO:teuthology.orchestra.run.smithi122.stdout:34359738373 2024-09-11T01:11:24.186 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-11T01:11:24.464 INFO:teuthology.orchestra.run.smithi122.stdout:34359738370 2024-09-11T01:11:24.482 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738372 got 34359738370 for osd.2 2024-09-11T01:11:24.513 INFO:teuthology.orchestra.run.smithi122.stdout:34359738371 2024-09-11T01:11:24.522 INFO:teuthology.orchestra.run.smithi122.stdout:38654705666 2024-09-11T01:11:24.530 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738373 got 34359738371 for osd.1 2024-09-11T01:11:24.540 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705667 got 38654705666 for osd.0 2024-09-11T01:11:25.482 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-11T01:11:25.530 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-11T01:11:25.541 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T01:11:25.792 INFO:teuthology.orchestra.run.smithi122.stdout:34359738372 2024-09-11T01:11:25.815 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738372 got 34359738372 for osd.2 2024-09-11T01:11:25.815 DEBUG:teuthology.parallel:result is None 2024-09-11T01:11:25.835 INFO:teuthology.orchestra.run.smithi122.stdout:34359738373 2024-09-11T01:11:25.851 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738373 got 34359738373 for osd.1 2024-09-11T01:11:25.852 DEBUG:teuthology.parallel:result is None 2024-09-11T01:11:25.871 INFO:teuthology.orchestra.run.smithi122.stdout:38654705667 2024-09-11T01:11:25.887 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705667 got 38654705667 for osd.0 2024-09-11T01:11:25.888 DEBUG:teuthology.parallel:result is None 2024-09-11T01:11:25.888 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-11T01:11:25.888 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:11:26.198 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:11:26.199 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:11:26.214 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-09-11T01:11:24.523341+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":4,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81432,"kb_used_data":360,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034088,"statfs":{"total":289910292480,"available":289826906112,"internally_reserved":0,"allocated":368640,"data_stored":84993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":82963791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:11:22.672724+0000","last_change":"2024-09-11T01:11:22.672724+0000","last_active":"2024-09-11T01:11:22.672724+0000","last_peered":"2024-09-11T01:11:22.672724+0000","last_clean":"2024-09-11T01:11:22.672724+0000","last_became_active":"2024-09-11T01:11:20.665799+0000","last_became_peered":"2024-09-11T01:11:20.665799+0000","last_unstale":"2024-09-11T01:11:22.672724+0000","last_undegraded":"2024-09-11T01:11:22.672724+0000","last_fullsized":"2024-09-11T01:11:22.672724+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:49.340044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:11:22.672920+0000","last_change":"2024-09-11T01:11:22.672920+0000","last_active":"2024-09-11T01:11:22.672920+0000","last_peered":"2024-09-11T01:11:22.672920+0000","last_clean":"2024-09-11T01:11:22.672920+0000","last_became_active":"2024-09-11T01:11:20.662201+0000","last_became_peered":"2024-09-11T01:11:20.662201+0000","last_unstale":"2024-09-11T01:11:22.672920+0000","last_undegraded":"2024-09-11T01:11:22.672920+0000","last_fullsized":"2024-09-11T01:11:22.672920+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:37.382862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:11:23.597735+0000","last_change":"2024-09-11T01:11:23.598199+0000","last_active":"2024-09-11T01:11:23.597735+0000","last_peered":"2024-09-11T01:11:23.597735+0000","last_clean":"2024-09-11T01:11:23.597735+0000","last_became_active":"2024-09-11T01:11:20.669850+0000","last_became_peered":"2024-09-11T01:11:20.669850+0000","last_unstale":"2024-09-11T01:11:23.597735+0000","last_undegraded":"2024-09-11T01:11:23.597735+0000","last_fullsized":"2024-09-11T01:11:23.597735+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:14:26.601987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097838900000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:11:22.672257+0000","last_change":"2024-09-11T01:11:22.672257+0000","last_active":"2024-09-11T01:11:22.672257+0000","last_peered":"2024-09-11T01:11:22.672257+0000","last_clean":"2024-09-11T01:11:22.672257+0000","last_became_active":"2024-09-11T01:11:20.659744+0000","last_became_peered":"2024-09-11T01:11:20.659744+0000","last_unstale":"2024-09-11T01:11:22.672257+0000","last_undegraded":"2024-09-11T01:11:22.672257+0000","last_fullsized":"2024-09-11T01:11:22.672257+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:12.118110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"12'1","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:11:22.672127+0000","last_change":"2024-09-11T01:11:22.672127+0000","last_active":"2024-09-11T01:11:22.672127+0000","last_peered":"2024-09-11T01:11:22.672127+0000","last_clean":"2024-09-11T01:11:22.672127+0000","last_became_active":"2024-09-11T01:11:20.664883+0000","last_became_peered":"2024-09-11T01:11:20.664883+0000","last_unstale":"2024-09-11T01:11:22.672127+0000","last_undegraded":"2024-09-11T01:11:22.672127+0000","last_fullsized":"2024-09-11T01:11:22.672127+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:48.181171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"14'2","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:11:22.676432+0000","last_change":"2024-09-11T01:11:22.670534+0000","last_active":"2024-09-11T01:11:22.676432+0000","last_peered":"2024-09-11T01:11:22.676432+0000","last_clean":"2024-09-11T01:11:22.676432+0000","last_became_active":"2024-09-11T01:11:20.654870+0000","last_became_peered":"2024-09-11T01:11:20.654870+0000","last_unstale":"2024-09-11T01:11:22.676432+0000","last_undegraded":"2024-09-11T01:11:22.676432+0000","last_fullsized":"2024-09-11T01:11:22.676432+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:01:43.922314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:11:23.597353+0000","last_change":"2024-09-11T01:11:23.597586+0000","last_active":"2024-09-11T01:11:23.597353+0000","last_peered":"2024-09-11T01:11:23.597353+0000","last_clean":"2024-09-11T01:11:23.597353+0000","last_became_active":"2024-09-11T01:11:20.669746+0000","last_became_peered":"2024-09-11T01:11:20.669746+0000","last_unstale":"2024-09-11T01:11:23.597353+0000","last_undegraded":"2024-09-11T01:11:23.597353+0000","last_fullsized":"2024-09-11T01:11:23.597353+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:29:28.780931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054732200000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:11:22.669819+0000","last_change":"2024-09-11T01:11:22.669819+0000","last_active":"2024-09-11T01:11:22.669819+0000","last_peered":"2024-09-11T01:11:22.669819+0000","last_clean":"2024-09-11T01:11:22.669819+0000","last_became_active":"2024-09-11T01:11:20.659393+0000","last_became_peered":"2024-09-11T01:11:20.659393+0000","last_unstale":"2024-09-11T01:11:22.669819+0000","last_undegraded":"2024-09-11T01:11:22.669819+0000","last_fullsized":"2024-09-11T01:11:22.669819+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:27.933594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":9,"seq":38654705667,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":126976,"data_stored":29256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":8,"seq":34359738372,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27136,"kb_used_data":112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344704,"statfs":{"total":96636764160,"available":96608976896,"internally_reserved":0,"allocated":114688,"data_stored":26481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738373,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":126976,"data_stored":29256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:11:26.215 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:11:26.483 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:11:26.483 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:11:26.499 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-09-11T01:11:24.523341+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":4,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81432,"kb_used_data":360,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034088,"statfs":{"total":289910292480,"available":289826906112,"internally_reserved":0,"allocated":368640,"data_stored":84993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":82963791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:11:22.672724+0000","last_change":"2024-09-11T01:11:22.672724+0000","last_active":"2024-09-11T01:11:22.672724+0000","last_peered":"2024-09-11T01:11:22.672724+0000","last_clean":"2024-09-11T01:11:22.672724+0000","last_became_active":"2024-09-11T01:11:20.665799+0000","last_became_peered":"2024-09-11T01:11:20.665799+0000","last_unstale":"2024-09-11T01:11:22.672724+0000","last_undegraded":"2024-09-11T01:11:22.672724+0000","last_fullsized":"2024-09-11T01:11:22.672724+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:49.340044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:11:22.672920+0000","last_change":"2024-09-11T01:11:22.672920+0000","last_active":"2024-09-11T01:11:22.672920+0000","last_peered":"2024-09-11T01:11:22.672920+0000","last_clean":"2024-09-11T01:11:22.672920+0000","last_became_active":"2024-09-11T01:11:20.662201+0000","last_became_peered":"2024-09-11T01:11:20.662201+0000","last_unstale":"2024-09-11T01:11:22.672920+0000","last_undegraded":"2024-09-11T01:11:22.672920+0000","last_fullsized":"2024-09-11T01:11:22.672920+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:37.382862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:11:23.597735+0000","last_change":"2024-09-11T01:11:23.598199+0000","last_active":"2024-09-11T01:11:23.597735+0000","last_peered":"2024-09-11T01:11:23.597735+0000","last_clean":"2024-09-11T01:11:23.597735+0000","last_became_active":"2024-09-11T01:11:20.669850+0000","last_became_peered":"2024-09-11T01:11:20.669850+0000","last_unstale":"2024-09-11T01:11:23.597735+0000","last_undegraded":"2024-09-11T01:11:23.597735+0000","last_fullsized":"2024-09-11T01:11:23.597735+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:14:26.601987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097838900000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:11:22.672257+0000","last_change":"2024-09-11T01:11:22.672257+0000","last_active":"2024-09-11T01:11:22.672257+0000","last_peered":"2024-09-11T01:11:22.672257+0000","last_clean":"2024-09-11T01:11:22.672257+0000","last_became_active":"2024-09-11T01:11:20.659744+0000","last_became_peered":"2024-09-11T01:11:20.659744+0000","last_unstale":"2024-09-11T01:11:22.672257+0000","last_undegraded":"2024-09-11T01:11:22.672257+0000","last_fullsized":"2024-09-11T01:11:22.672257+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:12.118110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"12'1","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:11:22.672127+0000","last_change":"2024-09-11T01:11:22.672127+0000","last_active":"2024-09-11T01:11:22.672127+0000","last_peered":"2024-09-11T01:11:22.672127+0000","last_clean":"2024-09-11T01:11:22.672127+0000","last_became_active":"2024-09-11T01:11:20.664883+0000","last_became_peered":"2024-09-11T01:11:20.664883+0000","last_unstale":"2024-09-11T01:11:22.672127+0000","last_undegraded":"2024-09-11T01:11:22.672127+0000","last_fullsized":"2024-09-11T01:11:22.672127+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:48.181171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"14'2","reported_seq":22,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:11:22.676432+0000","last_change":"2024-09-11T01:11:22.670534+0000","last_active":"2024-09-11T01:11:22.676432+0000","last_peered":"2024-09-11T01:11:22.676432+0000","last_clean":"2024-09-11T01:11:22.676432+0000","last_became_active":"2024-09-11T01:11:20.654870+0000","last_became_peered":"2024-09-11T01:11:20.654870+0000","last_unstale":"2024-09-11T01:11:22.676432+0000","last_undegraded":"2024-09-11T01:11:22.676432+0000","last_fullsized":"2024-09-11T01:11:22.676432+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:01:43.922314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:11:23.597353+0000","last_change":"2024-09-11T01:11:23.597586+0000","last_active":"2024-09-11T01:11:23.597353+0000","last_peered":"2024-09-11T01:11:23.597353+0000","last_clean":"2024-09-11T01:11:23.597353+0000","last_became_active":"2024-09-11T01:11:20.669746+0000","last_became_peered":"2024-09-11T01:11:20.669746+0000","last_unstale":"2024-09-11T01:11:23.597353+0000","last_undegraded":"2024-09-11T01:11:23.597353+0000","last_fullsized":"2024-09-11T01:11:23.597353+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:29:28.780931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054732200000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-11T01:11:22.669819+0000","last_change":"2024-09-11T01:11:22.669819+0000","last_active":"2024-09-11T01:11:22.669819+0000","last_peered":"2024-09-11T01:11:22.669819+0000","last_clean":"2024-09-11T01:11:22.669819+0000","last_became_active":"2024-09-11T01:11:20.659393+0000","last_became_peered":"2024-09-11T01:11:20.659393+0000","last_unstale":"2024-09-11T01:11:22.669819+0000","last_undegraded":"2024-09-11T01:11:22.669819+0000","last_fullsized":"2024-09-11T01:11:22.669819+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:27.933594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":9,"seq":38654705667,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":126976,"data_stored":29256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":8,"seq":34359738372,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27136,"kb_used_data":112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344704,"statfs":{"total":96636764160,"available":96608976896,"internally_reserved":0,"allocated":114688,"data_stored":26481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738373,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":126976,"data_stored":29256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:11:26.500 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-11T01:11:26.500 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-11T01:11:26.500 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-11T01:11:26.500 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-11T01:11:26.808 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:11:26.808 INFO:teuthology.orchestra.run.smithi122.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-11T01:11:26.825 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-11T01:11:26.825 INFO:teuthology.run_tasks:Running task rep_lost_unfound_delete... 2024-09-11T01:11:26.841 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T01:11:27.118 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:11:27.118 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":14,"fsid":"6d328773-3914-4412-91b7-393379984de3","created":"2024-09-11T01:11:03.508628+0000","modified":"2024-09-11T01:11:22.656113+0000","last_up_change":"2024-09-11T01:11:13.597327+0000","last_in_change":"2024-09-11T01:11:04.628439+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-11T01:11:18.999872+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e160603b-65bb-420f-9130-6d7c6506f2a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6808","nonce":3528172777}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6809","nonce":3528172777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6811","nonce":3528172777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6810","nonce":3528172777}]},"public_addr":"172.21.15.122:6808/3528172777","cluster_addr":"172.21.15.122:6809/3528172777","heartbeat_back_addr":"172.21.15.122:6811/3528172777","heartbeat_front_addr":"172.21.15.122:6810/3528172777","state":["exists","up"]},{"osd":1,"uuid":"6fbfe9e1-4255-4c44-b619-5fd57486e5fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6804","nonce":2638428424}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6805","nonce":2638428424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6807","nonce":2638428424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6806","nonce":2638428424}]},"public_addr":"172.21.15.122:6804/2638428424","cluster_addr":"172.21.15.122:6805/2638428424","heartbeat_back_addr":"172.21.15.122:6807/2638428424","heartbeat_front_addr":"172.21.15.122:6806/2638428424","state":["exists","up"]},{"osd":2,"uuid":"e27771fd-ac9d-4a64-a01f-465a5acade91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6800","nonce":2913015415}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6801","nonce":2913015415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6803","nonce":2913015415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6802","nonce":2913015415}]},"public_addr":"172.21.15.122:6800/2913015415","cluster_addr":"172.21.15.122:6801/2913015415","heartbeat_back_addr":"172.21.15.122:6803/2913015415","heartbeat_front_addr":"172.21.15.122:6802/2913015415","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:11:12.059865+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:11:11.519118+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:11:11.402743+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T01:11:27.135 INFO:tasks.rep_lost_unfound_delete.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-09-11T01:11:18.999872+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '14', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 2, 'snap_epoch': 14, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-11T01:11:27.135 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-09-11T01:11:27.420 INFO:teuthology.orchestra.run.smithi122.stdout:pg_num: 8 2024-09-11T01:11:27.436 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-09-11T01:11:27.720 INFO:teuthology.orchestra.run.smithi122.stdout:epoch 15 2024-09-11T01:11:27.721 INFO:teuthology.orchestra.run.smithi122.stdout:fsid 6d328773-3914-4412-91b7-393379984de3 2024-09-11T01:11:27.721 INFO:teuthology.orchestra.run.smithi122.stdout:created 2024-09-11T01:11:03.508628+0000 2024-09-11T01:11:27.721 INFO:teuthology.orchestra.run.smithi122.stdout:modified 2024-09-11T01:11:27.242110+0000 2024-09-11T01:11:27.721 INFO:teuthology.orchestra.run.smithi122.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-11T01:11:27.721 INFO:teuthology.orchestra.run.smithi122.stdout:crush_version 5 2024-09-11T01:11:27.721 INFO:teuthology.orchestra.run.smithi122.stdout:full_ratio 0.95 2024-09-11T01:11:27.721 INFO:teuthology.orchestra.run.smithi122.stdout:backfillfull_ratio 0.9 2024-09-11T01:11:27.721 INFO:teuthology.orchestra.run.smithi122.stdout:nearfull_ratio 0.85 2024-09-11T01:11:27.721 INFO:teuthology.orchestra.run.smithi122.stdout:require_min_compat_client luminous 2024-09-11T01:11:27.721 INFO:teuthology.orchestra.run.smithi122.stdout:min_compat_client jewel 2024-09-11T01:11:27.721 INFO:teuthology.orchestra.run.smithi122.stdout:require_osd_release squid 2024-09-11T01:11:27.722 INFO:teuthology.orchestra.run.smithi122.stdout:stretch_mode_enabled false 2024-09-11T01:11:27.722 INFO:teuthology.orchestra.run.smithi122.stdout:pool 1 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 14 flags hashpspool,selfmanaged_snaps stripe_width 0 application rbd read_balance_score 1.88 2024-09-11T01:11:27.722 INFO:teuthology.orchestra.run.smithi122.stdout:max_osd 3 2024-09-11T01:11:27.722 INFO:teuthology.orchestra.run.smithi122.stdout:osd.0 up in weight 1 up_from 9 up_thru 11 down_at 0 last_clean_interval [0,0) v1:172.21.15.122:6808/3528172777 v1:172.21.15.122:6809/3528172777 exists,up e160603b-65bb-420f-9130-6d7c6506f2a5 2024-09-11T01:11:27.722 INFO:teuthology.orchestra.run.smithi122.stdout:osd.1 up in weight 1 up_from 8 up_thru 11 down_at 0 last_clean_interval [0,0) v1:172.21.15.122:6804/2638428424 v1:172.21.15.122:6805/2638428424 exists,up 6fbfe9e1-4255-4c44-b619-5fd57486e5fe 2024-09-11T01:11:27.722 INFO:teuthology.orchestra.run.smithi122.stdout:osd.2 up in weight 1 up_from 8 up_thru 11 down_at 0 last_clean_interval [0,0) v1:172.21.15.122:6800/2913015415 v1:172.21.15.122:6801/2913015415 exists,up e27771fd-ac9d-4a64-a01f-465a5acade91 2024-09-11T01:11:27.737 INFO:tasks.rep_lost_unfound_delete.ceph_manager:['osd.0 up in weight 1 up_from 9 up_thru 11 down_at 0 last_clean_interval [0,0) v1:172.21.15.122:6808/3528172777 v1:172.21.15.122:6809/3528172777 exists,up e160603b-65bb-420f-9130-6d7c6506f2a5', 'osd.1 up in weight 1 up_from 8 up_thru 11 down_at 0 last_clean_interval [0,0) v1:172.21.15.122:6804/2638428424 v1:172.21.15.122:6805/2638428424 exists,up 6fbfe9e1-4255-4c44-b619-5fd57486e5fe', 'osd.2 up in weight 1 up_from 8 up_thru 11 down_at 0 last_clean_interval [0,0) v1:172.21.15.122:6800/2913015415 v1:172.21.15.122:6801/2913015415 exists,up e27771fd-ac9d-4a64-a01f-465a5acade91'] 2024-09-11T01:11:27.738 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-11T01:11:27.738 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-11T01:11:27.739 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-11T01:11:27.883 INFO:teuthology.orchestra.run.smithi122.stdout:34359738374 2024-09-11T01:11:27.883 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-11T01:11:27.899 INFO:teuthology.orchestra.run.smithi122.stdout:34359738373 2024-09-11T01:11:27.899 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-11T01:11:27.900 INFO:teuthology.orchestra.run.smithi122.stdout:38654705669 2024-09-11T01:11:27.900 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T01:11:28.200 INFO:teuthology.orchestra.run.smithi122.stdout:34359738373 2024-09-11T01:11:28.218 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738374 got 34359738373 for osd.1 2024-09-11T01:11:28.234 INFO:teuthology.orchestra.run.smithi122.stdout:34359738372 2024-09-11T01:11:28.251 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738373 got 34359738372 for osd.2 2024-09-11T01:11:28.259 INFO:teuthology.orchestra.run.smithi122.stdout:38654705668 2024-09-11T01:11:28.274 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 38654705669 got 38654705668 for osd.0 2024-09-11T01:11:29.219 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-11T01:11:29.252 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-11T01:11:29.275 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T01:11:29.533 INFO:teuthology.orchestra.run.smithi122.stdout:34359738374 2024-09-11T01:11:29.558 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738374 got 34359738374 for osd.1 2024-09-11T01:11:29.558 DEBUG:teuthology.parallel:result is None 2024-09-11T01:11:29.622 INFO:teuthology.orchestra.run.smithi122.stdout:34359738373 2024-09-11T01:11:29.639 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738373 got 34359738373 for osd.2 2024-09-11T01:11:29.639 DEBUG:teuthology.parallel:result is None 2024-09-11T01:11:29.648 INFO:teuthology.orchestra.run.smithi122.stdout:38654705669 2024-09-11T01:11:29.664 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 38654705669 got 38654705669 for osd.0 2024-09-11T01:11:29.664 DEBUG:teuthology.parallel:result is None 2024-09-11T01:11:29.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for clean 2024-09-11T01:11:29.665 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:11:29.983 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:11:29.983 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:11:29.999 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-11T01:11:28.524075+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81448,"kb_used_data":376,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034072,"statfs":{"total":289910292480,"available":289826889728,"internally_reserved":0,"allocated":385024,"data_stored":90139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":82963791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000733"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-11T01:11:27.248293+0000","last_change":"2024-09-11T01:11:22.672724+0000","last_active":"2024-09-11T01:11:27.248293+0000","last_peered":"2024-09-11T01:11:27.248293+0000","last_clean":"2024-09-11T01:11:27.248293+0000","last_became_active":"2024-09-11T01:11:20.665799+0000","last_became_peered":"2024-09-11T01:11:20.665799+0000","last_unstale":"2024-09-11T01:11:27.248293+0000","last_undegraded":"2024-09-11T01:11:27.248293+0000","last_fullsized":"2024-09-11T01:11:27.248293+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:49.340044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-11T01:11:27.248199+0000","last_change":"2024-09-11T01:11:22.672920+0000","last_active":"2024-09-11T01:11:27.248199+0000","last_peered":"2024-09-11T01:11:27.248199+0000","last_clean":"2024-09-11T01:11:27.248199+0000","last_became_active":"2024-09-11T01:11:20.662201+0000","last_became_peered":"2024-09-11T01:11:20.662201+0000","last_unstale":"2024-09-11T01:11:27.248199+0000","last_undegraded":"2024-09-11T01:11:27.248199+0000","last_fullsized":"2024-09-11T01:11:27.248199+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:37.382862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-11T01:11:27.662489+0000","last_change":"2024-09-11T01:11:23.598199+0000","last_active":"2024-09-11T01:11:27.662489+0000","last_peered":"2024-09-11T01:11:27.662489+0000","last_clean":"2024-09-11T01:11:27.662489+0000","last_became_active":"2024-09-11T01:11:20.669850+0000","last_became_peered":"2024-09-11T01:11:20.669850+0000","last_unstale":"2024-09-11T01:11:27.662489+0000","last_undegraded":"2024-09-11T01:11:27.662489+0000","last_fullsized":"2024-09-11T01:11:27.662489+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:14:26.601987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097838900000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-11T01:11:27.248329+0000","last_change":"2024-09-11T01:11:22.672257+0000","last_active":"2024-09-11T01:11:27.248329+0000","last_peered":"2024-09-11T01:11:27.248329+0000","last_clean":"2024-09-11T01:11:27.248329+0000","last_became_active":"2024-09-11T01:11:20.659744+0000","last_became_peered":"2024-09-11T01:11:20.659744+0000","last_unstale":"2024-09-11T01:11:27.248329+0000","last_undegraded":"2024-09-11T01:11:27.248329+0000","last_fullsized":"2024-09-11T01:11:27.248329+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:12.118110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":23,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-11T01:11:27.248313+0000","last_change":"2024-09-11T01:11:22.672127+0000","last_active":"2024-09-11T01:11:27.248313+0000","last_peered":"2024-09-11T01:11:27.248313+0000","last_clean":"2024-09-11T01:11:27.248313+0000","last_became_active":"2024-09-11T01:11:20.664883+0000","last_became_peered":"2024-09-11T01:11:20.664883+0000","last_unstale":"2024-09-11T01:11:27.248313+0000","last_undegraded":"2024-09-11T01:11:27.248313+0000","last_fullsized":"2024-09-11T01:11:27.248313+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:48.181171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":24,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-11T01:11:27.247545+0000","last_change":"2024-09-11T01:11:22.670534+0000","last_active":"2024-09-11T01:11:27.247545+0000","last_peered":"2024-09-11T01:11:27.247545+0000","last_clean":"2024-09-11T01:11:27.247545+0000","last_became_active":"2024-09-11T01:11:20.654870+0000","last_became_peered":"2024-09-11T01:11:20.654870+0000","last_unstale":"2024-09-11T01:11:27.247545+0000","last_undegraded":"2024-09-11T01:11:27.247545+0000","last_fullsized":"2024-09-11T01:11:27.247545+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:01:43.922314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-11T01:11:27.662352+0000","last_change":"2024-09-11T01:11:23.597586+0000","last_active":"2024-09-11T01:11:27.662352+0000","last_peered":"2024-09-11T01:11:27.662352+0000","last_clean":"2024-09-11T01:11:27.662352+0000","last_became_active":"2024-09-11T01:11:20.669746+0000","last_became_peered":"2024-09-11T01:11:20.669746+0000","last_unstale":"2024-09-11T01:11:27.662352+0000","last_undegraded":"2024-09-11T01:11:27.662352+0000","last_fullsized":"2024-09-11T01:11:27.662352+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:29:28.780931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054732200000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-11T01:11:27.248177+0000","last_change":"2024-09-11T01:11:22.669819+0000","last_active":"2024-09-11T01:11:27.248177+0000","last_peered":"2024-09-11T01:11:27.248177+0000","last_clean":"2024-09-11T01:11:27.248177+0000","last_became_active":"2024-09-11T01:11:20.659393+0000","last_became_peered":"2024-09-11T01:11:20.659393+0000","last_unstale":"2024-09-11T01:11:27.248177+0000","last_undegraded":"2024-09-11T01:11:27.248177+0000","last_fullsized":"2024-09-11T01:11:27.248177+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:27.933594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":126976,"data_stored":29256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":8,"seq":34359738373,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27144,"kb_used_data":120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344696,"statfs":{"total":96636764160,"available":96608968704,"internally_reserved":0,"allocated":122880,"data_stored":29237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738374,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27156,"kb_used_data":132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344684,"statfs":{"total":96636764160,"available":96608956416,"internally_reserved":0,"allocated":135168,"data_stored":31646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:11:30.000 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:11:30.290 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:11:30.291 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:11:30.308 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2024-09-11T01:11:28.524075+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":81448,"kb_used_data":376,"kb_used_omap":4,"kb_used_meta":81019,"kb_avail":283034072,"statfs":{"total":289910292480,"available":289826889728,"internally_reserved":0,"allocated":385024,"data_stored":90139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4785,"internal_metadata":82963791},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000733"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-11T01:11:27.248293+0000","last_change":"2024-09-11T01:11:22.672724+0000","last_active":"2024-09-11T01:11:27.248293+0000","last_peered":"2024-09-11T01:11:27.248293+0000","last_clean":"2024-09-11T01:11:27.248293+0000","last_became_active":"2024-09-11T01:11:20.665799+0000","last_became_peered":"2024-09-11T01:11:20.665799+0000","last_unstale":"2024-09-11T01:11:27.248293+0000","last_undegraded":"2024-09-11T01:11:27.248293+0000","last_fullsized":"2024-09-11T01:11:27.248293+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:11:49.340044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-11T01:11:27.248199+0000","last_change":"2024-09-11T01:11:22.672920+0000","last_active":"2024-09-11T01:11:27.248199+0000","last_peered":"2024-09-11T01:11:27.248199+0000","last_clean":"2024-09-11T01:11:27.248199+0000","last_became_active":"2024-09-11T01:11:20.662201+0000","last_became_peered":"2024-09-11T01:11:20.662201+0000","last_unstale":"2024-09-11T01:11:27.248199+0000","last_undegraded":"2024-09-11T01:11:27.248199+0000","last_fullsized":"2024-09-11T01:11:27.248199+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:37.382862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-11T01:11:27.662489+0000","last_change":"2024-09-11T01:11:23.598199+0000","last_active":"2024-09-11T01:11:27.662489+0000","last_peered":"2024-09-11T01:11:27.662489+0000","last_clean":"2024-09-11T01:11:27.662489+0000","last_became_active":"2024-09-11T01:11:20.669850+0000","last_became_peered":"2024-09-11T01:11:20.669850+0000","last_unstale":"2024-09-11T01:11:27.662489+0000","last_undegraded":"2024-09-11T01:11:27.662489+0000","last_fullsized":"2024-09-11T01:11:27.662489+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:14:26.601987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097838900000000008,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-11T01:11:27.248329+0000","last_change":"2024-09-11T01:11:22.672257+0000","last_active":"2024-09-11T01:11:27.248329+0000","last_peered":"2024-09-11T01:11:27.248329+0000","last_clean":"2024-09-11T01:11:27.248329+0000","last_became_active":"2024-09-11T01:11:20.659744+0000","last_became_peered":"2024-09-11T01:11:20.659744+0000","last_unstale":"2024-09-11T01:11:27.248329+0000","last_undegraded":"2024-09-11T01:11:27.248329+0000","last_fullsized":"2024-09-11T01:11:27.248329+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:12.118110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":23,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-11T01:11:27.248313+0000","last_change":"2024-09-11T01:11:22.672127+0000","last_active":"2024-09-11T01:11:27.248313+0000","last_peered":"2024-09-11T01:11:27.248313+0000","last_clean":"2024-09-11T01:11:27.248313+0000","last_became_active":"2024-09-11T01:11:20.664883+0000","last_became_peered":"2024-09-11T01:11:20.664883+0000","last_unstale":"2024-09-11T01:11:27.248313+0000","last_undegraded":"2024-09-11T01:11:27.248313+0000","last_fullsized":"2024-09-11T01:11:27.248313+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:13:48.181171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":24,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-11T01:11:27.247545+0000","last_change":"2024-09-11T01:11:22.670534+0000","last_active":"2024-09-11T01:11:27.247545+0000","last_peered":"2024-09-11T01:11:27.247545+0000","last_clean":"2024-09-11T01:11:27.247545+0000","last_became_active":"2024-09-11T01:11:20.654870+0000","last_became_peered":"2024-09-11T01:11:20.654870+0000","last_unstale":"2024-09-11T01:11:27.247545+0000","last_undegraded":"2024-09-11T01:11:27.247545+0000","last_fullsized":"2024-09-11T01:11:27.247545+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:01:43.922314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-11T01:11:27.662352+0000","last_change":"2024-09-11T01:11:23.597586+0000","last_active":"2024-09-11T01:11:27.662352+0000","last_peered":"2024-09-11T01:11:27.662352+0000","last_clean":"2024-09-11T01:11:27.662352+0000","last_became_active":"2024-09-11T01:11:20.669746+0000","last_became_peered":"2024-09-11T01:11:20.669746+0000","last_unstale":"2024-09-11T01:11:27.662352+0000","last_undegraded":"2024-09-11T01:11:27.662352+0000","last_fullsized":"2024-09-11T01:11:27.662352+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:29:28.780931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054732200000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-11T01:11:27.248177+0000","last_change":"2024-09-11T01:11:22.669819+0000","last_active":"2024-09-11T01:11:27.248177+0000","last_peered":"2024-09-11T01:11:27.248177+0000","last_clean":"2024-09-11T01:11:27.248177+0000","last_became_active":"2024-09-11T01:11:20.659393+0000","last_became_peered":"2024-09-11T01:11:20.659393+0000","last_unstale":"2024-09-11T01:11:27.248177+0000","last_undegraded":"2024-09-11T01:11:27.248177+0000","last_fullsized":"2024-09-11T01:11:27.248177+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:27.933594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":9,"seq":38654705669,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27148,"kb_used_data":124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344692,"statfs":{"total":96636764160,"available":96608964608,"internally_reserved":0,"allocated":126976,"data_stored":29256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":8,"seq":34359738373,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27144,"kb_used_data":120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344696,"statfs":{"total":96636764160,"available":96608968704,"internally_reserved":0,"allocated":122880,"data_stored":29237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738374,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27156,"kb_used_data":132,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344684,"statfs":{"total":96636764160,"available":96608956416,"internally_reserved":0,"allocated":135168,"data_stored":31646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:11:30.313 INFO:tasks.rep_lost_unfound_delete.ceph_manager:clean! 2024-09-11T01:11:30.313 INFO:tasks.rep_lost_unfound_delete.ceph_manager:creating pool_name unfounddel_pool 2024-09-11T01:11:30.313 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unfounddel_pool 16 2024-09-11T01:11:30.705 INFO:teuthology.orchestra.run.smithi122.stderr:pool 'unfounddel_pool' created 2024-09-11T01:11:30.731 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unfounddel_pool rados --yes-i-really-mean-it || true 2024-09-11T01:11:31.718 INFO:teuthology.orchestra.run.smithi122.stderr:enabled application 'rados' on pool 'unfounddel_pool' 2024-09-11T01:11:32.751 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-11T01:11:32.752 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T01:11:32.821 INFO:tasks.ceph.osd.2:Stopped 2024-09-11T01:11:32.821 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 2 2024-09-11T01:11:33.140 INFO:tasks.ceph.mon.a.smithi122.stderr:2024-09-11T01:11:33.136+0000 7f9dc4120640 -1 mon.a@0(leader).osd e18 definitely_dead 0 2024-09-11T01:11:33.748 INFO:teuthology.orchestra.run.smithi122.stderr:marked down osd.2. 2024-09-11T01:11:33.770 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 2 2024-09-11T01:11:34.075 INFO:tasks.ceph.mon.a.smithi122.stderr:2024-09-11T01:11:34.072+0000 7f9dc4120640 -1 mon.a@0(leader).osd e19 definitely_dead 0 2024-09-11T01:11:34.752 INFO:teuthology.orchestra.run.smithi122.stderr:marked out osd.2. 2024-09-11T01:11:34.776 INFO:tasks.util.rados:rados -p unfounddel_pool put dummy /etc/fstab 2024-09-11T01:11:34.776 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put dummy /etc/fstab 2024-09-11T01:11:34.820 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-11T01:11:34.821 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-11T01:11:35.016 INFO:teuthology.orchestra.run.smithi122.stdout:34359738377 2024-09-11T01:11:35.016 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-11T01:11:35.020 INFO:teuthology.orchestra.run.smithi122.stdout:38654705672 2024-09-11T01:11:35.020 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T01:11:35.305 INFO:teuthology.orchestra.run.smithi122.stdout:34359738376 2024-09-11T01:11:35.322 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738377 got 34359738376 for osd.1 2024-09-11T01:11:35.361 INFO:teuthology.orchestra.run.smithi122.stdout:38654705670 2024-09-11T01:11:35.379 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 38654705672 got 38654705670 for osd.0 2024-09-11T01:11:36.323 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-11T01:11:36.379 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T01:11:36.656 INFO:teuthology.orchestra.run.smithi122.stdout:34359738376 2024-09-11T01:11:36.674 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738377 got 34359738376 for osd.1 2024-09-11T01:11:36.684 INFO:teuthology.orchestra.run.smithi122.stdout:38654705670 2024-09-11T01:11:36.700 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 38654705672 got 38654705670 for osd.0 2024-09-11T01:11:37.675 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-11T01:11:37.701 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T01:11:37.974 INFO:teuthology.orchestra.run.smithi122.stdout:34359738377 2024-09-11T01:11:37.992 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738377 got 34359738377 for osd.1 2024-09-11T01:11:37.992 DEBUG:teuthology.parallel:result is None 2024-09-11T01:11:38.031 INFO:teuthology.orchestra.run.smithi122.stdout:38654705672 2024-09-11T01:11:38.049 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 38654705672 got 38654705672 for osd.0 2024-09-11T01:11:38.049 DEBUG:teuthology.parallel:result is None 2024-09-11T01:11:38.050 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for recovery to complete 2024-09-11T01:11:38.050 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:11:38.342 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:11:38.343 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:11:38.358 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2024-09-11T01:11:36.525947+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":24,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54376,"kb_used_data":328,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689304,"statfs":{"total":193273528320,"available":193217847296,"internally_reserved":0,"allocated":335872,"data_stored":87650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3190,"internal_metadata":55309194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000915"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:11:34.749748+0000","last_change":"2024-09-11T01:11:31.741068+0000","last_active":"2024-09-11T01:11:34.749748+0000","last_peered":"2024-09-11T01:11:34.749748+0000","last_clean":"2024-09-11T01:11:34.749748+0000","last_became_active":"2024-09-11T01:11:31.740203+0000","last_became_peered":"2024-09-11T01:11:31.740203+0000","last_unstale":"2024-09-11T01:11:34.749748+0000","last_undegraded":"2024-09-11T01:11:34.749748+0000","last_fullsized":"2024-09-11T01:11:34.749748+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:23.192014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:11:34.604177+0000","last_change":"2024-09-11T01:11:33.749559+0000","last_active":"2024-09-11T01:11:30.697577+0000","last_peered":"2024-09-11T01:11:30.697577+0000","last_clean":"2024-09-11T01:11:30.697577+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:11:34.604177+0000","last_undegraded":"2024-09-11T01:11:34.604177+0000","last_fullsized":"2024-09-11T01:11:34.604177+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:11:33.752614+0000","last_change":"2024-09-11T01:11:33.752614+0000","last_active":"2024-09-11T01:11:33.752614+0000","last_peered":"2024-09-11T01:11:33.591006+0000","last_clean":"2024-09-11T01:11:33.591006+0000","last_became_active":"2024-09-11T01:11:31.742309+0000","last_became_peered":"2024-09-11T01:11:31.742309+0000","last_unstale":"2024-09-11T01:11:33.752614+0000","last_undegraded":"2024-09-11T01:11:33.752614+0000","last_fullsized":"2024-09-11T01:11:33.752614+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:11:34.604116+0000","last_change":"2024-09-11T01:11:33.747864+0000","last_active":"2024-09-11T01:11:30.697577+0000","last_peered":"2024-09-11T01:11:30.697577+0000","last_clean":"2024-09-11T01:11:30.697577+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:11:34.604116+0000","last_undegraded":"2024-09-11T01:11:34.604116+0000","last_fullsized":"2024-09-11T01:11:34.604116+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:11:34.604096+0000","last_change":"2024-09-11T01:11:33.750982+0000","last_active":"2024-09-11T01:11:33.750982+0000","last_peered":"2024-09-11T01:11:32.736803+0000","last_clean":"2024-09-11T01:11:32.736803+0000","last_became_active":"2024-09-11T01:11:31.727424+0000","last_became_peered":"2024-09-11T01:11:31.727424+0000","last_unstale":"2024-09-11T01:11:34.604096+0000","last_undegraded":"2024-09-11T01:11:34.604096+0000","last_fullsized":"2024-09-11T01:11:34.604096+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:11:34.746979+0000","last_change":"2024-09-11T01:11:31.736403+0000","last_active":"2024-09-11T01:11:34.746979+0000","last_peered":"2024-09-11T01:11:34.746979+0000","last_clean":"2024-09-11T01:11:34.746979+0000","last_became_active":"2024-09-11T01:11:31.735830+0000","last_became_peered":"2024-09-11T01:11:31.735830+0000","last_unstale":"2024-09-11T01:11:34.746979+0000","last_undegraded":"2024-09-11T01:11:34.746979+0000","last_fullsized":"2024-09-11T01:11:34.746979+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:23:16.085781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:11:34.746812+0000","last_change":"2024-09-11T01:11:31.738452+0000","last_active":"2024-09-11T01:11:34.746812+0000","last_peered":"2024-09-11T01:11:34.746812+0000","last_clean":"2024-09-11T01:11:34.746812+0000","last_became_active":"2024-09-11T01:11:31.738044+0000","last_became_peered":"2024-09-11T01:11:31.738044+0000","last_unstale":"2024-09-11T01:11:34.746812+0000","last_undegraded":"2024-09-11T01:11:34.746812+0000","last_fullsized":"2024-09-11T01:11:34.746812+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:38:37.064918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:11:34.603998+0000","last_change":"2024-09-11T01:11:33.746462+0000","last_active":"2024-09-11T01:11:33.746462+0000","last_peered":"2024-09-11T01:11:32.735868+0000","last_clean":"2024-09-11T01:11:32.735868+0000","last_became_active":"2024-09-11T01:11:31.727523+0000","last_became_peered":"2024-09-11T01:11:31.727523+0000","last_unstale":"2024-09-11T01:11:34.603998+0000","last_undegraded":"2024-09-11T01:11:34.603998+0000","last_fullsized":"2024-09-11T01:11:34.603998+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:11:34.746714+0000","last_change":"2024-09-11T01:11:22.669819+0000","last_active":"2024-09-11T01:11:34.746714+0000","last_peered":"2024-09-11T01:11:34.746714+0000","last_clean":"2024-09-11T01:11:34.746714+0000","last_became_active":"2024-09-11T01:11:20.659393+0000","last_became_peered":"2024-09-11T01:11:20.659393+0000","last_unstale":"2024-09-11T01:11:34.746714+0000","last_undegraded":"2024-09-11T01:11:34.746714+0000","last_fullsized":"2024-09-11T01:11:34.746714+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:27.933594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:11:33.747567+0000","last_change":"2024-09-11T01:11:33.747567+0000","last_active":"2024-09-11T01:11:33.747567+0000","last_peered":"2024-09-11T01:11:33.590946+0000","last_clean":"2024-09-11T01:11:33.590946+0000","last_became_active":"2024-09-11T01:11:31.740446+0000","last_became_peered":"2024-09-11T01:11:31.740446+0000","last_unstale":"2024-09-11T01:11:33.747567+0000","last_undegraded":"2024-09-11T01:11:33.747567+0000","last_fullsized":"2024-09-11T01:11:33.747567+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:11:34.604047+0000","last_change":"2024-09-11T01:11:33.746784+0000","last_active":"2024-09-11T01:11:19.623673+0000","last_peered":"2024-09-11T01:11:19.623673+0000","last_clean":"2024-09-11T01:11:19.623673+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:11:34.604047+0000","last_undegraded":"2024-09-11T01:11:34.604047+0000","last_fullsized":"2024-09-11T01:11:34.604047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:11:34.747498+0000","last_change":"2024-09-11T01:11:31.735507+0000","last_active":"2024-09-11T01:11:34.747498+0000","last_peered":"2024-09-11T01:11:34.747498+0000","last_clean":"2024-09-11T01:11:34.747498+0000","last_became_active":"2024-09-11T01:11:31.735258+0000","last_became_peered":"2024-09-11T01:11:31.735258+0000","last_unstale":"2024-09-11T01:11:34.747498+0000","last_undegraded":"2024-09-11T01:11:34.747498+0000","last_fullsized":"2024-09-11T01:11:34.747498+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:39.007497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":34,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:11:34.747756+0000","last_change":"2024-09-11T01:11:22.670534+0000","last_active":"2024-09-11T01:11:34.747756+0000","last_peered":"2024-09-11T01:11:34.747756+0000","last_clean":"2024-09-11T01:11:34.747756+0000","last_became_active":"2024-09-11T01:11:20.654870+0000","last_became_peered":"2024-09-11T01:11:20.654870+0000","last_unstale":"2024-09-11T01:11:34.747756+0000","last_undegraded":"2024-09-11T01:11:34.747756+0000","last_fullsized":"2024-09-11T01:11:34.747756+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:01:43.922314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":4,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-11T01:11:34.747675+0000","last_change":"2024-09-11T01:11:33.747393+0000","last_active":"2024-09-11T01:11:30.697577+0000","last_peered":"2024-09-11T01:11:30.697577+0000","last_clean":"2024-09-11T01:11:30.697577+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:11:34.747675+0000","last_undegraded":"2024-09-11T01:11:34.747675+0000","last_fullsized":"2024-09-11T01:11:34.747675+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":31,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:11:33.749304+0000","last_change":"2024-09-11T01:11:33.749304+0000","last_active":"2024-09-11T01:11:33.749304+0000","last_peered":"2024-09-11T01:11:32.737376+0000","last_clean":"2024-09-11T01:11:32.737376+0000","last_became_active":"2024-09-11T01:11:20.664883+0000","last_became_peered":"2024-09-11T01:11:20.664883+0000","last_unstale":"2024-09-11T01:11:33.749304+0000","last_undegraded":"2024-09-11T01:11:33.749304+0000","last_fullsized":"2024-09-11T01:11:33.749304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":4,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-11T01:11:34.752466+0000","last_change":"2024-09-11T01:11:33.745510+0000","last_active":"2024-09-11T01:11:30.697577+0000","last_peered":"2024-09-11T01:11:30.697577+0000","last_clean":"2024-09-11T01:11:30.697577+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:11:34.752466+0000","last_undegraded":"2024-09-11T01:11:34.752466+0000","last_fullsized":"2024-09-11T01:11:34.752466+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:11:34.751470+0000","last_change":"2024-09-11T01:11:31.736495+0000","last_active":"2024-09-11T01:11:34.751470+0000","last_peered":"2024-09-11T01:11:34.751470+0000","last_clean":"2024-09-11T01:11:34.751470+0000","last_became_active":"2024-09-11T01:11:31.735795+0000","last_became_peered":"2024-09-11T01:11:31.735795+0000","last_unstale":"2024-09-11T01:11:34.751470+0000","last_undegraded":"2024-09-11T01:11:34.751470+0000","last_fullsized":"2024-09-11T01:11:34.751470+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:53.172912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:11:34.750358+0000","last_change":"2024-09-11T01:11:22.672257+0000","last_active":"2024-09-11T01:11:34.750358+0000","last_peered":"2024-09-11T01:11:34.750358+0000","last_clean":"2024-09-11T01:11:34.750358+0000","last_became_active":"2024-09-11T01:11:20.659744+0000","last_became_peered":"2024-09-11T01:11:20.659744+0000","last_unstale":"2024-09-11T01:11:34.750358+0000","last_undegraded":"2024-09-11T01:11:34.750358+0000","last_fullsized":"2024-09-11T01:11:34.750358+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:12.118110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:11:34.750769+0000","last_change":"2024-09-11T01:11:31.738293+0000","last_active":"2024-09-11T01:11:34.750769+0000","last_peered":"2024-09-11T01:11:34.750769+0000","last_clean":"2024-09-11T01:11:34.750769+0000","last_became_active":"2024-09-11T01:11:31.737939+0000","last_became_peered":"2024-09-11T01:11:31.737939+0000","last_unstale":"2024-09-11T01:11:34.750769+0000","last_undegraded":"2024-09-11T01:11:34.750769+0000","last_fullsized":"2024-09-11T01:11:34.750769+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:10:20.475024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:11:34.604140+0000","last_change":"2024-09-11T01:11:33.751300+0000","last_active":"2024-09-11T01:11:19.623673+0000","last_peered":"2024-09-11T01:11:19.623673+0000","last_clean":"2024-09-11T01:11:19.623673+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:11:34.604140+0000","last_undegraded":"2024-09-11T01:11:34.604140+0000","last_fullsized":"2024-09-11T01:11:34.604140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:11:34.750456+0000","last_change":"2024-09-11T01:11:31.735839+0000","last_active":"2024-09-11T01:11:34.750456+0000","last_peered":"2024-09-11T01:11:34.750456+0000","last_clean":"2024-09-11T01:11:34.750456+0000","last_became_active":"2024-09-11T01:11:31.735451+0000","last_became_peered":"2024-09-11T01:11:31.735451+0000","last_unstale":"2024-09-11T01:11:34.750456+0000","last_undegraded":"2024-09-11T01:11:34.750456+0000","last_fullsized":"2024-09-11T01:11:34.750456+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:01:57.389515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:11:34.751026+0000","last_change":"2024-09-11T01:11:22.672920+0000","last_active":"2024-09-11T01:11:34.751026+0000","last_peered":"2024-09-11T01:11:34.751026+0000","last_clean":"2024-09-11T01:11:34.751026+0000","last_became_active":"2024-09-11T01:11:20.662201+0000","last_became_peered":"2024-09-11T01:11:20.662201+0000","last_unstale":"2024-09-11T01:11:34.751026+0000","last_undegraded":"2024-09-11T01:11:34.751026+0000","last_fullsized":"2024-09-11T01:11:34.751026+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:37.382862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":30,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:11:33.750192+0000","last_change":"2024-09-11T01:11:33.750192+0000","last_active":"2024-09-11T01:11:33.750192+0000","last_peered":"2024-09-11T01:11:32.739392+0000","last_clean":"2024-09-11T01:11:32.739392+0000","last_became_active":"2024-09-11T01:11:20.665799+0000","last_became_peered":"2024-09-11T01:11:20.665799+0000","last_unstale":"2024-09-11T01:11:33.750192+0000","last_undegraded":"2024-09-11T01:11:33.750192+0000","last_fullsized":"2024-09-11T01:11:33.750192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:11:34.750122+0000","last_change":"2024-09-11T01:11:31.738742+0000","last_active":"2024-09-11T01:11:34.750122+0000","last_peered":"2024-09-11T01:11:34.750122+0000","last_clean":"2024-09-11T01:11:34.750122+0000","last_became_active":"2024-09-11T01:11:31.738088+0000","last_became_peered":"2024-09-11T01:11:31.738088+0000","last_unstale":"2024-09-11T01:11:34.750122+0000","last_undegraded":"2024-09-11T01:11:34.750122+0000","last_fullsized":"2024-09-11T01:11:34.750122+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:50:04.038198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":9,"seq":38654705672,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":167936,"data_stored":43825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738377,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":167936,"data_stored":43825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:11:38.360 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:11:38.638 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:11:38.638 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:11:38.654 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-09-11T01:11:38.526248+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":24,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54376,"kb_used_data":328,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689304,"statfs":{"total":193273528320,"available":193217847296,"internally_reserved":0,"allocated":335872,"data_stored":87650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3190,"internal_metadata":55309194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.773506"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:11:34.749748+0000","last_change":"2024-09-11T01:11:31.741068+0000","last_active":"2024-09-11T01:11:34.749748+0000","last_peered":"2024-09-11T01:11:34.749748+0000","last_clean":"2024-09-11T01:11:34.749748+0000","last_became_active":"2024-09-11T01:11:31.740203+0000","last_became_peered":"2024-09-11T01:11:31.740203+0000","last_unstale":"2024-09-11T01:11:34.749748+0000","last_undegraded":"2024-09-11T01:11:34.749748+0000","last_fullsized":"2024-09-11T01:11:34.749748+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:23.192014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:11:34.604177+0000","last_change":"2024-09-11T01:11:33.749559+0000","last_active":"2024-09-11T01:11:30.697577+0000","last_peered":"2024-09-11T01:11:30.697577+0000","last_clean":"2024-09-11T01:11:30.697577+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:11:34.604177+0000","last_undegraded":"2024-09-11T01:11:34.604177+0000","last_fullsized":"2024-09-11T01:11:34.604177+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:11:33.752614+0000","last_change":"2024-09-11T01:11:33.752614+0000","last_active":"2024-09-11T01:11:33.752614+0000","last_peered":"2024-09-11T01:11:33.591006+0000","last_clean":"2024-09-11T01:11:33.591006+0000","last_became_active":"2024-09-11T01:11:31.742309+0000","last_became_peered":"2024-09-11T01:11:31.742309+0000","last_unstale":"2024-09-11T01:11:33.752614+0000","last_undegraded":"2024-09-11T01:11:33.752614+0000","last_fullsized":"2024-09-11T01:11:33.752614+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:11:34.604116+0000","last_change":"2024-09-11T01:11:33.747864+0000","last_active":"2024-09-11T01:11:30.697577+0000","last_peered":"2024-09-11T01:11:30.697577+0000","last_clean":"2024-09-11T01:11:30.697577+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:11:34.604116+0000","last_undegraded":"2024-09-11T01:11:34.604116+0000","last_fullsized":"2024-09-11T01:11:34.604116+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:11:34.604096+0000","last_change":"2024-09-11T01:11:33.750982+0000","last_active":"2024-09-11T01:11:33.750982+0000","last_peered":"2024-09-11T01:11:32.736803+0000","last_clean":"2024-09-11T01:11:32.736803+0000","last_became_active":"2024-09-11T01:11:31.727424+0000","last_became_peered":"2024-09-11T01:11:31.727424+0000","last_unstale":"2024-09-11T01:11:34.604096+0000","last_undegraded":"2024-09-11T01:11:34.604096+0000","last_fullsized":"2024-09-11T01:11:34.604096+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:11:34.746979+0000","last_change":"2024-09-11T01:11:31.736403+0000","last_active":"2024-09-11T01:11:34.746979+0000","last_peered":"2024-09-11T01:11:34.746979+0000","last_clean":"2024-09-11T01:11:34.746979+0000","last_became_active":"2024-09-11T01:11:31.735830+0000","last_became_peered":"2024-09-11T01:11:31.735830+0000","last_unstale":"2024-09-11T01:11:34.746979+0000","last_undegraded":"2024-09-11T01:11:34.746979+0000","last_fullsized":"2024-09-11T01:11:34.746979+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:23:16.085781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:11:34.746812+0000","last_change":"2024-09-11T01:11:31.738452+0000","last_active":"2024-09-11T01:11:34.746812+0000","last_peered":"2024-09-11T01:11:34.746812+0000","last_clean":"2024-09-11T01:11:34.746812+0000","last_became_active":"2024-09-11T01:11:31.738044+0000","last_became_peered":"2024-09-11T01:11:31.738044+0000","last_unstale":"2024-09-11T01:11:34.746812+0000","last_undegraded":"2024-09-11T01:11:34.746812+0000","last_fullsized":"2024-09-11T01:11:34.746812+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:38:37.064918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:11:34.603998+0000","last_change":"2024-09-11T01:11:33.746462+0000","last_active":"2024-09-11T01:11:33.746462+0000","last_peered":"2024-09-11T01:11:32.735868+0000","last_clean":"2024-09-11T01:11:32.735868+0000","last_became_active":"2024-09-11T01:11:31.727523+0000","last_became_peered":"2024-09-11T01:11:31.727523+0000","last_unstale":"2024-09-11T01:11:34.603998+0000","last_undegraded":"2024-09-11T01:11:34.603998+0000","last_fullsized":"2024-09-11T01:11:34.603998+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:11:34.746714+0000","last_change":"2024-09-11T01:11:22.669819+0000","last_active":"2024-09-11T01:11:34.746714+0000","last_peered":"2024-09-11T01:11:34.746714+0000","last_clean":"2024-09-11T01:11:34.746714+0000","last_became_active":"2024-09-11T01:11:20.659393+0000","last_became_peered":"2024-09-11T01:11:20.659393+0000","last_unstale":"2024-09-11T01:11:34.746714+0000","last_undegraded":"2024-09-11T01:11:34.746714+0000","last_fullsized":"2024-09-11T01:11:34.746714+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:27.933594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:11:33.747567+0000","last_change":"2024-09-11T01:11:33.747567+0000","last_active":"2024-09-11T01:11:33.747567+0000","last_peered":"2024-09-11T01:11:33.590946+0000","last_clean":"2024-09-11T01:11:33.590946+0000","last_became_active":"2024-09-11T01:11:31.740446+0000","last_became_peered":"2024-09-11T01:11:31.740446+0000","last_unstale":"2024-09-11T01:11:33.747567+0000","last_undegraded":"2024-09-11T01:11:33.747567+0000","last_fullsized":"2024-09-11T01:11:33.747567+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:11:34.604047+0000","last_change":"2024-09-11T01:11:33.746784+0000","last_active":"2024-09-11T01:11:19.623673+0000","last_peered":"2024-09-11T01:11:19.623673+0000","last_clean":"2024-09-11T01:11:19.623673+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:11:34.604047+0000","last_undegraded":"2024-09-11T01:11:34.604047+0000","last_fullsized":"2024-09-11T01:11:34.604047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:11:34.747498+0000","last_change":"2024-09-11T01:11:31.735507+0000","last_active":"2024-09-11T01:11:34.747498+0000","last_peered":"2024-09-11T01:11:34.747498+0000","last_clean":"2024-09-11T01:11:34.747498+0000","last_became_active":"2024-09-11T01:11:31.735258+0000","last_became_peered":"2024-09-11T01:11:31.735258+0000","last_unstale":"2024-09-11T01:11:34.747498+0000","last_undegraded":"2024-09-11T01:11:34.747498+0000","last_fullsized":"2024-09-11T01:11:34.747498+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:39.007497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":34,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:11:34.747756+0000","last_change":"2024-09-11T01:11:22.670534+0000","last_active":"2024-09-11T01:11:34.747756+0000","last_peered":"2024-09-11T01:11:34.747756+0000","last_clean":"2024-09-11T01:11:34.747756+0000","last_became_active":"2024-09-11T01:11:20.654870+0000","last_became_peered":"2024-09-11T01:11:20.654870+0000","last_unstale":"2024-09-11T01:11:34.747756+0000","last_undegraded":"2024-09-11T01:11:34.747756+0000","last_fullsized":"2024-09-11T01:11:34.747756+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:01:43.922314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":4,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-11T01:11:34.747675+0000","last_change":"2024-09-11T01:11:33.747393+0000","last_active":"2024-09-11T01:11:30.697577+0000","last_peered":"2024-09-11T01:11:30.697577+0000","last_clean":"2024-09-11T01:11:30.697577+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:11:34.747675+0000","last_undegraded":"2024-09-11T01:11:34.747675+0000","last_fullsized":"2024-09-11T01:11:34.747675+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":31,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:11:33.749304+0000","last_change":"2024-09-11T01:11:33.749304+0000","last_active":"2024-09-11T01:11:33.749304+0000","last_peered":"2024-09-11T01:11:32.737376+0000","last_clean":"2024-09-11T01:11:32.737376+0000","last_became_active":"2024-09-11T01:11:20.664883+0000","last_became_peered":"2024-09-11T01:11:20.664883+0000","last_unstale":"2024-09-11T01:11:33.749304+0000","last_undegraded":"2024-09-11T01:11:33.749304+0000","last_fullsized":"2024-09-11T01:11:33.749304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":4,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-11T01:11:34.752466+0000","last_change":"2024-09-11T01:11:33.745510+0000","last_active":"2024-09-11T01:11:30.697577+0000","last_peered":"2024-09-11T01:11:30.697577+0000","last_clean":"2024-09-11T01:11:30.697577+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:11:34.752466+0000","last_undegraded":"2024-09-11T01:11:34.752466+0000","last_fullsized":"2024-09-11T01:11:34.752466+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:11:34.751470+0000","last_change":"2024-09-11T01:11:31.736495+0000","last_active":"2024-09-11T01:11:34.751470+0000","last_peered":"2024-09-11T01:11:34.751470+0000","last_clean":"2024-09-11T01:11:34.751470+0000","last_became_active":"2024-09-11T01:11:31.735795+0000","last_became_peered":"2024-09-11T01:11:31.735795+0000","last_unstale":"2024-09-11T01:11:34.751470+0000","last_undegraded":"2024-09-11T01:11:34.751470+0000","last_fullsized":"2024-09-11T01:11:34.751470+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:53.172912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:11:34.750358+0000","last_change":"2024-09-11T01:11:22.672257+0000","last_active":"2024-09-11T01:11:34.750358+0000","last_peered":"2024-09-11T01:11:34.750358+0000","last_clean":"2024-09-11T01:11:34.750358+0000","last_became_active":"2024-09-11T01:11:20.659744+0000","last_became_peered":"2024-09-11T01:11:20.659744+0000","last_unstale":"2024-09-11T01:11:34.750358+0000","last_undegraded":"2024-09-11T01:11:34.750358+0000","last_fullsized":"2024-09-11T01:11:34.750358+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:12.118110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:11:34.750769+0000","last_change":"2024-09-11T01:11:31.738293+0000","last_active":"2024-09-11T01:11:34.750769+0000","last_peered":"2024-09-11T01:11:34.750769+0000","last_clean":"2024-09-11T01:11:34.750769+0000","last_became_active":"2024-09-11T01:11:31.737939+0000","last_became_peered":"2024-09-11T01:11:31.737939+0000","last_unstale":"2024-09-11T01:11:34.750769+0000","last_undegraded":"2024-09-11T01:11:34.750769+0000","last_fullsized":"2024-09-11T01:11:34.750769+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:10:20.475024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:11:34.604140+0000","last_change":"2024-09-11T01:11:33.751300+0000","last_active":"2024-09-11T01:11:19.623673+0000","last_peered":"2024-09-11T01:11:19.623673+0000","last_clean":"2024-09-11T01:11:19.623673+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:11:34.604140+0000","last_undegraded":"2024-09-11T01:11:34.604140+0000","last_fullsized":"2024-09-11T01:11:34.604140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:11:34.750456+0000","last_change":"2024-09-11T01:11:31.735839+0000","last_active":"2024-09-11T01:11:34.750456+0000","last_peered":"2024-09-11T01:11:34.750456+0000","last_clean":"2024-09-11T01:11:34.750456+0000","last_became_active":"2024-09-11T01:11:31.735451+0000","last_became_peered":"2024-09-11T01:11:31.735451+0000","last_unstale":"2024-09-11T01:11:34.750456+0000","last_undegraded":"2024-09-11T01:11:34.750456+0000","last_fullsized":"2024-09-11T01:11:34.750456+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:01:57.389515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:11:34.751026+0000","last_change":"2024-09-11T01:11:22.672920+0000","last_active":"2024-09-11T01:11:34.751026+0000","last_peered":"2024-09-11T01:11:34.751026+0000","last_clean":"2024-09-11T01:11:34.751026+0000","last_became_active":"2024-09-11T01:11:20.662201+0000","last_became_peered":"2024-09-11T01:11:20.662201+0000","last_unstale":"2024-09-11T01:11:34.751026+0000","last_undegraded":"2024-09-11T01:11:34.751026+0000","last_fullsized":"2024-09-11T01:11:34.751026+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:37.382862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":30,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:11:33.750192+0000","last_change":"2024-09-11T01:11:33.750192+0000","last_active":"2024-09-11T01:11:33.750192+0000","last_peered":"2024-09-11T01:11:32.739392+0000","last_clean":"2024-09-11T01:11:32.739392+0000","last_became_active":"2024-09-11T01:11:20.665799+0000","last_became_peered":"2024-09-11T01:11:20.665799+0000","last_unstale":"2024-09-11T01:11:33.750192+0000","last_undegraded":"2024-09-11T01:11:33.750192+0000","last_fullsized":"2024-09-11T01:11:33.750192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:11:34.750122+0000","last_change":"2024-09-11T01:11:31.738742+0000","last_active":"2024-09-11T01:11:34.750122+0000","last_peered":"2024-09-11T01:11:34.750122+0000","last_clean":"2024-09-11T01:11:34.750122+0000","last_became_active":"2024-09-11T01:11:31.738088+0000","last_became_peered":"2024-09-11T01:11:31.738088+0000","last_unstale":"2024-09-11T01:11:34.750122+0000","last_undegraded":"2024-09-11T01:11:34.750122+0000","last_fullsized":"2024-09-11T01:11:34.750122+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:50:04.038198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":9,"seq":38654705672,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":167936,"data_stored":43825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738377,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":167936,"data_stored":43825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:11:38.655 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:11:38.923 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:11:38.923 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:11:38.939 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2024-09-11T01:11:38.526248+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":24,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54376,"kb_used_data":328,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689304,"statfs":{"total":193273528320,"available":193217847296,"internally_reserved":0,"allocated":335872,"data_stored":87650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3190,"internal_metadata":55309194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.773506"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:11:34.749748+0000","last_change":"2024-09-11T01:11:31.741068+0000","last_active":"2024-09-11T01:11:34.749748+0000","last_peered":"2024-09-11T01:11:34.749748+0000","last_clean":"2024-09-11T01:11:34.749748+0000","last_became_active":"2024-09-11T01:11:31.740203+0000","last_became_peered":"2024-09-11T01:11:31.740203+0000","last_unstale":"2024-09-11T01:11:34.749748+0000","last_undegraded":"2024-09-11T01:11:34.749748+0000","last_fullsized":"2024-09-11T01:11:34.749748+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:23.192014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:11:34.604177+0000","last_change":"2024-09-11T01:11:33.749559+0000","last_active":"2024-09-11T01:11:30.697577+0000","last_peered":"2024-09-11T01:11:30.697577+0000","last_clean":"2024-09-11T01:11:30.697577+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:11:34.604177+0000","last_undegraded":"2024-09-11T01:11:34.604177+0000","last_fullsized":"2024-09-11T01:11:34.604177+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:11:33.752614+0000","last_change":"2024-09-11T01:11:33.752614+0000","last_active":"2024-09-11T01:11:33.752614+0000","last_peered":"2024-09-11T01:11:33.591006+0000","last_clean":"2024-09-11T01:11:33.591006+0000","last_became_active":"2024-09-11T01:11:31.742309+0000","last_became_peered":"2024-09-11T01:11:31.742309+0000","last_unstale":"2024-09-11T01:11:33.752614+0000","last_undegraded":"2024-09-11T01:11:33.752614+0000","last_fullsized":"2024-09-11T01:11:33.752614+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:11:34.604116+0000","last_change":"2024-09-11T01:11:33.747864+0000","last_active":"2024-09-11T01:11:30.697577+0000","last_peered":"2024-09-11T01:11:30.697577+0000","last_clean":"2024-09-11T01:11:30.697577+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:11:34.604116+0000","last_undegraded":"2024-09-11T01:11:34.604116+0000","last_fullsized":"2024-09-11T01:11:34.604116+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:11:34.604096+0000","last_change":"2024-09-11T01:11:33.750982+0000","last_active":"2024-09-11T01:11:33.750982+0000","last_peered":"2024-09-11T01:11:32.736803+0000","last_clean":"2024-09-11T01:11:32.736803+0000","last_became_active":"2024-09-11T01:11:31.727424+0000","last_became_peered":"2024-09-11T01:11:31.727424+0000","last_unstale":"2024-09-11T01:11:34.604096+0000","last_undegraded":"2024-09-11T01:11:34.604096+0000","last_fullsized":"2024-09-11T01:11:34.604096+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:11:34.746979+0000","last_change":"2024-09-11T01:11:31.736403+0000","last_active":"2024-09-11T01:11:34.746979+0000","last_peered":"2024-09-11T01:11:34.746979+0000","last_clean":"2024-09-11T01:11:34.746979+0000","last_became_active":"2024-09-11T01:11:31.735830+0000","last_became_peered":"2024-09-11T01:11:31.735830+0000","last_unstale":"2024-09-11T01:11:34.746979+0000","last_undegraded":"2024-09-11T01:11:34.746979+0000","last_fullsized":"2024-09-11T01:11:34.746979+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:23:16.085781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:11:34.746812+0000","last_change":"2024-09-11T01:11:31.738452+0000","last_active":"2024-09-11T01:11:34.746812+0000","last_peered":"2024-09-11T01:11:34.746812+0000","last_clean":"2024-09-11T01:11:34.746812+0000","last_became_active":"2024-09-11T01:11:31.738044+0000","last_became_peered":"2024-09-11T01:11:31.738044+0000","last_unstale":"2024-09-11T01:11:34.746812+0000","last_undegraded":"2024-09-11T01:11:34.746812+0000","last_fullsized":"2024-09-11T01:11:34.746812+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:38:37.064918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:11:34.603998+0000","last_change":"2024-09-11T01:11:33.746462+0000","last_active":"2024-09-11T01:11:33.746462+0000","last_peered":"2024-09-11T01:11:32.735868+0000","last_clean":"2024-09-11T01:11:32.735868+0000","last_became_active":"2024-09-11T01:11:31.727523+0000","last_became_peered":"2024-09-11T01:11:31.727523+0000","last_unstale":"2024-09-11T01:11:34.603998+0000","last_undegraded":"2024-09-11T01:11:34.603998+0000","last_fullsized":"2024-09-11T01:11:34.603998+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:11:34.746714+0000","last_change":"2024-09-11T01:11:22.669819+0000","last_active":"2024-09-11T01:11:34.746714+0000","last_peered":"2024-09-11T01:11:34.746714+0000","last_clean":"2024-09-11T01:11:34.746714+0000","last_became_active":"2024-09-11T01:11:20.659393+0000","last_became_peered":"2024-09-11T01:11:20.659393+0000","last_unstale":"2024-09-11T01:11:34.746714+0000","last_undegraded":"2024-09-11T01:11:34.746714+0000","last_fullsized":"2024-09-11T01:11:34.746714+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:27.933594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:11:33.747567+0000","last_change":"2024-09-11T01:11:33.747567+0000","last_active":"2024-09-11T01:11:33.747567+0000","last_peered":"2024-09-11T01:11:33.590946+0000","last_clean":"2024-09-11T01:11:33.590946+0000","last_became_active":"2024-09-11T01:11:31.740446+0000","last_became_peered":"2024-09-11T01:11:31.740446+0000","last_unstale":"2024-09-11T01:11:33.747567+0000","last_undegraded":"2024-09-11T01:11:33.747567+0000","last_fullsized":"2024-09-11T01:11:33.747567+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:11:34.604047+0000","last_change":"2024-09-11T01:11:33.746784+0000","last_active":"2024-09-11T01:11:19.623673+0000","last_peered":"2024-09-11T01:11:19.623673+0000","last_clean":"2024-09-11T01:11:19.623673+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:11:34.604047+0000","last_undegraded":"2024-09-11T01:11:34.604047+0000","last_fullsized":"2024-09-11T01:11:34.604047+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:11:34.747498+0000","last_change":"2024-09-11T01:11:31.735507+0000","last_active":"2024-09-11T01:11:34.747498+0000","last_peered":"2024-09-11T01:11:34.747498+0000","last_clean":"2024-09-11T01:11:34.747498+0000","last_became_active":"2024-09-11T01:11:31.735258+0000","last_became_peered":"2024-09-11T01:11:31.735258+0000","last_unstale":"2024-09-11T01:11:34.747498+0000","last_undegraded":"2024-09-11T01:11:34.747498+0000","last_fullsized":"2024-09-11T01:11:34.747498+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:39.007497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":34,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:11:34.747756+0000","last_change":"2024-09-11T01:11:22.670534+0000","last_active":"2024-09-11T01:11:34.747756+0000","last_peered":"2024-09-11T01:11:34.747756+0000","last_clean":"2024-09-11T01:11:34.747756+0000","last_became_active":"2024-09-11T01:11:20.654870+0000","last_became_peered":"2024-09-11T01:11:20.654870+0000","last_unstale":"2024-09-11T01:11:34.747756+0000","last_undegraded":"2024-09-11T01:11:34.747756+0000","last_fullsized":"2024-09-11T01:11:34.747756+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:01:43.922314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":4,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-11T01:11:34.747675+0000","last_change":"2024-09-11T01:11:33.747393+0000","last_active":"2024-09-11T01:11:30.697577+0000","last_peered":"2024-09-11T01:11:30.697577+0000","last_clean":"2024-09-11T01:11:30.697577+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:11:34.747675+0000","last_undegraded":"2024-09-11T01:11:34.747675+0000","last_fullsized":"2024-09-11T01:11:34.747675+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":31,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:11:33.749304+0000","last_change":"2024-09-11T01:11:33.749304+0000","last_active":"2024-09-11T01:11:33.749304+0000","last_peered":"2024-09-11T01:11:32.737376+0000","last_clean":"2024-09-11T01:11:32.737376+0000","last_became_active":"2024-09-11T01:11:20.664883+0000","last_became_peered":"2024-09-11T01:11:20.664883+0000","last_unstale":"2024-09-11T01:11:33.749304+0000","last_undegraded":"2024-09-11T01:11:33.749304+0000","last_fullsized":"2024-09-11T01:11:33.749304+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":4,"reported_epoch":20,"state":"peering","last_fresh":"2024-09-11T01:11:34.752466+0000","last_change":"2024-09-11T01:11:33.745510+0000","last_active":"2024-09-11T01:11:30.697577+0000","last_peered":"2024-09-11T01:11:30.697577+0000","last_clean":"2024-09-11T01:11:30.697577+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:11:34.752466+0000","last_undegraded":"2024-09-11T01:11:34.752466+0000","last_fullsized":"2024-09-11T01:11:34.752466+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:11:34.751470+0000","last_change":"2024-09-11T01:11:31.736495+0000","last_active":"2024-09-11T01:11:34.751470+0000","last_peered":"2024-09-11T01:11:34.751470+0000","last_clean":"2024-09-11T01:11:34.751470+0000","last_became_active":"2024-09-11T01:11:31.735795+0000","last_became_peered":"2024-09-11T01:11:31.735795+0000","last_unstale":"2024-09-11T01:11:34.751470+0000","last_undegraded":"2024-09-11T01:11:34.751470+0000","last_fullsized":"2024-09-11T01:11:34.751470+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:53.172912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:11:34.750358+0000","last_change":"2024-09-11T01:11:22.672257+0000","last_active":"2024-09-11T01:11:34.750358+0000","last_peered":"2024-09-11T01:11:34.750358+0000","last_clean":"2024-09-11T01:11:34.750358+0000","last_became_active":"2024-09-11T01:11:20.659744+0000","last_became_peered":"2024-09-11T01:11:20.659744+0000","last_unstale":"2024-09-11T01:11:34.750358+0000","last_undegraded":"2024-09-11T01:11:34.750358+0000","last_fullsized":"2024-09-11T01:11:34.750358+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:12.118110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:11:34.750769+0000","last_change":"2024-09-11T01:11:31.738293+0000","last_active":"2024-09-11T01:11:34.750769+0000","last_peered":"2024-09-11T01:11:34.750769+0000","last_clean":"2024-09-11T01:11:34.750769+0000","last_became_active":"2024-09-11T01:11:31.737939+0000","last_became_peered":"2024-09-11T01:11:31.737939+0000","last_unstale":"2024-09-11T01:11:34.750769+0000","last_undegraded":"2024-09-11T01:11:34.750769+0000","last_fullsized":"2024-09-11T01:11:34.750769+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:10:20.475024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":4,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:11:34.604140+0000","last_change":"2024-09-11T01:11:33.751300+0000","last_active":"2024-09-11T01:11:19.623673+0000","last_peered":"2024-09-11T01:11:19.623673+0000","last_clean":"2024-09-11T01:11:19.623673+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-11T01:11:34.604140+0000","last_undegraded":"2024-09-11T01:11:34.604140+0000","last_fullsized":"2024-09-11T01:11:34.604140+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:11:34.750456+0000","last_change":"2024-09-11T01:11:31.735839+0000","last_active":"2024-09-11T01:11:34.750456+0000","last_peered":"2024-09-11T01:11:34.750456+0000","last_clean":"2024-09-11T01:11:34.750456+0000","last_became_active":"2024-09-11T01:11:31.735451+0000","last_became_peered":"2024-09-11T01:11:31.735451+0000","last_unstale":"2024-09-11T01:11:34.750456+0000","last_undegraded":"2024-09-11T01:11:34.750456+0000","last_fullsized":"2024-09-11T01:11:34.750456+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:01:57.389515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":32,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:11:34.751026+0000","last_change":"2024-09-11T01:11:22.672920+0000","last_active":"2024-09-11T01:11:34.751026+0000","last_peered":"2024-09-11T01:11:34.751026+0000","last_clean":"2024-09-11T01:11:34.751026+0000","last_became_active":"2024-09-11T01:11:20.662201+0000","last_became_peered":"2024-09-11T01:11:20.662201+0000","last_unstale":"2024-09-11T01:11:34.751026+0000","last_undegraded":"2024-09-11T01:11:34.751026+0000","last_fullsized":"2024-09-11T01:11:34.751026+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:37.382862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":30,"reported_epoch":19,"state":"peering","last_fresh":"2024-09-11T01:11:33.750192+0000","last_change":"2024-09-11T01:11:33.750192+0000","last_active":"2024-09-11T01:11:33.750192+0000","last_peered":"2024-09-11T01:11:32.739392+0000","last_clean":"2024-09-11T01:11:32.739392+0000","last_became_active":"2024-09-11T01:11:20.665799+0000","last_became_peered":"2024-09-11T01:11:20.665799+0000","last_unstale":"2024-09-11T01:11:33.750192+0000","last_undegraded":"2024-09-11T01:11:33.750192+0000","last_fullsized":"2024-09-11T01:11:33.750192+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-11T01:11:34.750122+0000","last_change":"2024-09-11T01:11:31.738742+0000","last_active":"2024-09-11T01:11:34.750122+0000","last_peered":"2024-09-11T01:11:34.750122+0000","last_clean":"2024-09-11T01:11:34.750122+0000","last_became_active":"2024-09-11T01:11:31.738088+0000","last_became_peered":"2024-09-11T01:11:31.738088+0000","last_unstale":"2024-09-11T01:11:34.750122+0000","last_undegraded":"2024-09-11T01:11:34.750122+0000","last_fullsized":"2024-09-11T01:11:34.750122+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:50:04.038198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":9,"seq":38654705672,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":167936,"data_stored":43825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738377,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27188,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344652,"statfs":{"total":96636764160,"available":96608923648,"internally_reserved":0,"allocated":167936,"data_stored":43825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:11:41.941 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:11:42.256 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:11:42.257 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:11:42.274 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2024-09-11T01:11:40.526648+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":42,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":54416,"kb_used_data":368,"kb_used_omap":3,"kb_used_meta":54012,"kb_avail":188689264,"statfs":{"total":193273528320,"available":193217806336,"internally_reserved":0,"allocated":376832,"data_stored":99868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3190,"internal_metadata":55309194},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.784390"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:11:35.762177+0000","last_change":"2024-09-11T01:11:31.741068+0000","last_active":"2024-09-11T01:11:35.762177+0000","last_peered":"2024-09-11T01:11:35.762177+0000","last_clean":"2024-09-11T01:11:35.762177+0000","last_became_active":"2024-09-11T01:11:31.740203+0000","last_became_peered":"2024-09-11T01:11:31.740203+0000","last_unstale":"2024-09-11T01:11:35.762177+0000","last_undegraded":"2024-09-11T01:11:35.762177+0000","last_fullsized":"2024-09-11T01:11:35.762177+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:23.192014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:11:35.773694+0000","last_change":"2024-09-11T01:11:35.773432+0000","last_active":"2024-09-11T01:11:35.773694+0000","last_peered":"2024-09-11T01:11:35.773694+0000","last_clean":"2024-09-11T01:11:35.773694+0000","last_became_active":"2024-09-11T01:11:35.772699+0000","last_became_peered":"2024-09-11T01:11:35.772699+0000","last_unstale":"2024-09-11T01:11:35.773694+0000","last_undegraded":"2024-09-11T01:11:35.773694+0000","last_fullsized":"2024-09-11T01:11:35.773694+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:37:16.393565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":24,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:11:35.769260+0000","last_change":"2024-09-11T01:11:35.769008+0000","last_active":"2024-09-11T01:11:35.769260+0000","last_peered":"2024-09-11T01:11:35.769260+0000","last_clean":"2024-09-11T01:11:35.769260+0000","last_became_active":"2024-09-11T01:11:35.768237+0000","last_became_peered":"2024-09-11T01:11:35.768237+0000","last_unstale":"2024-09-11T01:11:35.769260+0000","last_undegraded":"2024-09-11T01:11:35.769260+0000","last_fullsized":"2024-09-11T01:11:35.769260+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:05:07.319570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:11:35.773651+0000","last_change":"2024-09-11T01:11:35.773376+0000","last_active":"2024-09-11T01:11:35.773651+0000","last_peered":"2024-09-11T01:11:35.773651+0000","last_clean":"2024-09-11T01:11:35.773651+0000","last_became_active":"2024-09-11T01:11:35.772595+0000","last_became_peered":"2024-09-11T01:11:35.772595+0000","last_unstale":"2024-09-11T01:11:35.773651+0000","last_undegraded":"2024-09-11T01:11:35.773651+0000","last_fullsized":"2024-09-11T01:11:35.773651+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:29.338645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":24,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:11:35.774074+0000","last_change":"2024-09-11T01:11:35.773831+0000","last_active":"2024-09-11T01:11:35.774074+0000","last_peered":"2024-09-11T01:11:35.774074+0000","last_clean":"2024-09-11T01:11:35.774074+0000","last_became_active":"2024-09-11T01:11:35.772259+0000","last_became_peered":"2024-09-11T01:11:35.772259+0000","last_unstale":"2024-09-11T01:11:35.774074+0000","last_undegraded":"2024-09-11T01:11:35.774074+0000","last_fullsized":"2024-09-11T01:11:35.774074+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:25:50.336189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:11:35.756602+0000","last_change":"2024-09-11T01:11:31.736403+0000","last_active":"2024-09-11T01:11:35.756602+0000","last_peered":"2024-09-11T01:11:35.756602+0000","last_clean":"2024-09-11T01:11:35.756602+0000","last_became_active":"2024-09-11T01:11:31.735830+0000","last_became_peered":"2024-09-11T01:11:31.735830+0000","last_unstale":"2024-09-11T01:11:35.756602+0000","last_undegraded":"2024-09-11T01:11:35.756602+0000","last_fullsized":"2024-09-11T01:11:35.756602+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:23:16.085781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:11:35.757316+0000","last_change":"2024-09-11T01:11:31.738452+0000","last_active":"2024-09-11T01:11:35.757316+0000","last_peered":"2024-09-11T01:11:35.757316+0000","last_clean":"2024-09-11T01:11:35.757316+0000","last_became_active":"2024-09-11T01:11:31.738044+0000","last_became_peered":"2024-09-11T01:11:31.738044+0000","last_unstale":"2024-09-11T01:11:35.757316+0000","last_undegraded":"2024-09-11T01:11:35.757316+0000","last_fullsized":"2024-09-11T01:11:35.757316+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:38:37.064918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":24,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:11:35.767577+0000","last_change":"2024-09-11T01:11:35.767431+0000","last_active":"2024-09-11T01:11:35.767577+0000","last_peered":"2024-09-11T01:11:35.767577+0000","last_clean":"2024-09-11T01:11:35.767577+0000","last_became_active":"2024-09-11T01:11:35.766734+0000","last_became_peered":"2024-09-11T01:11:35.766734+0000","last_unstale":"2024-09-11T01:11:35.767577+0000","last_undegraded":"2024-09-11T01:11:35.767577+0000","last_fullsized":"2024-09-11T01:11:35.767577+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:49:28.101742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":34,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:11:35.757036+0000","last_change":"2024-09-11T01:11:22.669819+0000","last_active":"2024-09-11T01:11:35.757036+0000","last_peered":"2024-09-11T01:11:35.757036+0000","last_clean":"2024-09-11T01:11:35.757036+0000","last_became_active":"2024-09-11T01:11:20.659393+0000","last_became_peered":"2024-09-11T01:11:20.659393+0000","last_unstale":"2024-09-11T01:11:35.757036+0000","last_undegraded":"2024-09-11T01:11:35.757036+0000","last_fullsized":"2024-09-11T01:11:35.757036+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:27.933594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":24,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:11:35.767251+0000","last_change":"2024-09-11T01:11:35.767095+0000","last_active":"2024-09-11T01:11:35.767251+0000","last_peered":"2024-09-11T01:11:35.767251+0000","last_clean":"2024-09-11T01:11:35.767251+0000","last_became_active":"2024-09-11T01:11:35.766340+0000","last_became_peered":"2024-09-11T01:11:35.766340+0000","last_unstale":"2024-09-11T01:11:35.767251+0000","last_undegraded":"2024-09-11T01:11:35.767251+0000","last_fullsized":"2024-09-11T01:11:35.767251+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:56:31.918928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:11:35.770589+0000","last_change":"2024-09-11T01:11:35.770341+0000","last_active":"2024-09-11T01:11:35.770589+0000","last_peered":"2024-09-11T01:11:35.770589+0000","last_clean":"2024-09-11T01:11:35.770589+0000","last_became_active":"2024-09-11T01:11:35.769594+0000","last_became_peered":"2024-09-11T01:11:35.769594+0000","last_unstale":"2024-09-11T01:11:35.770589+0000","last_undegraded":"2024-09-11T01:11:35.770589+0000","last_fullsized":"2024-09-11T01:11:35.770589+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:35.313456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:11:35.761328+0000","last_change":"2024-09-11T01:11:31.735507+0000","last_active":"2024-09-11T01:11:35.761328+0000","last_peered":"2024-09-11T01:11:35.761328+0000","last_clean":"2024-09-11T01:11:35.761328+0000","last_became_active":"2024-09-11T01:11:31.735258+0000","last_became_peered":"2024-09-11T01:11:31.735258+0000","last_unstale":"2024-09-11T01:11:35.761328+0000","last_undegraded":"2024-09-11T01:11:35.761328+0000","last_fullsized":"2024-09-11T01:11:35.761328+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:05:39.007497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":36,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:11:35.761593+0000","last_change":"2024-09-11T01:11:22.670534+0000","last_active":"2024-09-11T01:11:35.761593+0000","last_peered":"2024-09-11T01:11:35.761593+0000","last_clean":"2024-09-11T01:11:35.761593+0000","last_became_active":"2024-09-11T01:11:20.654870+0000","last_became_peered":"2024-09-11T01:11:20.654870+0000","last_unstale":"2024-09-11T01:11:35.761593+0000","last_undegraded":"2024-09-11T01:11:35.761593+0000","last_fullsized":"2024-09-11T01:11:35.761593+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:01:43.922314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:11:35.765617+0000","last_change":"2024-09-11T01:11:35.765350+0000","last_active":"2024-09-11T01:11:35.765617+0000","last_peered":"2024-09-11T01:11:35.765617+0000","last_clean":"2024-09-11T01:11:35.765617+0000","last_became_active":"2024-09-11T01:11:35.764566+0000","last_became_peered":"2024-09-11T01:11:35.764566+0000","last_unstale":"2024-09-11T01:11:35.765617+0000","last_undegraded":"2024-09-11T01:11:35.765617+0000","last_fullsized":"2024-09-11T01:11:35.765617+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:49:07.785699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":44,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:11:35.780065+0000","last_change":"2024-09-11T01:11:35.779895+0000","last_active":"2024-09-11T01:11:35.780065+0000","last_peered":"2024-09-11T01:11:35.780065+0000","last_clean":"2024-09-11T01:11:35.780065+0000","last_became_active":"2024-09-11T01:11:35.768544+0000","last_became_peered":"2024-09-11T01:11:35.768544+0000","last_unstale":"2024-09-11T01:11:35.780065+0000","last_undegraded":"2024-09-11T01:11:35.780065+0000","last_fullsized":"2024-09-11T01:11:35.780065+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:07:24.332790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:11:35.765576+0000","last_change":"2024-09-11T01:11:35.765281+0000","last_active":"2024-09-11T01:11:35.765576+0000","last_peered":"2024-09-11T01:11:35.765576+0000","last_clean":"2024-09-11T01:11:35.765576+0000","last_became_active":"2024-09-11T01:11:35.764648+0000","last_became_peered":"2024-09-11T01:11:35.764648+0000","last_unstale":"2024-09-11T01:11:35.765576+0000","last_undegraded":"2024-09-11T01:11:35.765576+0000","last_fullsized":"2024-09-11T01:11:35.765576+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:57:12.494813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:11:35.758750+0000","last_change":"2024-09-11T01:11:31.736495+0000","last_active":"2024-09-11T01:11:35.758750+0000","last_peered":"2024-09-11T01:11:35.758750+0000","last_clean":"2024-09-11T01:11:35.758750+0000","last_became_active":"2024-09-11T01:11:31.735795+0000","last_became_peered":"2024-09-11T01:11:31.735795+0000","last_unstale":"2024-09-11T01:11:35.758750+0000","last_undegraded":"2024-09-11T01:11:35.758750+0000","last_fullsized":"2024-09-11T01:11:35.758750+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:38:53.172912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":34,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:11:35.757237+0000","last_change":"2024-09-11T01:11:22.672257+0000","last_active":"2024-09-11T01:11:35.757237+0000","last_peered":"2024-09-11T01:11:35.757237+0000","last_clean":"2024-09-11T01:11:35.757237+0000","last_became_active":"2024-09-11T01:11:20.659744+0000","last_became_peered":"2024-09-11T01:11:20.659744+0000","last_unstale":"2024-09-11T01:11:35.757237+0000","last_undegraded":"2024-09-11T01:11:35.757237+0000","last_fullsized":"2024-09-11T01:11:35.757237+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:59:12.118110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:11:35.758009+0000","last_change":"2024-09-11T01:11:31.738293+0000","last_active":"2024-09-11T01:11:35.758009+0000","last_peered":"2024-09-11T01:11:35.758009+0000","last_clean":"2024-09-11T01:11:35.758009+0000","last_became_active":"2024-09-11T01:11:31.737939+0000","last_became_peered":"2024-09-11T01:11:31.737939+0000","last_unstale":"2024-09-11T01:11:35.758009+0000","last_undegraded":"2024-09-11T01:11:35.758009+0000","last_fullsized":"2024-09-11T01:11:35.758009+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T13:10:20.475024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:11:35.773440+0000","last_change":"2024-09-11T01:11:35.773194+0000","last_active":"2024-09-11T01:11:35.773440+0000","last_peered":"2024-09-11T01:11:35.773440+0000","last_clean":"2024-09-11T01:11:35.773440+0000","last_became_active":"2024-09-11T01:11:35.772460+0000","last_became_peered":"2024-09-11T01:11:35.772460+0000","last_unstale":"2024-09-11T01:11:35.773440+0000","last_undegraded":"2024-09-11T01:11:35.773440+0000","last_fullsized":"2024-09-11T01:11:35.773440+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:20:07.064167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:11:35.758324+0000","last_change":"2024-09-11T01:11:31.735839+0000","last_active":"2024-09-11T01:11:35.758324+0000","last_peered":"2024-09-11T01:11:35.758324+0000","last_clean":"2024-09-11T01:11:35.758324+0000","last_became_active":"2024-09-11T01:11:31.735451+0000","last_became_peered":"2024-09-11T01:11:31.735451+0000","last_unstale":"2024-09-11T01:11:35.758324+0000","last_undegraded":"2024-09-11T01:11:35.758324+0000","last_fullsized":"2024-09-11T01:11:35.758324+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:01:57.389515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":34,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:11:35.758248+0000","last_change":"2024-09-11T01:11:22.672920+0000","last_active":"2024-09-11T01:11:35.758248+0000","last_peered":"2024-09-11T01:11:35.758248+0000","last_clean":"2024-09-11T01:11:35.758248+0000","last_became_active":"2024-09-11T01:11:20.662201+0000","last_became_peered":"2024-09-11T01:11:20.662201+0000","last_unstale":"2024-09-11T01:11:35.758248+0000","last_undegraded":"2024-09-11T01:11:35.758248+0000","last_fullsized":"2024-09-11T01:11:35.758248+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:37.382862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":39,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:11:35.769707+0000","last_change":"2024-09-11T01:11:35.769401+0000","last_active":"2024-09-11T01:11:35.769707+0000","last_peered":"2024-09-11T01:11:35.769707+0000","last_clean":"2024-09-11T01:11:35.769707+0000","last_became_active":"2024-09-11T01:11:35.768646+0000","last_became_peered":"2024-09-11T01:11:35.768646+0000","last_unstale":"2024-09-11T01:11:35.769707+0000","last_undegraded":"2024-09-11T01:11:35.769707+0000","last_fullsized":"2024-09-11T01:11:35.769707+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:06:09.019950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"20'1","reported_seq":20,"reported_epoch":21,"state":"active+clean","last_fresh":"2024-09-11T01:11:35.757043+0000","last_change":"2024-09-11T01:11:31.738742+0000","last_active":"2024-09-11T01:11:35.757043+0000","last_peered":"2024-09-11T01:11:35.757043+0000","last_clean":"2024-09-11T01:11:35.757043+0000","last_became_active":"2024-09-11T01:11:31.738088+0000","last_became_peered":"2024-09-11T01:11:31.738088+0000","last_unstale":"2024-09-11T01:11:35.757043+0000","last_undegraded":"2024-09-11T01:11:35.757043+0000","last_fullsized":"2024-09-11T01:11:35.757043+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:50:04.038198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":9,"seq":38654705673,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":188416,"data_stored":49934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738378,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27208,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344632,"statfs":{"total":96636764160,"available":96608903168,"internally_reserved":0,"allocated":188416,"data_stored":49934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:11:42.276 INFO:tasks.rep_lost_unfound_delete.ceph_manager:recovered! 2024-09-11T01:11:42.276 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_1 /etc/fstab 2024-09-11T01:11:42.276 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_1 /etc/fstab 2024-09-11T01:11:42.319 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_1 /etc/fstab 2024-09-11T01:11:42.319 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_1 /etc/fstab 2024-09-11T01:11:42.401 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_1 2024-09-11T01:11:42.401 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_1 2024-09-11T01:11:42.487 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_2 /etc/fstab 2024-09-11T01:11:42.487 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_2 /etc/fstab 2024-09-11T01:11:42.571 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_2 /etc/fstab 2024-09-11T01:11:42.571 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_2 /etc/fstab 2024-09-11T01:11:42.659 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_2 2024-09-11T01:11:42.659 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_2 2024-09-11T01:11:42.696 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_3 /etc/fstab 2024-09-11T01:11:42.696 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_3 /etc/fstab 2024-09-11T01:11:42.775 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_3 /etc/fstab 2024-09-11T01:11:42.776 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_3 /etc/fstab 2024-09-11T01:11:42.856 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_3 2024-09-11T01:11:42.856 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_3 2024-09-11T01:11:42.939 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_4 /etc/fstab 2024-09-11T01:11:42.939 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_4 /etc/fstab 2024-09-11T01:11:43.020 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_4 /etc/fstab 2024-09-11T01:11:43.020 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_4 /etc/fstab 2024-09-11T01:11:43.108 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_4 2024-09-11T01:11:43.108 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_4 2024-09-11T01:11:43.191 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_5 /etc/fstab 2024-09-11T01:11:43.191 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_5 /etc/fstab 2024-09-11T01:11:43.276 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_5 /etc/fstab 2024-09-11T01:11:43.276 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_5 /etc/fstab 2024-09-11T01:11:43.360 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_5 2024-09-11T01:11:43.361 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_5 2024-09-11T01:11:43.440 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_6 /etc/fstab 2024-09-11T01:11:43.440 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_6 /etc/fstab 2024-09-11T01:11:43.523 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_6 /etc/fstab 2024-09-11T01:11:43.523 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_6 /etc/fstab 2024-09-11T01:11:43.608 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_6 2024-09-11T01:11:43.608 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_6 2024-09-11T01:11:43.691 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_7 /etc/fstab 2024-09-11T01:11:43.691 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_7 /etc/fstab 2024-09-11T01:11:43.772 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_7 /etc/fstab 2024-09-11T01:11:43.772 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_7 /etc/fstab 2024-09-11T01:11:43.850 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_7 2024-09-11T01:11:43.850 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_7 2024-09-11T01:11:43.937 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_8 /etc/fstab 2024-09-11T01:11:43.937 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_8 /etc/fstab 2024-09-11T01:11:44.017 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_8 /etc/fstab 2024-09-11T01:11:44.017 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_8 /etc/fstab 2024-09-11T01:11:44.095 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_8 2024-09-11T01:11:44.095 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_8 2024-09-11T01:11:44.178 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_9 /etc/fstab 2024-09-11T01:11:44.179 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_9 /etc/fstab 2024-09-11T01:11:44.267 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_9 /etc/fstab 2024-09-11T01:11:44.268 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_9 /etc/fstab 2024-09-11T01:11:44.303 INFO:tasks.util.rados:rados -p unfounddel_pool rm existed_9 2024-09-11T01:11:44.304 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool rm existed_9 2024-09-11T01:11:44.382 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 injectargs '--osd-recovery-delay-start 1000 --osd-min-pg-log-entries 100000000' 2024-09-11T01:11:44.556 INFO:teuthology.orchestra.run.smithi122.stdout:{} 2024-09-11T01:11:44.556 INFO:teuthology.orchestra.run.smithi122.stderr:osd_min_pg_log_entries = '' (not observed, change may require restart) osd_recovery_delay_start = '' 2024-09-11T01:11:44.570 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-11T01:11:44.570 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T01:11:44.646 INFO:tasks.ceph.osd.0:Stopped 2024-09-11T01:11:44.646 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 0 2024-09-11T01:11:44.928 INFO:tasks.ceph.mon.a.smithi122.stderr:2024-09-11T01:11:44.924+0000 7f9dc4120640 -1 mon.a@0(leader).osd e21 definitely_dead 0 2024-09-11T01:11:45.602 INFO:teuthology.orchestra.run.smithi122.stderr:marked down osd.0. 2024-09-11T01:11:45.624 INFO:tasks.util.rados:rados -p unfounddel_pool put new_1 /etc/fstab 2024-09-11T01:11:45.624 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_1 /etc/fstab 2024-09-11T01:11:46.628 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_1 /etc/fstab 2024-09-11T01:11:46.628 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_1 /etc/fstab 2024-09-11T01:11:46.673 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_1 /etc/fstab 2024-09-11T01:11:46.674 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_1 /etc/fstab 2024-09-11T01:11:46.759 INFO:tasks.util.rados:rados -p unfounddel_pool put new_2 /etc/fstab 2024-09-11T01:11:46.760 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_2 /etc/fstab 2024-09-11T01:11:46.847 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_2 /etc/fstab 2024-09-11T01:11:46.847 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_2 /etc/fstab 2024-09-11T01:11:46.938 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_2 /etc/fstab 2024-09-11T01:11:46.938 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_2 /etc/fstab 2024-09-11T01:11:47.022 INFO:tasks.util.rados:rados -p unfounddel_pool put new_3 /etc/fstab 2024-09-11T01:11:47.022 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_3 /etc/fstab 2024-09-11T01:11:47.110 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_3 /etc/fstab 2024-09-11T01:11:47.110 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_3 /etc/fstab 2024-09-11T01:11:47.197 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_3 /etc/fstab 2024-09-11T01:11:47.198 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_3 /etc/fstab 2024-09-11T01:11:47.282 INFO:tasks.util.rados:rados -p unfounddel_pool put new_4 /etc/fstab 2024-09-11T01:11:47.282 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_4 /etc/fstab 2024-09-11T01:11:47.372 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_4 /etc/fstab 2024-09-11T01:11:47.372 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_4 /etc/fstab 2024-09-11T01:11:47.410 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_4 /etc/fstab 2024-09-11T01:11:47.410 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_4 /etc/fstab 2024-09-11T01:11:47.496 INFO:tasks.util.rados:rados -p unfounddel_pool put new_5 /etc/fstab 2024-09-11T01:11:47.496 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_5 /etc/fstab 2024-09-11T01:11:47.581 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_5 /etc/fstab 2024-09-11T01:11:47.581 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_5 /etc/fstab 2024-09-11T01:11:47.670 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_5 /etc/fstab 2024-09-11T01:11:47.670 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_5 /etc/fstab 2024-09-11T01:11:47.708 INFO:tasks.util.rados:rados -p unfounddel_pool put new_6 /etc/fstab 2024-09-11T01:11:47.708 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_6 /etc/fstab 2024-09-11T01:11:47.792 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_6 /etc/fstab 2024-09-11T01:11:47.792 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_6 /etc/fstab 2024-09-11T01:11:47.874 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_6 /etc/fstab 2024-09-11T01:11:47.874 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_6 /etc/fstab 2024-09-11T01:11:47.962 INFO:tasks.util.rados:rados -p unfounddel_pool put new_7 /etc/fstab 2024-09-11T01:11:47.963 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_7 /etc/fstab 2024-09-11T01:11:48.000 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_7 /etc/fstab 2024-09-11T01:11:48.000 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_7 /etc/fstab 2024-09-11T01:11:48.079 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_7 /etc/fstab 2024-09-11T01:11:48.079 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_7 /etc/fstab 2024-09-11T01:11:48.168 INFO:tasks.util.rados:rados -p unfounddel_pool put new_8 /etc/fstab 2024-09-11T01:11:48.168 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_8 /etc/fstab 2024-09-11T01:11:48.255 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_8 /etc/fstab 2024-09-11T01:11:48.255 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_8 /etc/fstab 2024-09-11T01:11:48.345 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_8 /etc/fstab 2024-09-11T01:11:48.345 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_8 /etc/fstab 2024-09-11T01:11:48.430 INFO:tasks.util.rados:rados -p unfounddel_pool put new_9 /etc/fstab 2024-09-11T01:11:48.431 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put new_9 /etc/fstab 2024-09-11T01:11:48.519 INFO:tasks.util.rados:rados -p unfounddel_pool put existed_9 /etc/fstab 2024-09-11T01:11:48.519 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existed_9 /etc/fstab 2024-09-11T01:11:48.557 INFO:tasks.util.rados:rados -p unfounddel_pool put existing_9 /etc/fstab 2024-09-11T01:11:48.557 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool put existing_9 /etc/fstab 2024-09-11T01:11:48.638 INFO:tasks.rep_lost_unfound_delete:osd.0 command_args is foo 2024-09-11T01:11:48.638 INFO:tasks.rep_lost_unfound_delete:() 2024-09-11T01:11:48.638 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-11T01:11:48.639 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 --osd-recovery-delay-start 1000 2024-09-11T01:11:48.683 INFO:tasks.ceph.osd.0:Started 2024-09-11T01:11:48.683 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T01:11:48.799 INFO:teuthology.orchestra.run.smithi122.stderr:no valid command found; 10 closest matches: 2024-09-11T01:11:48.800 INFO:teuthology.orchestra.run.smithi122.stderr:0 2024-09-11T01:11:48.800 INFO:teuthology.orchestra.run.smithi122.stderr:1 2024-09-11T01:11:48.800 INFO:teuthology.orchestra.run.smithi122.stderr:2 2024-09-11T01:11:48.800 INFO:teuthology.orchestra.run.smithi122.stderr:abort 2024-09-11T01:11:48.800 INFO:teuthology.orchestra.run.smithi122.stderr:assert 2024-09-11T01:11:48.800 INFO:teuthology.orchestra.run.smithi122.stderr:config diff 2024-09-11T01:11:48.800 INFO:teuthology.orchestra.run.smithi122.stderr:config diff get 2024-09-11T01:11:48.800 INFO:teuthology.orchestra.run.smithi122.stderr:config get 2024-09-11T01:11:48.800 INFO:teuthology.orchestra.run.smithi122.stderr:config help [] 2024-09-11T01:11:48.800 INFO:teuthology.orchestra.run.smithi122.stderr:config set ... 2024-09-11T01:11:48.800 INFO:teuthology.orchestra.run.smithi122.stderr:admin_socket: invalid command 2024-09-11T01:11:48.802 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-11T01:11:48.803 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-0, ['dump_ops_in_flight'] 2024-09-11T01:11:49.736 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:11:49.732+0000 7f702d8c07c0 -1 Falling back to public interface 2024-09-11T01:11:53.637 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:11:53.632+0000 7f702d8c07c0 -1 osd.0 21 log_to_monitors true 2024-09-11T01:11:53.681 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:11:53.676+0000 7f70256a3640 -1 osd.0 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T01:11:53.803 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-09-11T01:11:53.935 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-11T01:11:53.935 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-11T01:11:53.935 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-11T01:11:53.935 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-11T01:11:53.949 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 2024-09-11T01:11:54.258 INFO:tasks.ceph.mon.a.smithi122.stderr:2024-09-11T01:11:54.256+0000 7f9dc4120640 -1 mon.a@0(leader).osd e24 definitely_dead 0 2024-09-11T01:11:54.258 INFO:teuthology.orchestra.run.smithi122.stderr:osd.0 is already in. 2024-09-11T01:11:54.276 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for osd.0 to be up 2024-09-11T01:11:54.276 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T01:11:54.564 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:11:54.564 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":24,"fsid":"6d328773-3914-4412-91b7-393379984de3","created":"2024-09-11T01:11:03.508628+0000","modified":"2024-09-11T01:11:53.662522+0000","last_up_change":"2024-09-11T01:11:45.582549+0000","last_in_change":"2024-09-11T01:11:34.740626+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-11T01:11:18.999872+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":3.0299999713897705,"score_stable":3.0299999713897705,"optimal_score":0.33000001311302185,"raw_score_acting":1,"raw_score_stable":1,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfounddel_pool","create_time":"2024-09-11T01:11:30.626364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":3.0299999713897705,"score_stable":3.0299999713897705,"optimal_score":0.33000001311302185,"raw_score_acting":1,"raw_score_stable":1,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e160603b-65bb-420f-9130-6d7c6506f2a5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":20,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6808","nonce":3528172777}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6809","nonce":3528172777}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6811","nonce":3528172777}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6810","nonce":3528172777}]},"public_addr":"172.21.15.122:6808/3528172777","cluster_addr":"172.21.15.122:6809/3528172777","heartbeat_back_addr":"172.21.15.122:6811/3528172777","heartbeat_front_addr":"172.21.15.122:6810/3528172777","state":["exists"]},{"osd":1,"uuid":"6fbfe9e1-4255-4c44-b619-5fd57486e5fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6804","nonce":2638428424}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6805","nonce":2638428424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6807","nonce":2638428424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6806","nonce":2638428424}]},"public_addr":"172.21.15.122:6804/2638428424","cluster_addr":"172.21.15.122:6805/2638428424","heartbeat_back_addr":"172.21.15.122:6807/2638428424","heartbeat_front_addr":"172.21.15.122:6806/2638428424","state":["exists","up"]},{"osd":2,"uuid":"e27771fd-ac9d-4a64-a01f-465a5acade91","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":16,"down_at":19,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6800","nonce":2913015415}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6801","nonce":2913015415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6803","nonce":2913015415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6802","nonce":2913015415}]},"public_addr":"172.21.15.122:6800/2913015415","cluster_addr":"172.21.15.122:6801/2913015415","heartbeat_back_addr":"172.21.15.122:6803/2913015415","heartbeat_front_addr":"172.21.15.122:6802/2913015415","state":["exists"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:11:12.059865+0000","dead_epoch":22},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:11:11.519118+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":65536,"last_purged_snaps_scrub":"2024-09-11T01:11:11.402743+0000","dead_epoch":19}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T01:11:57.582 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T01:11:57.893 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:11:57.893 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":26,"fsid":"6d328773-3914-4412-91b7-393379984de3","created":"2024-09-11T01:11:03.508628+0000","modified":"2024-09-11T01:11:55.696483+0000","last_up_change":"2024-09-11T01:11:54.674465+0000","last_in_change":"2024-09-11T01:11:34.740626+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-11T01:11:18.999872+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8700000047683716,"score_stable":1.8700000047683716,"optimal_score":0.67000001668930054,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfounddel_pool","create_time":"2024-09-11T01:11:30.626364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8700000047683716,"score_stable":1.8700000047683716,"optimal_score":0.67000001668930054,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e160603b-65bb-420f-9130-6d7c6506f2a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":21,"up_from":25,"up_thru":25,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6800","nonce":2613596984}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6801","nonce":2613596984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6803","nonce":2613596984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6802","nonce":2613596984}]},"public_addr":"172.21.15.122:6800/2613596984","cluster_addr":"172.21.15.122:6801/2613596984","heartbeat_back_addr":"172.21.15.122:6803/2613596984","heartbeat_front_addr":"172.21.15.122:6802/2613596984","state":["exists","up"]},{"osd":1,"uuid":"6fbfe9e1-4255-4c44-b619-5fd57486e5fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6804","nonce":2638428424}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6805","nonce":2638428424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6807","nonce":2638428424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6806","nonce":2638428424}]},"public_addr":"172.21.15.122:6804/2638428424","cluster_addr":"172.21.15.122:6805/2638428424","heartbeat_back_addr":"172.21.15.122:6807/2638428424","heartbeat_front_addr":"172.21.15.122:6806/2638428424","state":["exists","up"]},{"osd":2,"uuid":"e27771fd-ac9d-4a64-a01f-465a5acade91","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":16,"down_at":19,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6800","nonce":2913015415}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6801","nonce":2913015415}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6803","nonce":2913015415}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6802","nonce":2913015415}]},"public_addr":"172.21.15.122:6800/2913015415","cluster_addr":"172.21.15.122:6801/2913015415","heartbeat_back_addr":"172.21.15.122:6803/2913015415","heartbeat_front_addr":"172.21.15.122:6802/2913015415","state":["exists"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:11:12.059865+0000","dead_epoch":22},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:11:11.519118+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":65536,"last_purged_snaps_scrub":"2024-09-11T01:11:11.402743+0000","dead_epoch":19}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T01:11:57.911 INFO:tasks.rep_lost_unfound_delete.ceph_manager:osd.0 is up 2024-09-11T01:11:57.911 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-11T01:11:57.912 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-11T01:11:58.064 INFO:teuthology.orchestra.run.smithi122.stdout:107374182402 2024-09-11T01:11:58.064 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T01:11:58.070 INFO:teuthology.orchestra.run.smithi122.stdout:34359738382 2024-09-11T01:11:58.070 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-11T01:11:58.336 INFO:teuthology.orchestra.run.smithi122.stdout:1 2024-09-11T01:11:58.353 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 107374182402 got 1 for osd.0 2024-09-11T01:11:58.409 INFO:teuthology.orchestra.run.smithi122.stdout:34359738381 2024-09-11T01:11:58.426 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738382 got 34359738381 for osd.1 2024-09-11T01:11:59.354 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T01:11:59.427 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-11T01:11:59.668 INFO:teuthology.orchestra.run.smithi122.stdout:107374182402 2024-09-11T01:11:59.686 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 107374182402 got 107374182402 for osd.0 2024-09-11T01:11:59.686 DEBUG:teuthology.parallel:result is None 2024-09-11T01:11:59.719 INFO:teuthology.orchestra.run.smithi122.stdout:34359738382 2024-09-11T01:11:59.736 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 34359738382 got 34359738382 for osd.1 2024-09-11T01:11:59.736 DEBUG:teuthology.parallel:result is None 2024-09-11T01:11:59.736 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting till active 2024-09-11T01:11:59.737 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:12:00.069 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:12:00.070 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:12:00.086 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":47,"stamp":"2024-09-11T01:11:58.531871+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":17295,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":27,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58,"ondisk_log_size":58,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55088,"kb_used_data":592,"kb_used_omap":5,"kb_used_meta":54458,"kb_avail":188688592,"statfs":{"total":193273528320,"available":193217118208,"internally_reserved":0,"allocated":606208,"data_stored":153048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6032,"internal_metadata":55765104},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":-3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001971"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":22,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-11T01:11:55.725865+0000","last_change":"2024-09-11T01:11:55.723939+0000","last_active":"2024-09-11T01:11:55.725865+0000","last_peered":"2024-09-11T01:11:55.725865+0000","last_clean":"2024-09-11T01:11:55.725865+0000","last_became_active":"2024-09-11T01:11:55.721149+0000","last_became_peered":"2024-09-11T01:11:55.721149+0000","last_unstale":"2024-09-11T01:11:55.725865+0000","last_undegraded":"2024-09-11T01:11:55.725865+0000","last_fullsized":"2024-09-11T01:11:55.725865+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:44:49.966680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"23'4","reported_seq":24,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:11:55.721830+0000","last_change":"2024-09-11T01:11:55.722447+0000","last_active":"2024-09-11T01:11:55.721830+0000","last_peered":"2024-09-11T01:11:55.721830+0000","last_clean":"2024-09-11T01:11:43.020052+0000","last_became_active":"2024-09-11T01:11:55.721566+0000","last_became_peered":"2024-09-11T01:11:55.721566+0000","last_unstale":"2024-09-11T01:11:55.721830+0000","last_undegraded":"2024-09-11T01:11:55.704460+0000","last_fullsized":"2024-09-11T01:11:55.721830+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"23'3","reported_seq":44,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:11:55.733836+0000","last_change":"2024-09-11T01:11:55.735473+0000","last_active":"2024-09-11T01:11:55.733836+0000","last_peered":"2024-09-11T01:11:55.733836+0000","last_clean":"2024-09-11T01:11:43.936435+0000","last_became_active":"2024-09-11T01:11:55.733606+0000","last_became_peered":"2024-09-11T01:11:55.733606+0000","last_unstale":"2024-09-11T01:11:55.733836+0000","last_undegraded":"2024-09-11T01:11:55.709377+0000","last_fullsized":"2024-09-11T01:11:55.733836+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-11T01:11:55.726136+0000","last_change":"2024-09-11T01:11:55.723776+0000","last_active":"2024-09-11T01:11:55.726136+0000","last_peered":"2024-09-11T01:11:55.726136+0000","last_clean":"2024-09-11T01:11:55.726136+0000","last_became_active":"2024-09-11T01:11:55.721708+0000","last_became_peered":"2024-09-11T01:11:55.721708+0000","last_unstale":"2024-09-11T01:11:55.726136+0000","last_undegraded":"2024-09-11T01:11:55.726136+0000","last_fullsized":"2024-09-11T01:11:55.726136+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:39:51.004747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"23'5","reported_seq":36,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:11:55.722481+0000","last_change":"2024-09-11T01:11:55.723050+0000","last_active":"2024-09-11T01:11:55.722481+0000","last_peered":"2024-09-11T01:11:55.722481+0000","last_clean":"2024-09-11T01:11:42.855496+0000","last_became_active":"2024-09-11T01:11:55.722105+0000","last_became_peered":"2024-09-11T01:11:55.722105+0000","last_unstale":"2024-09-11T01:11:55.722481+0000","last_undegraded":"2024-09-11T01:11:55.706044+0000","last_fullsized":"2024-09-11T01:11:55.722481+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"23'3","reported_seq":39,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:11:55.733847+0000","last_change":"2024-09-11T01:11:55.734783+0000","last_active":"2024-09-11T01:11:55.733847+0000","last_peered":"2024-09-11T01:11:55.733847+0000","last_clean":"2024-09-11T01:11:44.266820+0000","last_became_active":"2024-09-11T01:11:55.733724+0000","last_became_peered":"2024-09-11T01:11:55.733724+0000","last_unstale":"2024-09-11T01:11:55.733847+0000","last_undegraded":"2024-09-11T01:11:55.708063+0000","last_fullsized":"2024-09-11T01:11:55.733847+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"23'4","reported_seq":40,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:11:55.728704+0000","last_change":"2024-09-11T01:11:55.729133+0000","last_active":"2024-09-11T01:11:55.728704+0000","last_peered":"2024-09-11T01:11:55.728704+0000","last_clean":"2024-09-11T01:11:43.275607+0000","last_became_active":"2024-09-11T01:11:55.728475+0000","last_became_peered":"2024-09-11T01:11:55.728475+0000","last_unstale":"2024-09-11T01:11:55.728704+0000","last_undegraded":"2024-09-11T01:11:55.706009+0000","last_fullsized":"2024-09-11T01:11:55.728704+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"23'1","reported_seq":34,"reported_epoch":26,"state":"active+recovering+degraded","last_fresh":"2024-09-11T01:11:55.722151+0000","last_change":"2024-09-11T01:11:55.738157+0000","last_active":"2024-09-11T01:11:55.722151+0000","last_peered":"2024-09-11T01:11:55.722151+0000","last_clean":"2024-09-11T01:11:32.735868+0000","last_became_active":"2024-09-11T01:11:55.721435+0000","last_became_peered":"2024-09-11T01:11:55.721435+0000","last_unstale":"2024-09-11T01:11:55.722151+0000","last_undegraded":"2024-09-11T01:11:55.704513+0000","last_fullsized":"2024-09-11T01:11:55.722151+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":53,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-11T01:11:55.729304+0000","last_change":"2024-09-11T01:11:55.729063+0000","last_active":"2024-09-11T01:11:55.729304+0000","last_peered":"2024-09-11T01:11:55.729304+0000","last_clean":"2024-09-11T01:11:55.729304+0000","last_became_active":"2024-09-11T01:11:55.728281+0000","last_became_peered":"2024-09-11T01:11:55.728281+0000","last_unstale":"2024-09-11T01:11:55.729304+0000","last_undegraded":"2024-09-11T01:11:55.729304+0000","last_fullsized":"2024-09-11T01:11:55.729304+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:06:55.845313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"23'5","reported_seq":46,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:11:55.727550+0000","last_change":"2024-09-11T01:11:55.728050+0000","last_active":"2024-09-11T01:11:55.727550+0000","last_peered":"2024-09-11T01:11:55.727550+0000","last_clean":"2024-09-11T01:11:44.016870+0000","last_became_active":"2024-09-11T01:11:55.727293+0000","last_became_peered":"2024-09-11T01:11:55.727293+0000","last_unstale":"2024-09-11T01:11:55.727550+0000","last_undegraded":"2024-09-11T01:11:55.706230+0000","last_fullsized":"2024-09-11T01:11:55.727550+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-11T01:11:55.720819+0000","last_change":"2024-09-11T01:11:55.720550+0000","last_active":"2024-09-11T01:11:55.720819+0000","last_peered":"2024-09-11T01:11:55.720819+0000","last_clean":"2024-09-11T01:11:55.720819+0000","last_became_active":"2024-09-11T01:11:55.719706+0000","last_became_peered":"2024-09-11T01:11:55.719706+0000","last_unstale":"2024-09-11T01:11:55.720819+0000","last_undegraded":"2024-09-11T01:11:55.720819+0000","last_fullsized":"2024-09-11T01:11:55.720819+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:02:25.613039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'6","reported_seq":32,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:11:55.725362+0000","last_change":"2024-09-11T01:11:55.730005+0000","last_active":"2024-09-11T01:11:55.725362+0000","last_peered":"2024-09-11T01:11:55.725362+0000","last_clean":"2024-09-11T01:11:44.303167+0000","last_became_active":"2024-09-11T01:11:55.725112+0000","last_became_peered":"2024-09-11T01:11:55.725112+0000","last_unstale":"2024-09-11T01:11:55.725362+0000","last_undegraded":"2024-09-11T01:11:55.704432+0000","last_fullsized":"2024-09-11T01:11:55.725362+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":38,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-11T01:11:55.730591+0000","last_change":"2024-09-11T01:11:55.730254+0000","last_active":"2024-09-11T01:11:55.730591+0000","last_peered":"2024-09-11T01:11:55.730591+0000","last_clean":"2024-09-11T01:11:55.730591+0000","last_became_active":"2024-09-11T01:11:55.722467+0000","last_became_peered":"2024-09-11T01:11:55.722467+0000","last_unstale":"2024-09-11T01:11:55.730591+0000","last_undegraded":"2024-09-11T01:11:55.730591+0000","last_fullsized":"2024-09-11T01:11:55.730591+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:35:15.417814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'3","reported_seq":33,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:11:55.725871+0000","last_change":"2024-09-11T01:11:55.727762+0000","last_active":"2024-09-11T01:11:55.725871+0000","last_peered":"2024-09-11T01:11:55.725871+0000","last_clean":"2024-09-11T01:11:42.775010+0000","last_became_active":"2024-09-11T01:11:55.725663+0000","last_became_peered":"2024-09-11T01:11:55.725663+0000","last_unstale":"2024-09-11T01:11:55.725871+0000","last_undegraded":"2024-09-11T01:11:55.704850+0000","last_fullsized":"2024-09-11T01:11:55.725871+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":63,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-11T01:11:55.726692+0000","last_change":"2024-09-11T01:11:55.726423+0000","last_active":"2024-09-11T01:11:55.726692+0000","last_peered":"2024-09-11T01:11:55.726692+0000","last_clean":"2024-09-11T01:11:55.726692+0000","last_became_active":"2024-09-11T01:11:55.725618+0000","last_became_peered":"2024-09-11T01:11:55.725618+0000","last_unstale":"2024-09-11T01:11:55.726692+0000","last_undegraded":"2024-09-11T01:11:55.726692+0000","last_fullsized":"2024-09-11T01:11:55.726692+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:10:49.707688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"23'4","reported_seq":35,"reported_epoch":26,"state":"active+recovering+degraded","last_fresh":"2024-09-11T01:11:55.724451+0000","last_change":"2024-09-11T01:11:55.739032+0000","last_active":"2024-09-11T01:11:55.724451+0000","last_peered":"2024-09-11T01:11:55.724451+0000","last_clean":"2024-09-11T01:11:43.440047+0000","last_became_active":"2024-09-11T01:11:55.723698+0000","last_became_peered":"2024-09-11T01:11:55.723698+0000","last_unstale":"2024-09-11T01:11:55.724451+0000","last_undegraded":"2024-09-11T01:11:55.703198+0000","last_fullsized":"2024-09-11T01:11:55.724451+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"23'1","reported_seq":36,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:11:55.733927+0000","last_change":"2024-09-11T01:11:55.734512+0000","last_active":"2024-09-11T01:11:55.733927+0000","last_peered":"2024-09-11T01:11:55.733927+0000","last_clean":"2024-09-11T01:11:35.758866+0000","last_became_active":"2024-09-11T01:11:55.733697+0000","last_became_peered":"2024-09-11T01:11:55.733697+0000","last_unstale":"2024-09-11T01:11:55.733927+0000","last_undegraded":"2024-09-11T01:11:55.708164+0000","last_fullsized":"2024-09-11T01:11:55.733927+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":1}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":53,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-11T01:11:55.730610+0000","last_change":"2024-09-11T01:11:55.730298+0000","last_active":"2024-09-11T01:11:55.730610+0000","last_peered":"2024-09-11T01:11:55.730610+0000","last_clean":"2024-09-11T01:11:55.730610+0000","last_became_active":"2024-09-11T01:11:55.729717+0000","last_became_peered":"2024-09-11T01:11:55.729717+0000","last_unstale":"2024-09-11T01:11:55.730610+0000","last_undegraded":"2024-09-11T01:11:55.730610+0000","last_fullsized":"2024-09-11T01:11:55.730610+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:59:10.682114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"23'5","reported_seq":41,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:11:55.725945+0000","last_change":"2024-09-11T01:11:55.726397+0000","last_active":"2024-09-11T01:11:55.725945+0000","last_peered":"2024-09-11T01:11:55.725945+0000","last_clean":"2024-09-11T01:11:44.178063+0000","last_became_active":"2024-09-11T01:11:55.725693+0000","last_became_peered":"2024-09-11T01:11:55.725693+0000","last_unstale":"2024-09-11T01:11:55.725945+0000","last_undegraded":"2024-09-11T01:11:55.707090+0000","last_fullsized":"2024-09-11T01:11:55.725945+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":3}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":25,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-11T01:11:55.723491+0000","last_change":"2024-09-11T01:11:55.723075+0000","last_active":"2024-09-11T01:11:55.723491+0000","last_peered":"2024-09-11T01:11:55.723491+0000","last_clean":"2024-09-11T01:11:55.723491+0000","last_became_active":"2024-09-11T01:11:55.722028+0000","last_became_peered":"2024-09-11T01:11:55.722028+0000","last_unstale":"2024-09-11T01:11:55.723491+0000","last_undegraded":"2024-09-11T01:11:55.723491+0000","last_fullsized":"2024-09-11T01:11:55.723491+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:19:38.022542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"23'2","reported_seq":37,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:11:55.734830+0000","last_change":"2024-09-11T01:11:55.735459+0000","last_active":"2024-09-11T01:11:55.734830+0000","last_peered":"2024-09-11T01:11:55.734830+0000","last_clean":"2024-09-11T01:11:35.758438+0000","last_became_active":"2024-09-11T01:11:55.734597+0000","last_became_peered":"2024-09-11T01:11:55.734597+0000","last_unstale":"2024-09-11T01:11:55.734830+0000","last_undegraded":"2024-09-11T01:11:55.709407+0000","last_fullsized":"2024-09-11T01:11:55.734830+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":2}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":53,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-11T01:11:55.727825+0000","last_change":"2024-09-11T01:11:55.727553+0000","last_active":"2024-09-11T01:11:55.727825+0000","last_peered":"2024-09-11T01:11:55.727825+0000","last_clean":"2024-09-11T01:11:55.727825+0000","last_became_active":"2024-09-11T01:11:55.726797+0000","last_became_peered":"2024-09-11T01:11:55.726797+0000","last_unstale":"2024-09-11T01:11:55.727825+0000","last_undegraded":"2024-09-11T01:11:55.727825+0000","last_fullsized":"2024-09-11T01:11:55.727825+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:39:21.610229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":58,"reported_epoch":26,"state":"active+clean","last_fresh":"2024-09-11T01:11:55.734615+0000","last_change":"2024-09-11T01:11:55.734408+0000","last_active":"2024-09-11T01:11:55.734615+0000","last_peered":"2024-09-11T01:11:55.734615+0000","last_clean":"2024-09-11T01:11:55.734615+0000","last_became_active":"2024-09-11T01:11:55.733699+0000","last_became_peered":"2024-09-11T01:11:55.733699+0000","last_unstale":"2024-09-11T01:11:55.734615+0000","last_undegraded":"2024-09-11T01:11:55.734615+0000","last_fullsized":"2024-09-11T01:11:55.734615+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:52:53.222366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"23'9","reported_seq":45,"reported_epoch":26,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:11:55.726728+0000","last_change":"2024-09-11T01:11:55.727137+0000","last_active":"2024-09-11T01:11:55.726728+0000","last_peered":"2024-09-11T01:11:55.726728+0000","last_clean":"2024-09-11T01:11:43.771457+0000","last_became_active":"2024-09-11T01:11:55.726503+0000","last_became_peered":"2024-09-11T01:11:55.726503+0000","last_unstale":"2024-09-11T01:11:55.726728+0000","last_undegraded":"2024-09-11T01:11:55.707356+0000","last_fullsized":"2024-09-11T01:11:55.726728+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2468,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1"],"object_location_counts":[{"shards":"0,1","objects":1},{"shards":"1","objects":3}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":17276,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":7,"num_objects_missing":7,"num_objects_degraded":27,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":23446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":32,"acting":32,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182402,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27732,"kb_used_data":260,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344108,"statfs":{"total":96636764160,"available":96608366592,"internally_reserved":0,"allocated":266240,"data_stored":70971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":28110507},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738382,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27356,"kb_used_data":332,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344484,"statfs":{"total":96636764160,"available":96608751616,"internally_reserved":0,"allocated":339968,"data_stored":82077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":17276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:12:00.087 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:12:00.422 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:12:00.422 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 27/60 objects degraded (45.000%), 14 pgs degraded","count":14},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":56,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":26,"num_osds":3,"num_up_osds":2,"osd_up_since":1726017114,"num_in_osds":2,"osd_in_since":1726017094,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+recovery_wait+degraded","count":12},{"state_name":"active+clean","count":10},{"state_name":"active+recovering+degraded","count":2}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":17295,"bytes_used":56410112,"bytes_avail":193217118208,"bytes_total":193273528320,"degraded_objects":27,"degraded_total":60,"degraded_ratio":0.45000000000000001,"write_bytes_sec":170,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2024-09-11T01:11:03.510424+0000","services":{}},"progress_events":{}} 2024-09-11T01:12:00.438 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'fsid': '6d328773-3914-4412-91b7-393379984de3', 'health': {'status': 'HEALTH_WARN', 'checks': {'PG_DEGRADED': {'severity': 'HEALTH_WARN', 'summary': {'message': 'Degraded data redundancy: 27/60 objects degraded (45.000%), 14 pgs degraded', 'count': 14}, 'muted': False}}, 'mutes': []}, 'election_epoch': 8, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 56, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 26, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1726017114, 'num_in_osds': 2, 'osd_in_since': 1726017094, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'active+recovery_wait+degraded', 'count': 12}, {'state_name': 'active+clean', 'count': 10}, {'state_name': 'active+recovering+degraded', 'count': 2}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 17295, 'bytes_used': 56410112, 'bytes_avail': 193217118208, 'bytes_total': 193273528320, 'degraded_objects': 27, 'degraded_total': 60, 'degraded_ratio': 0.45, 'write_bytes_sec': 170, 'read_op_per_sec': 0, 'write_op_per_sec': 0}, 'fsmap': {'epoch': 1, 'btime': '2024-09-11T01:11:03:507954+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 1, 'modified': '2024-09-11T01:11:03.510424+0000', 'services': {}}, 'progress_events': {}} 2024-09-11T01:12:00.438 INFO:tasks.rep_lost_unfound_delete.ceph_manager:active! 2024-09-11T01:12:00.438 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-11T01:12:00.438 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T01:12:00.507 INFO:tasks.ceph.osd.1:Stopped 2024-09-11T01:12:00.507 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd down 1 2024-09-11T01:12:00.822 INFO:tasks.ceph.mon.a.smithi122.stderr:2024-09-11T01:12:00.824+0000 7f9dc4120640 -1 mon.a@0(leader).osd e27 definitely_dead 0 2024-09-11T01:12:00.822 INFO:teuthology.orchestra.run.smithi122.stderr:osd.1 is already down. 2024-09-11T01:12:00.841 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 1 2024-09-11T01:12:01.148 INFO:tasks.ceph.mon.a.smithi122.stderr:2024-09-11T01:12:01.148+0000 7f9dc4120640 -1 mon.a@0(leader).osd e27 definitely_dead 0 2024-09-11T01:12:01.750 INFO:teuthology.orchestra.run.smithi122.stderr:marked out osd.1. 2024-09-11T01:12:01.777 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd lost 1 --yes-i-really-mean-it 2024-09-11T01:12:02.761 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:02.756+0000 7f700fe67640 -1 log_channel(cluster) log [ERR] : 2.0 has 2 objects unfound and apparently lost 2024-09-11T01:12:02.761 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:02.756+0000 7f701366e640 -1 log_channel(cluster) log [ERR] : 2.1 has 2 objects unfound and apparently lost 2024-09-11T01:12:02.761 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:02.760+0000 7f7013e6f640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-09-11T01:12:02.761 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:02.760+0000 7f700f666640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-11T01:12:02.761 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:02.760+0000 7f700ee65640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-11T01:12:02.762 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:02.760+0000 7f700de63640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-11T01:12:02.762 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:02.760+0000 7f7012e6d640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-11T01:12:02.762 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:02.760+0000 7f701266c640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-09-11T01:12:02.762 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:02.760+0000 7f701166a640 -1 log_channel(cluster) log [ERR] : 2.d has 1 objects unfound and apparently lost 2024-09-11T01:12:02.762 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:02.760+0000 7f7010668640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-11T01:12:02.762 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:02.760+0000 7f701166a640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-11T01:12:02.762 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:02.760+0000 7f700ce61640 -1 log_channel(cluster) log [ERR] : 2.e has 2 objects unfound and apparently lost 2024-09-11T01:12:02.762 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:02.760+0000 7f7010e69640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-11T01:12:02.762 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:02.760+0000 7f700e664640 -1 log_channel(cluster) log [ERR] : 2.3 has 2 objects unfound and apparently lost 2024-09-11T01:12:02.762 INFO:teuthology.orchestra.run.smithi122.stderr:marked osd lost in epoch 27 2024-09-11T01:12:02.781 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-11T01:12:02.782 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-11T01:12:02.784 INFO:tasks.ceph.osd.2:Started 2024-09-11T01:12:02.784 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T01:12:02.905 INFO:teuthology.orchestra.run.smithi122.stderr:no valid command found; 10 closest matches: 2024-09-11T01:12:02.905 INFO:teuthology.orchestra.run.smithi122.stderr:0 2024-09-11T01:12:02.905 INFO:teuthology.orchestra.run.smithi122.stderr:1 2024-09-11T01:12:02.905 INFO:teuthology.orchestra.run.smithi122.stderr:2 2024-09-11T01:12:02.906 INFO:teuthology.orchestra.run.smithi122.stderr:abort 2024-09-11T01:12:02.906 INFO:teuthology.orchestra.run.smithi122.stderr:assert 2024-09-11T01:12:02.906 INFO:teuthology.orchestra.run.smithi122.stderr:config diff 2024-09-11T01:12:02.906 INFO:teuthology.orchestra.run.smithi122.stderr:config diff get 2024-09-11T01:12:02.906 INFO:teuthology.orchestra.run.smithi122.stderr:config get 2024-09-11T01:12:02.906 INFO:teuthology.orchestra.run.smithi122.stderr:config help [] 2024-09-11T01:12:02.907 INFO:teuthology.orchestra.run.smithi122.stderr:config set ... 2024-09-11T01:12:02.907 INFO:teuthology.orchestra.run.smithi122.stderr:admin_socket: invalid command 2024-09-11T01:12:02.908 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-11T01:12:02.908 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-11T01:12:04.095 INFO:tasks.ceph.osd.2.smithi122.stderr:2024-09-11T01:12:04.096+0000 7f90708d97c0 -1 Falling back to public interface 2024-09-11T01:12:07.909 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T01:12:07.971 INFO:tasks.ceph.osd.2.smithi122.stderr:2024-09-11T01:12:07.972+0000 7f90708d97c0 -1 osd.2 17 log_to_monitors true 2024-09-11T01:12:08.030 INFO:teuthology.orchestra.run.smithi122.stderr:no valid command found; 10 closest matches: 2024-09-11T01:12:08.030 INFO:teuthology.orchestra.run.smithi122.stderr:0 2024-09-11T01:12:08.030 INFO:teuthology.orchestra.run.smithi122.stderr:1 2024-09-11T01:12:08.030 INFO:teuthology.orchestra.run.smithi122.stderr:2 2024-09-11T01:12:08.030 INFO:teuthology.orchestra.run.smithi122.stderr:abort 2024-09-11T01:12:08.030 INFO:teuthology.orchestra.run.smithi122.stderr:assert 2024-09-11T01:12:08.030 INFO:teuthology.orchestra.run.smithi122.stderr:bluefs debug_inject_read_zeros 2024-09-11T01:12:08.030 INFO:teuthology.orchestra.run.smithi122.stderr:bluefs files list 2024-09-11T01:12:08.030 INFO:teuthology.orchestra.run.smithi122.stderr:bluefs stats 2024-09-11T01:12:08.030 INFO:teuthology.orchestra.run.smithi122.stderr:bluestore allocator dump block 2024-09-11T01:12:08.031 INFO:teuthology.orchestra.run.smithi122.stderr:bluestore allocator fragmentation block 2024-09-11T01:12:08.031 INFO:teuthology.orchestra.run.smithi122.stderr:admin_socket: invalid command 2024-09-11T01:12:08.032 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-11T01:12:08.033 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-2, ['dump_ops_in_flight'] 2024-09-11T01:12:08.035 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:08.036+0000 7f7013e6f640 -1 log_channel(cluster) log [ERR] : 2.0 has 2 objects unfound and apparently lost 2024-09-11T01:12:08.035 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:08.036+0000 7f7013e6f640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-09-11T01:12:08.036 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:08.036+0000 7f700f666640 -1 log_channel(cluster) log [ERR] : 2.1 has 2 objects unfound and apparently lost 2024-09-11T01:12:08.036 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:08.036+0000 7f700f666640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-11T01:12:08.038 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:08.040+0000 7f7012e6d640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-11T01:12:08.038 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:08.040+0000 7f700ee65640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-11T01:12:08.038 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:08.040+0000 7f701266c640 -1 log_channel(cluster) log [ERR] : 2.3 has 2 objects unfound and apparently lost 2024-09-11T01:12:08.038 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:08.040+0000 7f700e664640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-09-11T01:12:08.038 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:08.040+0000 7f700de63640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-11T01:12:08.038 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:08.040+0000 7f701166a640 -1 log_channel(cluster) log [ERR] : 2.d has 1 objects unfound and apparently lost 2024-09-11T01:12:08.039 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:08.040+0000 7f701166a640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-11T01:12:08.039 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:08.040+0000 7f700ce61640 -1 log_channel(cluster) log [ERR] : 2.e has 2 objects unfound and apparently lost 2024-09-11T01:12:08.040 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:08.040+0000 7f7010e69640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-11T01:12:08.040 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:08.040+0000 7f7010668640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-11T01:12:08.040 INFO:tasks.ceph.osd.2.smithi122.stderr:2024-09-11T01:12:08.040+0000 7f90686bc640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T01:12:09.047 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:09.044+0000 7f700fe67640 -1 log_channel(cluster) log [ERR] : 2.0 has 2 objects unfound and apparently lost 2024-09-11T01:12:09.047 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:09.044+0000 7f700fe67640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-09-11T01:12:09.048 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:09.044+0000 7f700de63640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-11T01:12:09.048 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:09.044+0000 7f700e664640 -1 log_channel(cluster) log [ERR] : 2.3 has 2 objects unfound and apparently lost 2024-09-11T01:12:09.049 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:09.044+0000 7f700e664640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-09-11T01:12:09.049 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:09.048+0000 7f700d662640 -1 log_channel(cluster) log [ERR] : 2.d has 1 objects unfound and apparently lost 2024-09-11T01:12:09.050 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:09.048+0000 7f700d662640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-11T01:12:09.050 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:09.048+0000 7f7010668640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-11T01:12:09.051 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:09.048+0000 7f701366e640 -1 log_channel(cluster) log [ERR] : 2.1 has 2 objects unfound and apparently lost 2024-09-11T01:12:09.051 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:09.048+0000 7f7012e6d640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-11T01:12:09.051 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:09.048+0000 7f700f666640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-11T01:12:09.052 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:09.048+0000 7f7010e69640 -1 log_channel(cluster) log [ERR] : 2.e has 2 objects unfound and apparently lost 2024-09-11T01:12:09.052 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:09.048+0000 7f7012e6d640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-11T01:12:09.052 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:09.048+0000 7f7010e69640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-11T01:12:10.064 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:10.064+0000 7f700ee65640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-11T01:12:10.072 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:10.068+0000 7f700de63640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-11T01:12:10.075 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:10.068+0000 7f7010668640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-11T01:12:10.075 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:10.068+0000 7f700f666640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-11T01:12:10.077 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:10.068+0000 7f700ce61640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-11T01:12:10.079 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:10.072+0000 7f700d662640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-11T01:12:10.080 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:10.072+0000 7f700ee65640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-11T01:12:11.090 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:11.084+0000 7f7011e6b640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-11T01:12:11.091 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:11.088+0000 7f700ee65640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-11T01:12:11.093 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:11.092+0000 7f700f666640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-11T01:12:11.094 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:11.092+0000 7f7012e6d640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-11T01:12:11.094 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:11.092+0000 7f7010e69640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-11T01:12:11.095 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:11.092+0000 7f700c660640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-11T01:12:11.096 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:11.092+0000 7f701166a640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-11T01:12:13.033 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-09-11T01:12:13.169 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-11T01:12:13.169 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [], 2024-09-11T01:12:13.169 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 0 2024-09-11T01:12:13.169 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-11T01:12:13.184 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 2 2024-09-11T01:12:13.496 INFO:tasks.ceph.mon.a.smithi122.stderr:2024-09-11T01:12:13.496+0000 7f9dc4120640 -1 mon.a@0(leader).osd e33 definitely_dead 0 2024-09-11T01:12:14.111 INFO:teuthology.orchestra.run.smithi122.stderr:marked in osd.2. 2024-09-11T01:12:14.138 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for osd.2 to be up 2024-09-11T01:12:14.138 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T01:12:14.439 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:12:14.439 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":34,"fsid":"6d328773-3914-4412-91b7-393379984de3","created":"2024-09-11T01:11:03.508628+0000","modified":"2024-09-11T01:12:14.088560+0000","last_up_change":"2024-09-11T01:12:09.036840+0000","last_in_change":"2024-09-11T01:12:14.088560+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-11T01:11:18.999872+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":1.4900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":1,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfounddel_pool","create_time":"2024-09-11T01:11:30.626364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":1.8700000047683716,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e160603b-65bb-420f-9130-6d7c6506f2a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":21,"up_from":25,"up_thru":32,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6800","nonce":2613596984}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6801","nonce":2613596984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6803","nonce":2613596984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6802","nonce":2613596984}]},"public_addr":"172.21.15.122:6800/2613596984","cluster_addr":"172.21.15.122:6801/2613596984","heartbeat_back_addr":"172.21.15.122:6803/2613596984","heartbeat_front_addr":"172.21.15.122:6802/2613596984","state":["exists","up"]},{"osd":1,"uuid":"6fbfe9e1-4255-4c44-b619-5fd57486e5fe","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":25,"down_at":27,"lost_at":27,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6804","nonce":2638428424}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6805","nonce":2638428424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6807","nonce":2638428424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6806","nonce":2638428424}]},"public_addr":"172.21.15.122:6804/2638428424","cluster_addr":"172.21.15.122:6805/2638428424","heartbeat_back_addr":"172.21.15.122:6807/2638428424","heartbeat_front_addr":"172.21.15.122:6806/2638428424","state":["exists"]},{"osd":2,"uuid":"e27771fd-ac9d-4a64-a01f-465a5acade91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":17,"up_from":31,"up_thru":16,"down_at":19,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6804","nonce":1506556606}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6805","nonce":1506556606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6807","nonce":1506556606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6806","nonce":1506556606}]},"public_addr":"172.21.15.122:6804/1506556606","cluster_addr":"172.21.15.122:6805/1506556606","heartbeat_back_addr":"172.21.15.122:6807/1506556606","heartbeat_front_addr":"172.21.15.122:6806/1506556606","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:11:12.059865+0000","dead_epoch":22},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":65536,"last_purged_snaps_scrub":"2024-09-11T01:11:11.519118+0000","dead_epoch":27},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:11:11.402743+0000","dead_epoch":19}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[{"pgid":"1.1","osds":[0,2]},{"pgid":"1.3","osds":[0,2]},{"pgid":"1.5","osds":[0,2]},{"pgid":"1.7","osds":[0,2]},{"pgid":"2.0","osds":[0,2]},{"pgid":"2.1","osds":[0,2]},{"pgid":"2.3","osds":[0,2]},{"pgid":"2.c","osds":[0,2]},{"pgid":"2.d","osds":[0,2]},{"pgid":"2.e","osds":[0,2]}],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T01:12:14.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:osd.2 is up 2024-09-11T01:12:14.458 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-11T01:12:14.458 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-11T01:12:14.613 INFO:teuthology.orchestra.run.smithi122.stdout:107374182407 2024-09-11T01:12:14.613 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T01:12:14.619 INFO:teuthology.orchestra.run.smithi122.stdout:133143986179 2024-09-11T01:12:14.619 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-11T01:12:14.912 INFO:teuthology.orchestra.run.smithi122.stdout:107374182405 2024-09-11T01:12:14.929 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 107374182407 got 107374182405 for osd.0 2024-09-11T01:12:14.945 INFO:teuthology.orchestra.run.smithi122.stdout:1 2024-09-11T01:12:14.961 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 133143986179 got 1 for osd.2 2024-09-11T01:12:15.929 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T01:12:15.962 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-11T01:12:16.116 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:16.111+0000 7f7013e6f640 -1 log_channel(cluster) log [ERR] : 2.8 has 1 objects unfound and apparently lost 2024-09-11T01:12:16.116 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:16.111+0000 7f701366e640 -1 log_channel(cluster) log [ERR] : 2.9 has 2 objects unfound and apparently lost 2024-09-11T01:12:16.116 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:16.111+0000 7f700ee65640 -1 log_channel(cluster) log [ERR] : 2.a has 2 objects unfound and apparently lost 2024-09-11T01:12:16.116 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:16.111+0000 7f7012e6d640 -1 log_channel(cluster) log [ERR] : 2.2 has 2 objects unfound and apparently lost 2024-09-11T01:12:16.116 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:16.111+0000 7f701266c640 -1 log_channel(cluster) log [ERR] : 2.b has 2 objects unfound and apparently lost 2024-09-11T01:12:16.116 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:16.111+0000 7f7011e6b640 -1 log_channel(cluster) log [ERR] : 2.4 has 3 objects unfound and apparently lost 2024-09-11T01:12:16.116 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:16.111+0000 7f700d662640 -1 log_channel(cluster) log [ERR] : 2.5 has 2 objects unfound and apparently lost 2024-09-11T01:12:16.116 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:16.111+0000 7f700ce61640 -1 log_channel(cluster) log [ERR] : 2.6 has 3 objects unfound and apparently lost 2024-09-11T01:12:16.116 INFO:tasks.ceph.osd.0.smithi122.stderr:2024-09-11T01:12:16.111+0000 7f7010668640 -1 log_channel(cluster) log [ERR] : 2.7 has 1 objects unfound and apparently lost 2024-09-11T01:12:16.251 INFO:teuthology.orchestra.run.smithi122.stdout:107374182406 2024-09-11T01:12:16.269 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 107374182407 got 107374182406 for osd.0 2024-09-11T01:12:16.276 INFO:teuthology.orchestra.run.smithi122.stdout:133143986178 2024-09-11T01:12:16.293 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 133143986179 got 133143986178 for osd.2 2024-09-11T01:12:17.270 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T01:12:17.293 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-11T01:12:17.592 INFO:teuthology.orchestra.run.smithi122.stdout:107374182407 2024-09-11T01:12:17.610 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 107374182407 got 107374182407 for osd.0 2024-09-11T01:12:17.610 DEBUG:teuthology.parallel:result is None 2024-09-11T01:12:17.623 INFO:teuthology.orchestra.run.smithi122.stdout:133143986179 2024-09-11T01:12:17.640 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 133143986179 got 133143986179 for osd.2 2024-09-11T01:12:17.640 DEBUG:teuthology.parallel:result is None 2024-09-11T01:12:17.640 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting till active 2024-09-11T01:12:17.640 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:12:17.976 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:12:17.977 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:12:17.993 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":66,"stamp":"2024-09-11T01:12:16.538541+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":17295,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58,"ondisk_log_size":58,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":36,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55540,"kb_used_data":596,"kb_used_omap":8,"kb_used_meta":54903,"kb_avail":188688140,"statfs":{"total":193273528320,"available":193216655360,"internally_reserved":0,"allocated":610304,"data_stored":182080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8875,"internal_metadata":56221013},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-57,"num_objects_misplaced":0,"num_objects_unfound":-27,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.455673"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":40,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-11T01:12:11.095543+0000","last_change":"2024-09-11T01:12:14.116085+0000","last_active":"2024-09-11T01:12:14.116085+0000","last_peered":"2024-09-11T01:12:11.095543+0000","last_clean":"2024-09-11T01:11:55.731006+0000","last_became_active":"2024-09-11T01:12:01.761776+0000","last_became_peered":"2024-09-11T01:12:01.761776+0000","last_unstale":"2024-09-11T01:12:11.095543+0000","last_undegraded":"2024-09-11T01:12:11.095543+0000","last_fullsized":"2024-09-11T01:12:01.756532+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"23'4","reported_seq":44,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-11T01:12:11.098213+0000","last_change":"2024-09-11T01:12:14.117881+0000","last_active":"2024-09-11T01:12:14.117881+0000","last_peered":"2024-09-11T01:12:11.098213+0000","last_clean":"2024-09-11T01:11:43.020052+0000","last_became_active":"2024-09-11T01:12:11.097479+0000","last_became_peered":"2024-09-11T01:12:11.097479+0000","last_unstale":"2024-09-11T01:12:11.098213+0000","last_undegraded":"2024-09-11T01:12:11.088753+0000","last_fullsized":"2024-09-11T01:12:11.098213+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"23'3","reported_seq":60,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-11T01:12:11.104277+0000","last_change":"2024-09-11T01:12:14.117910+0000","last_active":"2024-09-11T01:12:14.117910+0000","last_peered":"2024-09-11T01:12:11.104277+0000","last_clean":"2024-09-11T01:11:43.936435+0000","last_became_active":"2024-09-11T01:12:11.103508+0000","last_became_peered":"2024-09-11T01:12:11.103508+0000","last_unstale":"2024-09-11T01:12:11.104277+0000","last_undegraded":"2024-09-11T01:12:11.094379+0000","last_fullsized":"2024-09-11T01:12:11.104277+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":49,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-11T01:12:11.103819+0000","last_change":"2024-09-11T01:12:14.122073+0000","last_active":"2024-09-11T01:12:14.122073+0000","last_peered":"2024-09-11T01:12:11.103819+0000","last_clean":"2024-09-11T01:12:11.103819+0000","last_became_active":"2024-09-11T01:12:11.099113+0000","last_became_peered":"2024-09-11T01:12:11.099113+0000","last_unstale":"2024-09-11T01:12:11.103819+0000","last_undegraded":"2024-09-11T01:12:11.103819+0000","last_fullsized":"2024-09-11T01:12:11.103819+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.b","version":"23'5","reported_seq":57,"reported_epoch":34,"state":"peering","last_fresh":"2024-09-11T01:12:14.121081+0000","last_change":"2024-09-11T01:12:14.121081+0000","last_active":"2024-09-11T01:12:14.121081+0000","last_peered":"2024-09-11T01:12:11.100663+0000","last_clean":"2024-09-11T01:11:42.855496+0000","last_became_active":"2024-09-11T01:12:11.099946+0000","last_became_peered":"2024-09-11T01:12:11.099946+0000","last_unstale":"2024-09-11T01:12:14.121081+0000","last_undegraded":"2024-09-11T01:12:14.121081+0000","last_fullsized":"2024-09-11T01:12:14.121081+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"23'3","reported_seq":53,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-11T01:12:11.088500+0000","last_change":"2024-09-11T01:12:14.117669+0000","last_active":"2024-09-11T01:12:14.117669+0000","last_peered":"2024-09-11T01:12:11.088500+0000","last_clean":"2024-09-11T01:11:44.266820+0000","last_became_active":"2024-09-11T01:12:01.758594+0000","last_became_peered":"2024-09-11T01:12:01.758594+0000","last_unstale":"2024-09-11T01:12:11.088500+0000","last_undegraded":"2024-09-11T01:12:01.754811+0000","last_fullsized":"2024-09-11T01:12:01.754458+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"23'4","reported_seq":53,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-11T01:12:11.092880+0000","last_change":"2024-09-11T01:12:14.122664+0000","last_active":"2024-09-11T01:12:14.122664+0000","last_peered":"2024-09-11T01:12:11.092880+0000","last_clean":"2024-09-11T01:11:43.275607+0000","last_became_active":"2024-09-11T01:12:01.764412+0000","last_became_peered":"2024-09-11T01:12:01.764412+0000","last_unstale":"2024-09-11T01:12:11.092880+0000","last_undegraded":"2024-09-11T01:12:01.755871+0000","last_fullsized":"2024-09-11T01:12:01.755617+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"23'1","reported_seq":55,"reported_epoch":34,"state":"peering","last_fresh":"2024-09-11T01:12:14.118515+0000","last_change":"2024-09-11T01:12:14.118515+0000","last_active":"2024-09-11T01:12:14.118515+0000","last_peered":"2024-09-11T01:12:11.098085+0000","last_clean":"2024-09-11T01:11:32.735868+0000","last_became_active":"2024-09-11T01:12:11.097348+0000","last_became_peered":"2024-09-11T01:12:11.097348+0000","last_unstale":"2024-09-11T01:12:14.118515+0000","last_undegraded":"2024-09-11T01:12:14.118515+0000","last_fullsized":"2024-09-11T01:12:14.118515+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-11T01:12:11.085877+0000","last_change":"2024-09-11T01:12:14.116127+0000","last_active":"2024-09-11T01:12:14.116127+0000","last_peered":"2024-09-11T01:12:11.085877+0000","last_clean":"2024-09-11T01:11:35.757152+0000","last_became_active":"2024-09-11T01:12:01.756841+0000","last_became_peered":"2024-09-11T01:12:01.756841+0000","last_unstale":"2024-09-11T01:12:11.085877+0000","last_undegraded":"2024-09-11T01:12:11.085877+0000","last_fullsized":"2024-09-11T01:12:01.753579+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"23'5","reported_seq":62,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-11T01:12:11.106623+0000","last_change":"2024-09-11T01:12:14.117027+0000","last_active":"2024-09-11T01:12:14.117027+0000","last_peered":"2024-09-11T01:12:11.106623+0000","last_clean":"2024-09-11T01:11:44.016870+0000","last_became_active":"2024-09-11T01:12:11.106070+0000","last_became_peered":"2024-09-11T01:12:11.106070+0000","last_unstale":"2024-09-11T01:12:11.106623+0000","last_undegraded":"2024-09-11T01:12:11.095731+0000","last_fullsized":"2024-09-11T01:12:11.106623+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":49,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-11T01:12:11.101215+0000","last_change":"2024-09-11T01:12:14.110131+0000","last_active":"2024-09-11T01:12:14.110131+0000","last_peered":"2024-09-11T01:12:11.101215+0000","last_clean":"2024-09-11T01:12:11.101215+0000","last_became_active":"2024-09-11T01:12:11.096402+0000","last_became_peered":"2024-09-11T01:12:11.096402+0000","last_unstale":"2024-09-11T01:12:11.101215+0000","last_undegraded":"2024-09-11T01:12:11.101215+0000","last_fullsized":"2024-09-11T01:12:11.101215+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"23'6","reported_seq":49,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-11T01:12:11.093950+0000","last_change":"2024-09-11T01:12:14.121533+0000","last_active":"2024-09-11T01:12:14.121533+0000","last_peered":"2024-09-11T01:12:11.093950+0000","last_clean":"2024-09-11T01:11:44.303167+0000","last_became_active":"2024-09-11T01:12:01.759535+0000","last_became_peered":"2024-09-11T01:12:01.759535+0000","last_unstale":"2024-09-11T01:12:11.093950+0000","last_undegraded":"2024-09-11T01:12:01.755741+0000","last_fullsized":"2024-09-11T01:12:01.755433+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":56,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-11T01:12:11.088235+0000","last_change":"2024-09-11T01:12:14.116450+0000","last_active":"2024-09-11T01:12:14.116450+0000","last_peered":"2024-09-11T01:12:11.088235+0000","last_clean":"2024-09-11T01:11:55.731962+0000","last_became_active":"2024-09-11T01:12:01.759536+0000","last_became_peered":"2024-09-11T01:12:01.759536+0000","last_unstale":"2024-09-11T01:12:11.088235+0000","last_undegraded":"2024-09-11T01:12:01.754412+0000","last_fullsized":"2024-09-11T01:12:01.754301+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'3","reported_seq":49,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-11T01:12:11.099725+0000","last_change":"2024-09-11T01:12:14.111744+0000","last_active":"2024-09-11T01:12:14.111744+0000","last_peered":"2024-09-11T01:12:11.099725+0000","last_clean":"2024-09-11T01:11:42.775010+0000","last_became_active":"2024-09-11T01:12:11.098954+0000","last_became_peered":"2024-09-11T01:12:11.098954+0000","last_unstale":"2024-09-11T01:12:11.099725+0000","last_undegraded":"2024-09-11T01:12:11.087854+0000","last_fullsized":"2024-09-11T01:12:11.099725+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":57,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-11T01:12:11.103589+0000","last_change":"2024-09-11T01:12:14.115817+0000","last_active":"2024-09-11T01:12:14.115817+0000","last_peered":"2024-09-11T01:12:11.103589+0000","last_clean":"2024-09-11T01:12:11.103589+0000","last_became_active":"2024-09-11T01:12:11.098752+0000","last_became_peered":"2024-09-11T01:12:11.098752+0000","last_unstale":"2024-09-11T01:12:11.103589+0000","last_undegraded":"2024-09-11T01:12:11.103589+0000","last_fullsized":"2024-09-11T01:12:11.103589+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"23'4","reported_seq":51,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-11T01:12:11.098968+0000","last_change":"2024-09-11T01:12:14.117399+0000","last_active":"2024-09-11T01:12:14.117399+0000","last_peered":"2024-09-11T01:12:11.098968+0000","last_clean":"2024-09-11T01:11:43.440047+0000","last_became_active":"2024-09-11T01:12:11.097923+0000","last_became_peered":"2024-09-11T01:12:11.097923+0000","last_unstale":"2024-09-11T01:12:11.098968+0000","last_undegraded":"2024-09-11T01:12:11.086475+0000","last_fullsized":"2024-09-11T01:12:11.098968+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"23'1","reported_seq":49,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-11T01:12:11.095198+0000","last_change":"2024-09-11T01:12:14.118202+0000","last_active":"2024-09-11T01:12:14.118202+0000","last_peered":"2024-09-11T01:12:11.095198+0000","last_clean":"2024-09-11T01:11:35.758866+0000","last_became_active":"2024-09-11T01:12:01.761784+0000","last_became_peered":"2024-09-11T01:12:01.761784+0000","last_unstale":"2024-09-11T01:12:11.095198+0000","last_undegraded":"2024-09-11T01:12:01.756883+0000","last_fullsized":"2024-09-11T01:12:01.756571+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-11T01:12:11.087728+0000","last_change":"2024-09-11T01:12:14.117997+0000","last_active":"2024-09-11T01:12:14.117997+0000","last_peered":"2024-09-11T01:12:11.087728+0000","last_clean":"2024-09-11T01:11:35.757292+0000","last_became_active":"2024-09-11T01:12:01.761485+0000","last_became_peered":"2024-09-11T01:12:01.761485+0000","last_unstale":"2024-09-11T01:12:11.087728+0000","last_undegraded":"2024-09-11T01:12:11.087728+0000","last_fullsized":"2024-09-11T01:12:01.755011+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"23'5","reported_seq":54,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-11T01:12:11.093965+0000","last_change":"2024-09-11T01:12:14.118530+0000","last_active":"2024-09-11T01:12:14.118530+0000","last_peered":"2024-09-11T01:12:11.093965+0000","last_clean":"2024-09-11T01:11:44.178063+0000","last_became_active":"2024-09-11T01:12:01.759541+0000","last_became_peered":"2024-09-11T01:12:01.759541+0000","last_unstale":"2024-09-11T01:12:11.093965+0000","last_undegraded":"2024-09-11T01:12:01.755705+0000","last_fullsized":"2024-09-11T01:12:01.755507+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":49,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-11T01:12:11.102432+0000","last_change":"2024-09-11T01:12:14.114570+0000","last_active":"2024-09-11T01:12:14.114570+0000","last_peered":"2024-09-11T01:12:11.102432+0000","last_clean":"2024-09-11T01:12:11.102432+0000","last_became_active":"2024-09-11T01:12:11.097658+0000","last_became_peered":"2024-09-11T01:12:11.097658+0000","last_unstale":"2024-09-11T01:12:11.102432+0000","last_undegraded":"2024-09-11T01:12:11.102432+0000","last_fullsized":"2024-09-11T01:12:11.102432+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"23'2","reported_seq":50,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-11T01:12:11.095337+0000","last_change":"2024-09-11T01:12:14.115488+0000","last_active":"2024-09-11T01:12:14.115488+0000","last_peered":"2024-09-11T01:12:11.095337+0000","last_clean":"2024-09-11T01:11:35.758438+0000","last_became_active":"2024-09-11T01:12:01.759186+0000","last_became_peered":"2024-09-11T01:12:01.759186+0000","last_unstale":"2024-09-11T01:12:11.095337+0000","last_undegraded":"2024-09-11T01:12:01.756381+0000","last_fullsized":"2024-09-11T01:12:01.756250+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-11T01:12:11.087704+0000","last_change":"2024-09-11T01:12:14.121430+0000","last_active":"2024-09-11T01:12:14.121430+0000","last_peered":"2024-09-11T01:12:11.087704+0000","last_clean":"2024-09-11T01:11:35.758347+0000","last_became_active":"2024-09-11T01:12:01.758594+0000","last_became_peered":"2024-09-11T01:12:01.758594+0000","last_unstale":"2024-09-11T01:12:11.087704+0000","last_undegraded":"2024-09-11T01:12:11.087704+0000","last_fullsized":"2024-09-11T01:12:01.754792+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":33,"state":"remapped+peering","last_fresh":"2024-09-11T01:12:11.106836+0000","last_change":"2024-09-11T01:12:14.115196+0000","last_active":"2024-09-11T01:12:14.115196+0000","last_peered":"2024-09-11T01:12:11.106836+0000","last_clean":"2024-09-11T01:12:11.106836+0000","last_became_active":"2024-09-11T01:12:11.102928+0000","last_became_peered":"2024-09-11T01:12:11.102928+0000","last_unstale":"2024-09-11T01:12:11.106836+0000","last_undegraded":"2024-09-11T01:12:11.106836+0000","last_fullsized":"2024-09-11T01:12:11.106836+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":2,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"23'9","reported_seq":58,"reported_epoch":33,"state":"peering","last_fresh":"2024-09-11T01:12:11.088091+0000","last_change":"2024-09-11T01:12:14.122800+0000","last_active":"2024-09-11T01:12:14.122800+0000","last_peered":"2024-09-11T01:12:11.088091+0000","last_clean":"2024-09-11T01:11:43.771457+0000","last_became_active":"2024-09-11T01:12:01.759155+0000","last_became_peered":"2024-09-11T01:12:01.759155+0000","last_unstale":"2024-09-11T01:12:11.088091+0000","last_undegraded":"2024-09-11T01:12:01.754727+0000","last_fullsized":"2024-09-11T01:12:01.754566+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2468,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":17276,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":23446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182407,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27796,"kb_used_data":324,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344044,"statfs":{"total":96636764160,"available":96608301056,"internally_reserved":0,"allocated":331776,"data_stored":95806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":28110507},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986179,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27744,"kb_used_data":272,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344096,"statfs":{"total":96636764160,"available":96608354304,"internally_reserved":0,"allocated":278528,"data_stored":86274,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":28110506},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":17276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:12:17.995 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:12:18.359 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:12:18.359 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":74,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":36,"num_osds":3,"num_up_osds":2,"osd_up_since":1726017129,"num_in_osds":2,"osd_in_since":1726017134,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"peering","count":14},{"state_name":"remapped+peering","count":10}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":17295,"bytes_used":56872960,"bytes_avail":193216655360,"bytes_total":193273528320,"inactive_pgs_ratio":1},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-11T01:12:14.537806+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T01:12:18.377 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'fsid': '6d328773-3914-4412-91b7-393379984de3', 'health': {'status': 'HEALTH_OK', 'checks': {}, 'mutes': []}, 'election_epoch': 8, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 74, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 36, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1726017129, 'num_in_osds': 2, 'osd_in_since': 1726017134, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'peering', 'count': 14}, {'state_name': 'remapped+peering', 'count': 10}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 17295, 'bytes_used': 56872960, 'bytes_avail': 193216655360, 'bytes_total': 193273528320, 'inactive_pgs_ratio': 1}, 'fsmap': {'epoch': 1, 'btime': '2024-09-11T01:11:03:507954+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 3, 'modified': '2024-09-11T01:12:14.537806+0000', 'services': {'osd': {'daemons': {'summary': '', '0': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, '2': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}}}, 'progress_events': {}} 2024-09-11T01:12:19.117 INFO:tasks.ceph.mon.a.smithi122.stderr:2024-09-11T01:12:19.115+0000 7f9dc6925640 -1 log_channel(cluster) log [ERR] : Health check failed: Possible data damage: 5 pgs recovery_unfound (PG_DAMAGED) 2024-09-11T01:12:21.378 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:12:21.690 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:12:21.691 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:12:21.708 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":68,"stamp":"2024-09-11T01:12:20.539763+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":17295,"num_objects":30,"num_object_clones":0,"num_object_copies":60,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":50,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":30,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":58,"ondisk_log_size":58,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55580,"kb_used_data":636,"kb_used_omap":8,"kb_used_meta":54903,"kb_avail":188688100,"statfs":{"total":193273528320,"available":193216614400,"internally_reserved":0,"allocated":651264,"data_stored":196679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8875,"internal_metadata":56221013},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-6,"num_objects_misplaced":-1,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.433481"},"pg_stats":[{"pgid":"2.f","version":"0'0","reported_seq":51,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.115887+0000","last_change":"2024-09-11T01:12:15.141793+0000","last_active":"2024-09-11T01:12:16.115887+0000","last_peered":"2024-09-11T01:12:16.115887+0000","last_clean":"2024-09-11T01:12:16.115887+0000","last_became_active":"2024-09-11T01:12:15.141081+0000","last_became_peered":"2024-09-11T01:12:15.141081+0000","last_unstale":"2024-09-11T01:12:16.115887+0000","last_undegraded":"2024-09-11T01:12:16.115887+0000","last_fullsized":"2024-09-11T01:12:16.115887+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:18:33.495210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"23'4","reported_seq":48,"reported_epoch":36,"state":"active+recovery_unfound","last_fresh":"2024-09-11T01:12:16.124880+0000","last_change":"2024-09-11T01:12:16.125066+0000","last_active":"2024-09-11T01:12:16.124880+0000","last_peered":"2024-09-11T01:12:16.124880+0000","last_clean":"2024-09-11T01:11:43.020052+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:12:16.124880+0000","last_undegraded":"2024-09-11T01:12:16.116369+0000","last_fullsized":"2024-09-11T01:12:16.124880+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"23'3","reported_seq":65,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-11T01:12:16.125491+0000","last_change":"2024-09-11T01:12:18.044299+0000","last_active":"2024-09-11T01:12:16.125491+0000","last_peered":"2024-09-11T01:12:16.125491+0000","last_clean":"2024-09-11T01:11:43.936435+0000","last_became_active":"2024-09-11T01:12:16.123243+0000","last_became_peered":"2024-09-11T01:12:16.123243+0000","last_unstale":"2024-09-11T01:12:16.125491+0000","last_undegraded":"2024-09-11T01:12:16.125491+0000","last_fullsized":"2024-09-11T01:12:16.125491+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.122402+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:12:16.122402+0000","last_peered":"2024-09-11T01:12:16.122402+0000","last_clean":"2024-09-11T01:12:16.122402+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:12:16.122402+0000","last_undegraded":"2024-09-11T01:12:16.122402+0000","last_fullsized":"2024-09-11T01:12:16.122402+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"23'5","reported_seq":65,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:16.114984+0000","last_change":"2024-09-11T01:12:15.135722+0000","last_active":"2024-09-11T01:12:16.114984+0000","last_peered":"2024-09-11T01:12:16.114984+0000","last_clean":"2024-09-11T01:11:42.855496+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:12:16.114984+0000","last_undegraded":"2024-09-11T01:12:15.117592+0000","last_fullsized":"2024-09-11T01:12:16.114984+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"23'3","reported_seq":61,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:16.114806+0000","last_change":"2024-09-11T01:12:15.141906+0000","last_active":"2024-09-11T01:12:16.114806+0000","last_peered":"2024-09-11T01:12:16.114806+0000","last_clean":"2024-09-11T01:11:44.266820+0000","last_became_active":"2024-09-11T01:12:15.141377+0000","last_became_peered":"2024-09-11T01:12:15.141377+0000","last_unstale":"2024-09-11T01:12:16.114806+0000","last_undegraded":"2024-09-11T01:12:15.124660+0000","last_fullsized":"2024-09-11T01:12:16.114806+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"23'4","reported_seq":61,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:16.114610+0000","last_change":"2024-09-11T01:12:15.136057+0000","last_active":"2024-09-11T01:12:16.114610+0000","last_peered":"2024-09-11T01:12:16.114610+0000","last_clean":"2024-09-11T01:11:43.275607+0000","last_became_active":"2024-09-11T01:12:15.135367+0000","last_became_peered":"2024-09-11T01:12:15.135367+0000","last_unstale":"2024-09-11T01:12:16.114610+0000","last_undegraded":"2024-09-11T01:12:15.117856+0000","last_fullsized":"2024-09-11T01:12:16.114610+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"23'1","reported_seq":63,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:16.114529+0000","last_change":"2024-09-11T01:12:15.136010+0000","last_active":"2024-09-11T01:12:16.114529+0000","last_peered":"2024-09-11T01:12:16.114529+0000","last_clean":"2024-09-11T01:11:32.735868+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:12:16.114529+0000","last_undegraded":"2024-09-11T01:12:15.117050+0000","last_fullsized":"2024-09-11T01:12:16.114529+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.114317+0000","last_change":"2024-09-11T01:12:15.135174+0000","last_active":"2024-09-11T01:12:16.114317+0000","last_peered":"2024-09-11T01:12:16.114317+0000","last_clean":"2024-09-11T01:12:16.114317+0000","last_became_active":"2024-09-11T01:12:15.133597+0000","last_became_peered":"2024-09-11T01:12:15.133597+0000","last_unstale":"2024-09-11T01:12:16.114317+0000","last_undegraded":"2024-09-11T01:12:16.114317+0000","last_fullsized":"2024-09-11T01:12:16.114317+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:29:50.913351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"23'5","reported_seq":67,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-11T01:12:16.124562+0000","last_change":"2024-09-11T01:12:18.044248+0000","last_active":"2024-09-11T01:12:16.124562+0000","last_peered":"2024-09-11T01:12:16.124562+0000","last_clean":"2024-09-11T01:11:44.016870+0000","last_became_active":"2024-09-11T01:12:16.122119+0000","last_became_peered":"2024-09-11T01:12:16.122119+0000","last_unstale":"2024-09-11T01:12:16.124562+0000","last_undegraded":"2024-09-11T01:12:16.124562+0000","last_fullsized":"2024-09-11T01:12:16.124562+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.121218+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:12:16.121218+0000","last_peered":"2024-09-11T01:12:16.121218+0000","last_clean":"2024-09-11T01:12:16.121218+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:12:16.121218+0000","last_undegraded":"2024-09-11T01:12:16.121218+0000","last_fullsized":"2024-09-11T01:12:16.121218+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"23'6","reported_seq":57,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:16.114832+0000","last_change":"2024-09-11T01:12:15.141700+0000","last_active":"2024-09-11T01:12:16.114832+0000","last_peered":"2024-09-11T01:12:16.114832+0000","last_clean":"2024-09-11T01:11:44.303167+0000","last_became_active":"2024-09-11T01:12:15.140878+0000","last_became_peered":"2024-09-11T01:12:15.140878+0000","last_unstale":"2024-09-11T01:12:16.114832+0000","last_undegraded":"2024-09-11T01:12:15.122467+0000","last_fullsized":"2024-09-11T01:12:16.114832+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":64,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:16.114348+0000","last_change":"2024-09-11T01:12:15.141439+0000","last_active":"2024-09-11T01:12:16.114348+0000","last_peered":"2024-09-11T01:12:16.114348+0000","last_clean":"2024-09-11T01:11:55.731962+0000","last_became_active":"2024-09-11T01:12:15.139222+0000","last_became_peered":"2024-09-11T01:12:15.139222+0000","last_unstale":"2024-09-11T01:12:16.114348+0000","last_undegraded":"2024-09-11T01:12:15.121340+0000","last_fullsized":"2024-09-11T01:12:16.114348+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"23'3","reported_seq":53,"reported_epoch":36,"state":"active+recovery_unfound","last_fresh":"2024-09-11T01:12:16.123276+0000","last_change":"2024-09-11T01:12:16.123531+0000","last_active":"2024-09-11T01:12:16.123276+0000","last_peered":"2024-09-11T01:12:16.123276+0000","last_clean":"2024-09-11T01:11:42.775010+0000","last_became_active":"2024-09-11T01:12:16.120717+0000","last_became_peered":"2024-09-11T01:12:16.120717+0000","last_unstale":"2024-09-11T01:12:16.123276+0000","last_undegraded":"2024-09-11T01:12:16.113366+0000","last_fullsized":"2024-09-11T01:12:16.123276+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.122445+0000","last_change":"2024-09-11T01:12:16.118776+0000","last_active":"2024-09-11T01:12:16.122445+0000","last_peered":"2024-09-11T01:12:16.122445+0000","last_clean":"2024-09-11T01:12:16.122445+0000","last_became_active":"2024-09-11T01:12:16.117613+0000","last_became_peered":"2024-09-11T01:12:16.117613+0000","last_unstale":"2024-09-11T01:12:16.122445+0000","last_undegraded":"2024-09-11T01:12:16.122445+0000","last_fullsized":"2024-09-11T01:12:16.122445+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:46.399169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"23'4","reported_seq":55,"reported_epoch":36,"state":"active+recovery_unfound","last_fresh":"2024-09-11T01:12:16.121806+0000","last_change":"2024-09-11T01:12:16.122039+0000","last_active":"2024-09-11T01:12:16.121806+0000","last_peered":"2024-09-11T01:12:16.121806+0000","last_clean":"2024-09-11T01:11:43.440047+0000","last_became_active":"2024-09-11T01:12:16.119017+0000","last_became_peered":"2024-09-11T01:12:16.119017+0000","last_unstale":"2024-09-11T01:12:16.121806+0000","last_undegraded":"2024-09-11T01:12:16.113221+0000","last_fullsized":"2024-09-11T01:12:16.121806+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"23'1","reported_seq":57,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:16.115685+0000","last_change":"2024-09-11T01:12:15.136135+0000","last_active":"2024-09-11T01:12:16.115685+0000","last_peered":"2024-09-11T01:12:16.115685+0000","last_clean":"2024-09-11T01:11:35.758866+0000","last_became_active":"2024-09-11T01:12:15.134484+0000","last_became_peered":"2024-09-11T01:12:15.134484+0000","last_unstale":"2024-09-11T01:12:16.115685+0000","last_undegraded":"2024-09-11T01:12:15.118487+0000","last_fullsized":"2024-09-11T01:12:16.115685+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.115043+0000","last_change":"2024-09-11T01:12:15.137159+0000","last_active":"2024-09-11T01:12:16.115043+0000","last_peered":"2024-09-11T01:12:16.115043+0000","last_clean":"2024-09-11T01:12:16.115043+0000","last_became_active":"2024-09-11T01:12:15.136405+0000","last_became_peered":"2024-09-11T01:12:15.136405+0000","last_unstale":"2024-09-11T01:12:16.115043+0000","last_undegraded":"2024-09-11T01:12:16.115043+0000","last_fullsized":"2024-09-11T01:12:16.115043+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:32:43.772100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"23'5","reported_seq":62,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:16.115458+0000","last_change":"2024-09-11T01:12:15.141481+0000","last_active":"2024-09-11T01:12:16.115458+0000","last_peered":"2024-09-11T01:12:16.115458+0000","last_clean":"2024-09-11T01:11:44.178063+0000","last_became_active":"2024-09-11T01:12:15.140760+0000","last_became_peered":"2024-09-11T01:12:15.140760+0000","last_unstale":"2024-09-11T01:12:16.115458+0000","last_undegraded":"2024-09-11T01:12:15.124957+0000","last_fullsized":"2024-09-11T01:12:16.115458+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1851,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":3}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.123536+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:12:16.123536+0000","last_peered":"2024-09-11T01:12:16.123536+0000","last_clean":"2024-09-11T01:12:16.123536+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:12:16.123536+0000","last_undegraded":"2024-09-11T01:12:16.123536+0000","last_fullsized":"2024-09-11T01:12:16.123536+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"23'2","reported_seq":58,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:16.115386+0000","last_change":"2024-09-11T01:12:15.140853+0000","last_active":"2024-09-11T01:12:16.115386+0000","last_peered":"2024-09-11T01:12:16.115386+0000","last_clean":"2024-09-11T01:11:35.758438+0000","last_became_active":"2024-09-11T01:12:15.138823+0000","last_became_peered":"2024-09-11T01:12:15.138823+0000","last_unstale":"2024-09-11T01:12:16.115386+0000","last_undegraded":"2024-09-11T01:12:15.123630+0000","last_fullsized":"2024-09-11T01:12:16.115386+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1234,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.115056+0000","last_change":"2024-09-11T01:12:15.137008+0000","last_active":"2024-09-11T01:12:16.115056+0000","last_peered":"2024-09-11T01:12:16.115056+0000","last_clean":"2024-09-11T01:12:16.115056+0000","last_became_active":"2024-09-11T01:12:15.136222+0000","last_became_peered":"2024-09-11T01:12:15.136222+0000","last_unstale":"2024-09-11T01:12:16.115056+0000","last_undegraded":"2024-09-11T01:12:16.115056+0000","last_fullsized":"2024-09-11T01:12:16.115056+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:07:25.284013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.123820+0000","last_change":"2024-09-11T01:12:16.122836+0000","last_active":"2024-09-11T01:12:16.123820+0000","last_peered":"2024-09-11T01:12:16.123820+0000","last_clean":"2024-09-11T01:12:16.123820+0000","last_became_active":"2024-09-11T01:12:16.122572+0000","last_became_peered":"2024-09-11T01:12:16.122572+0000","last_unstale":"2024-09-11T01:12:16.123820+0000","last_undegraded":"2024-09-11T01:12:16.123820+0000","last_fullsized":"2024-09-11T01:12:16.123820+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:36:29.939442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"23'9","reported_seq":67,"reported_epoch":36,"state":"active+recovering+degraded","last_fresh":"2024-09-11T01:12:16.114983+0000","last_change":"2024-09-11T01:12:15.142056+0000","last_active":"2024-09-11T01:12:16.114983+0000","last_peered":"2024-09-11T01:12:16.114983+0000","last_clean":"2024-09-11T01:11:43.771457+0000","last_became_active":"2024-09-11T01:12:15.133897+0000","last_became_peered":"2024-09-11T01:12:15.133897+0000","last_unstale":"2024-09-11T01:12:16.114983+0000","last_undegraded":"2024-09-11T01:12:15.116261+0000","last_fullsized":"2024-09-11T01:12:16.114983+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2468,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":3}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":17276,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":49,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55,"num_write_kb":46,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":23446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182408,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27812,"kb_used_data":340,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344028,"statfs":{"total":96636764160,"available":96608284672,"internally_reserved":0,"allocated":348160,"data_stored":101434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":28110507},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986180,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27768,"kb_used_data":296,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344072,"statfs":{"total":96636764160,"available":96608329728,"internally_reserved":0,"allocated":303104,"data_stored":95245,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":28110506},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":17276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:12:21.713 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:12:22.076 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:12:22.076 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_ERR","checks":{"OBJECT_UNFOUND":{"severity":"HEALTH_WARN","summary":{"message":"27/30 objects unfound (90.000%)","count":27},"muted":false},"PG_DAMAGED":{"severity":"HEALTH_ERR","summary":{"message":"Possible data damage: 5 pgs recovery_unfound","count":5},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 50/60 objects degraded (83.333%), 12 pgs degraded","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":78,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":36,"num_osds":3,"num_up_osds":2,"osd_up_since":1726017129,"num_in_osds":2,"osd_in_since":1726017134,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":9},{"state_name":"active+recovery_wait+degraded","count":9},{"state_name":"active+recovery_unfound","count":3},{"state_name":"active+recovery_unfound+degraded","count":2},{"state_name":"active+recovering+degraded","count":1}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":17295,"bytes_used":56913920,"bytes_avail":193216614400,"bytes_total":193273528320,"degraded_objects":50,"degraded_total":60,"degraded_ratio":0.83333333333333348,"unfound_objects":27,"unfound_total":30,"unfound_ratio":0.90000000000000002},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-11T01:12:14.537806+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T01:12:22.095 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'fsid': '6d328773-3914-4412-91b7-393379984de3', 'health': {'status': 'HEALTH_ERR', 'checks': {'OBJECT_UNFOUND': {'severity': 'HEALTH_WARN', 'summary': {'message': '27/30 objects unfound (90.000%)', 'count': 27}, 'muted': False}, 'PG_DAMAGED': {'severity': 'HEALTH_ERR', 'summary': {'message': 'Possible data damage: 5 pgs recovery_unfound', 'count': 5}, 'muted': False}, 'PG_DEGRADED': {'severity': 'HEALTH_WARN', 'summary': {'message': 'Degraded data redundancy: 50/60 objects degraded (83.333%), 12 pgs degraded', 'count': 12}, 'muted': False}}, 'mutes': []}, 'election_epoch': 8, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 78, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 36, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1726017129, 'num_in_osds': 2, 'osd_in_since': 1726017134, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'active+clean', 'count': 9}, {'state_name': 'active+recovery_wait+degraded', 'count': 9}, {'state_name': 'active+recovery_unfound', 'count': 3}, {'state_name': 'active+recovery_unfound+degraded', 'count': 2}, {'state_name': 'active+recovering+degraded', 'count': 1}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 17295, 'bytes_used': 56913920, 'bytes_avail': 193216614400, 'bytes_total': 193273528320, 'degraded_objects': 50, 'degraded_total': 60, 'degraded_ratio': 0.8333333333333335, 'unfound_objects': 27, 'unfound_total': 30, 'unfound_ratio': 0.9}, 'fsmap': {'epoch': 1, 'btime': '2024-09-11T01:11:03:507954+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 3, 'modified': '2024-09-11T01:12:14.537806+0000', 'services': {'osd': {'daemons': {'summary': '', '0': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, '2': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}}}, 'progress_events': {}} 2024-09-11T01:12:22.095 INFO:tasks.rep_lost_unfound_delete.ceph_manager:active! 2024-09-11T01:12:22.095 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-11T01:12:22.096 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-11T01:12:22.247 INFO:teuthology.orchestra.run.smithi122.stdout:107374182409 2024-09-11T01:12:22.248 INFO:teuthology.orchestra.run.smithi122.stdout:133143986181 2024-09-11T01:12:22.248 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-11T01:12:22.249 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T01:12:22.529 INFO:teuthology.orchestra.run.smithi122.stdout:133143986180 2024-09-11T01:12:22.529 INFO:teuthology.orchestra.run.smithi122.stdout:107374182408 2024-09-11T01:12:22.548 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 133143986181 got 133143986180 for osd.2 2024-09-11T01:12:22.549 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 107374182409 got 107374182408 for osd.0 2024-09-11T01:12:23.552 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-11T01:12:23.553 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T01:12:23.847 INFO:teuthology.orchestra.run.smithi122.stdout:107374182409 2024-09-11T01:12:23.856 INFO:teuthology.orchestra.run.smithi122.stdout:133143986181 2024-09-11T01:12:23.865 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 107374182409 got 107374182409 for osd.0 2024-09-11T01:12:23.865 DEBUG:teuthology.parallel:result is None 2024-09-11T01:12:23.873 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 133143986181 got 133143986181 for osd.2 2024-09-11T01:12:23.873 DEBUG:teuthology.parallel:result is None 2024-09-11T01:12:23.873 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:12:24.279 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:12:24.279 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_ERR","checks":{"OBJECT_UNFOUND":{"severity":"HEALTH_WARN","summary":{"message":"27/30 objects unfound (90.000%)","count":27},"muted":false},"PG_DAMAGED":{"severity":"HEALTH_ERR","summary":{"message":"Possible data damage: 5 pgs recovery_unfound","count":5},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 50/60 objects degraded (83.333%), 12 pgs degraded","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":80,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":36,"num_osds":3,"num_up_osds":2,"osd_up_since":1726017129,"num_in_osds":2,"osd_in_since":1726017134,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":9},{"state_name":"active+recovery_wait+degraded","count":9},{"state_name":"active+recovery_unfound","count":3},{"state_name":"active+recovery_unfound+degraded","count":2},{"state_name":"active+recovering+degraded","count":1}],"num_pgs":24,"num_pools":2,"num_objects":30,"data_bytes":17295,"bytes_used":56913920,"bytes_avail":193216614400,"bytes_total":193273528320,"degraded_objects":50,"degraded_total":60,"degraded_ratio":0.83333333333333348,"unfound_objects":27,"unfound_total":30,"unfound_ratio":0.90000000000000002},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-11T01:12:14.537806+0000","services":{"osd":{"daemons":{"summary":"","0":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"2":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T01:12:24.296 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'fsid': '6d328773-3914-4412-91b7-393379984de3', 'health': {'status': 'HEALTH_ERR', 'checks': {'OBJECT_UNFOUND': {'severity': 'HEALTH_WARN', 'summary': {'message': '27/30 objects unfound (90.000%)', 'count': 27}, 'muted': False}, 'PG_DAMAGED': {'severity': 'HEALTH_ERR', 'summary': {'message': 'Possible data damage: 5 pgs recovery_unfound', 'count': 5}, 'muted': False}, 'PG_DEGRADED': {'severity': 'HEALTH_WARN', 'summary': {'message': 'Degraded data redundancy: 50/60 objects degraded (83.333%), 12 pgs degraded', 'count': 12}, 'muted': False}}, 'mutes': []}, 'election_epoch': 8, 'quorum': [0, 1, 2], 'quorum_names': ['a', 'b', 'c'], 'quorum_age': 80, 'monmap': {'epoch': 1, 'min_mon_release_name': 'squid', 'num_mons': 3}, 'osdmap': {'epoch': 36, 'num_osds': 3, 'num_up_osds': 2, 'osd_up_since': 1726017129, 'num_in_osds': 2, 'osd_in_since': 1726017134, 'num_remapped_pgs': 0}, 'pgmap': {'pgs_by_state': [{'state_name': 'active+clean', 'count': 9}, {'state_name': 'active+recovery_wait+degraded', 'count': 9}, {'state_name': 'active+recovery_unfound', 'count': 3}, {'state_name': 'active+recovery_unfound+degraded', 'count': 2}, {'state_name': 'active+recovering+degraded', 'count': 1}], 'num_pgs': 24, 'num_pools': 2, 'num_objects': 30, 'data_bytes': 17295, 'bytes_used': 56913920, 'bytes_avail': 193216614400, 'bytes_total': 193273528320, 'degraded_objects': 50, 'degraded_total': 60, 'degraded_ratio': 0.8333333333333335, 'unfound_objects': 27, 'unfound_total': 30, 'unfound_ratio': 0.9}, 'fsmap': {'epoch': 1, 'btime': '2024-09-11T01:11:03:507954+0000', 'by_rank': [], 'up:standby': 0}, 'mgrmap': {'available': True, 'num_standbys': 0, 'modules': ['iostat', 'nfs', 'restful'], 'services': {}}, 'servicemap': {'epoch': 3, 'modified': '2024-09-11T01:12:14.537806+0000', 'services': {'osd': {'daemons': {'summary': '', '0': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}, '2': {'start_epoch': 0, 'start_stamp': '0.000000', 'gid': 0, 'addr': '(unrecognized address family 0)/0', 'metadata': {}, 'task_status': {}}}}}}, 'progress_events': {}} 2024-09-11T01:12:24.296 INFO:tasks.rep_lost_unfound_delete:there are 27 unfound objects 2024-09-11T01:12:24.297 DEBUG:teuthology.orchestra.run.smithi122:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.admin -b 4096 -p unfounddel_pool -t 20 bench 240 write' 2024-09-11T01:12:24.339 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout:hints = 1 2024-09-11T01:12:24.339 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout:Maintaining 20 concurrent writes of 4096 bytes to objects of size 4096 for up to 240 seconds or 0 objects 2024-09-11T01:12:24.340 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout:Object prefix: benchmark_data_smithi122_33997 2024-09-11T01:12:24.340 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T01:12:24.340 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 0 0 0 0 0 0 - 0 2024-09-11T01:12:25.340 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 1 20 2809 2789 10.8939 10.8945 0.0074226 0.00713915 2024-09-11T01:12:26.340 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 2 20 5631 5611 10.9582 11.0234 0.00972686 0.00710818 2024-09-11T01:12:27.340 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 3 20 8118 8098 10.5435 9.71484 0.018472 0.00739716 2024-09-11T01:12:28.340 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 4 20 10621 10601 10.3518 9.77734 0.00421828 0.00752645 2024-09-11T01:12:29.340 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 5 20 12991 12971 10.1329 9.25781 0.0152079 0.00769483 2024-09-11T01:12:30.341 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 6 20 15189 15169 9.87399 8.58594 0.00992482 0.00790131 2024-09-11T01:12:31.341 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 7 20 17402 17382 9.69827 8.64453 0.00863129 0.00804876 2024-09-11T01:12:32.342 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 8 20 19554 19534 9.53548 8.40625 0.0117341 0.00818633 2024-09-11T01:12:33.343 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 9 20 21619 21599 9.37221 8.06641 0.0110847 0.00832922 2024-09-11T01:12:34.299 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:12:34.343 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 10 20 23589 23569 9.20403 7.69531 0.00734695 0.00847968 2024-09-11T01:12:34.784 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:12:34.785 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:12:34.814 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2024-09-11T01:12:34.543522+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":89478031,"num_objects":21871,"num_object_clones":0,"num_object_copies":43742,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":50,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":21871,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21898,"num_write_kb":87412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21899,"ondisk_log_size":21899,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":312624,"kb_used_data":168840,"kb_used_omap":8,"kb_used_meta":143735,"kb_avail":188431056,"statfs":{"total":193273528320,"available":192953401344,"internally_reserved":0,"allocated":172892160,"data_stored":172437575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8875,"internal_metadata":147184981},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,9,10,9,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":89460736,"num_objects":21841,"num_object_clones":0,"num_object_copies":43682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21841,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21841,"num_write_kb":87364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002986"},"pg_stats":[{"pgid":"2.f","version":"36'1385","reported_seq":1436,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:33.694974+0000","last_change":"2024-09-11T01:12:15.141793+0000","last_active":"2024-09-11T01:12:33.694974+0000","last_peered":"2024-09-11T01:12:33.694974+0000","last_clean":"2024-09-11T01:12:33.694974+0000","last_became_active":"2024-09-11T01:12:15.141081+0000","last_became_peered":"2024-09-11T01:12:15.141081+0000","last_unstale":"2024-09-11T01:12:33.694974+0000","last_undegraded":"2024-09-11T01:12:33.694974+0000","last_fullsized":"2024-09-11T01:12:33.694974+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":1385,"log_dups_size":0,"ondisk_log_size":1385,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:18:33.495210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5672960,"num_objects":1385,"num_object_clones":0,"num_object_copies":2770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1385,"num_write_kb":5540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"36'1379","reported_seq":1423,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-11T01:12:33.045040+0000","last_change":"2024-09-11T01:12:24.350643+0000","last_active":"2024-09-11T01:12:33.045040+0000","last_peered":"2024-09-11T01:12:33.045040+0000","last_clean":"2024-09-11T01:11:43.020052+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:12:33.045040+0000","last_undegraded":"2024-09-11T01:12:16.125066+0000","last_fullsized":"2024-09-11T01:12:33.045040+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":1379,"log_dups_size":0,"ondisk_log_size":1379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5633234,"num_objects":1377,"num_object_clones":0,"num_object_copies":2754,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1379,"num_write_kb":5504,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1377}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'1306","reported_seq":1368,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-11T01:12:33.042023+0000","last_change":"2024-09-11T01:12:18.044299+0000","last_active":"2024-09-11T01:12:33.042023+0000","last_peered":"2024-09-11T01:12:33.042023+0000","last_clean":"2024-09-11T01:11:43.936435+0000","last_became_active":"2024-09-11T01:12:16.123243+0000","last_became_peered":"2024-09-11T01:12:16.123243+0000","last_unstale":"2024-09-11T01:12:33.042023+0000","last_undegraded":"2024-09-11T01:12:16.125491+0000","last_fullsized":"2024-09-11T01:12:33.042023+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":1306,"log_dups_size":0,"ondisk_log_size":1306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5337705,"num_objects":1304,"num_object_clones":0,"num_object_copies":2608,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1304,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1306,"num_write_kb":5214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1304}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"36'1298","reported_seq":1348,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:33.044144+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:12:33.044144+0000","last_peered":"2024-09-11T01:12:33.044144+0000","last_clean":"2024-09-11T01:12:33.044144+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:12:33.044144+0000","last_undegraded":"2024-09-11T01:12:33.044144+0000","last_fullsized":"2024-09-11T01:12:33.044144+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":1298,"log_dups_size":0,"ondisk_log_size":1298,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5316608,"num_objects":1298,"num_object_clones":0,"num_object_copies":2596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1298,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1298,"num_write_kb":5192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"36'1411","reported_seq":1470,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:33.694580+0000","last_change":"2024-09-11T01:12:15.135722+0000","last_active":"2024-09-11T01:12:33.694580+0000","last_peered":"2024-09-11T01:12:33.694580+0000","last_clean":"2024-09-11T01:11:42.855496+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:12:33.694580+0000","last_undegraded":"2024-09-11T01:12:15.117592+0000","last_fullsized":"2024-09-11T01:12:33.694580+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":1411,"log_dups_size":0,"ondisk_log_size":1411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5760210,"num_objects":1408,"num_object_clones":0,"num_object_copies":2816,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1411,"num_write_kb":5628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1408}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'1394","reported_seq":1452,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:33.702248+0000","last_change":"2024-09-11T01:12:15.141906+0000","last_active":"2024-09-11T01:12:33.702248+0000","last_peered":"2024-09-11T01:12:33.702248+0000","last_clean":"2024-09-11T01:11:44.266820+0000","last_became_active":"2024-09-11T01:12:15.141377+0000","last_became_peered":"2024-09-11T01:12:15.141377+0000","last_unstale":"2024-09-11T01:12:33.702248+0000","last_undegraded":"2024-09-11T01:12:15.124660+0000","last_fullsized":"2024-09-11T01:12:33.702248+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":1394,"log_dups_size":0,"ondisk_log_size":1394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5698770,"num_objects":1393,"num_object_clones":0,"num_object_copies":2786,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1393,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1394,"num_write_kb":5567,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1393}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'1419","reported_seq":1476,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:33.695270+0000","last_change":"2024-09-11T01:12:15.136057+0000","last_active":"2024-09-11T01:12:33.695270+0000","last_peered":"2024-09-11T01:12:33.695270+0000","last_clean":"2024-09-11T01:11:43.275607+0000","last_became_active":"2024-09-11T01:12:15.135367+0000","last_became_peered":"2024-09-11T01:12:15.135367+0000","last_unstale":"2024-09-11T01:12:33.695270+0000","last_undegraded":"2024-09-11T01:12:15.117856+0000","last_fullsized":"2024-09-11T01:12:33.695270+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":1419,"log_dups_size":0,"ondisk_log_size":1419,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5797074,"num_objects":1417,"num_object_clones":0,"num_object_copies":2834,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1417,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1419,"num_write_kb":5664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1417}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'1427","reported_seq":1488,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:33.693323+0000","last_change":"2024-09-11T01:12:15.136010+0000","last_active":"2024-09-11T01:12:33.693323+0000","last_peered":"2024-09-11T01:12:33.693323+0000","last_clean":"2024-09-11T01:11:32.735868+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:12:33.693323+0000","last_undegraded":"2024-09-11T01:12:15.117050+0000","last_fullsized":"2024-09-11T01:12:33.693323+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":1427,"log_dups_size":0,"ondisk_log_size":1427,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5841513,"num_objects":1427,"num_object_clones":0,"num_object_copies":2854,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1427,"num_write_kb":5705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1427}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.114317+0000","last_change":"2024-09-11T01:12:15.135174+0000","last_active":"2024-09-11T01:12:16.114317+0000","last_peered":"2024-09-11T01:12:16.114317+0000","last_clean":"2024-09-11T01:12:16.114317+0000","last_became_active":"2024-09-11T01:12:15.133597+0000","last_became_peered":"2024-09-11T01:12:15.133597+0000","last_unstale":"2024-09-11T01:12:16.114317+0000","last_undegraded":"2024-09-11T01:12:16.114317+0000","last_fullsized":"2024-09-11T01:12:16.114317+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:29:50.913351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"36'1253","reported_seq":1315,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-11T01:12:33.033782+0000","last_change":"2024-09-11T01:12:18.044248+0000","last_active":"2024-09-11T01:12:33.033782+0000","last_peered":"2024-09-11T01:12:33.033782+0000","last_clean":"2024-09-11T01:11:44.016870+0000","last_became_active":"2024-09-11T01:12:16.122119+0000","last_became_peered":"2024-09-11T01:12:16.122119+0000","last_unstale":"2024-09-11T01:12:33.033782+0000","last_undegraded":"2024-09-11T01:12:16.124562+0000","last_fullsized":"2024-09-11T01:12:33.033782+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":1253,"log_dups_size":0,"ondisk_log_size":1253,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5113042,"num_objects":1250,"num_object_clones":0,"num_object_copies":2500,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1253,"num_write_kb":4996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1250}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.121218+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:12:16.121218+0000","last_peered":"2024-09-11T01:12:16.121218+0000","last_clean":"2024-09-11T01:12:16.121218+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:12:16.121218+0000","last_undegraded":"2024-09-11T01:12:16.121218+0000","last_fullsized":"2024-09-11T01:12:16.121218+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'1399","reported_seq":1450,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:33.686185+0000","last_change":"2024-09-11T01:12:15.141700+0000","last_active":"2024-09-11T01:12:33.686185+0000","last_peered":"2024-09-11T01:12:33.686185+0000","last_clean":"2024-09-11T01:11:44.303167+0000","last_became_active":"2024-09-11T01:12:15.140878+0000","last_became_peered":"2024-09-11T01:12:15.140878+0000","last_unstale":"2024-09-11T01:12:33.686185+0000","last_undegraded":"2024-09-11T01:12:15.122467+0000","last_fullsized":"2024-09-11T01:12:33.686185+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":1399,"log_dups_size":0,"ondisk_log_size":1399,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5706962,"num_objects":1395,"num_object_clones":0,"num_object_copies":2790,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1399,"num_write_kb":5576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1395}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":64,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:16.114348+0000","last_change":"2024-09-11T01:12:15.141439+0000","last_active":"2024-09-11T01:12:16.114348+0000","last_peered":"2024-09-11T01:12:16.114348+0000","last_clean":"2024-09-11T01:11:55.731962+0000","last_became_active":"2024-09-11T01:12:15.139222+0000","last_became_peered":"2024-09-11T01:12:15.139222+0000","last_unstale":"2024-09-11T01:12:16.114348+0000","last_undegraded":"2024-09-11T01:12:15.121340+0000","last_fullsized":"2024-09-11T01:12:16.114348+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'1339","reported_seq":1389,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-11T01:12:32.992040+0000","last_change":"2024-09-11T01:12:24.347263+0000","last_active":"2024-09-11T01:12:32.992040+0000","last_peered":"2024-09-11T01:12:32.992040+0000","last_clean":"2024-09-11T01:11:42.775010+0000","last_became_active":"2024-09-11T01:12:16.120717+0000","last_became_peered":"2024-09-11T01:12:16.120717+0000","last_unstale":"2024-09-11T01:12:32.992040+0000","last_undegraded":"2024-09-11T01:12:16.123531+0000","last_fullsized":"2024-09-11T01:12:32.992040+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":1339,"log_dups_size":0,"ondisk_log_size":1339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5473490,"num_objects":1338,"num_object_clones":0,"num_object_copies":2676,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1339,"num_write_kb":5347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1338}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.122445+0000","last_change":"2024-09-11T01:12:16.118776+0000","last_active":"2024-09-11T01:12:16.122445+0000","last_peered":"2024-09-11T01:12:16.122445+0000","last_clean":"2024-09-11T01:12:16.122445+0000","last_became_active":"2024-09-11T01:12:16.117613+0000","last_became_peered":"2024-09-11T01:12:16.117613+0000","last_unstale":"2024-09-11T01:12:16.122445+0000","last_undegraded":"2024-09-11T01:12:16.122445+0000","last_fullsized":"2024-09-11T01:12:16.122445+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:46.399169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"36'1281","reported_seq":1332,"reported_epoch":36,"state":"active+recovery_unfound+degraded","last_fresh":"2024-09-11T01:12:33.043993+0000","last_change":"2024-09-11T01:12:24.347085+0000","last_active":"2024-09-11T01:12:33.043993+0000","last_peered":"2024-09-11T01:12:33.043993+0000","last_clean":"2024-09-11T01:11:43.440047+0000","last_became_active":"2024-09-11T01:12:16.119017+0000","last_became_peered":"2024-09-11T01:12:16.119017+0000","last_unstale":"2024-09-11T01:12:33.043993+0000","last_undegraded":"2024-09-11T01:12:16.122039+0000","last_fullsized":"2024-09-11T01:12:33.043993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":1281,"log_dups_size":0,"ondisk_log_size":1281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5231826,"num_objects":1279,"num_object_clones":0,"num_object_copies":2558,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1281,"num_write_kb":5111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1279}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'1443","reported_seq":1499,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:33.693522+0000","last_change":"2024-09-11T01:12:15.136135+0000","last_active":"2024-09-11T01:12:33.693522+0000","last_peered":"2024-09-11T01:12:33.693522+0000","last_clean":"2024-09-11T01:11:35.758866+0000","last_became_active":"2024-09-11T01:12:15.134484+0000","last_became_peered":"2024-09-11T01:12:15.134484+0000","last_unstale":"2024-09-11T01:12:33.693522+0000","last_undegraded":"2024-09-11T01:12:15.118487+0000","last_fullsized":"2024-09-11T01:12:33.693522+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":1443,"log_dups_size":0,"ondisk_log_size":1443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5907049,"num_objects":1443,"num_object_clones":0,"num_object_copies":2886,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":1,"num_objects_dirty":1443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1443,"num_write_kb":5769,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1443}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.115043+0000","last_change":"2024-09-11T01:12:15.137159+0000","last_active":"2024-09-11T01:12:16.115043+0000","last_peered":"2024-09-11T01:12:16.115043+0000","last_clean":"2024-09-11T01:12:16.115043+0000","last_became_active":"2024-09-11T01:12:15.136405+0000","last_became_peered":"2024-09-11T01:12:15.136405+0000","last_unstale":"2024-09-11T01:12:16.115043+0000","last_undegraded":"2024-09-11T01:12:16.115043+0000","last_fullsized":"2024-09-11T01:12:16.115043+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:32:43.772100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"36'1369","reported_seq":1424,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:33.700419+0000","last_change":"2024-09-11T01:12:15.141481+0000","last_active":"2024-09-11T01:12:33.700419+0000","last_peered":"2024-09-11T01:12:33.700419+0000","last_clean":"2024-09-11T01:11:44.178063+0000","last_became_active":"2024-09-11T01:12:15.140760+0000","last_became_peered":"2024-09-11T01:12:15.140760+0000","last_unstale":"2024-09-11T01:12:33.700419+0000","last_undegraded":"2024-09-11T01:12:15.124957+0000","last_fullsized":"2024-09-11T01:12:33.700419+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":1369,"log_dups_size":0,"ondisk_log_size":1369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5588795,"num_objects":1367,"num_object_clones":0,"num_object_copies":2734,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":1367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1369,"num_write_kb":5460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1367}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.123536+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:12:16.123536+0000","last_peered":"2024-09-11T01:12:16.123536+0000","last_clean":"2024-09-11T01:12:16.123536+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:12:16.123536+0000","last_undegraded":"2024-09-11T01:12:16.123536+0000","last_fullsized":"2024-09-11T01:12:16.123536+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'1413","reported_seq":1469,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:33.681727+0000","last_change":"2024-09-11T01:12:15.140853+0000","last_active":"2024-09-11T01:12:33.681727+0000","last_peered":"2024-09-11T01:12:33.681727+0000","last_clean":"2024-09-11T01:11:35.758438+0000","last_became_active":"2024-09-11T01:12:15.138823+0000","last_became_peered":"2024-09-11T01:12:15.138823+0000","last_unstale":"2024-09-11T01:12:33.681727+0000","last_undegraded":"2024-09-11T01:12:15.123630+0000","last_fullsized":"2024-09-11T01:12:33.681727+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":1413,"log_dups_size":0,"ondisk_log_size":1413,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5780690,"num_objects":1413,"num_object_clones":0,"num_object_copies":2826,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":2,"num_objects_dirty":1413,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1413,"num_write_kb":5646,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":1413}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.115056+0000","last_change":"2024-09-11T01:12:15.137008+0000","last_active":"2024-09-11T01:12:16.115056+0000","last_peered":"2024-09-11T01:12:16.115056+0000","last_clean":"2024-09-11T01:12:16.115056+0000","last_became_active":"2024-09-11T01:12:15.136222+0000","last_became_peered":"2024-09-11T01:12:15.136222+0000","last_unstale":"2024-09-11T01:12:16.115056+0000","last_undegraded":"2024-09-11T01:12:16.115056+0000","last_fullsized":"2024-09-11T01:12:16.115056+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:07:25.284013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.123820+0000","last_change":"2024-09-11T01:12:16.122836+0000","last_active":"2024-09-11T01:12:16.123820+0000","last_peered":"2024-09-11T01:12:16.123820+0000","last_clean":"2024-09-11T01:12:16.123820+0000","last_became_active":"2024-09-11T01:12:16.122572+0000","last_became_peered":"2024-09-11T01:12:16.122572+0000","last_unstale":"2024-09-11T01:12:16.123820+0000","last_undegraded":"2024-09-11T01:12:16.123820+0000","last_fullsized":"2024-09-11T01:12:16.123820+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:36:29.939442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"36'1380","reported_seq":1438,"reported_epoch":36,"state":"active+recovering+degraded","last_fresh":"2024-09-11T01:12:33.701177+0000","last_change":"2024-09-11T01:12:15.142056+0000","last_active":"2024-09-11T01:12:33.701177+0000","last_peered":"2024-09-11T01:12:33.701177+0000","last_clean":"2024-09-11T01:11:43.771457+0000","last_became_active":"2024-09-11T01:12:15.133897+0000","last_became_peered":"2024-09-11T01:12:15.133897+0000","last_unstale":"2024-09-11T01:12:33.701177+0000","last_undegraded":"2024-09-11T01:12:15.116261+0000","last_fullsized":"2024-09-11T01:12:33.701177+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":1380,"log_dups_size":0,"ondisk_log_size":1380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5618084,"num_objects":1375,"num_object_clones":0,"num_object_copies":2750,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":3,"num_objects_dirty":1375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1380,"num_write_kb":5491,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":1374}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":89478012,"num_objects":21869,"num_object_clones":0,"num_object_copies":43738,"num_objects_missing_on_primary":27,"num_objects_missing":27,"num_objects_degraded":49,"num_objects_misplaced":0,"num_objects_unfound":27,"num_objects_dirty":21869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21896,"num_write_kb":87410,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":177676288,"data_stored":177544086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21896,"ondisk_log_size":21896,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182412,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153220,"kb_used_data":85492,"kb_used_omap":4,"kb_used_meta":67707,"kb_avail":94218620,"statfs":{"total":96636764160,"available":96479866880,"internally_reserved":0,"allocated":87543808,"data_stored":87297082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":69332651},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,5,5,6,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986184,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":159404,"kb_used_data":83348,"kb_used_omap":4,"kb_used_meta":76027,"kb_avail":94212436,"statfs":{"total":96636764160,"available":96473534464,"internally_reserved":0,"allocated":85348352,"data_stored":85140493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":77852330},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,4,5,3,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":91537408,"data_stored":91502618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":17276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86024192,"data_stored":86024192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:12:34.816 INFO:tasks.rep_lost_unfound_delete:no unfound in 2.f 2024-09-11T01:12:34.816 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.e state active+recovery_unfound+degraded 2024-09-11T01:12:34.816 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.e list_unfound '{}' 2024-09-11T01:12:35.090 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-11T01:12:35.091 INFO:teuthology.orchestra.run.smithi122.stdout: "offset": { 2024-09-11T01:12:35.091 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": "", 2024-09-11T01:12:35.091 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "", 2024-09-11T01:12:35.091 INFO:teuthology.orchestra.run.smithi122.stdout: "snapid": 0, 2024-09-11T01:12:35.091 INFO:teuthology.orchestra.run.smithi122.stdout: "hash": 0, 2024-09-11T01:12:35.091 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 0, 2024-09-11T01:12:35.091 INFO:teuthology.orchestra.run.smithi122.stdout: "pool": -9223372036854775808, 2024-09-11T01:12:35.091 INFO:teuthology.orchestra.run.smithi122.stdout: "namespace": "" 2024-09-11T01:12:35.091 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:35.091 INFO:teuthology.orchestra.run.smithi122.stdout: "num_missing": 2, 2024-09-11T01:12:35.091 INFO:teuthology.orchestra.run.smithi122.stdout: "num_unfound": 2, 2024-09-11T01:12:35.091 INFO:teuthology.orchestra.run.smithi122.stdout: "objects": [ 2024-09-11T01:12:35.091 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:12:35.092 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": { 2024-09-11T01:12:35.092 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": "existing_4", 2024-09-11T01:12:35.092 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "", 2024-09-11T01:12:35.092 INFO:teuthology.orchestra.run.smithi122.stdout: "snapid": -2, 2024-09-11T01:12:35.092 INFO:teuthology.orchestra.run.smithi122.stdout: "hash": 1224592654, 2024-09-11T01:12:35.092 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 0, 2024-09-11T01:12:35.092 INFO:teuthology.orchestra.run.smithi122.stdout: "pool": 2, 2024-09-11T01:12:35.092 INFO:teuthology.orchestra.run.smithi122.stdout: "namespace": "" 2024-09-11T01:12:35.092 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:35.092 INFO:teuthology.orchestra.run.smithi122.stdout: "need": "23'3", 2024-09-11T01:12:35.092 INFO:teuthology.orchestra.run.smithi122.stdout: "have": "21'1", 2024-09-11T01:12:35.092 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": "none", 2024-09-11T01:12:35.092 INFO:teuthology.orchestra.run.smithi122.stdout: "clean_regions": "clean_offsets: [(617, 18446744073709550998)], clean_omap: true, new_object: false", 2024-09-11T01:12:35.093 INFO:teuthology.orchestra.run.smithi122.stdout: "locations": [] 2024-09-11T01:12:35.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:35.093 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:12:35.093 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": { 2024-09-11T01:12:35.093 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": "existing_6", 2024-09-11T01:12:35.093 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "", 2024-09-11T01:12:35.093 INFO:teuthology.orchestra.run.smithi122.stdout: "snapid": -2, 2024-09-11T01:12:35.093 INFO:teuthology.orchestra.run.smithi122.stdout: "hash": 3101443614, 2024-09-11T01:12:35.093 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 0, 2024-09-11T01:12:35.093 INFO:teuthology.orchestra.run.smithi122.stdout: "pool": 2, 2024-09-11T01:12:35.093 INFO:teuthology.orchestra.run.smithi122.stdout: "namespace": "" 2024-09-11T01:12:35.093 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:35.093 INFO:teuthology.orchestra.run.smithi122.stdout: "need": "23'4", 2024-09-11T01:12:35.093 INFO:teuthology.orchestra.run.smithi122.stdout: "have": "21'2", 2024-09-11T01:12:35.094 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": "none", 2024-09-11T01:12:35.094 INFO:teuthology.orchestra.run.smithi122.stdout: "clean_regions": "clean_offsets: [(617, 18446744073709550998)], clean_omap: true, new_object: false", 2024-09-11T01:12:35.094 INFO:teuthology.orchestra.run.smithi122.stdout: "locations": [] 2024-09-11T01:12:35.094 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:12:35.094 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-11T01:12:35.094 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "NotRecovering", 2024-09-11T01:12:35.094 INFO:teuthology.orchestra.run.smithi122.stdout: "available_might_have_unfound": true, 2024-09-11T01:12:35.094 INFO:teuthology.orchestra.run.smithi122.stdout: "might_have_unfound": [ 2024-09-11T01:12:35.094 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:12:35.094 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "1", 2024-09-11T01:12:35.094 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "osd is down" 2024-09-11T01:12:35.094 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:12:35.094 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-11T01:12:35.094 INFO:teuthology.orchestra.run.smithi122.stdout: "more": false 2024-09-11T01:12:35.095 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-11T01:12:35.115 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.e on osd.2 2024-09-11T01:12:35.115 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.e mark_unfound_lost delete 2024-09-11T01:12:35.343 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 11 20 25259 25239 8.96034 6.52344 0.0165041 0.00871156 2024-09-11T01:12:35.364 INFO:teuthology.orchestra.run.smithi122.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-11T01:12:35.405 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.d state active+recovery_unfound+degraded 2024-09-11T01:12:35.405 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.d list_unfound '{}' 2024-09-11T01:12:35.664 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-11T01:12:35.664 INFO:teuthology.orchestra.run.smithi122.stdout: "offset": { 2024-09-11T01:12:35.664 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": "", 2024-09-11T01:12:35.664 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "", 2024-09-11T01:12:35.664 INFO:teuthology.orchestra.run.smithi122.stdout: "snapid": 0, 2024-09-11T01:12:35.664 INFO:teuthology.orchestra.run.smithi122.stdout: "hash": 0, 2024-09-11T01:12:35.664 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 0, 2024-09-11T01:12:35.664 INFO:teuthology.orchestra.run.smithi122.stdout: "pool": -9223372036854775808, 2024-09-11T01:12:35.664 INFO:teuthology.orchestra.run.smithi122.stdout: "namespace": "" 2024-09-11T01:12:35.664 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:35.664 INFO:teuthology.orchestra.run.smithi122.stdout: "num_missing": 1, 2024-09-11T01:12:35.664 INFO:teuthology.orchestra.run.smithi122.stdout: "num_unfound": 1, 2024-09-11T01:12:35.665 INFO:teuthology.orchestra.run.smithi122.stdout: "objects": [ 2024-09-11T01:12:35.665 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:12:35.665 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": { 2024-09-11T01:12:35.665 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": "existed_7", 2024-09-11T01:12:35.665 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "", 2024-09-11T01:12:35.665 INFO:teuthology.orchestra.run.smithi122.stdout: "snapid": -2, 2024-09-11T01:12:35.665 INFO:teuthology.orchestra.run.smithi122.stdout: "hash": 2358579165, 2024-09-11T01:12:35.665 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 0, 2024-09-11T01:12:35.665 INFO:teuthology.orchestra.run.smithi122.stdout: "pool": 2, 2024-09-11T01:12:35.665 INFO:teuthology.orchestra.run.smithi122.stdout: "namespace": "" 2024-09-11T01:12:35.665 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:35.665 INFO:teuthology.orchestra.run.smithi122.stdout: "need": "23'3", 2024-09-11T01:12:35.666 INFO:teuthology.orchestra.run.smithi122.stdout: "have": "0'0", 2024-09-11T01:12:35.666 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": "none", 2024-09-11T01:12:35.666 INFO:teuthology.orchestra.run.smithi122.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T01:12:35.666 INFO:teuthology.orchestra.run.smithi122.stdout: "locations": [] 2024-09-11T01:12:35.666 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:12:35.666 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-11T01:12:35.666 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "NotRecovering", 2024-09-11T01:12:35.666 INFO:teuthology.orchestra.run.smithi122.stdout: "available_might_have_unfound": true, 2024-09-11T01:12:35.666 INFO:teuthology.orchestra.run.smithi122.stdout: "might_have_unfound": [ 2024-09-11T01:12:35.666 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:12:35.666 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "1", 2024-09-11T01:12:35.666 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "osd is down" 2024-09-11T01:12:35.666 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:12:35.666 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-11T01:12:35.666 INFO:teuthology.orchestra.run.smithi122.stdout: "more": false 2024-09-11T01:12:35.667 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-11T01:12:35.689 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.d on osd.2 2024-09-11T01:12:35.689 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.d mark_unfound_lost delete 2024-09-11T01:12:35.949 INFO:teuthology.orchestra.run.smithi122.stderr:pg has 1 objects unfound and apparently lost marking 2024-09-11T01:12:35.978 INFO:tasks.rep_lost_unfound_delete:no unfound in 2.c 2024-09-11T01:12:35.978 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.b state active+recovery_wait+degraded 2024-09-11T01:12:35.978 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.b list_unfound '{}' 2024-09-11T01:12:36.194 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-11T01:12:36.195 INFO:teuthology.orchestra.run.smithi122.stdout: "offset": { 2024-09-11T01:12:36.195 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": "", 2024-09-11T01:12:36.195 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "", 2024-09-11T01:12:36.195 INFO:teuthology.orchestra.run.smithi122.stdout: "snapid": 0, 2024-09-11T01:12:36.195 INFO:teuthology.orchestra.run.smithi122.stdout: "hash": 0, 2024-09-11T01:12:36.195 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 0, 2024-09-11T01:12:36.195 INFO:teuthology.orchestra.run.smithi122.stdout: "pool": -9223372036854775808, 2024-09-11T01:12:36.195 INFO:teuthology.orchestra.run.smithi122.stdout: "namespace": "" 2024-09-11T01:12:36.195 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:36.195 INFO:teuthology.orchestra.run.smithi122.stdout: "num_missing": 2, 2024-09-11T01:12:36.195 INFO:teuthology.orchestra.run.smithi122.stdout: "num_unfound": 2, 2024-09-11T01:12:36.195 INFO:teuthology.orchestra.run.smithi122.stdout: "objects": [ 2024-09-11T01:12:36.195 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:12:36.196 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": { 2024-09-11T01:12:36.196 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": "existed_3", 2024-09-11T01:12:36.196 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "", 2024-09-11T01:12:36.196 INFO:teuthology.orchestra.run.smithi122.stdout: "snapid": -2, 2024-09-11T01:12:36.196 INFO:teuthology.orchestra.run.smithi122.stdout: "hash": 943960171, 2024-09-11T01:12:36.196 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 0, 2024-09-11T01:12:36.196 INFO:teuthology.orchestra.run.smithi122.stdout: "pool": 2, 2024-09-11T01:12:36.196 INFO:teuthology.orchestra.run.smithi122.stdout: "namespace": "" 2024-09-11T01:12:36.196 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:36.196 INFO:teuthology.orchestra.run.smithi122.stdout: "need": "23'5", 2024-09-11T01:12:36.196 INFO:teuthology.orchestra.run.smithi122.stdout: "have": "0'0", 2024-09-11T01:12:36.196 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": "none", 2024-09-11T01:12:36.196 INFO:teuthology.orchestra.run.smithi122.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T01:12:36.196 INFO:teuthology.orchestra.run.smithi122.stdout: "locations": [] 2024-09-11T01:12:36.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:36.197 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:12:36.197 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": { 2024-09-11T01:12:36.197 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": "existing_1", 2024-09-11T01:12:36.197 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "", 2024-09-11T01:12:36.197 INFO:teuthology.orchestra.run.smithi122.stdout: "snapid": -2, 2024-09-11T01:12:36.197 INFO:teuthology.orchestra.run.smithi122.stdout: "hash": 2885884443, 2024-09-11T01:12:36.197 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 0, 2024-09-11T01:12:36.197 INFO:teuthology.orchestra.run.smithi122.stdout: "pool": 2, 2024-09-11T01:12:36.197 INFO:teuthology.orchestra.run.smithi122.stdout: "namespace": "" 2024-09-11T01:12:36.197 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:36.197 INFO:teuthology.orchestra.run.smithi122.stdout: "need": "23'4", 2024-09-11T01:12:36.197 INFO:teuthology.orchestra.run.smithi122.stdout: "have": "21'1", 2024-09-11T01:12:36.197 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": "none", 2024-09-11T01:12:36.198 INFO:teuthology.orchestra.run.smithi122.stdout: "clean_regions": "clean_offsets: [(617, 18446744073709550998)], clean_omap: true, new_object: false", 2024-09-11T01:12:36.198 INFO:teuthology.orchestra.run.smithi122.stdout: "locations": [] 2024-09-11T01:12:36.198 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:12:36.198 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-11T01:12:36.198 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "Active", 2024-09-11T01:12:36.198 INFO:teuthology.orchestra.run.smithi122.stdout: "available_might_have_unfound": true, 2024-09-11T01:12:36.198 INFO:teuthology.orchestra.run.smithi122.stdout: "might_have_unfound": [ 2024-09-11T01:12:36.198 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:12:36.198 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "1", 2024-09-11T01:12:36.198 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "osd is down" 2024-09-11T01:12:36.198 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:12:36.198 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-11T01:12:36.198 INFO:teuthology.orchestra.run.smithi122.stdout: "more": false 2024-09-11T01:12:36.198 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-11T01:12:36.221 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.b on osd.0 2024-09-11T01:12:36.221 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.b mark_unfound_lost delete 2024-09-11T01:12:36.343 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 12 20 26809 26789 8.7182 6.05469 0.0079858 0.0089535 2024-09-11T01:12:36.454 INFO:teuthology.orchestra.run.smithi122.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-11T01:12:36.518 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.a state active+recovery_wait+degraded 2024-09-11T01:12:36.519 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.a list_unfound '{}' 2024-09-11T01:12:36.783 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-11T01:12:36.783 INFO:teuthology.orchestra.run.smithi122.stdout: "offset": { 2024-09-11T01:12:36.783 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": "", 2024-09-11T01:12:36.784 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "", 2024-09-11T01:12:36.784 INFO:teuthology.orchestra.run.smithi122.stdout: "snapid": 0, 2024-09-11T01:12:36.784 INFO:teuthology.orchestra.run.smithi122.stdout: "hash": 0, 2024-09-11T01:12:36.784 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 0, 2024-09-11T01:12:36.784 INFO:teuthology.orchestra.run.smithi122.stdout: "pool": -9223372036854775808, 2024-09-11T01:12:36.784 INFO:teuthology.orchestra.run.smithi122.stdout: "namespace": "" 2024-09-11T01:12:36.784 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:36.784 INFO:teuthology.orchestra.run.smithi122.stdout: "num_missing": 2, 2024-09-11T01:12:36.784 INFO:teuthology.orchestra.run.smithi122.stdout: "num_unfound": 2, 2024-09-11T01:12:36.784 INFO:teuthology.orchestra.run.smithi122.stdout: "objects": [ 2024-09-11T01:12:36.784 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:12:36.784 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": { 2024-09-11T01:12:36.784 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": "existing_9", 2024-09-11T01:12:36.785 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "", 2024-09-11T01:12:36.785 INFO:teuthology.orchestra.run.smithi122.stdout: "snapid": -2, 2024-09-11T01:12:36.785 INFO:teuthology.orchestra.run.smithi122.stdout: "hash": 1217469130, 2024-09-11T01:12:36.785 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 0, 2024-09-11T01:12:36.785 INFO:teuthology.orchestra.run.smithi122.stdout: "pool": 2, 2024-09-11T01:12:36.785 INFO:teuthology.orchestra.run.smithi122.stdout: "namespace": "" 2024-09-11T01:12:36.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:36.785 INFO:teuthology.orchestra.run.smithi122.stdout: "need": "23'3", 2024-09-11T01:12:36.785 INFO:teuthology.orchestra.run.smithi122.stdout: "have": "21'1", 2024-09-11T01:12:36.785 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": "none", 2024-09-11T01:12:36.785 INFO:teuthology.orchestra.run.smithi122.stdout: "clean_regions": "clean_offsets: [(617, 18446744073709550998)], clean_omap: true, new_object: false", 2024-09-11T01:12:36.785 INFO:teuthology.orchestra.run.smithi122.stdout: "locations": [] 2024-09-11T01:12:36.785 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:36.786 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:12:36.786 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": { 2024-09-11T01:12:36.786 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": "new_9", 2024-09-11T01:12:36.786 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "", 2024-09-11T01:12:36.786 INFO:teuthology.orchestra.run.smithi122.stdout: "snapid": -2, 2024-09-11T01:12:36.786 INFO:teuthology.orchestra.run.smithi122.stdout: "hash": 170713514, 2024-09-11T01:12:36.786 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 0, 2024-09-11T01:12:36.786 INFO:teuthology.orchestra.run.smithi122.stdout: "pool": 2, 2024-09-11T01:12:36.786 INFO:teuthology.orchestra.run.smithi122.stdout: "namespace": "" 2024-09-11T01:12:36.786 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:36.786 INFO:teuthology.orchestra.run.smithi122.stdout: "need": "23'2", 2024-09-11T01:12:36.786 INFO:teuthology.orchestra.run.smithi122.stdout: "have": "0'0", 2024-09-11T01:12:36.786 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": "none", 2024-09-11T01:12:36.786 INFO:teuthology.orchestra.run.smithi122.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T01:12:36.787 INFO:teuthology.orchestra.run.smithi122.stdout: "locations": [] 2024-09-11T01:12:36.787 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:12:36.787 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-11T01:12:36.787 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "Active", 2024-09-11T01:12:36.787 INFO:teuthology.orchestra.run.smithi122.stdout: "available_might_have_unfound": true, 2024-09-11T01:12:36.787 INFO:teuthology.orchestra.run.smithi122.stdout: "might_have_unfound": [ 2024-09-11T01:12:36.787 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:12:36.787 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "1", 2024-09-11T01:12:36.787 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "osd is down" 2024-09-11T01:12:36.787 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:12:36.787 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-11T01:12:36.787 INFO:teuthology.orchestra.run.smithi122.stdout: "more": false 2024-09-11T01:12:36.787 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-11T01:12:36.810 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.a on osd.0 2024-09-11T01:12:36.811 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.a mark_unfound_lost delete 2024-09-11T01:12:37.093 INFO:teuthology.orchestra.run.smithi122.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-11T01:12:37.149 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.9 state active+recovery_wait+degraded 2024-09-11T01:12:37.150 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.9 list_unfound '{}' 2024-09-11T01:12:37.343 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 13 20 28297 28277 8.49467 5.8125 0.0157987 0.00918723 2024-09-11T01:12:37.446 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-11T01:12:37.446 INFO:teuthology.orchestra.run.smithi122.stdout: "offset": { 2024-09-11T01:12:37.446 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": "", 2024-09-11T01:12:37.446 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "", 2024-09-11T01:12:37.446 INFO:teuthology.orchestra.run.smithi122.stdout: "snapid": 0, 2024-09-11T01:12:37.446 INFO:teuthology.orchestra.run.smithi122.stdout: "hash": 0, 2024-09-11T01:12:37.446 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 0, 2024-09-11T01:12:37.446 INFO:teuthology.orchestra.run.smithi122.stdout: "pool": -9223372036854775808, 2024-09-11T01:12:37.446 INFO:teuthology.orchestra.run.smithi122.stdout: "namespace": "" 2024-09-11T01:12:37.447 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:37.447 INFO:teuthology.orchestra.run.smithi122.stdout: "num_missing": 2, 2024-09-11T01:12:37.447 INFO:teuthology.orchestra.run.smithi122.stdout: "num_unfound": 2, 2024-09-11T01:12:37.447 INFO:teuthology.orchestra.run.smithi122.stdout: "objects": [ 2024-09-11T01:12:37.447 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:12:37.447 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": { 2024-09-11T01:12:37.447 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": "existing_5", 2024-09-11T01:12:37.447 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "", 2024-09-11T01:12:37.447 INFO:teuthology.orchestra.run.smithi122.stdout: "snapid": -2, 2024-09-11T01:12:37.447 INFO:teuthology.orchestra.run.smithi122.stdout: "hash": 2998549577, 2024-09-11T01:12:37.447 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 0, 2024-09-11T01:12:37.447 INFO:teuthology.orchestra.run.smithi122.stdout: "pool": 2, 2024-09-11T01:12:37.448 INFO:teuthology.orchestra.run.smithi122.stdout: "namespace": "" 2024-09-11T01:12:37.448 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:37.448 INFO:teuthology.orchestra.run.smithi122.stdout: "need": "23'4", 2024-09-11T01:12:37.448 INFO:teuthology.orchestra.run.smithi122.stdout: "have": "21'2", 2024-09-11T01:12:37.448 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": "none", 2024-09-11T01:12:37.448 INFO:teuthology.orchestra.run.smithi122.stdout: "clean_regions": "clean_offsets: [(617, 18446744073709550998)], clean_omap: true, new_object: false", 2024-09-11T01:12:37.448 INFO:teuthology.orchestra.run.smithi122.stdout: "locations": [] 2024-09-11T01:12:37.448 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:37.448 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:12:37.448 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": { 2024-09-11T01:12:37.448 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": "existing_2", 2024-09-11T01:12:37.448 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "", 2024-09-11T01:12:37.448 INFO:teuthology.orchestra.run.smithi122.stdout: "snapid": -2, 2024-09-11T01:12:37.448 INFO:teuthology.orchestra.run.smithi122.stdout: "hash": 4032093161, 2024-09-11T01:12:37.449 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 0, 2024-09-11T01:12:37.449 INFO:teuthology.orchestra.run.smithi122.stdout: "pool": 2, 2024-09-11T01:12:37.449 INFO:teuthology.orchestra.run.smithi122.stdout: "namespace": "" 2024-09-11T01:12:37.449 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:37.449 INFO:teuthology.orchestra.run.smithi122.stdout: "need": "23'3", 2024-09-11T01:12:37.449 INFO:teuthology.orchestra.run.smithi122.stdout: "have": "21'1", 2024-09-11T01:12:37.449 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": "none", 2024-09-11T01:12:37.449 INFO:teuthology.orchestra.run.smithi122.stdout: "clean_regions": "clean_offsets: [(617, 18446744073709550998)], clean_omap: true, new_object: false", 2024-09-11T01:12:37.449 INFO:teuthology.orchestra.run.smithi122.stdout: "locations": [] 2024-09-11T01:12:37.449 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:12:37.449 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-11T01:12:37.449 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "Active", 2024-09-11T01:12:37.449 INFO:teuthology.orchestra.run.smithi122.stdout: "available_might_have_unfound": true, 2024-09-11T01:12:37.450 INFO:teuthology.orchestra.run.smithi122.stdout: "might_have_unfound": [ 2024-09-11T01:12:37.450 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:12:37.450 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "1", 2024-09-11T01:12:37.450 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "osd is down" 2024-09-11T01:12:37.450 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:12:37.450 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-11T01:12:37.450 INFO:teuthology.orchestra.run.smithi122.stdout: "more": false 2024-09-11T01:12:37.450 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-11T01:12:37.473 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.9 on osd.0 2024-09-11T01:12:37.474 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.9 mark_unfound_lost delete 2024-09-11T01:12:37.730 INFO:teuthology.orchestra.run.smithi122.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-11T01:12:37.772 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.8 state active+recovery_wait+degraded 2024-09-11T01:12:37.772 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.8 list_unfound '{}' 2024-09-11T01:12:38.121 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-11T01:12:38.121 INFO:teuthology.orchestra.run.smithi122.stdout: "offset": { 2024-09-11T01:12:38.122 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": "", 2024-09-11T01:12:38.122 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "", 2024-09-11T01:12:38.122 INFO:teuthology.orchestra.run.smithi122.stdout: "snapid": 0, 2024-09-11T01:12:38.122 INFO:teuthology.orchestra.run.smithi122.stdout: "hash": 0, 2024-09-11T01:12:38.122 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 0, 2024-09-11T01:12:38.122 INFO:teuthology.orchestra.run.smithi122.stdout: "pool": -9223372036854775808, 2024-09-11T01:12:38.122 INFO:teuthology.orchestra.run.smithi122.stdout: "namespace": "" 2024-09-11T01:12:38.122 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:38.122 INFO:teuthology.orchestra.run.smithi122.stdout: "num_missing": 1, 2024-09-11T01:12:38.122 INFO:teuthology.orchestra.run.smithi122.stdout: "num_unfound": 1, 2024-09-11T01:12:38.122 INFO:teuthology.orchestra.run.smithi122.stdout: "objects": [ 2024-09-11T01:12:38.122 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:12:38.122 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": { 2024-09-11T01:12:38.123 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": "new_8", 2024-09-11T01:12:38.123 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "", 2024-09-11T01:12:38.123 INFO:teuthology.orchestra.run.smithi122.stdout: "snapid": -2, 2024-09-11T01:12:38.123 INFO:teuthology.orchestra.run.smithi122.stdout: "hash": 2730327960, 2024-09-11T01:12:38.123 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 0, 2024-09-11T01:12:38.123 INFO:teuthology.orchestra.run.smithi122.stdout: "pool": 2, 2024-09-11T01:12:38.123 INFO:teuthology.orchestra.run.smithi122.stdout: "namespace": "" 2024-09-11T01:12:38.123 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:38.123 INFO:teuthology.orchestra.run.smithi122.stdout: "need": "23'1", 2024-09-11T01:12:38.123 INFO:teuthology.orchestra.run.smithi122.stdout: "have": "0'0", 2024-09-11T01:12:38.123 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": "none", 2024-09-11T01:12:38.123 INFO:teuthology.orchestra.run.smithi122.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T01:12:38.123 INFO:teuthology.orchestra.run.smithi122.stdout: "locations": [] 2024-09-11T01:12:38.123 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:12:38.124 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-11T01:12:38.124 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "Active", 2024-09-11T01:12:38.124 INFO:teuthology.orchestra.run.smithi122.stdout: "available_might_have_unfound": true, 2024-09-11T01:12:38.124 INFO:teuthology.orchestra.run.smithi122.stdout: "might_have_unfound": [ 2024-09-11T01:12:38.124 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:12:38.124 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "1", 2024-09-11T01:12:38.124 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "osd is down" 2024-09-11T01:12:38.124 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:12:38.124 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-11T01:12:38.124 INFO:teuthology.orchestra.run.smithi122.stdout: "more": false 2024-09-11T01:12:38.124 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-11T01:12:38.188 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.8 on osd.0 2024-09-11T01:12:38.188 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.8 mark_unfound_lost delete 2024-09-11T01:12:38.344 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 14 20 29821 29801 8.31311 5.95312 0.0172212 0.0093901 2024-09-11T01:12:38.510 INFO:teuthology.orchestra.run.smithi122.stderr:pg has 1 objects unfound and apparently lost marking 2024-09-11T01:12:38.533 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.0 2024-09-11T01:12:38.533 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.3 state active+recovery_unfound+degraded 2024-09-11T01:12:38.533 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.3 list_unfound '{}' 2024-09-11T01:12:38.635 INFO:tasks.ceph.mon.a.smithi122.stderr:2024-09-11T01:12:38.629+0000 7f9dc6925640 -1 log_channel(cluster) log [ERR] : Health check update: Possible data damage: 3 pgs recovery_unfound (PG_DAMAGED) 2024-09-11T01:12:38.779 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-11T01:12:38.779 INFO:teuthology.orchestra.run.smithi122.stdout: "offset": { 2024-09-11T01:12:38.779 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": "", 2024-09-11T01:12:38.779 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "", 2024-09-11T01:12:38.780 INFO:teuthology.orchestra.run.smithi122.stdout: "snapid": 0, 2024-09-11T01:12:38.780 INFO:teuthology.orchestra.run.smithi122.stdout: "hash": 0, 2024-09-11T01:12:38.780 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 0, 2024-09-11T01:12:38.780 INFO:teuthology.orchestra.run.smithi122.stdout: "pool": -9223372036854775808, 2024-09-11T01:12:38.780 INFO:teuthology.orchestra.run.smithi122.stdout: "namespace": "" 2024-09-11T01:12:38.780 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:38.780 INFO:teuthology.orchestra.run.smithi122.stdout: "num_missing": 2, 2024-09-11T01:12:38.780 INFO:teuthology.orchestra.run.smithi122.stdout: "num_unfound": 2, 2024-09-11T01:12:38.780 INFO:teuthology.orchestra.run.smithi122.stdout: "objects": [ 2024-09-11T01:12:38.780 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:12:38.780 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": { 2024-09-11T01:12:38.780 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": "existing_8", 2024-09-11T01:12:38.780 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "", 2024-09-11T01:12:38.781 INFO:teuthology.orchestra.run.smithi122.stdout: "snapid": -2, 2024-09-11T01:12:38.781 INFO:teuthology.orchestra.run.smithi122.stdout: "hash": 1901787523, 2024-09-11T01:12:38.781 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 0, 2024-09-11T01:12:38.781 INFO:teuthology.orchestra.run.smithi122.stdout: "pool": 2, 2024-09-11T01:12:38.781 INFO:teuthology.orchestra.run.smithi122.stdout: "namespace": "" 2024-09-11T01:12:38.781 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:38.781 INFO:teuthology.orchestra.run.smithi122.stdout: "need": "23'5", 2024-09-11T01:12:38.781 INFO:teuthology.orchestra.run.smithi122.stdout: "have": "21'3", 2024-09-11T01:12:38.781 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": "none", 2024-09-11T01:12:38.781 INFO:teuthology.orchestra.run.smithi122.stdout: "clean_regions": "clean_offsets: [(617, 18446744073709550998)], clean_omap: true, new_object: false", 2024-09-11T01:12:38.781 INFO:teuthology.orchestra.run.smithi122.stdout: "locations": [] 2024-09-11T01:12:38.781 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:38.782 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:12:38.782 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": { 2024-09-11T01:12:38.782 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": "existed_1", 2024-09-11T01:12:38.782 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "", 2024-09-11T01:12:38.782 INFO:teuthology.orchestra.run.smithi122.stdout: "snapid": -2, 2024-09-11T01:12:38.782 INFO:teuthology.orchestra.run.smithi122.stdout: "hash": 3122379363, 2024-09-11T01:12:38.782 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 0, 2024-09-11T01:12:38.782 INFO:teuthology.orchestra.run.smithi122.stdout: "pool": 2, 2024-09-11T01:12:38.782 INFO:teuthology.orchestra.run.smithi122.stdout: "namespace": "" 2024-09-11T01:12:38.782 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:38.782 INFO:teuthology.orchestra.run.smithi122.stdout: "need": "23'4", 2024-09-11T01:12:38.782 INFO:teuthology.orchestra.run.smithi122.stdout: "have": "0'0", 2024-09-11T01:12:38.782 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": "none", 2024-09-11T01:12:38.782 INFO:teuthology.orchestra.run.smithi122.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T01:12:38.782 INFO:teuthology.orchestra.run.smithi122.stdout: "locations": [] 2024-09-11T01:12:38.783 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:12:38.783 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-11T01:12:38.783 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "NotRecovering", 2024-09-11T01:12:38.783 INFO:teuthology.orchestra.run.smithi122.stdout: "available_might_have_unfound": true, 2024-09-11T01:12:38.783 INFO:teuthology.orchestra.run.smithi122.stdout: "might_have_unfound": [ 2024-09-11T01:12:38.783 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:12:38.783 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "1", 2024-09-11T01:12:38.783 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "osd is down" 2024-09-11T01:12:38.783 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:12:38.783 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-11T01:12:38.783 INFO:teuthology.orchestra.run.smithi122.stdout: "more": false 2024-09-11T01:12:38.783 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-11T01:12:38.845 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.3 on osd.2 2024-09-11T01:12:38.846 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.3 mark_unfound_lost delete 2024-09-11T01:12:39.209 INFO:teuthology.orchestra.run.smithi122.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-11T01:12:39.251 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.1 2024-09-11T01:12:39.251 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.2 state active+recovery_wait+degraded 2024-09-11T01:12:39.252 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.2 list_unfound '{}' 2024-09-11T01:12:39.344 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 15 20 31365 31345 8.16098 6.03125 0.0120994 0.00956551 2024-09-11T01:12:39.503 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-11T01:12:39.504 INFO:teuthology.orchestra.run.smithi122.stdout: "offset": { 2024-09-11T01:12:39.504 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": "", 2024-09-11T01:12:39.504 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "", 2024-09-11T01:12:39.504 INFO:teuthology.orchestra.run.smithi122.stdout: "snapid": 0, 2024-09-11T01:12:39.505 INFO:teuthology.orchestra.run.smithi122.stdout: "hash": 0, 2024-09-11T01:12:39.505 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 0, 2024-09-11T01:12:39.505 INFO:teuthology.orchestra.run.smithi122.stdout: "pool": -9223372036854775808, 2024-09-11T01:12:39.505 INFO:teuthology.orchestra.run.smithi122.stdout: "namespace": "" 2024-09-11T01:12:39.505 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:39.506 INFO:teuthology.orchestra.run.smithi122.stdout: "num_missing": 2, 2024-09-11T01:12:39.506 INFO:teuthology.orchestra.run.smithi122.stdout: "num_unfound": 2, 2024-09-11T01:12:39.506 INFO:teuthology.orchestra.run.smithi122.stdout: "objects": [ 2024-09-11T01:12:39.506 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:12:39.506 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": { 2024-09-11T01:12:39.506 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": "existed_6", 2024-09-11T01:12:39.507 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "", 2024-09-11T01:12:39.507 INFO:teuthology.orchestra.run.smithi122.stdout: "snapid": -2, 2024-09-11T01:12:39.507 INFO:teuthology.orchestra.run.smithi122.stdout: "hash": 1708673474, 2024-09-11T01:12:39.507 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 0, 2024-09-11T01:12:39.507 INFO:teuthology.orchestra.run.smithi122.stdout: "pool": 2, 2024-09-11T01:12:39.508 INFO:teuthology.orchestra.run.smithi122.stdout: "namespace": "" 2024-09-11T01:12:39.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:39.508 INFO:teuthology.orchestra.run.smithi122.stdout: "need": "23'5", 2024-09-11T01:12:39.508 INFO:teuthology.orchestra.run.smithi122.stdout: "have": "0'0", 2024-09-11T01:12:39.508 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": "none", 2024-09-11T01:12:39.508 INFO:teuthology.orchestra.run.smithi122.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T01:12:39.508 INFO:teuthology.orchestra.run.smithi122.stdout: "locations": [] 2024-09-11T01:12:39.508 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:39.508 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:12:39.508 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": { 2024-09-11T01:12:39.509 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": "existed_9", 2024-09-11T01:12:39.509 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "", 2024-09-11T01:12:39.509 INFO:teuthology.orchestra.run.smithi122.stdout: "snapid": -2, 2024-09-11T01:12:39.509 INFO:teuthology.orchestra.run.smithi122.stdout: "hash": 831402034, 2024-09-11T01:12:39.509 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 0, 2024-09-11T01:12:39.509 INFO:teuthology.orchestra.run.smithi122.stdout: "pool": 2, 2024-09-11T01:12:39.509 INFO:teuthology.orchestra.run.smithi122.stdout: "namespace": "" 2024-09-11T01:12:39.509 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:39.509 INFO:teuthology.orchestra.run.smithi122.stdout: "need": "23'6", 2024-09-11T01:12:39.509 INFO:teuthology.orchestra.run.smithi122.stdout: "have": "0'0", 2024-09-11T01:12:39.509 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": "none", 2024-09-11T01:12:39.509 INFO:teuthology.orchestra.run.smithi122.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T01:12:39.509 INFO:teuthology.orchestra.run.smithi122.stdout: "locations": [] 2024-09-11T01:12:39.510 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:12:39.510 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-11T01:12:39.510 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "Active", 2024-09-11T01:12:39.510 INFO:teuthology.orchestra.run.smithi122.stdout: "available_might_have_unfound": true, 2024-09-11T01:12:39.510 INFO:teuthology.orchestra.run.smithi122.stdout: "might_have_unfound": [ 2024-09-11T01:12:39.510 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:12:39.510 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "1", 2024-09-11T01:12:39.510 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "osd is down" 2024-09-11T01:12:39.510 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:12:39.510 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-11T01:12:39.510 INFO:teuthology.orchestra.run.smithi122.stdout: "more": false 2024-09-11T01:12:39.510 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-11T01:12:39.549 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.2 on osd.0 2024-09-11T01:12:39.550 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.2 mark_unfound_lost delete 2024-09-11T01:12:39.898 INFO:teuthology.orchestra.run.smithi122.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-11T01:12:39.938 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.2 2024-09-11T01:12:39.938 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.1 state active+recovery_unfound+degraded 2024-09-11T01:12:39.938 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.1 list_unfound '{}' 2024-09-11T01:12:40.175 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-11T01:12:40.175 INFO:teuthology.orchestra.run.smithi122.stdout: "offset": { 2024-09-11T01:12:40.175 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": "", 2024-09-11T01:12:40.175 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "", 2024-09-11T01:12:40.176 INFO:teuthology.orchestra.run.smithi122.stdout: "snapid": 0, 2024-09-11T01:12:40.176 INFO:teuthology.orchestra.run.smithi122.stdout: "hash": 0, 2024-09-11T01:12:40.176 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 0, 2024-09-11T01:12:40.176 INFO:teuthology.orchestra.run.smithi122.stdout: "pool": -9223372036854775808, 2024-09-11T01:12:40.176 INFO:teuthology.orchestra.run.smithi122.stdout: "namespace": "" 2024-09-11T01:12:40.176 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:40.176 INFO:teuthology.orchestra.run.smithi122.stdout: "num_missing": 2, 2024-09-11T01:12:40.176 INFO:teuthology.orchestra.run.smithi122.stdout: "num_unfound": 2, 2024-09-11T01:12:40.177 INFO:teuthology.orchestra.run.smithi122.stdout: "objects": [ 2024-09-11T01:12:40.177 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:12:40.177 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": { 2024-09-11T01:12:40.177 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": "existing_3", 2024-09-11T01:12:40.177 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "", 2024-09-11T01:12:40.177 INFO:teuthology.orchestra.run.smithi122.stdout: "snapid": -2, 2024-09-11T01:12:40.177 INFO:teuthology.orchestra.run.smithi122.stdout: "hash": 459768081, 2024-09-11T01:12:40.177 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 0, 2024-09-11T01:12:40.177 INFO:teuthology.orchestra.run.smithi122.stdout: "pool": 2, 2024-09-11T01:12:40.177 INFO:teuthology.orchestra.run.smithi122.stdout: "namespace": "" 2024-09-11T01:12:40.177 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:40.177 INFO:teuthology.orchestra.run.smithi122.stdout: "need": "23'3", 2024-09-11T01:12:40.177 INFO:teuthology.orchestra.run.smithi122.stdout: "have": "21'1", 2024-09-11T01:12:40.178 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": "none", 2024-09-11T01:12:40.178 INFO:teuthology.orchestra.run.smithi122.stdout: "clean_regions": "clean_offsets: [(617, 18446744073709550998)], clean_omap: true, new_object: false", 2024-09-11T01:12:40.178 INFO:teuthology.orchestra.run.smithi122.stdout: "locations": [] 2024-09-11T01:12:40.178 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:40.178 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:12:40.178 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": { 2024-09-11T01:12:40.178 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": "new_3", 2024-09-11T01:12:40.178 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "", 2024-09-11T01:12:40.178 INFO:teuthology.orchestra.run.smithi122.stdout: "snapid": -2, 2024-09-11T01:12:40.178 INFO:teuthology.orchestra.run.smithi122.stdout: "hash": 480221105, 2024-09-11T01:12:40.178 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 0, 2024-09-11T01:12:40.178 INFO:teuthology.orchestra.run.smithi122.stdout: "pool": 2, 2024-09-11T01:12:40.178 INFO:teuthology.orchestra.run.smithi122.stdout: "namespace": "" 2024-09-11T01:12:40.179 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:40.179 INFO:teuthology.orchestra.run.smithi122.stdout: "need": "23'2", 2024-09-11T01:12:40.179 INFO:teuthology.orchestra.run.smithi122.stdout: "have": "0'0", 2024-09-11T01:12:40.179 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": "none", 2024-09-11T01:12:40.179 INFO:teuthology.orchestra.run.smithi122.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T01:12:40.179 INFO:teuthology.orchestra.run.smithi122.stdout: "locations": [] 2024-09-11T01:12:40.179 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:12:40.179 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-11T01:12:40.179 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "NotRecovering", 2024-09-11T01:12:40.179 INFO:teuthology.orchestra.run.smithi122.stdout: "available_might_have_unfound": true, 2024-09-11T01:12:40.179 INFO:teuthology.orchestra.run.smithi122.stdout: "might_have_unfound": [ 2024-09-11T01:12:40.179 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:12:40.179 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "1", 2024-09-11T01:12:40.179 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "osd is down" 2024-09-11T01:12:40.180 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:12:40.180 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-11T01:12:40.180 INFO:teuthology.orchestra.run.smithi122.stdout: "more": false 2024-09-11T01:12:40.180 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-11T01:12:40.207 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.1 on osd.2 2024-09-11T01:12:40.207 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.1 mark_unfound_lost delete 2024-09-11T01:12:40.344 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 16 20 32864 32844 8.01687 5.85547 0.0135537 0.00973535 2024-09-11T01:12:40.488 INFO:teuthology.orchestra.run.smithi122.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-11T01:12:40.540 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.3 2024-09-11T01:12:40.540 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.0 state active+recovery_unfound+degraded 2024-09-11T01:12:40.540 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.0 list_unfound '{}' 2024-09-11T01:12:40.806 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-11T01:12:40.806 INFO:teuthology.orchestra.run.smithi122.stdout: "offset": { 2024-09-11T01:12:40.806 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": "", 2024-09-11T01:12:40.807 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "", 2024-09-11T01:12:40.807 INFO:teuthology.orchestra.run.smithi122.stdout: "snapid": 0, 2024-09-11T01:12:40.807 INFO:teuthology.orchestra.run.smithi122.stdout: "hash": 0, 2024-09-11T01:12:40.807 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 0, 2024-09-11T01:12:40.807 INFO:teuthology.orchestra.run.smithi122.stdout: "pool": -9223372036854775808, 2024-09-11T01:12:40.807 INFO:teuthology.orchestra.run.smithi122.stdout: "namespace": "" 2024-09-11T01:12:40.807 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:40.807 INFO:teuthology.orchestra.run.smithi122.stdout: "num_missing": 2, 2024-09-11T01:12:40.807 INFO:teuthology.orchestra.run.smithi122.stdout: "num_unfound": 2, 2024-09-11T01:12:40.807 INFO:teuthology.orchestra.run.smithi122.stdout: "objects": [ 2024-09-11T01:12:40.807 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:12:40.808 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": { 2024-09-11T01:12:40.808 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": "new_5", 2024-09-11T01:12:40.808 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "", 2024-09-11T01:12:40.808 INFO:teuthology.orchestra.run.smithi122.stdout: "snapid": -2, 2024-09-11T01:12:40.808 INFO:teuthology.orchestra.run.smithi122.stdout: "hash": 4248945696, 2024-09-11T01:12:40.808 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 0, 2024-09-11T01:12:40.808 INFO:teuthology.orchestra.run.smithi122.stdout: "pool": 2, 2024-09-11T01:12:40.808 INFO:teuthology.orchestra.run.smithi122.stdout: "namespace": "" 2024-09-11T01:12:40.808 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:40.808 INFO:teuthology.orchestra.run.smithi122.stdout: "need": "23'3", 2024-09-11T01:12:40.808 INFO:teuthology.orchestra.run.smithi122.stdout: "have": "0'0", 2024-09-11T01:12:40.808 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": "none", 2024-09-11T01:12:40.809 INFO:teuthology.orchestra.run.smithi122.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T01:12:40.809 INFO:teuthology.orchestra.run.smithi122.stdout: "locations": [] 2024-09-11T01:12:40.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:40.809 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:12:40.809 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": { 2024-09-11T01:12:40.809 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": "existed_5", 2024-09-11T01:12:40.809 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "", 2024-09-11T01:12:40.809 INFO:teuthology.orchestra.run.smithi122.stdout: "snapid": -2, 2024-09-11T01:12:40.809 INFO:teuthology.orchestra.run.smithi122.stdout: "hash": 4212789968, 2024-09-11T01:12:40.809 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 0, 2024-09-11T01:12:40.809 INFO:teuthology.orchestra.run.smithi122.stdout: "pool": 2, 2024-09-11T01:12:40.809 INFO:teuthology.orchestra.run.smithi122.stdout: "namespace": "" 2024-09-11T01:12:40.809 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:40.810 INFO:teuthology.orchestra.run.smithi122.stdout: "need": "23'4", 2024-09-11T01:12:40.810 INFO:teuthology.orchestra.run.smithi122.stdout: "have": "0'0", 2024-09-11T01:12:40.810 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": "none", 2024-09-11T01:12:40.810 INFO:teuthology.orchestra.run.smithi122.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T01:12:40.810 INFO:teuthology.orchestra.run.smithi122.stdout: "locations": [] 2024-09-11T01:12:40.810 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:12:40.810 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-11T01:12:40.810 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "NotRecovering", 2024-09-11T01:12:40.810 INFO:teuthology.orchestra.run.smithi122.stdout: "available_might_have_unfound": true, 2024-09-11T01:12:40.810 INFO:teuthology.orchestra.run.smithi122.stdout: "might_have_unfound": [ 2024-09-11T01:12:40.810 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:12:40.810 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "1", 2024-09-11T01:12:40.810 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "osd is down" 2024-09-11T01:12:40.810 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:12:40.811 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-11T01:12:40.811 INFO:teuthology.orchestra.run.smithi122.stdout: "more": false 2024-09-11T01:12:40.811 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-11T01:12:40.836 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.0 on osd.2 2024-09-11T01:12:40.836 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.0 mark_unfound_lost delete 2024-09-11T01:12:41.162 INFO:teuthology.orchestra.run.smithi122.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-11T01:12:41.194 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.7 state active+recovery_wait+degraded 2024-09-11T01:12:41.195 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.7 list_unfound '{}' 2024-09-11T01:12:41.344 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 17 19 34244 34225 7.86261 5.39453 0.0106296 0.00992803 2024-09-11T01:12:41.451 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-11T01:12:41.451 INFO:teuthology.orchestra.run.smithi122.stdout: "offset": { 2024-09-11T01:12:41.451 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": "", 2024-09-11T01:12:41.451 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "", 2024-09-11T01:12:41.451 INFO:teuthology.orchestra.run.smithi122.stdout: "snapid": 0, 2024-09-11T01:12:41.451 INFO:teuthology.orchestra.run.smithi122.stdout: "hash": 0, 2024-09-11T01:12:41.451 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 0, 2024-09-11T01:12:41.452 INFO:teuthology.orchestra.run.smithi122.stdout: "pool": -9223372036854775808, 2024-09-11T01:12:41.452 INFO:teuthology.orchestra.run.smithi122.stdout: "namespace": "" 2024-09-11T01:12:41.452 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:41.452 INFO:teuthology.orchestra.run.smithi122.stdout: "num_missing": 1, 2024-09-11T01:12:41.452 INFO:teuthology.orchestra.run.smithi122.stdout: "num_unfound": 1, 2024-09-11T01:12:41.452 INFO:teuthology.orchestra.run.smithi122.stdout: "objects": [ 2024-09-11T01:12:41.452 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:12:41.452 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": { 2024-09-11T01:12:41.452 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": "new_7", 2024-09-11T01:12:41.452 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "", 2024-09-11T01:12:41.452 INFO:teuthology.orchestra.run.smithi122.stdout: "snapid": -2, 2024-09-11T01:12:41.453 INFO:teuthology.orchestra.run.smithi122.stdout: "hash": 3761017527, 2024-09-11T01:12:41.453 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 0, 2024-09-11T01:12:41.453 INFO:teuthology.orchestra.run.smithi122.stdout: "pool": 2, 2024-09-11T01:12:41.453 INFO:teuthology.orchestra.run.smithi122.stdout: "namespace": "" 2024-09-11T01:12:41.453 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:41.453 INFO:teuthology.orchestra.run.smithi122.stdout: "need": "23'1", 2024-09-11T01:12:41.453 INFO:teuthology.orchestra.run.smithi122.stdout: "have": "0'0", 2024-09-11T01:12:41.453 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": "none", 2024-09-11T01:12:41.453 INFO:teuthology.orchestra.run.smithi122.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T01:12:41.453 INFO:teuthology.orchestra.run.smithi122.stdout: "locations": [] 2024-09-11T01:12:41.453 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:12:41.453 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-11T01:12:41.453 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "Active", 2024-09-11T01:12:41.453 INFO:teuthology.orchestra.run.smithi122.stdout: "available_might_have_unfound": true, 2024-09-11T01:12:41.454 INFO:teuthology.orchestra.run.smithi122.stdout: "might_have_unfound": [ 2024-09-11T01:12:41.454 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:12:41.454 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "1", 2024-09-11T01:12:41.454 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "osd is down" 2024-09-11T01:12:41.454 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:12:41.454 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-11T01:12:41.454 INFO:teuthology.orchestra.run.smithi122.stdout: "more": false 2024-09-11T01:12:41.454 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-11T01:12:41.477 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.7 on osd.0 2024-09-11T01:12:41.477 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.7 mark_unfound_lost delete 2024-09-11T01:12:41.754 INFO:teuthology.orchestra.run.smithi122.stderr:pg has 1 objects unfound and apparently lost marking 2024-09-11T01:12:41.795 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.4 2024-09-11T01:12:41.795 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.6 state active+recovery_wait+degraded 2024-09-11T01:12:41.795 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.6 list_unfound '{}' 2024-09-11T01:12:42.023 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-11T01:12:42.023 INFO:teuthology.orchestra.run.smithi122.stdout: "offset": { 2024-09-11T01:12:42.023 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": "", 2024-09-11T01:12:42.023 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "", 2024-09-11T01:12:42.023 INFO:teuthology.orchestra.run.smithi122.stdout: "snapid": 0, 2024-09-11T01:12:42.023 INFO:teuthology.orchestra.run.smithi122.stdout: "hash": 0, 2024-09-11T01:12:42.024 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 0, 2024-09-11T01:12:42.024 INFO:teuthology.orchestra.run.smithi122.stdout: "pool": -9223372036854775808, 2024-09-11T01:12:42.024 INFO:teuthology.orchestra.run.smithi122.stdout: "namespace": "" 2024-09-11T01:12:42.024 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:42.024 INFO:teuthology.orchestra.run.smithi122.stdout: "num_missing": 3, 2024-09-11T01:12:42.024 INFO:teuthology.orchestra.run.smithi122.stdout: "num_unfound": 3, 2024-09-11T01:12:42.024 INFO:teuthology.orchestra.run.smithi122.stdout: "objects": [ 2024-09-11T01:12:42.024 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:12:42.024 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": { 2024-09-11T01:12:42.024 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": "existed_8", 2024-09-11T01:12:42.024 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "", 2024-09-11T01:12:42.024 INFO:teuthology.orchestra.run.smithi122.stdout: "snapid": -2, 2024-09-11T01:12:42.025 INFO:teuthology.orchestra.run.smithi122.stdout: "hash": 1861424966, 2024-09-11T01:12:42.025 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 0, 2024-09-11T01:12:42.025 INFO:teuthology.orchestra.run.smithi122.stdout: "pool": 2, 2024-09-11T01:12:42.025 INFO:teuthology.orchestra.run.smithi122.stdout: "namespace": "" 2024-09-11T01:12:42.025 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:42.025 INFO:teuthology.orchestra.run.smithi122.stdout: "need": "23'5", 2024-09-11T01:12:42.025 INFO:teuthology.orchestra.run.smithi122.stdout: "have": "0'0", 2024-09-11T01:12:42.025 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": "none", 2024-09-11T01:12:42.025 INFO:teuthology.orchestra.run.smithi122.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T01:12:42.025 INFO:teuthology.orchestra.run.smithi122.stdout: "locations": [] 2024-09-11T01:12:42.025 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:42.025 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:12:42.025 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": { 2024-09-11T01:12:42.025 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": "new_6", 2024-09-11T01:12:42.026 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "", 2024-09-11T01:12:42.026 INFO:teuthology.orchestra.run.smithi122.stdout: "snapid": -2, 2024-09-11T01:12:42.026 INFO:teuthology.orchestra.run.smithi122.stdout: "hash": 3910131350, 2024-09-11T01:12:42.026 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 0, 2024-09-11T01:12:42.026 INFO:teuthology.orchestra.run.smithi122.stdout: "pool": 2, 2024-09-11T01:12:42.026 INFO:teuthology.orchestra.run.smithi122.stdout: "namespace": "" 2024-09-11T01:12:42.026 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:42.026 INFO:teuthology.orchestra.run.smithi122.stdout: "need": "23'4", 2024-09-11T01:12:42.026 INFO:teuthology.orchestra.run.smithi122.stdout: "have": "0'0", 2024-09-11T01:12:42.026 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": "none", 2024-09-11T01:12:42.026 INFO:teuthology.orchestra.run.smithi122.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T01:12:42.026 INFO:teuthology.orchestra.run.smithi122.stdout: "locations": [] 2024-09-11T01:12:42.026 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:42.027 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:12:42.027 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": { 2024-09-11T01:12:42.027 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": "new_4", 2024-09-11T01:12:42.027 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "", 2024-09-11T01:12:42.027 INFO:teuthology.orchestra.run.smithi122.stdout: "snapid": -2, 2024-09-11T01:12:42.027 INFO:teuthology.orchestra.run.smithi122.stdout: "hash": 2787839990, 2024-09-11T01:12:42.027 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 0, 2024-09-11T01:12:42.027 INFO:teuthology.orchestra.run.smithi122.stdout: "pool": 2, 2024-09-11T01:12:42.027 INFO:teuthology.orchestra.run.smithi122.stdout: "namespace": "" 2024-09-11T01:12:42.027 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:42.027 INFO:teuthology.orchestra.run.smithi122.stdout: "need": "23'3", 2024-09-11T01:12:42.027 INFO:teuthology.orchestra.run.smithi122.stdout: "have": "0'0", 2024-09-11T01:12:42.027 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": "none", 2024-09-11T01:12:42.028 INFO:teuthology.orchestra.run.smithi122.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T01:12:42.028 INFO:teuthology.orchestra.run.smithi122.stdout: "locations": [] 2024-09-11T01:12:42.028 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:12:42.028 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-11T01:12:42.028 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "Active", 2024-09-11T01:12:42.028 INFO:teuthology.orchestra.run.smithi122.stdout: "available_might_have_unfound": true, 2024-09-11T01:12:42.028 INFO:teuthology.orchestra.run.smithi122.stdout: "might_have_unfound": [ 2024-09-11T01:12:42.028 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:12:42.028 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "1", 2024-09-11T01:12:42.028 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "osd is down" 2024-09-11T01:12:42.028 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:12:42.029 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-11T01:12:42.029 INFO:teuthology.orchestra.run.smithi122.stdout: "more": false 2024-09-11T01:12:42.029 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-11T01:12:42.052 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.6 on osd.0 2024-09-11T01:12:42.052 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.6 mark_unfound_lost delete 2024-09-11T01:12:42.335 INFO:teuthology.orchestra.run.smithi122.stderr:pg has 3 objects unfound and apparently lost marking 2024-09-11T01:12:42.344 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 18 20 35548 35528 7.70846 5.08984 0.0122542 0.0101269 2024-09-11T01:12:42.359 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.5 2024-09-11T01:12:42.359 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.5 state active+recovery_wait+degraded 2024-09-11T01:12:42.359 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.5 list_unfound '{}' 2024-09-11T01:12:42.603 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-11T01:12:42.604 INFO:teuthology.orchestra.run.smithi122.stdout: "offset": { 2024-09-11T01:12:42.604 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": "", 2024-09-11T01:12:42.604 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "", 2024-09-11T01:12:42.604 INFO:teuthology.orchestra.run.smithi122.stdout: "snapid": 0, 2024-09-11T01:12:42.604 INFO:teuthology.orchestra.run.smithi122.stdout: "hash": 0, 2024-09-11T01:12:42.604 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 0, 2024-09-11T01:12:42.604 INFO:teuthology.orchestra.run.smithi122.stdout: "pool": -9223372036854775808, 2024-09-11T01:12:42.604 INFO:teuthology.orchestra.run.smithi122.stdout: "namespace": "" 2024-09-11T01:12:42.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:42.604 INFO:teuthology.orchestra.run.smithi122.stdout: "num_missing": 2, 2024-09-11T01:12:42.604 INFO:teuthology.orchestra.run.smithi122.stdout: "num_unfound": 2, 2024-09-11T01:12:42.604 INFO:teuthology.orchestra.run.smithi122.stdout: "objects": [ 2024-09-11T01:12:42.605 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:12:42.605 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": { 2024-09-11T01:12:42.605 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": "new_1", 2024-09-11T01:12:42.605 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "", 2024-09-11T01:12:42.605 INFO:teuthology.orchestra.run.smithi122.stdout: "snapid": -2, 2024-09-11T01:12:42.605 INFO:teuthology.orchestra.run.smithi122.stdout: "hash": 26281365, 2024-09-11T01:12:42.605 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 0, 2024-09-11T01:12:42.605 INFO:teuthology.orchestra.run.smithi122.stdout: "pool": 2, 2024-09-11T01:12:42.605 INFO:teuthology.orchestra.run.smithi122.stdout: "namespace": "" 2024-09-11T01:12:42.605 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:42.605 INFO:teuthology.orchestra.run.smithi122.stdout: "need": "23'1", 2024-09-11T01:12:42.605 INFO:teuthology.orchestra.run.smithi122.stdout: "have": "0'0", 2024-09-11T01:12:42.605 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": "none", 2024-09-11T01:12:42.605 INFO:teuthology.orchestra.run.smithi122.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T01:12:42.606 INFO:teuthology.orchestra.run.smithi122.stdout: "locations": [] 2024-09-11T01:12:42.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:42.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:12:42.606 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": { 2024-09-11T01:12:42.606 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": "new_2", 2024-09-11T01:12:42.606 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "", 2024-09-11T01:12:42.606 INFO:teuthology.orchestra.run.smithi122.stdout: "snapid": -2, 2024-09-11T01:12:42.606 INFO:teuthology.orchestra.run.smithi122.stdout: "hash": 3196340981, 2024-09-11T01:12:42.606 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 0, 2024-09-11T01:12:42.606 INFO:teuthology.orchestra.run.smithi122.stdout: "pool": 2, 2024-09-11T01:12:42.606 INFO:teuthology.orchestra.run.smithi122.stdout: "namespace": "" 2024-09-11T01:12:42.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:42.606 INFO:teuthology.orchestra.run.smithi122.stdout: "need": "23'2", 2024-09-11T01:12:42.606 INFO:teuthology.orchestra.run.smithi122.stdout: "have": "0'0", 2024-09-11T01:12:42.606 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": "none", 2024-09-11T01:12:42.607 INFO:teuthology.orchestra.run.smithi122.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T01:12:42.607 INFO:teuthology.orchestra.run.smithi122.stdout: "locations": [] 2024-09-11T01:12:42.607 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:12:42.607 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-11T01:12:42.607 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "Active", 2024-09-11T01:12:42.607 INFO:teuthology.orchestra.run.smithi122.stdout: "available_might_have_unfound": true, 2024-09-11T01:12:42.607 INFO:teuthology.orchestra.run.smithi122.stdout: "might_have_unfound": [ 2024-09-11T01:12:42.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:12:42.607 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "1", 2024-09-11T01:12:42.607 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "osd is down" 2024-09-11T01:12:42.607 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:12:42.607 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-11T01:12:42.607 INFO:teuthology.orchestra.run.smithi122.stdout: "more": false 2024-09-11T01:12:42.607 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-11T01:12:42.639 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.5 on osd.0 2024-09-11T01:12:42.640 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.5 mark_unfound_lost delete 2024-09-11T01:12:42.971 INFO:teuthology.orchestra.run.smithi122.stderr:pg has 2 objects unfound and apparently lost marking 2024-09-11T01:12:43.006 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.6 2024-09-11T01:12:43.006 INFO:tasks.rep_lost_unfound_delete:no unfound in 1.7 2024-09-11T01:12:43.006 INFO:tasks.rep_lost_unfound_delete:listing missing/lost in 2.4 state active+recovering+degraded 2024-09-11T01:12:43.006 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- pg 2.4 list_unfound '{}' 2024-09-11T01:12:43.236 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-11T01:12:43.237 INFO:teuthology.orchestra.run.smithi122.stdout: "offset": { 2024-09-11T01:12:43.237 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": "", 2024-09-11T01:12:43.237 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "", 2024-09-11T01:12:43.237 INFO:teuthology.orchestra.run.smithi122.stdout: "snapid": 0, 2024-09-11T01:12:43.237 INFO:teuthology.orchestra.run.smithi122.stdout: "hash": 0, 2024-09-11T01:12:43.237 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 0, 2024-09-11T01:12:43.237 INFO:teuthology.orchestra.run.smithi122.stdout: "pool": -9223372036854775808, 2024-09-11T01:12:43.237 INFO:teuthology.orchestra.run.smithi122.stdout: "namespace": "" 2024-09-11T01:12:43.237 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:43.237 INFO:teuthology.orchestra.run.smithi122.stdout: "num_missing": 4, 2024-09-11T01:12:43.237 INFO:teuthology.orchestra.run.smithi122.stdout: "num_unfound": 3, 2024-09-11T01:12:43.237 INFO:teuthology.orchestra.run.smithi122.stdout: "objects": [ 2024-09-11T01:12:43.238 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:12:43.238 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": { 2024-09-11T01:12:43.238 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": "existed_4", 2024-09-11T01:12:43.238 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "", 2024-09-11T01:12:43.238 INFO:teuthology.orchestra.run.smithi122.stdout: "snapid": -2, 2024-09-11T01:12:43.238 INFO:teuthology.orchestra.run.smithi122.stdout: "hash": 873891716, 2024-09-11T01:12:43.238 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 0, 2024-09-11T01:12:43.238 INFO:teuthology.orchestra.run.smithi122.stdout: "pool": 2, 2024-09-11T01:12:43.238 INFO:teuthology.orchestra.run.smithi122.stdout: "namespace": "" 2024-09-11T01:12:43.238 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:43.238 INFO:teuthology.orchestra.run.smithi122.stdout: "need": "23'8", 2024-09-11T01:12:43.238 INFO:teuthology.orchestra.run.smithi122.stdout: "have": "0'0", 2024-09-11T01:12:43.238 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": "none", 2024-09-11T01:12:43.238 INFO:teuthology.orchestra.run.smithi122.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T01:12:43.239 INFO:teuthology.orchestra.run.smithi122.stdout: "locations": [] 2024-09-11T01:12:43.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:43.239 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:12:43.239 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": { 2024-09-11T01:12:43.239 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": "existing_7", 2024-09-11T01:12:43.239 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "", 2024-09-11T01:12:43.239 INFO:teuthology.orchestra.run.smithi122.stdout: "snapid": -2, 2024-09-11T01:12:43.239 INFO:teuthology.orchestra.run.smithi122.stdout: "hash": 3996155460, 2024-09-11T01:12:43.239 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 0, 2024-09-11T01:12:43.239 INFO:teuthology.orchestra.run.smithi122.stdout: "pool": 2, 2024-09-11T01:12:43.239 INFO:teuthology.orchestra.run.smithi122.stdout: "namespace": "" 2024-09-11T01:12:43.239 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:43.239 INFO:teuthology.orchestra.run.smithi122.stdout: "need": "23'9", 2024-09-11T01:12:43.239 INFO:teuthology.orchestra.run.smithi122.stdout: "have": "21'6", 2024-09-11T01:12:43.240 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": "none", 2024-09-11T01:12:43.240 INFO:teuthology.orchestra.run.smithi122.stdout: "clean_regions": "clean_offsets: [(617, 18446744073709550998)], clean_omap: true, new_object: false", 2024-09-11T01:12:43.240 INFO:teuthology.orchestra.run.smithi122.stdout: "locations": [] 2024-09-11T01:12:43.240 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:43.240 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:12:43.240 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": { 2024-09-11T01:12:43.240 INFO:teuthology.orchestra.run.smithi122.stdout: "oid": "existed_2", 2024-09-11T01:12:43.240 INFO:teuthology.orchestra.run.smithi122.stdout: "key": "", 2024-09-11T01:12:43.240 INFO:teuthology.orchestra.run.smithi122.stdout: "snapid": -2, 2024-09-11T01:12:43.240 INFO:teuthology.orchestra.run.smithi122.stdout: "hash": 1767667508, 2024-09-11T01:12:43.240 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 0, 2024-09-11T01:12:43.240 INFO:teuthology.orchestra.run.smithi122.stdout: "pool": 2, 2024-09-11T01:12:43.240 INFO:teuthology.orchestra.run.smithi122.stdout: "namespace": "" 2024-09-11T01:12:43.240 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:12:43.241 INFO:teuthology.orchestra.run.smithi122.stdout: "need": "23'7", 2024-09-11T01:12:43.241 INFO:teuthology.orchestra.run.smithi122.stdout: "have": "0'0", 2024-09-11T01:12:43.241 INFO:teuthology.orchestra.run.smithi122.stdout: "flags": "none", 2024-09-11T01:12:43.241 INFO:teuthology.orchestra.run.smithi122.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2024-09-11T01:12:43.241 INFO:teuthology.orchestra.run.smithi122.stdout: "locations": [] 2024-09-11T01:12:43.241 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:12:43.241 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-11T01:12:43.241 INFO:teuthology.orchestra.run.smithi122.stdout: "state": "Active", 2024-09-11T01:12:43.241 INFO:teuthology.orchestra.run.smithi122.stdout: "available_might_have_unfound": true, 2024-09-11T01:12:43.241 INFO:teuthology.orchestra.run.smithi122.stdout: "might_have_unfound": [ 2024-09-11T01:12:43.241 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:12:43.241 INFO:teuthology.orchestra.run.smithi122.stdout: "osd": "1", 2024-09-11T01:12:43.241 INFO:teuthology.orchestra.run.smithi122.stdout: "status": "osd is down" 2024-09-11T01:12:43.242 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:12:43.242 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-11T01:12:43.242 INFO:teuthology.orchestra.run.smithi122.stdout: "more": false 2024-09-11T01:12:43.242 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-11T01:12:43.284 INFO:tasks.rep_lost_unfound_delete:reverting unfound in 2.4 on osd.0 2024-09-11T01:12:43.284 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg 2.4 mark_unfound_lost delete 2024-09-11T01:12:43.344 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 19 20 36910 36890 7.58276 5.32031 0.0130634 0.0102935 2024-09-11T01:12:43.608 INFO:teuthology.orchestra.run.smithi122.stderr:pg has 3 objects unfound and apparently lost marking 2024-09-11T01:12:43.643 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 debug kick_recovery_wq 5 2024-09-11T01:12:43.912 INFO:teuthology.orchestra.run.smithi122.stderr:kicking recovery queue. set osd_recovery_delay_start to 5 2024-09-11T01:12:43.943 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 debug kick_recovery_wq 5 2024-09-11T01:12:44.189 INFO:teuthology.orchestra.run.smithi122.stderr:kicking recovery queue. set osd_recovery_delay_start to 5 2024-09-11T01:12:44.217 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-11T01:12:44.218 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-11T01:12:44.345 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout:2024-09-11T01:12:44.346015+0000 min lat: 0.00219431 max lat: 0.0609487 avg lat: 0.0104545 2024-09-11T01:12:44.345 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T01:12:44.345 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 20 20 38263 38243 7.4677 5.28516 0.0107084 0.0104545 2024-09-11T01:12:44.514 INFO:teuthology.orchestra.run.smithi122.stdout:133143986187 2024-09-11T01:12:44.515 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-11T01:12:44.516 INFO:teuthology.orchestra.run.smithi122.stdout:107374182415 2024-09-11T01:12:44.516 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-11T01:12:45.198 INFO:teuthology.orchestra.run.smithi122.stdout:107374182415 2024-09-11T01:12:45.261 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 107374182415 got 107374182415 for osd.0 2024-09-11T01:12:45.261 DEBUG:teuthology.parallel:result is None 2024-09-11T01:12:45.345 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 21 20 39559 39539 7.35316 5.0625 0.0194394 0.0106176 2024-09-11T01:12:45.355 INFO:teuthology.orchestra.run.smithi122.stdout:133143986187 2024-09-11T01:12:45.409 INFO:tasks.rep_lost_unfound_delete.ceph_manager:need seq 133143986187 got 133143986187 for osd.2 2024-09-11T01:12:45.409 DEBUG:teuthology.parallel:result is None 2024-09-11T01:12:45.409 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for recovery to complete 2024-09-11T01:12:45.409 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:12:45.976 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:12:45.977 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:12:46.012 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":80,"stamp":"2024-09-11T01:12:44.548289+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":157269470,"num_objects":38414,"num_object_clones":0,"num_object_copies":76828,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28572,"num_write_kb":114168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38478,"ondisk_log_size":38478,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":523756,"kb_used_data":303628,"kb_used_omap":8,"kb_used_meta":220087,"kb_avail":188219924,"statfs":{"total":193273528320,"available":192737202176,"internally_reserved":0,"allocated":310915072,"data_stored":310474403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8875,"internal_metadata":225369429},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,1,8,14,8,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":112794191,"num_objects":27530,"num_object_clones":0,"num_object_copies":55060,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-27,"num_objects_dirty":27530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17661,"num_write_kb":70704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005222"},"pg_stats":[{"pgid":"2.f","version":"36'2423","reported_seq":2474,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:44.454128+0000","last_change":"2024-09-11T01:12:15.141793+0000","last_active":"2024-09-11T01:12:44.454128+0000","last_peered":"2024-09-11T01:12:44.454128+0000","last_clean":"2024-09-11T01:12:44.454128+0000","last_became_active":"2024-09-11T01:12:15.141081+0000","last_became_peered":"2024-09-11T01:12:15.141081+0000","last_unstale":"2024-09-11T01:12:44.454128+0000","last_undegraded":"2024-09-11T01:12:44.454128+0000","last_fullsized":"2024-09-11T01:12:44.454128+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2423,"log_dups_size":0,"ondisk_log_size":2423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:18:33.495210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9924608,"num_objects":2423,"num_object_clones":0,"num_object_copies":4846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2423,"num_write_kb":9692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"36'2508","reported_seq":2590,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:44.457181+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:12:44.457181+0000","last_peered":"2024-09-11T01:12:44.457181+0000","last_clean":"2024-09-11T01:12:44.457181+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:12:44.457181+0000","last_undegraded":"2024-09-11T01:12:44.457181+0000","last_fullsized":"2024-09-11T01:12:44.457181+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":2508,"log_dups_size":0,"ondisk_log_size":2508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10248192,"num_objects":2502,"num_object_clones":0,"num_object_copies":5004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":747,"num_write_kb":2988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'2384","reported_seq":2480,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:44.432183+0000","last_change":"2024-09-11T01:12:37.612304+0000","last_active":"2024-09-11T01:12:44.432183+0000","last_peered":"2024-09-11T01:12:44.432183+0000","last_clean":"2024-09-11T01:12:44.432183+0000","last_became_active":"2024-09-11T01:12:16.123243+0000","last_became_peered":"2024-09-11T01:12:16.123243+0000","last_unstale":"2024-09-11T01:12:44.432183+0000","last_undegraded":"2024-09-11T01:12:44.432183+0000","last_fullsized":"2024-09-11T01:12:44.432183+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":2384,"log_dups_size":0,"ondisk_log_size":2384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:39.057954+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9748480,"num_objects":2380,"num_object_clones":0,"num_object_copies":4760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":685,"num_write_kb":2740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"36'2356","reported_seq":2406,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:44.445156+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:12:44.445156+0000","last_peered":"2024-09-11T01:12:44.445156+0000","last_clean":"2024-09-11T01:12:44.445156+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:12:44.445156+0000","last_undegraded":"2024-09-11T01:12:44.445156+0000","last_fullsized":"2024-09-11T01:12:44.445156+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2356,"log_dups_size":0,"ondisk_log_size":2356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9650176,"num_objects":2356,"num_object_clones":0,"num_object_copies":4712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2356,"num_write_kb":9424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"36'2384","reported_seq":2441,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:44.444329+0000","last_change":"2024-09-11T01:12:15.135722+0000","last_active":"2024-09-11T01:12:44.444329+0000","last_peered":"2024-09-11T01:12:44.444329+0000","last_clean":"2024-09-11T01:11:42.855496+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:12:44.444329+0000","last_undegraded":"2024-09-11T01:12:15.117592+0000","last_fullsized":"2024-09-11T01:12:44.444329+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2384,"log_dups_size":0,"ondisk_log_size":2384,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9737426,"num_objects":2379,"num_object_clones":0,"num_object_copies":4758,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2382,"num_write_kb":9512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2379}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'2441","reported_seq":2498,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:44.463102+0000","last_change":"2024-09-11T01:12:15.141906+0000","last_active":"2024-09-11T01:12:44.463102+0000","last_peered":"2024-09-11T01:12:44.463102+0000","last_clean":"2024-09-11T01:11:44.266820+0000","last_became_active":"2024-09-11T01:12:15.141377+0000","last_became_peered":"2024-09-11T01:12:15.141377+0000","last_unstale":"2024-09-11T01:12:44.463102+0000","last_undegraded":"2024-09-11T01:12:15.124660+0000","last_fullsized":"2024-09-11T01:12:44.463102+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2441,"log_dups_size":0,"ondisk_log_size":2441,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9979090,"num_objects":2438,"num_object_clones":0,"num_object_copies":4876,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2438,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2439,"num_write_kb":9747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2438}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'2372","reported_seq":2428,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:44.445129+0000","last_change":"2024-09-11T01:12:15.136057+0000","last_active":"2024-09-11T01:12:44.445129+0000","last_peered":"2024-09-11T01:12:44.445129+0000","last_clean":"2024-09-11T01:11:43.275607+0000","last_became_active":"2024-09-11T01:12:15.135367+0000","last_became_peered":"2024-09-11T01:12:15.135367+0000","last_unstale":"2024-09-11T01:12:44.445129+0000","last_undegraded":"2024-09-11T01:12:15.117856+0000","last_fullsized":"2024-09-11T01:12:44.445129+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2372,"log_dups_size":0,"ondisk_log_size":2372,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9692370,"num_objects":2368,"num_object_clones":0,"num_object_copies":4736,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2368,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2370,"num_write_kb":9468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2368}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'2422","reported_seq":2484,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:44.435637+0000","last_change":"2024-09-11T01:12:15.136010+0000","last_active":"2024-09-11T01:12:44.435637+0000","last_peered":"2024-09-11T01:12:44.435637+0000","last_clean":"2024-09-11T01:11:32.735868+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:12:44.435637+0000","last_undegraded":"2024-09-11T01:12:15.117050+0000","last_fullsized":"2024-09-11T01:12:44.435637+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2422,"log_dups_size":0,"ondisk_log_size":2422,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9912937,"num_objects":2421,"num_object_clones":0,"num_object_copies":4842,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2421,"num_write_kb":9681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2421}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.114317+0000","last_change":"2024-09-11T01:12:15.135174+0000","last_active":"2024-09-11T01:12:16.114317+0000","last_peered":"2024-09-11T01:12:16.114317+0000","last_clean":"2024-09-11T01:12:16.114317+0000","last_became_active":"2024-09-11T01:12:15.133597+0000","last_became_peered":"2024-09-11T01:12:15.133597+0000","last_unstale":"2024-09-11T01:12:16.114317+0000","last_undegraded":"2024-09-11T01:12:16.114317+0000","last_fullsized":"2024-09-11T01:12:16.114317+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:29:50.913351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"36'2384","reported_seq":2487,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:44.445270+0000","last_change":"2024-09-11T01:12:40.856584+0000","last_active":"2024-09-11T01:12:44.445270+0000","last_peered":"2024-09-11T01:12:44.445270+0000","last_clean":"2024-09-11T01:12:44.445270+0000","last_became_active":"2024-09-11T01:12:16.122119+0000","last_became_peered":"2024-09-11T01:12:16.122119+0000","last_unstale":"2024-09-11T01:12:44.445270+0000","last_undegraded":"2024-09-11T01:12:44.445270+0000","last_fullsized":"2024-09-11T01:12:44.445270+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":2384,"log_dups_size":0,"ondisk_log_size":2384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:43.522530+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9736192,"num_objects":2377,"num_object_clones":0,"num_object_copies":4754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":355,"num_write_kb":1420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.121218+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:12:16.121218+0000","last_peered":"2024-09-11T01:12:16.121218+0000","last_clean":"2024-09-11T01:12:16.121218+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:12:16.121218+0000","last_undegraded":"2024-09-11T01:12:16.121218+0000","last_fullsized":"2024-09-11T01:12:16.121218+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'2391","reported_seq":2441,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:44.463227+0000","last_change":"2024-09-11T01:12:15.141700+0000","last_active":"2024-09-11T01:12:44.463227+0000","last_peered":"2024-09-11T01:12:44.463227+0000","last_clean":"2024-09-11T01:11:44.303167+0000","last_became_active":"2024-09-11T01:12:15.140878+0000","last_became_peered":"2024-09-11T01:12:15.140878+0000","last_unstale":"2024-09-11T01:12:44.463227+0000","last_undegraded":"2024-09-11T01:12:15.122467+0000","last_fullsized":"2024-09-11T01:12:44.463227+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2391,"log_dups_size":0,"ondisk_log_size":2391,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9762002,"num_objects":2385,"num_object_clones":0,"num_object_copies":4770,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2389,"num_write_kb":9536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2385}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":64,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:16.114348+0000","last_change":"2024-09-11T01:12:15.141439+0000","last_active":"2024-09-11T01:12:16.114348+0000","last_peered":"2024-09-11T01:12:16.114348+0000","last_clean":"2024-09-11T01:11:55.731962+0000","last_became_active":"2024-09-11T01:12:15.139222+0000","last_became_peered":"2024-09-11T01:12:15.139222+0000","last_unstale":"2024-09-11T01:12:16.114348+0000","last_undegraded":"2024-09-11T01:12:15.121340+0000","last_fullsized":"2024-09-11T01:12:16.114348+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'2444","reported_seq":2536,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:44.453935+0000","last_change":"2024-09-11T01:12:41.868636+0000","last_active":"2024-09-11T01:12:44.453935+0000","last_peered":"2024-09-11T01:12:44.453935+0000","last_clean":"2024-09-11T01:12:44.453935+0000","last_became_active":"2024-09-11T01:12:16.120717+0000","last_became_peered":"2024-09-11T01:12:16.120717+0000","last_unstale":"2024-09-11T01:12:44.453935+0000","last_undegraded":"2024-09-11T01:12:44.453935+0000","last_fullsized":"2024-09-11T01:12:44.453935+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":2444,"log_dups_size":0,"ondisk_log_size":2444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:54:47.497574+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9990144,"num_objects":2439,"num_object_clones":0,"num_object_copies":4878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2439,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":261,"num_write_kb":1044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.122445+0000","last_change":"2024-09-11T01:12:16.118776+0000","last_active":"2024-09-11T01:12:16.122445+0000","last_peered":"2024-09-11T01:12:16.122445+0000","last_clean":"2024-09-11T01:12:16.122445+0000","last_became_active":"2024-09-11T01:12:16.117613+0000","last_became_peered":"2024-09-11T01:12:16.117613+0000","last_unstale":"2024-09-11T01:12:16.122445+0000","last_undegraded":"2024-09-11T01:12:16.122445+0000","last_fullsized":"2024-09-11T01:12:16.122445+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:46.399169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"36'2390","reported_seq":2482,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:44.427175+0000","last_change":"2024-09-11T01:12:42.891057+0000","last_active":"2024-09-11T01:12:44.427175+0000","last_peered":"2024-09-11T01:12:44.427175+0000","last_clean":"2024-09-11T01:12:44.427175+0000","last_became_active":"2024-09-11T01:12:16.119017+0000","last_became_peered":"2024-09-11T01:12:16.119017+0000","last_unstale":"2024-09-11T01:12:44.427175+0000","last_undegraded":"2024-09-11T01:12:44.427175+0000","last_fullsized":"2024-09-11T01:12:44.427175+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":2390,"log_dups_size":0,"ondisk_log_size":2390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:20:57.577195+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9764864,"num_objects":2384,"num_object_clones":0,"num_object_copies":4768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":175,"num_write_kb":700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'2433","reported_seq":2489,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:44.446838+0000","last_change":"2024-09-11T01:12:15.136135+0000","last_active":"2024-09-11T01:12:44.446838+0000","last_peered":"2024-09-11T01:12:44.446838+0000","last_clean":"2024-09-11T01:11:35.758866+0000","last_became_active":"2024-09-11T01:12:15.134484+0000","last_became_peered":"2024-09-11T01:12:15.134484+0000","last_unstale":"2024-09-11T01:12:44.446838+0000","last_undegraded":"2024-09-11T01:12:15.118487+0000","last_fullsized":"2024-09-11T01:12:44.446838+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2433,"log_dups_size":0,"ondisk_log_size":2433,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9957993,"num_objects":2432,"num_object_clones":0,"num_object_copies":4864,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2432,"num_write_kb":9725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2432}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.115043+0000","last_change":"2024-09-11T01:12:15.137159+0000","last_active":"2024-09-11T01:12:16.115043+0000","last_peered":"2024-09-11T01:12:16.115043+0000","last_clean":"2024-09-11T01:12:16.115043+0000","last_became_active":"2024-09-11T01:12:15.136405+0000","last_became_peered":"2024-09-11T01:12:15.136405+0000","last_unstale":"2024-09-11T01:12:16.115043+0000","last_undegraded":"2024-09-11T01:12:16.115043+0000","last_fullsized":"2024-09-11T01:12:16.115043+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:32:43.772100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"36'2374","reported_seq":2428,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:44.468001+0000","last_change":"2024-09-11T01:12:15.141481+0000","last_active":"2024-09-11T01:12:44.468001+0000","last_peered":"2024-09-11T01:12:44.468001+0000","last_clean":"2024-09-11T01:11:44.178063+0000","last_became_active":"2024-09-11T01:12:15.140760+0000","last_became_peered":"2024-09-11T01:12:15.140760+0000","last_unstale":"2024-09-11T01:12:44.468001+0000","last_undegraded":"2024-09-11T01:12:15.124957+0000","last_fullsized":"2024-09-11T01:12:44.468001+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2374,"log_dups_size":0,"ondisk_log_size":2374,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9692987,"num_objects":2369,"num_object_clones":0,"num_object_copies":4738,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2371,"num_write_kb":9468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2369}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.123536+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:12:16.123536+0000","last_peered":"2024-09-11T01:12:16.123536+0000","last_clean":"2024-09-11T01:12:16.123536+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:12:16.123536+0000","last_undegraded":"2024-09-11T01:12:16.123536+0000","last_fullsized":"2024-09-11T01:12:16.123536+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'2422","reported_seq":2477,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:44.419230+0000","last_change":"2024-09-11T01:12:15.140853+0000","last_active":"2024-09-11T01:12:44.419230+0000","last_peered":"2024-09-11T01:12:44.419230+0000","last_clean":"2024-09-11T01:11:35.758438+0000","last_became_active":"2024-09-11T01:12:15.138823+0000","last_became_peered":"2024-09-11T01:12:15.138823+0000","last_unstale":"2024-09-11T01:12:44.419230+0000","last_undegraded":"2024-09-11T01:12:15.123630+0000","last_fullsized":"2024-09-11T01:12:44.419230+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2422,"log_dups_size":0,"ondisk_log_size":2422,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9905362,"num_objects":2420,"num_object_clones":0,"num_object_copies":4840,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2420,"num_write_kb":9674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.115056+0000","last_change":"2024-09-11T01:12:15.137008+0000","last_active":"2024-09-11T01:12:16.115056+0000","last_peered":"2024-09-11T01:12:16.115056+0000","last_clean":"2024-09-11T01:12:16.115056+0000","last_became_active":"2024-09-11T01:12:15.136222+0000","last_became_peered":"2024-09-11T01:12:15.136222+0000","last_unstale":"2024-09-11T01:12:16.115056+0000","last_undegraded":"2024-09-11T01:12:16.115056+0000","last_fullsized":"2024-09-11T01:12:16.115056+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:07:25.284013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.123820+0000","last_change":"2024-09-11T01:12:16.122836+0000","last_active":"2024-09-11T01:12:16.123820+0000","last_peered":"2024-09-11T01:12:16.123820+0000","last_clean":"2024-09-11T01:12:16.123820+0000","last_became_active":"2024-09-11T01:12:16.122572+0000","last_became_peered":"2024-09-11T01:12:16.122572+0000","last_unstale":"2024-09-11T01:12:16.123820+0000","last_undegraded":"2024-09-11T01:12:16.123820+0000","last_fullsized":"2024-09-11T01:12:16.123820+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:36:29.939442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"36'2347","reported_seq":2403,"reported_epoch":36,"state":"active+recovering+degraded","last_fresh":"2024-09-11T01:12:44.435287+0000","last_change":"2024-09-11T01:12:15.142056+0000","last_active":"2024-09-11T01:12:44.435287+0000","last_peered":"2024-09-11T01:12:44.435287+0000","last_clean":"2024-09-11T01:11:43.771457+0000","last_became_active":"2024-09-11T01:12:15.133897+0000","last_became_peered":"2024-09-11T01:12:15.133897+0000","last_unstale":"2024-09-11T01:12:44.435287+0000","last_undegraded":"2024-09-11T01:12:15.116261+0000","last_fullsized":"2024-09-11T01:12:44.435287+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2347,"log_dups_size":0,"ondisk_log_size":2347,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9566628,"num_objects":2339,"num_object_clones":0,"num_object_copies":4678,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2344,"num_write_kb":9347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":2338}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":157269451,"num_objects":38412,"num_object_clones":0,"num_object_copies":76824,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":31,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28570,"num_write_kb":114166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":314748928,"data_stored":314630642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38475,"ondisk_log_size":38475,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182415,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":260548,"kb_used_data":152564,"kb_used_omap":4,"kb_used_meta":107963,"kb_avail":94111292,"statfs":{"total":96636764160,"available":96369963008,"internally_reserved":0,"allocated":156225536,"data_stored":155992726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":110554795},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,9,4,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986187,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":263208,"kb_used_data":151064,"kb_used_omap":4,"kb_used_meta":112123,"kb_avail":94108632,"statfs":{"total":96636764160,"available":96367239168,"internally_reserved":0,"allocated":154689536,"data_stored":154481677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":114814634},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1,3,5,4,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":157335552,"data_stored":157314678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":17276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":157298688,"data_stored":157298688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:12:46.013 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:12:46.345 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 22 20 40873 40853 7.25223 5.13281 0.0132496 0.0107655 2024-09-11T01:12:46.599 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:12:46.600 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:12:46.636 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2024-09-11T01:12:46.548751+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":157269470,"num_objects":38414,"num_object_clones":0,"num_object_copies":76828,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28572,"num_write_kb":114168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38478,"ondisk_log_size":38478,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":523756,"kb_used_data":303628,"kb_used_omap":8,"kb_used_meta":220087,"kb_avail":188219924,"statfs":{"total":193273528320,"available":192737202176,"internally_reserved":0,"allocated":310915072,"data_stored":310474403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8875,"internal_metadata":225369429},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,1,8,14,8,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":67791439,"num_objects":16543,"num_object_clones":0,"num_object_copies":33086,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-27,"num_objects_dirty":16543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6674,"num_write_kb":26756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005229"},"pg_stats":[{"pgid":"2.f","version":"36'2423","reported_seq":2474,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:44.454128+0000","last_change":"2024-09-11T01:12:15.141793+0000","last_active":"2024-09-11T01:12:44.454128+0000","last_peered":"2024-09-11T01:12:44.454128+0000","last_clean":"2024-09-11T01:12:44.454128+0000","last_became_active":"2024-09-11T01:12:15.141081+0000","last_became_peered":"2024-09-11T01:12:15.141081+0000","last_unstale":"2024-09-11T01:12:44.454128+0000","last_undegraded":"2024-09-11T01:12:44.454128+0000","last_fullsized":"2024-09-11T01:12:44.454128+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2423,"log_dups_size":0,"ondisk_log_size":2423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:18:33.495210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9924608,"num_objects":2423,"num_object_clones":0,"num_object_copies":4846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2423,"num_write_kb":9692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"36'2508","reported_seq":2590,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:44.457181+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:12:44.457181+0000","last_peered":"2024-09-11T01:12:44.457181+0000","last_clean":"2024-09-11T01:12:44.457181+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:12:44.457181+0000","last_undegraded":"2024-09-11T01:12:44.457181+0000","last_fullsized":"2024-09-11T01:12:44.457181+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":2508,"log_dups_size":0,"ondisk_log_size":2508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10248192,"num_objects":2502,"num_object_clones":0,"num_object_copies":5004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":747,"num_write_kb":2988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'2384","reported_seq":2480,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:44.432183+0000","last_change":"2024-09-11T01:12:37.612304+0000","last_active":"2024-09-11T01:12:44.432183+0000","last_peered":"2024-09-11T01:12:44.432183+0000","last_clean":"2024-09-11T01:12:44.432183+0000","last_became_active":"2024-09-11T01:12:16.123243+0000","last_became_peered":"2024-09-11T01:12:16.123243+0000","last_unstale":"2024-09-11T01:12:44.432183+0000","last_undegraded":"2024-09-11T01:12:44.432183+0000","last_fullsized":"2024-09-11T01:12:44.432183+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":2384,"log_dups_size":0,"ondisk_log_size":2384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:39.057954+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9748480,"num_objects":2380,"num_object_clones":0,"num_object_copies":4760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":685,"num_write_kb":2740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"36'2356","reported_seq":2406,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:44.445156+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:12:44.445156+0000","last_peered":"2024-09-11T01:12:44.445156+0000","last_clean":"2024-09-11T01:12:44.445156+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:12:44.445156+0000","last_undegraded":"2024-09-11T01:12:44.445156+0000","last_fullsized":"2024-09-11T01:12:44.445156+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2356,"log_dups_size":0,"ondisk_log_size":2356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9650176,"num_objects":2356,"num_object_clones":0,"num_object_copies":4712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2356,"num_write_kb":9424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"36'2384","reported_seq":2441,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:44.444329+0000","last_change":"2024-09-11T01:12:15.135722+0000","last_active":"2024-09-11T01:12:44.444329+0000","last_peered":"2024-09-11T01:12:44.444329+0000","last_clean":"2024-09-11T01:11:42.855496+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:12:44.444329+0000","last_undegraded":"2024-09-11T01:12:15.117592+0000","last_fullsized":"2024-09-11T01:12:44.444329+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2384,"log_dups_size":0,"ondisk_log_size":2384,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9737426,"num_objects":2379,"num_object_clones":0,"num_object_copies":4758,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2382,"num_write_kb":9512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2379}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'2441","reported_seq":2498,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:44.463102+0000","last_change":"2024-09-11T01:12:15.141906+0000","last_active":"2024-09-11T01:12:44.463102+0000","last_peered":"2024-09-11T01:12:44.463102+0000","last_clean":"2024-09-11T01:11:44.266820+0000","last_became_active":"2024-09-11T01:12:15.141377+0000","last_became_peered":"2024-09-11T01:12:15.141377+0000","last_unstale":"2024-09-11T01:12:44.463102+0000","last_undegraded":"2024-09-11T01:12:15.124660+0000","last_fullsized":"2024-09-11T01:12:44.463102+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2441,"log_dups_size":0,"ondisk_log_size":2441,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9979090,"num_objects":2438,"num_object_clones":0,"num_object_copies":4876,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2438,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2439,"num_write_kb":9747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2438}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'2372","reported_seq":2428,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:44.445129+0000","last_change":"2024-09-11T01:12:15.136057+0000","last_active":"2024-09-11T01:12:44.445129+0000","last_peered":"2024-09-11T01:12:44.445129+0000","last_clean":"2024-09-11T01:11:43.275607+0000","last_became_active":"2024-09-11T01:12:15.135367+0000","last_became_peered":"2024-09-11T01:12:15.135367+0000","last_unstale":"2024-09-11T01:12:44.445129+0000","last_undegraded":"2024-09-11T01:12:15.117856+0000","last_fullsized":"2024-09-11T01:12:44.445129+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2372,"log_dups_size":0,"ondisk_log_size":2372,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9692370,"num_objects":2368,"num_object_clones":0,"num_object_copies":4736,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2368,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2370,"num_write_kb":9468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2368}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'2422","reported_seq":2484,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:44.435637+0000","last_change":"2024-09-11T01:12:15.136010+0000","last_active":"2024-09-11T01:12:44.435637+0000","last_peered":"2024-09-11T01:12:44.435637+0000","last_clean":"2024-09-11T01:11:32.735868+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:12:44.435637+0000","last_undegraded":"2024-09-11T01:12:15.117050+0000","last_fullsized":"2024-09-11T01:12:44.435637+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2422,"log_dups_size":0,"ondisk_log_size":2422,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9912937,"num_objects":2421,"num_object_clones":0,"num_object_copies":4842,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2421,"num_write_kb":9681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2421}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.114317+0000","last_change":"2024-09-11T01:12:15.135174+0000","last_active":"2024-09-11T01:12:16.114317+0000","last_peered":"2024-09-11T01:12:16.114317+0000","last_clean":"2024-09-11T01:12:16.114317+0000","last_became_active":"2024-09-11T01:12:15.133597+0000","last_became_peered":"2024-09-11T01:12:15.133597+0000","last_unstale":"2024-09-11T01:12:16.114317+0000","last_undegraded":"2024-09-11T01:12:16.114317+0000","last_fullsized":"2024-09-11T01:12:16.114317+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:29:50.913351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"36'2384","reported_seq":2487,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:44.445270+0000","last_change":"2024-09-11T01:12:40.856584+0000","last_active":"2024-09-11T01:12:44.445270+0000","last_peered":"2024-09-11T01:12:44.445270+0000","last_clean":"2024-09-11T01:12:44.445270+0000","last_became_active":"2024-09-11T01:12:16.122119+0000","last_became_peered":"2024-09-11T01:12:16.122119+0000","last_unstale":"2024-09-11T01:12:44.445270+0000","last_undegraded":"2024-09-11T01:12:44.445270+0000","last_fullsized":"2024-09-11T01:12:44.445270+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":2384,"log_dups_size":0,"ondisk_log_size":2384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:43.522530+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9736192,"num_objects":2377,"num_object_clones":0,"num_object_copies":4754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":355,"num_write_kb":1420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.121218+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:12:16.121218+0000","last_peered":"2024-09-11T01:12:16.121218+0000","last_clean":"2024-09-11T01:12:16.121218+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:12:16.121218+0000","last_undegraded":"2024-09-11T01:12:16.121218+0000","last_fullsized":"2024-09-11T01:12:16.121218+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'2391","reported_seq":2441,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:44.463227+0000","last_change":"2024-09-11T01:12:15.141700+0000","last_active":"2024-09-11T01:12:44.463227+0000","last_peered":"2024-09-11T01:12:44.463227+0000","last_clean":"2024-09-11T01:11:44.303167+0000","last_became_active":"2024-09-11T01:12:15.140878+0000","last_became_peered":"2024-09-11T01:12:15.140878+0000","last_unstale":"2024-09-11T01:12:44.463227+0000","last_undegraded":"2024-09-11T01:12:15.122467+0000","last_fullsized":"2024-09-11T01:12:44.463227+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2391,"log_dups_size":0,"ondisk_log_size":2391,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9762002,"num_objects":2385,"num_object_clones":0,"num_object_copies":4770,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2389,"num_write_kb":9536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2385}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":64,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:16.114348+0000","last_change":"2024-09-11T01:12:15.141439+0000","last_active":"2024-09-11T01:12:16.114348+0000","last_peered":"2024-09-11T01:12:16.114348+0000","last_clean":"2024-09-11T01:11:55.731962+0000","last_became_active":"2024-09-11T01:12:15.139222+0000","last_became_peered":"2024-09-11T01:12:15.139222+0000","last_unstale":"2024-09-11T01:12:16.114348+0000","last_undegraded":"2024-09-11T01:12:15.121340+0000","last_fullsized":"2024-09-11T01:12:16.114348+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'2444","reported_seq":2536,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:44.453935+0000","last_change":"2024-09-11T01:12:41.868636+0000","last_active":"2024-09-11T01:12:44.453935+0000","last_peered":"2024-09-11T01:12:44.453935+0000","last_clean":"2024-09-11T01:12:44.453935+0000","last_became_active":"2024-09-11T01:12:16.120717+0000","last_became_peered":"2024-09-11T01:12:16.120717+0000","last_unstale":"2024-09-11T01:12:44.453935+0000","last_undegraded":"2024-09-11T01:12:44.453935+0000","last_fullsized":"2024-09-11T01:12:44.453935+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":2444,"log_dups_size":0,"ondisk_log_size":2444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:54:47.497574+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9990144,"num_objects":2439,"num_object_clones":0,"num_object_copies":4878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2439,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":261,"num_write_kb":1044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.122445+0000","last_change":"2024-09-11T01:12:16.118776+0000","last_active":"2024-09-11T01:12:16.122445+0000","last_peered":"2024-09-11T01:12:16.122445+0000","last_clean":"2024-09-11T01:12:16.122445+0000","last_became_active":"2024-09-11T01:12:16.117613+0000","last_became_peered":"2024-09-11T01:12:16.117613+0000","last_unstale":"2024-09-11T01:12:16.122445+0000","last_undegraded":"2024-09-11T01:12:16.122445+0000","last_fullsized":"2024-09-11T01:12:16.122445+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:46.399169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"36'2390","reported_seq":2482,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:44.427175+0000","last_change":"2024-09-11T01:12:42.891057+0000","last_active":"2024-09-11T01:12:44.427175+0000","last_peered":"2024-09-11T01:12:44.427175+0000","last_clean":"2024-09-11T01:12:44.427175+0000","last_became_active":"2024-09-11T01:12:16.119017+0000","last_became_peered":"2024-09-11T01:12:16.119017+0000","last_unstale":"2024-09-11T01:12:44.427175+0000","last_undegraded":"2024-09-11T01:12:44.427175+0000","last_fullsized":"2024-09-11T01:12:44.427175+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":2390,"log_dups_size":0,"ondisk_log_size":2390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:20:57.577195+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9764864,"num_objects":2384,"num_object_clones":0,"num_object_copies":4768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":175,"num_write_kb":700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'2433","reported_seq":2489,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:44.446838+0000","last_change":"2024-09-11T01:12:15.136135+0000","last_active":"2024-09-11T01:12:44.446838+0000","last_peered":"2024-09-11T01:12:44.446838+0000","last_clean":"2024-09-11T01:11:35.758866+0000","last_became_active":"2024-09-11T01:12:15.134484+0000","last_became_peered":"2024-09-11T01:12:15.134484+0000","last_unstale":"2024-09-11T01:12:44.446838+0000","last_undegraded":"2024-09-11T01:12:15.118487+0000","last_fullsized":"2024-09-11T01:12:44.446838+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2433,"log_dups_size":0,"ondisk_log_size":2433,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9957993,"num_objects":2432,"num_object_clones":0,"num_object_copies":4864,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2432,"num_write_kb":9725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2432}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.115043+0000","last_change":"2024-09-11T01:12:15.137159+0000","last_active":"2024-09-11T01:12:16.115043+0000","last_peered":"2024-09-11T01:12:16.115043+0000","last_clean":"2024-09-11T01:12:16.115043+0000","last_became_active":"2024-09-11T01:12:15.136405+0000","last_became_peered":"2024-09-11T01:12:15.136405+0000","last_unstale":"2024-09-11T01:12:16.115043+0000","last_undegraded":"2024-09-11T01:12:16.115043+0000","last_fullsized":"2024-09-11T01:12:16.115043+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:32:43.772100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"36'2374","reported_seq":2428,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:44.468001+0000","last_change":"2024-09-11T01:12:15.141481+0000","last_active":"2024-09-11T01:12:44.468001+0000","last_peered":"2024-09-11T01:12:44.468001+0000","last_clean":"2024-09-11T01:11:44.178063+0000","last_became_active":"2024-09-11T01:12:15.140760+0000","last_became_peered":"2024-09-11T01:12:15.140760+0000","last_unstale":"2024-09-11T01:12:44.468001+0000","last_undegraded":"2024-09-11T01:12:15.124957+0000","last_fullsized":"2024-09-11T01:12:44.468001+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2374,"log_dups_size":0,"ondisk_log_size":2374,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9692987,"num_objects":2369,"num_object_clones":0,"num_object_copies":4738,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2371,"num_write_kb":9468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2369}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.123536+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:12:16.123536+0000","last_peered":"2024-09-11T01:12:16.123536+0000","last_clean":"2024-09-11T01:12:16.123536+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:12:16.123536+0000","last_undegraded":"2024-09-11T01:12:16.123536+0000","last_fullsized":"2024-09-11T01:12:16.123536+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'2422","reported_seq":2477,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:44.419230+0000","last_change":"2024-09-11T01:12:15.140853+0000","last_active":"2024-09-11T01:12:44.419230+0000","last_peered":"2024-09-11T01:12:44.419230+0000","last_clean":"2024-09-11T01:11:35.758438+0000","last_became_active":"2024-09-11T01:12:15.138823+0000","last_became_peered":"2024-09-11T01:12:15.138823+0000","last_unstale":"2024-09-11T01:12:44.419230+0000","last_undegraded":"2024-09-11T01:12:15.123630+0000","last_fullsized":"2024-09-11T01:12:44.419230+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2422,"log_dups_size":0,"ondisk_log_size":2422,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9905362,"num_objects":2420,"num_object_clones":0,"num_object_copies":4840,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2420,"num_write_kb":9674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.115056+0000","last_change":"2024-09-11T01:12:15.137008+0000","last_active":"2024-09-11T01:12:16.115056+0000","last_peered":"2024-09-11T01:12:16.115056+0000","last_clean":"2024-09-11T01:12:16.115056+0000","last_became_active":"2024-09-11T01:12:15.136222+0000","last_became_peered":"2024-09-11T01:12:15.136222+0000","last_unstale":"2024-09-11T01:12:16.115056+0000","last_undegraded":"2024-09-11T01:12:16.115056+0000","last_fullsized":"2024-09-11T01:12:16.115056+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:07:25.284013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.123820+0000","last_change":"2024-09-11T01:12:16.122836+0000","last_active":"2024-09-11T01:12:16.123820+0000","last_peered":"2024-09-11T01:12:16.123820+0000","last_clean":"2024-09-11T01:12:16.123820+0000","last_became_active":"2024-09-11T01:12:16.122572+0000","last_became_peered":"2024-09-11T01:12:16.122572+0000","last_unstale":"2024-09-11T01:12:16.123820+0000","last_undegraded":"2024-09-11T01:12:16.123820+0000","last_fullsized":"2024-09-11T01:12:16.123820+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:36:29.939442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"36'2347","reported_seq":2403,"reported_epoch":36,"state":"active+recovering+degraded","last_fresh":"2024-09-11T01:12:44.435287+0000","last_change":"2024-09-11T01:12:15.142056+0000","last_active":"2024-09-11T01:12:44.435287+0000","last_peered":"2024-09-11T01:12:44.435287+0000","last_clean":"2024-09-11T01:11:43.771457+0000","last_became_active":"2024-09-11T01:12:15.133897+0000","last_became_peered":"2024-09-11T01:12:15.133897+0000","last_unstale":"2024-09-11T01:12:44.435287+0000","last_undegraded":"2024-09-11T01:12:15.116261+0000","last_fullsized":"2024-09-11T01:12:44.435287+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2347,"log_dups_size":0,"ondisk_log_size":2347,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9566628,"num_objects":2339,"num_object_clones":0,"num_object_copies":4678,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2344,"num_write_kb":9347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":2338}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":157269451,"num_objects":38412,"num_object_clones":0,"num_object_copies":76824,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":31,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28570,"num_write_kb":114166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":314748928,"data_stored":314630642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38475,"ondisk_log_size":38475,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182415,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":260548,"kb_used_data":152564,"kb_used_omap":4,"kb_used_meta":107963,"kb_avail":94111292,"statfs":{"total":96636764160,"available":96369963008,"internally_reserved":0,"allocated":156225536,"data_stored":155992726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":110554795},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,9,4,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986187,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":263208,"kb_used_data":151064,"kb_used_omap":4,"kb_used_meta":112123,"kb_avail":94108632,"statfs":{"total":96636764160,"available":96367239168,"internally_reserved":0,"allocated":154689536,"data_stored":154481677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":114814634},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1,3,5,4,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":157335552,"data_stored":157314678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":17276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":157298688,"data_stored":157298688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:12:46.638 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:12:47.241 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:12:47.242 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:12:47.270 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2024-09-11T01:12:46.548751+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":157269470,"num_objects":38414,"num_object_clones":0,"num_object_copies":76828,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28572,"num_write_kb":114168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38478,"ondisk_log_size":38478,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":523756,"kb_used_data":303628,"kb_used_omap":8,"kb_used_meta":220087,"kb_avail":188219924,"statfs":{"total":193273528320,"available":192737202176,"internally_reserved":0,"allocated":310915072,"data_stored":310474403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8875,"internal_metadata":225369429},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,1,8,14,8,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":67791439,"num_objects":16543,"num_object_clones":0,"num_object_copies":33086,"num_objects_missing_on_primary":-9,"num_objects_missing":-9,"num_objects_degraded":-18,"num_objects_misplaced":0,"num_objects_unfound":-27,"num_objects_dirty":16543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6674,"num_write_kb":26756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005229"},"pg_stats":[{"pgid":"2.f","version":"36'2423","reported_seq":2474,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:44.454128+0000","last_change":"2024-09-11T01:12:15.141793+0000","last_active":"2024-09-11T01:12:44.454128+0000","last_peered":"2024-09-11T01:12:44.454128+0000","last_clean":"2024-09-11T01:12:44.454128+0000","last_became_active":"2024-09-11T01:12:15.141081+0000","last_became_peered":"2024-09-11T01:12:15.141081+0000","last_unstale":"2024-09-11T01:12:44.454128+0000","last_undegraded":"2024-09-11T01:12:44.454128+0000","last_fullsized":"2024-09-11T01:12:44.454128+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2423,"log_dups_size":0,"ondisk_log_size":2423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:18:33.495210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9924608,"num_objects":2423,"num_object_clones":0,"num_object_copies":4846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2423,"num_write_kb":9692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"36'2508","reported_seq":2590,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:44.457181+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:12:44.457181+0000","last_peered":"2024-09-11T01:12:44.457181+0000","last_clean":"2024-09-11T01:12:44.457181+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:12:44.457181+0000","last_undegraded":"2024-09-11T01:12:44.457181+0000","last_fullsized":"2024-09-11T01:12:44.457181+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":2508,"log_dups_size":0,"ondisk_log_size":2508,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10248192,"num_objects":2502,"num_object_clones":0,"num_object_copies":5004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":747,"num_write_kb":2988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'2384","reported_seq":2480,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:44.432183+0000","last_change":"2024-09-11T01:12:37.612304+0000","last_active":"2024-09-11T01:12:44.432183+0000","last_peered":"2024-09-11T01:12:44.432183+0000","last_clean":"2024-09-11T01:12:44.432183+0000","last_became_active":"2024-09-11T01:12:16.123243+0000","last_became_peered":"2024-09-11T01:12:16.123243+0000","last_unstale":"2024-09-11T01:12:44.432183+0000","last_undegraded":"2024-09-11T01:12:44.432183+0000","last_fullsized":"2024-09-11T01:12:44.432183+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":2384,"log_dups_size":0,"ondisk_log_size":2384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:39.057954+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9748480,"num_objects":2380,"num_object_clones":0,"num_object_copies":4760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":685,"num_write_kb":2740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"36'2356","reported_seq":2406,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:44.445156+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:12:44.445156+0000","last_peered":"2024-09-11T01:12:44.445156+0000","last_clean":"2024-09-11T01:12:44.445156+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:12:44.445156+0000","last_undegraded":"2024-09-11T01:12:44.445156+0000","last_fullsized":"2024-09-11T01:12:44.445156+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2356,"log_dups_size":0,"ondisk_log_size":2356,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9650176,"num_objects":2356,"num_object_clones":0,"num_object_copies":4712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2356,"num_write_kb":9424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"36'2384","reported_seq":2441,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:44.444329+0000","last_change":"2024-09-11T01:12:15.135722+0000","last_active":"2024-09-11T01:12:44.444329+0000","last_peered":"2024-09-11T01:12:44.444329+0000","last_clean":"2024-09-11T01:11:42.855496+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:12:44.444329+0000","last_undegraded":"2024-09-11T01:12:15.117592+0000","last_fullsized":"2024-09-11T01:12:44.444329+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2384,"log_dups_size":0,"ondisk_log_size":2384,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9737426,"num_objects":2379,"num_object_clones":0,"num_object_copies":4758,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2382,"num_write_kb":9512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2379}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'2441","reported_seq":2498,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:44.463102+0000","last_change":"2024-09-11T01:12:15.141906+0000","last_active":"2024-09-11T01:12:44.463102+0000","last_peered":"2024-09-11T01:12:44.463102+0000","last_clean":"2024-09-11T01:11:44.266820+0000","last_became_active":"2024-09-11T01:12:15.141377+0000","last_became_peered":"2024-09-11T01:12:15.141377+0000","last_unstale":"2024-09-11T01:12:44.463102+0000","last_undegraded":"2024-09-11T01:12:15.124660+0000","last_fullsized":"2024-09-11T01:12:44.463102+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2441,"log_dups_size":0,"ondisk_log_size":2441,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9979090,"num_objects":2438,"num_object_clones":0,"num_object_copies":4876,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2438,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2439,"num_write_kb":9747,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2438}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'2372","reported_seq":2428,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:44.445129+0000","last_change":"2024-09-11T01:12:15.136057+0000","last_active":"2024-09-11T01:12:44.445129+0000","last_peered":"2024-09-11T01:12:44.445129+0000","last_clean":"2024-09-11T01:11:43.275607+0000","last_became_active":"2024-09-11T01:12:15.135367+0000","last_became_peered":"2024-09-11T01:12:15.135367+0000","last_unstale":"2024-09-11T01:12:44.445129+0000","last_undegraded":"2024-09-11T01:12:15.117856+0000","last_fullsized":"2024-09-11T01:12:44.445129+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2372,"log_dups_size":0,"ondisk_log_size":2372,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9692370,"num_objects":2368,"num_object_clones":0,"num_object_copies":4736,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2368,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2370,"num_write_kb":9468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2368}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'2422","reported_seq":2484,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:44.435637+0000","last_change":"2024-09-11T01:12:15.136010+0000","last_active":"2024-09-11T01:12:44.435637+0000","last_peered":"2024-09-11T01:12:44.435637+0000","last_clean":"2024-09-11T01:11:32.735868+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:12:44.435637+0000","last_undegraded":"2024-09-11T01:12:15.117050+0000","last_fullsized":"2024-09-11T01:12:44.435637+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2422,"log_dups_size":0,"ondisk_log_size":2422,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9912937,"num_objects":2421,"num_object_clones":0,"num_object_copies":4842,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2421,"num_write_kb":9681,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2421}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.114317+0000","last_change":"2024-09-11T01:12:15.135174+0000","last_active":"2024-09-11T01:12:16.114317+0000","last_peered":"2024-09-11T01:12:16.114317+0000","last_clean":"2024-09-11T01:12:16.114317+0000","last_became_active":"2024-09-11T01:12:15.133597+0000","last_became_peered":"2024-09-11T01:12:15.133597+0000","last_unstale":"2024-09-11T01:12:16.114317+0000","last_undegraded":"2024-09-11T01:12:16.114317+0000","last_fullsized":"2024-09-11T01:12:16.114317+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:29:50.913351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"36'2384","reported_seq":2487,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:44.445270+0000","last_change":"2024-09-11T01:12:40.856584+0000","last_active":"2024-09-11T01:12:44.445270+0000","last_peered":"2024-09-11T01:12:44.445270+0000","last_clean":"2024-09-11T01:12:44.445270+0000","last_became_active":"2024-09-11T01:12:16.122119+0000","last_became_peered":"2024-09-11T01:12:16.122119+0000","last_unstale":"2024-09-11T01:12:44.445270+0000","last_undegraded":"2024-09-11T01:12:44.445270+0000","last_fullsized":"2024-09-11T01:12:44.445270+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":2384,"log_dups_size":0,"ondisk_log_size":2384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:43.522530+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9736192,"num_objects":2377,"num_object_clones":0,"num_object_copies":4754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":355,"num_write_kb":1420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.121218+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:12:16.121218+0000","last_peered":"2024-09-11T01:12:16.121218+0000","last_clean":"2024-09-11T01:12:16.121218+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:12:16.121218+0000","last_undegraded":"2024-09-11T01:12:16.121218+0000","last_fullsized":"2024-09-11T01:12:16.121218+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'2391","reported_seq":2441,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:44.463227+0000","last_change":"2024-09-11T01:12:15.141700+0000","last_active":"2024-09-11T01:12:44.463227+0000","last_peered":"2024-09-11T01:12:44.463227+0000","last_clean":"2024-09-11T01:11:44.303167+0000","last_became_active":"2024-09-11T01:12:15.140878+0000","last_became_peered":"2024-09-11T01:12:15.140878+0000","last_unstale":"2024-09-11T01:12:44.463227+0000","last_undegraded":"2024-09-11T01:12:15.122467+0000","last_fullsized":"2024-09-11T01:12:44.463227+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2391,"log_dups_size":0,"ondisk_log_size":2391,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9762002,"num_objects":2385,"num_object_clones":0,"num_object_copies":4770,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2389,"num_write_kb":9536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2385}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":64,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:16.114348+0000","last_change":"2024-09-11T01:12:15.141439+0000","last_active":"2024-09-11T01:12:16.114348+0000","last_peered":"2024-09-11T01:12:16.114348+0000","last_clean":"2024-09-11T01:11:55.731962+0000","last_became_active":"2024-09-11T01:12:15.139222+0000","last_became_peered":"2024-09-11T01:12:15.139222+0000","last_unstale":"2024-09-11T01:12:16.114348+0000","last_undegraded":"2024-09-11T01:12:15.121340+0000","last_fullsized":"2024-09-11T01:12:16.114348+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'2444","reported_seq":2536,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:44.453935+0000","last_change":"2024-09-11T01:12:41.868636+0000","last_active":"2024-09-11T01:12:44.453935+0000","last_peered":"2024-09-11T01:12:44.453935+0000","last_clean":"2024-09-11T01:12:44.453935+0000","last_became_active":"2024-09-11T01:12:16.120717+0000","last_became_peered":"2024-09-11T01:12:16.120717+0000","last_unstale":"2024-09-11T01:12:44.453935+0000","last_undegraded":"2024-09-11T01:12:44.453935+0000","last_fullsized":"2024-09-11T01:12:44.453935+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":2444,"log_dups_size":0,"ondisk_log_size":2444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:54:47.497574+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9990144,"num_objects":2439,"num_object_clones":0,"num_object_copies":4878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2439,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":261,"num_write_kb":1044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.122445+0000","last_change":"2024-09-11T01:12:16.118776+0000","last_active":"2024-09-11T01:12:16.122445+0000","last_peered":"2024-09-11T01:12:16.122445+0000","last_clean":"2024-09-11T01:12:16.122445+0000","last_became_active":"2024-09-11T01:12:16.117613+0000","last_became_peered":"2024-09-11T01:12:16.117613+0000","last_unstale":"2024-09-11T01:12:16.122445+0000","last_undegraded":"2024-09-11T01:12:16.122445+0000","last_fullsized":"2024-09-11T01:12:16.122445+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:46.399169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"36'2390","reported_seq":2482,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:44.427175+0000","last_change":"2024-09-11T01:12:42.891057+0000","last_active":"2024-09-11T01:12:44.427175+0000","last_peered":"2024-09-11T01:12:44.427175+0000","last_clean":"2024-09-11T01:12:44.427175+0000","last_became_active":"2024-09-11T01:12:16.119017+0000","last_became_peered":"2024-09-11T01:12:16.119017+0000","last_unstale":"2024-09-11T01:12:44.427175+0000","last_undegraded":"2024-09-11T01:12:44.427175+0000","last_fullsized":"2024-09-11T01:12:44.427175+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":2390,"log_dups_size":0,"ondisk_log_size":2390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:20:57.577195+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9764864,"num_objects":2384,"num_object_clones":0,"num_object_copies":4768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2384,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":175,"num_write_kb":700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'2433","reported_seq":2489,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:44.446838+0000","last_change":"2024-09-11T01:12:15.136135+0000","last_active":"2024-09-11T01:12:44.446838+0000","last_peered":"2024-09-11T01:12:44.446838+0000","last_clean":"2024-09-11T01:11:35.758866+0000","last_became_active":"2024-09-11T01:12:15.134484+0000","last_became_peered":"2024-09-11T01:12:15.134484+0000","last_unstale":"2024-09-11T01:12:44.446838+0000","last_undegraded":"2024-09-11T01:12:15.118487+0000","last_fullsized":"2024-09-11T01:12:44.446838+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2433,"log_dups_size":0,"ondisk_log_size":2433,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9957993,"num_objects":2432,"num_object_clones":0,"num_object_copies":4864,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2432,"num_write_kb":9725,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2432}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.115043+0000","last_change":"2024-09-11T01:12:15.137159+0000","last_active":"2024-09-11T01:12:16.115043+0000","last_peered":"2024-09-11T01:12:16.115043+0000","last_clean":"2024-09-11T01:12:16.115043+0000","last_became_active":"2024-09-11T01:12:15.136405+0000","last_became_peered":"2024-09-11T01:12:15.136405+0000","last_unstale":"2024-09-11T01:12:16.115043+0000","last_undegraded":"2024-09-11T01:12:16.115043+0000","last_fullsized":"2024-09-11T01:12:16.115043+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:32:43.772100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"36'2374","reported_seq":2428,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:44.468001+0000","last_change":"2024-09-11T01:12:15.141481+0000","last_active":"2024-09-11T01:12:44.468001+0000","last_peered":"2024-09-11T01:12:44.468001+0000","last_clean":"2024-09-11T01:11:44.178063+0000","last_became_active":"2024-09-11T01:12:15.140760+0000","last_became_peered":"2024-09-11T01:12:15.140760+0000","last_unstale":"2024-09-11T01:12:44.468001+0000","last_undegraded":"2024-09-11T01:12:15.124957+0000","last_fullsized":"2024-09-11T01:12:44.468001+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2374,"log_dups_size":0,"ondisk_log_size":2374,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9692987,"num_objects":2369,"num_object_clones":0,"num_object_copies":4738,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2371,"num_write_kb":9468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2369}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.123536+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:12:16.123536+0000","last_peered":"2024-09-11T01:12:16.123536+0000","last_clean":"2024-09-11T01:12:16.123536+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:12:16.123536+0000","last_undegraded":"2024-09-11T01:12:16.123536+0000","last_fullsized":"2024-09-11T01:12:16.123536+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'2422","reported_seq":2477,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:44.419230+0000","last_change":"2024-09-11T01:12:15.140853+0000","last_active":"2024-09-11T01:12:44.419230+0000","last_peered":"2024-09-11T01:12:44.419230+0000","last_clean":"2024-09-11T01:11:35.758438+0000","last_became_active":"2024-09-11T01:12:15.138823+0000","last_became_peered":"2024-09-11T01:12:15.138823+0000","last_unstale":"2024-09-11T01:12:44.419230+0000","last_undegraded":"2024-09-11T01:12:15.123630+0000","last_fullsized":"2024-09-11T01:12:44.419230+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2422,"log_dups_size":0,"ondisk_log_size":2422,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9905362,"num_objects":2420,"num_object_clones":0,"num_object_copies":4840,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2420,"num_write_kb":9674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2420}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.115056+0000","last_change":"2024-09-11T01:12:15.137008+0000","last_active":"2024-09-11T01:12:16.115056+0000","last_peered":"2024-09-11T01:12:16.115056+0000","last_clean":"2024-09-11T01:12:16.115056+0000","last_became_active":"2024-09-11T01:12:15.136222+0000","last_became_peered":"2024-09-11T01:12:15.136222+0000","last_unstale":"2024-09-11T01:12:16.115056+0000","last_undegraded":"2024-09-11T01:12:16.115056+0000","last_fullsized":"2024-09-11T01:12:16.115056+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:07:25.284013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.123820+0000","last_change":"2024-09-11T01:12:16.122836+0000","last_active":"2024-09-11T01:12:16.123820+0000","last_peered":"2024-09-11T01:12:16.123820+0000","last_clean":"2024-09-11T01:12:16.123820+0000","last_became_active":"2024-09-11T01:12:16.122572+0000","last_became_peered":"2024-09-11T01:12:16.122572+0000","last_unstale":"2024-09-11T01:12:16.123820+0000","last_undegraded":"2024-09-11T01:12:16.123820+0000","last_fullsized":"2024-09-11T01:12:16.123820+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:36:29.939442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"36'2347","reported_seq":2403,"reported_epoch":36,"state":"active+recovering+degraded","last_fresh":"2024-09-11T01:12:44.435287+0000","last_change":"2024-09-11T01:12:15.142056+0000","last_active":"2024-09-11T01:12:44.435287+0000","last_peered":"2024-09-11T01:12:44.435287+0000","last_clean":"2024-09-11T01:11:43.771457+0000","last_became_active":"2024-09-11T01:12:15.133897+0000","last_became_peered":"2024-09-11T01:12:15.133897+0000","last_unstale":"2024-09-11T01:12:44.435287+0000","last_undegraded":"2024-09-11T01:12:15.116261+0000","last_fullsized":"2024-09-11T01:12:44.435287+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2347,"log_dups_size":0,"ondisk_log_size":2347,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9566628,"num_objects":2339,"num_object_clones":0,"num_object_copies":4678,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2344,"num_write_kb":9347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":2338}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":157269451,"num_objects":38412,"num_object_clones":0,"num_object_copies":76824,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":31,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28570,"num_write_kb":114166,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":314748928,"data_stored":314630642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38475,"ondisk_log_size":38475,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182415,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":260548,"kb_used_data":152564,"kb_used_omap":4,"kb_used_meta":107963,"kb_avail":94111292,"statfs":{"total":96636764160,"available":96369963008,"internally_reserved":0,"allocated":156225536,"data_stored":155992726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":110554795},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,9,4,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986187,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":263208,"kb_used_data":151064,"kb_used_omap":4,"kb_used_meta":112123,"kb_avail":94108632,"statfs":{"total":96636764160,"available":96367239168,"internally_reserved":0,"allocated":154689536,"data_stored":154481677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":114814634},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1,3,5,4,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":157335552,"data_stored":157314678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":17276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":157298688,"data_stored":157298688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:12:47.346 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 23 20 42213 42193 7.16449 5.23438 0.00988086 0.0108969 2024-09-11T01:12:48.345 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 24 20 43689 43669 7.10619 5.76562 0.00667762 0.0109859 2024-09-11T01:12:49.346 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 25 20 45165 45145 7.05255 5.76562 0.0279516 0.0110692 2024-09-11T01:12:50.272 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:12:50.346 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 26 20 46478 46458 6.97856 5.12891 0.0175945 0.0111891 2024-09-11T01:12:51.044 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:12:51.045 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:12:51.075 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2024-09-11T01:12:50.552084+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":179740126,"num_objects":43900,"num_object_clones":0,"num_object_copies":87800,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34058,"num_write_kb":136112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43964,"ondisk_log_size":43964,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":553468,"kb_used_data":345820,"kb_used_omap":8,"kb_used_meta":207607,"kb_avail":188190212,"statfs":{"total":193273528320,"available":192706777088,"internally_reserved":0,"allocated":354119680,"data_stored":353679011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8875,"internal_metadata":212589909},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,1,3,13,7,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":77443466,"num_objects":18902,"num_object_clones":0,"num_object_copies":37804,"num_objects_missing_on_primary":-6,"num_objects_missing":-6,"num_objects_degraded":-12,"num_objects_misplaced":0,"num_objects_unfound":-24,"num_objects_dirty":18902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12487,"num_write_kb":49986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007446"},"pg_stats":[{"pgid":"2.f","version":"36'2794","reported_seq":2844,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:48.742333+0000","last_change":"2024-09-11T01:12:15.141793+0000","last_active":"2024-09-11T01:12:48.742333+0000","last_peered":"2024-09-11T01:12:48.742333+0000","last_clean":"2024-09-11T01:12:48.742333+0000","last_became_active":"2024-09-11T01:12:15.141081+0000","last_became_peered":"2024-09-11T01:12:15.141081+0000","last_unstale":"2024-09-11T01:12:48.742333+0000","last_undegraded":"2024-09-11T01:12:48.742333+0000","last_fullsized":"2024-09-11T01:12:48.742333+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2794,"log_dups_size":0,"ondisk_log_size":2794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:18:33.495210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11444224,"num_objects":2794,"num_object_clones":0,"num_object_copies":5588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2794,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2794,"num_write_kb":11176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"36'2804","reported_seq":2886,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:48.037443+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:12:48.037443+0000","last_peered":"2024-09-11T01:12:48.037443+0000","last_clean":"2024-09-11T01:12:48.037443+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:12:48.037443+0000","last_undegraded":"2024-09-11T01:12:48.037443+0000","last_fullsized":"2024-09-11T01:12:48.037443+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":2804,"log_dups_size":0,"ondisk_log_size":2804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11460608,"num_objects":2798,"num_object_clones":0,"num_object_copies":5596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1043,"num_write_kb":4172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'2685","reported_seq":2779,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:48.059963+0000","last_change":"2024-09-11T01:12:37.612304+0000","last_active":"2024-09-11T01:12:48.059963+0000","last_peered":"2024-09-11T01:12:48.059963+0000","last_clean":"2024-09-11T01:12:48.059963+0000","last_became_active":"2024-09-11T01:12:16.123243+0000","last_became_peered":"2024-09-11T01:12:16.123243+0000","last_unstale":"2024-09-11T01:12:48.059963+0000","last_undegraded":"2024-09-11T01:12:48.059963+0000","last_fullsized":"2024-09-11T01:12:48.059963+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":2685,"log_dups_size":0,"ondisk_log_size":2685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:39.057954+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10981376,"num_objects":2681,"num_object_clones":0,"num_object_copies":5362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":986,"num_write_kb":3944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"36'2666","reported_seq":2716,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:48.030715+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:12:48.030715+0000","last_peered":"2024-09-11T01:12:48.030715+0000","last_clean":"2024-09-11T01:12:48.030715+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:12:48.030715+0000","last_undegraded":"2024-09-11T01:12:48.030715+0000","last_fullsized":"2024-09-11T01:12:48.030715+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2666,"log_dups_size":0,"ondisk_log_size":2666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10919936,"num_objects":2666,"num_object_clones":0,"num_object_copies":5332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2666,"num_write_kb":10664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"36'2766","reported_seq":2825,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:48.721843+0000","last_change":"2024-09-11T01:12:15.135722+0000","last_active":"2024-09-11T01:12:48.721843+0000","last_peered":"2024-09-11T01:12:48.721843+0000","last_clean":"2024-09-11T01:11:42.855496+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:12:48.721843+0000","last_undegraded":"2024-09-11T01:12:15.117592+0000","last_fullsized":"2024-09-11T01:12:48.721843+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2766,"log_dups_size":0,"ondisk_log_size":2766,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11302098,"num_objects":2761,"num_object_clones":0,"num_object_copies":5522,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2764,"num_write_kb":11040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2761}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'2805","reported_seq":2862,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:48.703149+0000","last_change":"2024-09-11T01:12:15.141906+0000","last_active":"2024-09-11T01:12:48.703149+0000","last_peered":"2024-09-11T01:12:48.703149+0000","last_clean":"2024-09-11T01:11:44.266820+0000","last_became_active":"2024-09-11T01:12:15.141377+0000","last_became_peered":"2024-09-11T01:12:15.141377+0000","last_unstale":"2024-09-11T01:12:48.703149+0000","last_undegraded":"2024-09-11T01:12:15.124660+0000","last_fullsized":"2024-09-11T01:12:48.703149+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2805,"log_dups_size":0,"ondisk_log_size":2805,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11470034,"num_objects":2802,"num_object_clones":0,"num_object_copies":5604,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2803,"num_write_kb":11203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2802}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'2731","reported_seq":2786,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:48.735705+0000","last_change":"2024-09-11T01:12:15.136057+0000","last_active":"2024-09-11T01:12:48.735705+0000","last_peered":"2024-09-11T01:12:48.735705+0000","last_clean":"2024-09-11T01:11:43.275607+0000","last_became_active":"2024-09-11T01:12:15.135367+0000","last_became_peered":"2024-09-11T01:12:15.135367+0000","last_unstale":"2024-09-11T01:12:48.735705+0000","last_undegraded":"2024-09-11T01:12:15.117856+0000","last_fullsized":"2024-09-11T01:12:48.735705+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2731,"log_dups_size":0,"ondisk_log_size":2731,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11162834,"num_objects":2727,"num_object_clones":0,"num_object_copies":5454,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2729,"num_write_kb":10904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2727}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'2782","reported_seq":2844,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:48.731504+0000","last_change":"2024-09-11T01:12:15.136010+0000","last_active":"2024-09-11T01:12:48.731504+0000","last_peered":"2024-09-11T01:12:48.731504+0000","last_clean":"2024-09-11T01:11:32.735868+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:12:48.731504+0000","last_undegraded":"2024-09-11T01:12:15.117050+0000","last_fullsized":"2024-09-11T01:12:48.731504+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2782,"log_dups_size":0,"ondisk_log_size":2782,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11387497,"num_objects":2781,"num_object_clones":0,"num_object_copies":5562,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2781,"num_write_kb":11121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2781}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.114317+0000","last_change":"2024-09-11T01:12:15.135174+0000","last_active":"2024-09-11T01:12:16.114317+0000","last_peered":"2024-09-11T01:12:16.114317+0000","last_clean":"2024-09-11T01:12:16.114317+0000","last_became_active":"2024-09-11T01:12:15.133597+0000","last_became_peered":"2024-09-11T01:12:15.133597+0000","last_unstale":"2024-09-11T01:12:16.114317+0000","last_undegraded":"2024-09-11T01:12:16.114317+0000","last_fullsized":"2024-09-11T01:12:16.114317+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:29:50.913351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"36'2679","reported_seq":2781,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:48.064133+0000","last_change":"2024-09-11T01:12:40.856584+0000","last_active":"2024-09-11T01:12:48.064133+0000","last_peered":"2024-09-11T01:12:48.064133+0000","last_clean":"2024-09-11T01:12:48.064133+0000","last_became_active":"2024-09-11T01:12:16.122119+0000","last_became_peered":"2024-09-11T01:12:16.122119+0000","last_unstale":"2024-09-11T01:12:48.064133+0000","last_undegraded":"2024-09-11T01:12:48.064133+0000","last_fullsized":"2024-09-11T01:12:48.064133+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":2679,"log_dups_size":0,"ondisk_log_size":2679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:43.522530+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10944512,"num_objects":2672,"num_object_clones":0,"num_object_copies":5344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":650,"num_write_kb":2600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.121218+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:12:16.121218+0000","last_peered":"2024-09-11T01:12:16.121218+0000","last_clean":"2024-09-11T01:12:16.121218+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:12:16.121218+0000","last_undegraded":"2024-09-11T01:12:16.121218+0000","last_fullsized":"2024-09-11T01:12:16.121218+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'2751","reported_seq":2801,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:48.721670+0000","last_change":"2024-09-11T01:12:15.141700+0000","last_active":"2024-09-11T01:12:48.721670+0000","last_peered":"2024-09-11T01:12:48.721670+0000","last_clean":"2024-09-11T01:11:44.303167+0000","last_became_active":"2024-09-11T01:12:15.140878+0000","last_became_peered":"2024-09-11T01:12:15.140878+0000","last_unstale":"2024-09-11T01:12:48.721670+0000","last_undegraded":"2024-09-11T01:12:15.122467+0000","last_fullsized":"2024-09-11T01:12:48.721670+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2751,"log_dups_size":0,"ondisk_log_size":2751,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11236562,"num_objects":2745,"num_object_clones":0,"num_object_copies":5490,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2749,"num_write_kb":10976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2745}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":64,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:16.114348+0000","last_change":"2024-09-11T01:12:15.141439+0000","last_active":"2024-09-11T01:12:16.114348+0000","last_peered":"2024-09-11T01:12:16.114348+0000","last_clean":"2024-09-11T01:11:55.731962+0000","last_became_active":"2024-09-11T01:12:15.139222+0000","last_became_peered":"2024-09-11T01:12:15.139222+0000","last_unstale":"2024-09-11T01:12:16.114348+0000","last_undegraded":"2024-09-11T01:12:15.121340+0000","last_fullsized":"2024-09-11T01:12:16.114348+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'2743","reported_seq":2835,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:48.060793+0000","last_change":"2024-09-11T01:12:41.868636+0000","last_active":"2024-09-11T01:12:48.060793+0000","last_peered":"2024-09-11T01:12:48.060793+0000","last_clean":"2024-09-11T01:12:48.060793+0000","last_became_active":"2024-09-11T01:12:16.120717+0000","last_became_peered":"2024-09-11T01:12:16.120717+0000","last_unstale":"2024-09-11T01:12:48.060793+0000","last_undegraded":"2024-09-11T01:12:48.060793+0000","last_fullsized":"2024-09-11T01:12:48.060793+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":2743,"log_dups_size":0,"ondisk_log_size":2743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:54:47.497574+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11214848,"num_objects":2738,"num_object_clones":0,"num_object_copies":5476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2738,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":560,"num_write_kb":2240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.122445+0000","last_change":"2024-09-11T01:12:16.118776+0000","last_active":"2024-09-11T01:12:16.122445+0000","last_peered":"2024-09-11T01:12:16.122445+0000","last_clean":"2024-09-11T01:12:16.122445+0000","last_became_active":"2024-09-11T01:12:16.117613+0000","last_became_peered":"2024-09-11T01:12:16.117613+0000","last_unstale":"2024-09-11T01:12:16.122445+0000","last_undegraded":"2024-09-11T01:12:16.122445+0000","last_fullsized":"2024-09-11T01:12:16.122445+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:46.399169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"36'2684","reported_seq":2776,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:48.053695+0000","last_change":"2024-09-11T01:12:42.891057+0000","last_active":"2024-09-11T01:12:48.053695+0000","last_peered":"2024-09-11T01:12:48.053695+0000","last_clean":"2024-09-11T01:12:48.053695+0000","last_became_active":"2024-09-11T01:12:16.119017+0000","last_became_peered":"2024-09-11T01:12:16.119017+0000","last_unstale":"2024-09-11T01:12:48.053695+0000","last_undegraded":"2024-09-11T01:12:48.053695+0000","last_fullsized":"2024-09-11T01:12:48.053695+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":2684,"log_dups_size":0,"ondisk_log_size":2684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:20:57.577195+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10969088,"num_objects":2678,"num_object_clones":0,"num_object_copies":5356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2678,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":469,"num_write_kb":1876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'2758","reported_seq":2814,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:48.706811+0000","last_change":"2024-09-11T01:12:15.136135+0000","last_active":"2024-09-11T01:12:48.706811+0000","last_peered":"2024-09-11T01:12:48.706811+0000","last_clean":"2024-09-11T01:11:35.758866+0000","last_became_active":"2024-09-11T01:12:15.134484+0000","last_became_peered":"2024-09-11T01:12:15.134484+0000","last_unstale":"2024-09-11T01:12:48.706811+0000","last_undegraded":"2024-09-11T01:12:15.118487+0000","last_fullsized":"2024-09-11T01:12:48.706811+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2758,"log_dups_size":0,"ondisk_log_size":2758,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11289193,"num_objects":2757,"num_object_clones":0,"num_object_copies":5514,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2757,"num_write_kb":11025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2757}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.115043+0000","last_change":"2024-09-11T01:12:15.137159+0000","last_active":"2024-09-11T01:12:16.115043+0000","last_peered":"2024-09-11T01:12:16.115043+0000","last_clean":"2024-09-11T01:12:16.115043+0000","last_became_active":"2024-09-11T01:12:15.136405+0000","last_became_peered":"2024-09-11T01:12:15.136405+0000","last_unstale":"2024-09-11T01:12:16.115043+0000","last_undegraded":"2024-09-11T01:12:16.115043+0000","last_fullsized":"2024-09-11T01:12:16.115043+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:32:43.772100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"36'2743","reported_seq":2798,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:48.731312+0000","last_change":"2024-09-11T01:12:15.141481+0000","last_active":"2024-09-11T01:12:48.731312+0000","last_peered":"2024-09-11T01:12:48.731312+0000","last_clean":"2024-09-11T01:11:44.178063+0000","last_became_active":"2024-09-11T01:12:15.140760+0000","last_became_peered":"2024-09-11T01:12:15.140760+0000","last_unstale":"2024-09-11T01:12:48.731312+0000","last_undegraded":"2024-09-11T01:12:15.124957+0000","last_fullsized":"2024-09-11T01:12:48.731312+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2743,"log_dups_size":0,"ondisk_log_size":2743,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11204411,"num_objects":2738,"num_object_clones":0,"num_object_copies":5476,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2738,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2740,"num_write_kb":10944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2738}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.123536+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:12:16.123536+0000","last_peered":"2024-09-11T01:12:16.123536+0000","last_clean":"2024-09-11T01:12:16.123536+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:12:16.123536+0000","last_undegraded":"2024-09-11T01:12:16.123536+0000","last_fullsized":"2024-09-11T01:12:16.123536+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'2833","reported_seq":2887,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:48.745089+0000","last_change":"2024-09-11T01:12:15.140853+0000","last_active":"2024-09-11T01:12:48.745089+0000","last_peered":"2024-09-11T01:12:48.745089+0000","last_clean":"2024-09-11T01:11:35.758438+0000","last_became_active":"2024-09-11T01:12:15.138823+0000","last_became_peered":"2024-09-11T01:12:15.138823+0000","last_unstale":"2024-09-11T01:12:48.745089+0000","last_undegraded":"2024-09-11T01:12:15.123630+0000","last_fullsized":"2024-09-11T01:12:48.745089+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2833,"log_dups_size":0,"ondisk_log_size":2833,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11588818,"num_objects":2831,"num_object_clones":0,"num_object_copies":5662,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2831,"num_write_kb":11318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2831}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.115056+0000","last_change":"2024-09-11T01:12:15.137008+0000","last_active":"2024-09-11T01:12:16.115056+0000","last_peered":"2024-09-11T01:12:16.115056+0000","last_clean":"2024-09-11T01:12:16.115056+0000","last_became_active":"2024-09-11T01:12:15.136222+0000","last_became_peered":"2024-09-11T01:12:15.136222+0000","last_unstale":"2024-09-11T01:12:16.115056+0000","last_undegraded":"2024-09-11T01:12:16.115056+0000","last_fullsized":"2024-09-11T01:12:16.115056+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:07:25.284013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.123820+0000","last_change":"2024-09-11T01:12:16.122836+0000","last_active":"2024-09-11T01:12:16.123820+0000","last_peered":"2024-09-11T01:12:16.123820+0000","last_clean":"2024-09-11T01:12:16.123820+0000","last_became_active":"2024-09-11T01:12:16.122572+0000","last_became_peered":"2024-09-11T01:12:16.122572+0000","last_unstale":"2024-09-11T01:12:16.123820+0000","last_undegraded":"2024-09-11T01:12:16.123820+0000","last_fullsized":"2024-09-11T01:12:16.123820+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:36:29.939442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"36'2737","reported_seq":2793,"reported_epoch":36,"state":"active+recovering+degraded","last_fresh":"2024-09-11T01:12:48.742257+0000","last_change":"2024-09-11T01:12:15.142056+0000","last_active":"2024-09-11T01:12:48.742257+0000","last_peered":"2024-09-11T01:12:48.742257+0000","last_clean":"2024-09-11T01:11:43.771457+0000","last_became_active":"2024-09-11T01:12:15.133897+0000","last_became_peered":"2024-09-11T01:12:15.133897+0000","last_unstale":"2024-09-11T01:12:48.742257+0000","last_undegraded":"2024-09-11T01:12:15.116261+0000","last_fullsized":"2024-09-11T01:12:48.742257+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2737,"log_dups_size":0,"ondisk_log_size":2737,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11164068,"num_objects":2729,"num_object_clones":0,"num_object_copies":5458,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2734,"num_write_kb":10907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":2728}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":179740107,"num_objects":43898,"num_object_clones":0,"num_object_copies":87796,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":31,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34056,"num_write_kb":136110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":358699008,"data_stored":358580722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43961,"ondisk_log_size":43961,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182416,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":273012,"kb_used_data":173348,"kb_used_omap":4,"kb_used_meta":99643,"kb_avail":94098828,"statfs":{"total":96636764160,"available":96357199872,"internally_reserved":0,"allocated":177508352,"data_stored":177275542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":102035115},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,1,4,4,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986188,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":280456,"kb_used_data":172472,"kb_used_omap":4,"kb_used_meta":107963,"kb_avail":94091384,"statfs":{"total":96636764160,"available":96349577216,"internally_reserved":0,"allocated":176611328,"data_stored":176403469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":110554794},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,2,9,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":181399552,"data_stored":181378678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":17276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":177184768,"data_stored":177184768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:12:51.077 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:12:51.346 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 27 20 47651 47631 6.88979 4.58203 0.00867792 0.0113295 2024-09-11T01:12:51.720 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:12:51.721 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:12:51.750 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2024-09-11T01:12:50.552084+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":179740126,"num_objects":43900,"num_object_clones":0,"num_object_copies":87800,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":32,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34058,"num_write_kb":136112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43964,"ondisk_log_size":43964,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":553468,"kb_used_data":345820,"kb_used_omap":8,"kb_used_meta":207607,"kb_avail":188190212,"statfs":{"total":193273528320,"available":192706777088,"internally_reserved":0,"allocated":354119680,"data_stored":353679011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8875,"internal_metadata":212589909},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,1,3,13,7,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":77443466,"num_objects":18902,"num_object_clones":0,"num_object_copies":37804,"num_objects_missing_on_primary":-6,"num_objects_missing":-6,"num_objects_degraded":-12,"num_objects_misplaced":0,"num_objects_unfound":-24,"num_objects_dirty":18902,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12487,"num_write_kb":49986,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007446"},"pg_stats":[{"pgid":"2.f","version":"36'2794","reported_seq":2844,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:48.742333+0000","last_change":"2024-09-11T01:12:15.141793+0000","last_active":"2024-09-11T01:12:48.742333+0000","last_peered":"2024-09-11T01:12:48.742333+0000","last_clean":"2024-09-11T01:12:48.742333+0000","last_became_active":"2024-09-11T01:12:15.141081+0000","last_became_peered":"2024-09-11T01:12:15.141081+0000","last_unstale":"2024-09-11T01:12:48.742333+0000","last_undegraded":"2024-09-11T01:12:48.742333+0000","last_fullsized":"2024-09-11T01:12:48.742333+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2794,"log_dups_size":0,"ondisk_log_size":2794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:18:33.495210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11444224,"num_objects":2794,"num_object_clones":0,"num_object_copies":5588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2794,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2794,"num_write_kb":11176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"36'2804","reported_seq":2886,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:48.037443+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:12:48.037443+0000","last_peered":"2024-09-11T01:12:48.037443+0000","last_clean":"2024-09-11T01:12:48.037443+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:12:48.037443+0000","last_undegraded":"2024-09-11T01:12:48.037443+0000","last_fullsized":"2024-09-11T01:12:48.037443+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":2804,"log_dups_size":0,"ondisk_log_size":2804,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11460608,"num_objects":2798,"num_object_clones":0,"num_object_copies":5596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1043,"num_write_kb":4172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'2685","reported_seq":2779,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:48.059963+0000","last_change":"2024-09-11T01:12:37.612304+0000","last_active":"2024-09-11T01:12:48.059963+0000","last_peered":"2024-09-11T01:12:48.059963+0000","last_clean":"2024-09-11T01:12:48.059963+0000","last_became_active":"2024-09-11T01:12:16.123243+0000","last_became_peered":"2024-09-11T01:12:16.123243+0000","last_unstale":"2024-09-11T01:12:48.059963+0000","last_undegraded":"2024-09-11T01:12:48.059963+0000","last_fullsized":"2024-09-11T01:12:48.059963+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":2685,"log_dups_size":0,"ondisk_log_size":2685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:39.057954+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10981376,"num_objects":2681,"num_object_clones":0,"num_object_copies":5362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":986,"num_write_kb":3944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"36'2666","reported_seq":2716,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:48.030715+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:12:48.030715+0000","last_peered":"2024-09-11T01:12:48.030715+0000","last_clean":"2024-09-11T01:12:48.030715+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:12:48.030715+0000","last_undegraded":"2024-09-11T01:12:48.030715+0000","last_fullsized":"2024-09-11T01:12:48.030715+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2666,"log_dups_size":0,"ondisk_log_size":2666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10919936,"num_objects":2666,"num_object_clones":0,"num_object_copies":5332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2666,"num_write_kb":10664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"36'2766","reported_seq":2825,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:48.721843+0000","last_change":"2024-09-11T01:12:15.135722+0000","last_active":"2024-09-11T01:12:48.721843+0000","last_peered":"2024-09-11T01:12:48.721843+0000","last_clean":"2024-09-11T01:11:42.855496+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:12:48.721843+0000","last_undegraded":"2024-09-11T01:12:15.117592+0000","last_fullsized":"2024-09-11T01:12:48.721843+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2766,"log_dups_size":0,"ondisk_log_size":2766,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11302098,"num_objects":2761,"num_object_clones":0,"num_object_copies":5522,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2764,"num_write_kb":11040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2761}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'2805","reported_seq":2862,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:48.703149+0000","last_change":"2024-09-11T01:12:15.141906+0000","last_active":"2024-09-11T01:12:48.703149+0000","last_peered":"2024-09-11T01:12:48.703149+0000","last_clean":"2024-09-11T01:11:44.266820+0000","last_became_active":"2024-09-11T01:12:15.141377+0000","last_became_peered":"2024-09-11T01:12:15.141377+0000","last_unstale":"2024-09-11T01:12:48.703149+0000","last_undegraded":"2024-09-11T01:12:15.124660+0000","last_fullsized":"2024-09-11T01:12:48.703149+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2805,"log_dups_size":0,"ondisk_log_size":2805,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11470034,"num_objects":2802,"num_object_clones":0,"num_object_copies":5604,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2803,"num_write_kb":11203,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2802}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'2731","reported_seq":2786,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:48.735705+0000","last_change":"2024-09-11T01:12:15.136057+0000","last_active":"2024-09-11T01:12:48.735705+0000","last_peered":"2024-09-11T01:12:48.735705+0000","last_clean":"2024-09-11T01:11:43.275607+0000","last_became_active":"2024-09-11T01:12:15.135367+0000","last_became_peered":"2024-09-11T01:12:15.135367+0000","last_unstale":"2024-09-11T01:12:48.735705+0000","last_undegraded":"2024-09-11T01:12:15.117856+0000","last_fullsized":"2024-09-11T01:12:48.735705+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2731,"log_dups_size":0,"ondisk_log_size":2731,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11162834,"num_objects":2727,"num_object_clones":0,"num_object_copies":5454,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2729,"num_write_kb":10904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2727}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'2782","reported_seq":2844,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:48.731504+0000","last_change":"2024-09-11T01:12:15.136010+0000","last_active":"2024-09-11T01:12:48.731504+0000","last_peered":"2024-09-11T01:12:48.731504+0000","last_clean":"2024-09-11T01:11:32.735868+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:12:48.731504+0000","last_undegraded":"2024-09-11T01:12:15.117050+0000","last_fullsized":"2024-09-11T01:12:48.731504+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2782,"log_dups_size":0,"ondisk_log_size":2782,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11387497,"num_objects":2781,"num_object_clones":0,"num_object_copies":5562,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2781,"num_write_kb":11121,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2781}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.114317+0000","last_change":"2024-09-11T01:12:15.135174+0000","last_active":"2024-09-11T01:12:16.114317+0000","last_peered":"2024-09-11T01:12:16.114317+0000","last_clean":"2024-09-11T01:12:16.114317+0000","last_became_active":"2024-09-11T01:12:15.133597+0000","last_became_peered":"2024-09-11T01:12:15.133597+0000","last_unstale":"2024-09-11T01:12:16.114317+0000","last_undegraded":"2024-09-11T01:12:16.114317+0000","last_fullsized":"2024-09-11T01:12:16.114317+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:29:50.913351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"36'2679","reported_seq":2781,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:48.064133+0000","last_change":"2024-09-11T01:12:40.856584+0000","last_active":"2024-09-11T01:12:48.064133+0000","last_peered":"2024-09-11T01:12:48.064133+0000","last_clean":"2024-09-11T01:12:48.064133+0000","last_became_active":"2024-09-11T01:12:16.122119+0000","last_became_peered":"2024-09-11T01:12:16.122119+0000","last_unstale":"2024-09-11T01:12:48.064133+0000","last_undegraded":"2024-09-11T01:12:48.064133+0000","last_fullsized":"2024-09-11T01:12:48.064133+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":2679,"log_dups_size":0,"ondisk_log_size":2679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:43.522530+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10944512,"num_objects":2672,"num_object_clones":0,"num_object_copies":5344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":650,"num_write_kb":2600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.121218+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:12:16.121218+0000","last_peered":"2024-09-11T01:12:16.121218+0000","last_clean":"2024-09-11T01:12:16.121218+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:12:16.121218+0000","last_undegraded":"2024-09-11T01:12:16.121218+0000","last_fullsized":"2024-09-11T01:12:16.121218+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'2751","reported_seq":2801,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:48.721670+0000","last_change":"2024-09-11T01:12:15.141700+0000","last_active":"2024-09-11T01:12:48.721670+0000","last_peered":"2024-09-11T01:12:48.721670+0000","last_clean":"2024-09-11T01:11:44.303167+0000","last_became_active":"2024-09-11T01:12:15.140878+0000","last_became_peered":"2024-09-11T01:12:15.140878+0000","last_unstale":"2024-09-11T01:12:48.721670+0000","last_undegraded":"2024-09-11T01:12:15.122467+0000","last_fullsized":"2024-09-11T01:12:48.721670+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2751,"log_dups_size":0,"ondisk_log_size":2751,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11236562,"num_objects":2745,"num_object_clones":0,"num_object_copies":5490,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2749,"num_write_kb":10976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2745}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":64,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:16.114348+0000","last_change":"2024-09-11T01:12:15.141439+0000","last_active":"2024-09-11T01:12:16.114348+0000","last_peered":"2024-09-11T01:12:16.114348+0000","last_clean":"2024-09-11T01:11:55.731962+0000","last_became_active":"2024-09-11T01:12:15.139222+0000","last_became_peered":"2024-09-11T01:12:15.139222+0000","last_unstale":"2024-09-11T01:12:16.114348+0000","last_undegraded":"2024-09-11T01:12:15.121340+0000","last_fullsized":"2024-09-11T01:12:16.114348+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0"],"object_location_counts":[{"shards":"0","objects":1}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'2743","reported_seq":2835,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:48.060793+0000","last_change":"2024-09-11T01:12:41.868636+0000","last_active":"2024-09-11T01:12:48.060793+0000","last_peered":"2024-09-11T01:12:48.060793+0000","last_clean":"2024-09-11T01:12:48.060793+0000","last_became_active":"2024-09-11T01:12:16.120717+0000","last_became_peered":"2024-09-11T01:12:16.120717+0000","last_unstale":"2024-09-11T01:12:48.060793+0000","last_undegraded":"2024-09-11T01:12:48.060793+0000","last_fullsized":"2024-09-11T01:12:48.060793+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":2743,"log_dups_size":0,"ondisk_log_size":2743,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:54:47.497574+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11214848,"num_objects":2738,"num_object_clones":0,"num_object_copies":5476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2738,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":560,"num_write_kb":2240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.122445+0000","last_change":"2024-09-11T01:12:16.118776+0000","last_active":"2024-09-11T01:12:16.122445+0000","last_peered":"2024-09-11T01:12:16.122445+0000","last_clean":"2024-09-11T01:12:16.122445+0000","last_became_active":"2024-09-11T01:12:16.117613+0000","last_became_peered":"2024-09-11T01:12:16.117613+0000","last_unstale":"2024-09-11T01:12:16.122445+0000","last_undegraded":"2024-09-11T01:12:16.122445+0000","last_fullsized":"2024-09-11T01:12:16.122445+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:46.399169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"36'2684","reported_seq":2776,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:48.053695+0000","last_change":"2024-09-11T01:12:42.891057+0000","last_active":"2024-09-11T01:12:48.053695+0000","last_peered":"2024-09-11T01:12:48.053695+0000","last_clean":"2024-09-11T01:12:48.053695+0000","last_became_active":"2024-09-11T01:12:16.119017+0000","last_became_peered":"2024-09-11T01:12:16.119017+0000","last_unstale":"2024-09-11T01:12:48.053695+0000","last_undegraded":"2024-09-11T01:12:48.053695+0000","last_fullsized":"2024-09-11T01:12:48.053695+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":2684,"log_dups_size":0,"ondisk_log_size":2684,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:20:57.577195+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10969088,"num_objects":2678,"num_object_clones":0,"num_object_copies":5356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2678,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":469,"num_write_kb":1876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'2758","reported_seq":2814,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:48.706811+0000","last_change":"2024-09-11T01:12:15.136135+0000","last_active":"2024-09-11T01:12:48.706811+0000","last_peered":"2024-09-11T01:12:48.706811+0000","last_clean":"2024-09-11T01:11:35.758866+0000","last_became_active":"2024-09-11T01:12:15.134484+0000","last_became_peered":"2024-09-11T01:12:15.134484+0000","last_unstale":"2024-09-11T01:12:48.706811+0000","last_undegraded":"2024-09-11T01:12:15.118487+0000","last_fullsized":"2024-09-11T01:12:48.706811+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2758,"log_dups_size":0,"ondisk_log_size":2758,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11289193,"num_objects":2757,"num_object_clones":0,"num_object_copies":5514,"num_objects_missing_on_primary":1,"num_objects_missing":1,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2757,"num_write_kb":11025,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2757}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.115043+0000","last_change":"2024-09-11T01:12:15.137159+0000","last_active":"2024-09-11T01:12:16.115043+0000","last_peered":"2024-09-11T01:12:16.115043+0000","last_clean":"2024-09-11T01:12:16.115043+0000","last_became_active":"2024-09-11T01:12:15.136405+0000","last_became_peered":"2024-09-11T01:12:15.136405+0000","last_unstale":"2024-09-11T01:12:16.115043+0000","last_undegraded":"2024-09-11T01:12:16.115043+0000","last_fullsized":"2024-09-11T01:12:16.115043+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:32:43.772100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"36'2743","reported_seq":2798,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:48.731312+0000","last_change":"2024-09-11T01:12:15.141481+0000","last_active":"2024-09-11T01:12:48.731312+0000","last_peered":"2024-09-11T01:12:48.731312+0000","last_clean":"2024-09-11T01:11:44.178063+0000","last_became_active":"2024-09-11T01:12:15.140760+0000","last_became_peered":"2024-09-11T01:12:15.140760+0000","last_unstale":"2024-09-11T01:12:48.731312+0000","last_undegraded":"2024-09-11T01:12:15.124957+0000","last_fullsized":"2024-09-11T01:12:48.731312+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2743,"log_dups_size":0,"ondisk_log_size":2743,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11204411,"num_objects":2738,"num_object_clones":0,"num_object_copies":5476,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":6,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2738,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2740,"num_write_kb":10944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2738}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.123536+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:12:16.123536+0000","last_peered":"2024-09-11T01:12:16.123536+0000","last_clean":"2024-09-11T01:12:16.123536+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:12:16.123536+0000","last_undegraded":"2024-09-11T01:12:16.123536+0000","last_fullsized":"2024-09-11T01:12:16.123536+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'2833","reported_seq":2887,"reported_epoch":36,"state":"active+recovery_wait+degraded","last_fresh":"2024-09-11T01:12:48.745089+0000","last_change":"2024-09-11T01:12:15.140853+0000","last_active":"2024-09-11T01:12:48.745089+0000","last_peered":"2024-09-11T01:12:48.745089+0000","last_clean":"2024-09-11T01:11:35.758438+0000","last_became_active":"2024-09-11T01:12:15.138823+0000","last_became_peered":"2024-09-11T01:12:15.138823+0000","last_unstale":"2024-09-11T01:12:48.745089+0000","last_undegraded":"2024-09-11T01:12:15.123630+0000","last_fullsized":"2024-09-11T01:12:48.745089+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2833,"log_dups_size":0,"ondisk_log_size":2833,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11588818,"num_objects":2831,"num_object_clones":0,"num_object_copies":5662,"num_objects_missing_on_primary":2,"num_objects_missing":2,"num_objects_degraded":4,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2831,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2831,"num_write_kb":11318,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0,2","objects":2831}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.115056+0000","last_change":"2024-09-11T01:12:15.137008+0000","last_active":"2024-09-11T01:12:16.115056+0000","last_peered":"2024-09-11T01:12:16.115056+0000","last_clean":"2024-09-11T01:12:16.115056+0000","last_became_active":"2024-09-11T01:12:15.136222+0000","last_became_peered":"2024-09-11T01:12:15.136222+0000","last_unstale":"2024-09-11T01:12:16.115056+0000","last_undegraded":"2024-09-11T01:12:16.115056+0000","last_fullsized":"2024-09-11T01:12:16.115056+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:07:25.284013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.123820+0000","last_change":"2024-09-11T01:12:16.122836+0000","last_active":"2024-09-11T01:12:16.123820+0000","last_peered":"2024-09-11T01:12:16.123820+0000","last_clean":"2024-09-11T01:12:16.123820+0000","last_became_active":"2024-09-11T01:12:16.122572+0000","last_became_peered":"2024-09-11T01:12:16.122572+0000","last_unstale":"2024-09-11T01:12:16.123820+0000","last_undegraded":"2024-09-11T01:12:16.123820+0000","last_fullsized":"2024-09-11T01:12:16.123820+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:36:29.939442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"36'2737","reported_seq":2793,"reported_epoch":36,"state":"active+recovering+degraded","last_fresh":"2024-09-11T01:12:48.742257+0000","last_change":"2024-09-11T01:12:15.142056+0000","last_active":"2024-09-11T01:12:48.742257+0000","last_peered":"2024-09-11T01:12:48.742257+0000","last_clean":"2024-09-11T01:11:43.771457+0000","last_became_active":"2024-09-11T01:12:15.133897+0000","last_became_peered":"2024-09-11T01:12:15.133897+0000","last_unstale":"2024-09-11T01:12:48.742257+0000","last_undegraded":"2024-09-11T01:12:15.116261+0000","last_fullsized":"2024-09-11T01:12:48.742257+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":2737,"log_dups_size":0,"ondisk_log_size":2737,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11164068,"num_objects":2729,"num_object_clones":0,"num_object_copies":5458,"num_objects_missing_on_primary":3,"num_objects_missing":3,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2734,"num_write_kb":10907,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[{"shards":"0","objects":1},{"shards":"0,2","objects":2728}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":179740107,"num_objects":43898,"num_object_clones":0,"num_object_copies":87796,"num_objects_missing_on_primary":18,"num_objects_missing":18,"num_objects_degraded":31,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":43898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":34056,"num_write_kb":136110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":358699008,"data_stored":358580722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43961,"ondisk_log_size":43961,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182416,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":273012,"kb_used_data":173348,"kb_used_omap":4,"kb_used_meta":99643,"kb_avail":94098828,"statfs":{"total":96636764160,"available":96357199872,"internally_reserved":0,"allocated":177508352,"data_stored":177275542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":102035115},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,1,1,4,4,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986188,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":280456,"kb_used_data":172472,"kb_used_omap":4,"kb_used_meta":107963,"kb_avail":94091384,"statfs":{"total":96636764160,"available":96349577216,"internally_reserved":0,"allocated":176611328,"data_stored":176403469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":110554794},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,2,9,3],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":181399552,"data_stored":181378678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":17276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":177184768,"data_stored":177184768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:12:52.346 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 28 20 48768 48748 6.79956 4.36328 0.0185123 0.0114824 2024-09-11T01:12:53.346 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 29 20 49990 49970 6.72968 4.77344 0.010981 0.0116027 2024-09-11T01:12:54.347 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 30 20 51204 51184 6.66324 4.74219 0.0146432 0.0117185 2024-09-11T01:12:54.752 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:12:55.280 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:12:55.284 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:12:55.316 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":85,"stamp":"2024-09-11T01:12:54.552937+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":205594472,"num_objects":50207,"num_object_clones":0,"num_object_copies":100414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31328,"num_write_kb":125240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":50277,"ondisk_log_size":50277,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":613000,"kb_used_data":397032,"kb_used_omap":8,"kb_used_meta":215927,"kb_avail":188130680,"statfs":{"total":193273528320,"available":192645816320,"internally_reserved":0,"allocated":406560768,"data_stored":406129938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":8875,"internal_metadata":221109589},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,5,6,6,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":82748180,"num_objects":20192,"num_object_clones":0,"num_object_copies":40384,"num_objects_missing_on_primary":-24,"num_objects_missing":-24,"num_objects_degraded":-44,"num_objects_misplaced":0,"num_objects_unfound":-17,"num_objects_dirty":20192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4740,"num_write_kb":19046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006133"},"pg_stats":[{"pgid":"2.f","version":"36'3195","reported_seq":3246,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:53.748703+0000","last_change":"2024-09-11T01:12:15.141793+0000","last_active":"2024-09-11T01:12:53.748703+0000","last_peered":"2024-09-11T01:12:53.748703+0000","last_clean":"2024-09-11T01:12:53.748703+0000","last_became_active":"2024-09-11T01:12:15.141081+0000","last_became_peered":"2024-09-11T01:12:15.141081+0000","last_unstale":"2024-09-11T01:12:53.748703+0000","last_undegraded":"2024-09-11T01:12:53.748703+0000","last_fullsized":"2024-09-11T01:12:53.748703+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":3195,"log_dups_size":0,"ondisk_log_size":3195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:18:33.495210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13086720,"num_objects":3195,"num_object_clones":0,"num_object_copies":6390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3195,"num_write_kb":12780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"36'3193","reported_seq":3275,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:53.047752+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:12:53.047752+0000","last_peered":"2024-09-11T01:12:53.047752+0000","last_clean":"2024-09-11T01:12:53.047752+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:12:53.047752+0000","last_undegraded":"2024-09-11T01:12:53.047752+0000","last_fullsized":"2024-09-11T01:12:53.047752+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":3193,"log_dups_size":0,"ondisk_log_size":3193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13053952,"num_objects":3187,"num_object_clones":0,"num_object_copies":6374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1432,"num_write_kb":5728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"36'3038","reported_seq":3134,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:53.012581+0000","last_change":"2024-09-11T01:12:37.612304+0000","last_active":"2024-09-11T01:12:53.012581+0000","last_peered":"2024-09-11T01:12:53.012581+0000","last_clean":"2024-09-11T01:12:53.012581+0000","last_became_active":"2024-09-11T01:12:16.123243+0000","last_became_peered":"2024-09-11T01:12:16.123243+0000","last_unstale":"2024-09-11T01:12:53.012581+0000","last_undegraded":"2024-09-11T01:12:53.012581+0000","last_fullsized":"2024-09-11T01:12:53.012581+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":3038,"log_dups_size":0,"ondisk_log_size":3038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:22:39.057954+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12427264,"num_objects":3034,"num_object_clones":0,"num_object_copies":6068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1339,"num_write_kb":5356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"36'3092","reported_seq":3142,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:53.047254+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:12:53.047254+0000","last_peered":"2024-09-11T01:12:53.047254+0000","last_clean":"2024-09-11T01:12:53.047254+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:12:53.047254+0000","last_undegraded":"2024-09-11T01:12:53.047254+0000","last_fullsized":"2024-09-11T01:12:53.047254+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":3092,"log_dups_size":0,"ondisk_log_size":3092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12664832,"num_objects":3092,"num_object_clones":0,"num_object_copies":6184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3092,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3092,"num_write_kb":12368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"36'3129","reported_seq":3237,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:53.750333+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:12:53.750333+0000","last_peered":"2024-09-11T01:12:53.750333+0000","last_clean":"2024-09-11T01:12:53.750333+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:12:53.750333+0000","last_undegraded":"2024-09-11T01:12:53.750333+0000","last_fullsized":"2024-09-11T01:12:53.750333+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":3129,"log_dups_size":0,"ondisk_log_size":3129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12787712,"num_objects":3122,"num_object_clones":0,"num_object_copies":6244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":134,"num_write_kb":536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"36'3197","reported_seq":3265,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:53.692120+0000","last_change":"2024-09-11T01:12:49.870635+0000","last_active":"2024-09-11T01:12:53.692120+0000","last_peered":"2024-09-11T01:12:53.692120+0000","last_clean":"2024-09-11T01:12:53.692120+0000","last_became_active":"2024-09-11T01:12:15.141377+0000","last_became_peered":"2024-09-11T01:12:15.141377+0000","last_unstale":"2024-09-11T01:12:53.692120+0000","last_undegraded":"2024-09-11T01:12:53.692120+0000","last_fullsized":"2024-09-11T01:12:53.692120+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":3197,"log_dups_size":0,"ondisk_log_size":3197,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13075666,"num_objects":3194,"num_object_clones":0,"num_object_copies":6388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3195,"num_write_kb":12771,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"36'3142","reported_seq":3227,"reported_epoch":36,"state":"active+clean+scrubbing","last_fresh":"2024-09-11T01:12:53.761374+0000","last_change":"2024-09-11T01:12:53.091194+0000","last_active":"2024-09-11T01:12:53.761374+0000","last_peered":"2024-09-11T01:12:53.761374+0000","last_clean":"2024-09-11T01:12:53.761374+0000","last_became_active":"2024-09-11T01:12:15.135367+0000","last_became_peered":"2024-09-11T01:12:15.135367+0000","last_unstale":"2024-09-11T01:12:53.761374+0000","last_undegraded":"2024-09-11T01:12:53.761374+0000","last_fullsized":"2024-09-11T01:12:53.761374+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":1499,"log_size":3142,"log_dups_size":0,"ondisk_log_size":3142,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"scrubbing for 0s","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12846290,"num_objects":3138,"num_object_clones":0,"num_object_copies":6276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3140,"num_write_kb":12548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"36'3192","reported_seq":3302,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:53.750374+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:12:53.750374+0000","last_peered":"2024-09-11T01:12:53.750374+0000","last_clean":"2024-09-11T01:12:53.750374+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:12:53.750374+0000","last_undegraded":"2024-09-11T01:12:53.750374+0000","last_fullsized":"2024-09-11T01:12:53.750374+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":3192,"log_dups_size":0,"ondisk_log_size":3192,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13066240,"num_objects":3190,"num_object_clones":0,"num_object_copies":6380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68,"num_write_kb":272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.114317+0000","last_change":"2024-09-11T01:12:15.135174+0000","last_active":"2024-09-11T01:12:16.114317+0000","last_peered":"2024-09-11T01:12:16.114317+0000","last_clean":"2024-09-11T01:12:16.114317+0000","last_became_active":"2024-09-11T01:12:15.133597+0000","last_became_peered":"2024-09-11T01:12:15.133597+0000","last_unstale":"2024-09-11T01:12:16.114317+0000","last_undegraded":"2024-09-11T01:12:16.114317+0000","last_fullsized":"2024-09-11T01:12:16.114317+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:29:50.913351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"36'3057","reported_seq":3160,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:53.012721+0000","last_change":"2024-09-11T01:12:40.856584+0000","last_active":"2024-09-11T01:12:53.012721+0000","last_peered":"2024-09-11T01:12:53.012721+0000","last_clean":"2024-09-11T01:12:53.012721+0000","last_became_active":"2024-09-11T01:12:16.122119+0000","last_became_peered":"2024-09-11T01:12:16.122119+0000","last_unstale":"2024-09-11T01:12:53.012721+0000","last_undegraded":"2024-09-11T01:12:53.012721+0000","last_fullsized":"2024-09-11T01:12:53.012721+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":3057,"log_dups_size":0,"ondisk_log_size":3057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:55:43.522530+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12492800,"num_objects":3050,"num_object_clones":0,"num_object_copies":6100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1028,"num_write_kb":4112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.121218+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:12:16.121218+0000","last_peered":"2024-09-11T01:12:16.121218+0000","last_clean":"2024-09-11T01:12:16.121218+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:12:16.121218+0000","last_undegraded":"2024-09-11T01:12:16.121218+0000","last_fullsized":"2024-09-11T01:12:16.121218+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"36'3153","reported_seq":3214,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:53.735407+0000","last_change":"2024-09-11T01:12:49.762631+0000","last_active":"2024-09-11T01:12:53.735407+0000","last_peered":"2024-09-11T01:12:53.735407+0000","last_clean":"2024-09-11T01:12:53.735407+0000","last_became_active":"2024-09-11T01:12:15.140878+0000","last_became_peered":"2024-09-11T01:12:15.140878+0000","last_unstale":"2024-09-11T01:12:53.735407+0000","last_undegraded":"2024-09-11T01:12:53.735407+0000","last_fullsized":"2024-09-11T01:12:53.735407+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":3153,"log_dups_size":0,"ondisk_log_size":3153,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12883154,"num_objects":3147,"num_object_clones":0,"num_object_copies":6294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3151,"num_write_kb":12584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":71,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:49.898678+0000","last_change":"2024-09-11T01:12:49.898496+0000","last_active":"2024-09-11T01:12:49.898678+0000","last_peered":"2024-09-11T01:12:49.898678+0000","last_clean":"2024-09-11T01:12:49.898678+0000","last_became_active":"2024-09-11T01:12:15.139222+0000","last_became_peered":"2024-09-11T01:12:15.139222+0000","last_unstale":"2024-09-11T01:12:49.898678+0000","last_undegraded":"2024-09-11T01:12:49.898678+0000","last_fullsized":"2024-09-11T01:12:49.898678+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:58:51.211773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"36'3157","reported_seq":3249,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:53.067358+0000","last_change":"2024-09-11T01:12:41.868636+0000","last_active":"2024-09-11T01:12:53.067358+0000","last_peered":"2024-09-11T01:12:53.067358+0000","last_clean":"2024-09-11T01:12:53.067358+0000","last_became_active":"2024-09-11T01:12:16.120717+0000","last_became_peered":"2024-09-11T01:12:16.120717+0000","last_unstale":"2024-09-11T01:12:53.067358+0000","last_undegraded":"2024-09-11T01:12:53.067358+0000","last_fullsized":"2024-09-11T01:12:53.067358+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":3157,"log_dups_size":0,"ondisk_log_size":3157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:54:47.497574+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12910592,"num_objects":3152,"num_object_clones":0,"num_object_copies":6304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":974,"num_write_kb":3896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":16,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.122445+0000","last_change":"2024-09-11T01:12:16.118776+0000","last_active":"2024-09-11T01:12:16.122445+0000","last_peered":"2024-09-11T01:12:16.122445+0000","last_clean":"2024-09-11T01:12:16.122445+0000","last_became_active":"2024-09-11T01:12:16.117613+0000","last_became_peered":"2024-09-11T01:12:16.117613+0000","last_unstale":"2024-09-11T01:12:16.122445+0000","last_undegraded":"2024-09-11T01:12:16.122445+0000","last_fullsized":"2024-09-11T01:12:16.122445+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:23:46.399169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"36'3085","reported_seq":3177,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:53.041626+0000","last_change":"2024-09-11T01:12:42.891057+0000","last_active":"2024-09-11T01:12:53.041626+0000","last_peered":"2024-09-11T01:12:53.041626+0000","last_clean":"2024-09-11T01:12:53.041626+0000","last_became_active":"2024-09-11T01:12:16.119017+0000","last_became_peered":"2024-09-11T01:12:16.119017+0000","last_unstale":"2024-09-11T01:12:53.041626+0000","last_undegraded":"2024-09-11T01:12:53.041626+0000","last_fullsized":"2024-09-11T01:12:53.041626+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":3085,"log_dups_size":0,"ondisk_log_size":3085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:20:57.577195+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12611584,"num_objects":3079,"num_object_clones":0,"num_object_copies":6158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":870,"num_write_kb":3480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"36'3152","reported_seq":3212,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:53.765662+0000","last_change":"2024-09-11T01:12:49.519670+0000","last_active":"2024-09-11T01:12:53.765662+0000","last_peered":"2024-09-11T01:12:53.765662+0000","last_clean":"2024-09-11T01:12:53.765662+0000","last_became_active":"2024-09-11T01:12:15.134484+0000","last_became_peered":"2024-09-11T01:12:15.134484+0000","last_unstale":"2024-09-11T01:12:53.765662+0000","last_undegraded":"2024-09-11T01:12:53.765662+0000","last_fullsized":"2024-09-11T01:12:53.765662+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":3152,"log_dups_size":0,"ondisk_log_size":3152,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12903017,"num_objects":3151,"num_object_clones":0,"num_object_copies":6302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3151,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3151,"num_write_kb":12601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.115043+0000","last_change":"2024-09-11T01:12:15.137159+0000","last_active":"2024-09-11T01:12:16.115043+0000","last_peered":"2024-09-11T01:12:16.115043+0000","last_clean":"2024-09-11T01:12:16.115043+0000","last_became_active":"2024-09-11T01:12:15.136405+0000","last_became_peered":"2024-09-11T01:12:15.136405+0000","last_unstale":"2024-09-11T01:12:16.115043+0000","last_undegraded":"2024-09-11T01:12:16.115043+0000","last_fullsized":"2024-09-11T01:12:16.115043+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:32:43.772100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"36'3127","reported_seq":3196,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:53.736425+0000","last_change":"2024-09-11T01:12:49.682796+0000","last_active":"2024-09-11T01:12:53.736425+0000","last_peered":"2024-09-11T01:12:53.736425+0000","last_clean":"2024-09-11T01:12:53.736425+0000","last_became_active":"2024-09-11T01:12:15.140760+0000","last_became_peered":"2024-09-11T01:12:15.140760+0000","last_unstale":"2024-09-11T01:12:53.736425+0000","last_undegraded":"2024-09-11T01:12:53.736425+0000","last_fullsized":"2024-09-11T01:12:53.736425+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":3127,"log_dups_size":0,"ondisk_log_size":3127,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12777275,"num_objects":3122,"num_object_clones":0,"num_object_copies":6244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3124,"num_write_kb":12480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.123536+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:12:16.123536+0000","last_peered":"2024-09-11T01:12:16.123536+0000","last_clean":"2024-09-11T01:12:16.123536+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:12:16.123536+0000","last_undegraded":"2024-09-11T01:12:16.123536+0000","last_fullsized":"2024-09-11T01:12:16.123536+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"36'3226","reported_seq":3292,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:53.742514+0000","last_change":"2024-09-11T01:12:49.584819+0000","last_active":"2024-09-11T01:12:53.742514+0000","last_peered":"2024-09-11T01:12:53.742514+0000","last_clean":"2024-09-11T01:12:53.742514+0000","last_became_active":"2024-09-11T01:12:15.138823+0000","last_became_peered":"2024-09-11T01:12:15.138823+0000","last_unstale":"2024-09-11T01:12:53.742514+0000","last_undegraded":"2024-09-11T01:12:53.742514+0000","last_fullsized":"2024-09-11T01:12:53.742514+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":3226,"log_dups_size":0,"ondisk_log_size":3226,"stats_invalid":true,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13198546,"num_objects":3224,"num_object_clones":0,"num_object_copies":6448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3224,"num_write_kb":12890,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.115056+0000","last_change":"2024-09-11T01:12:15.137008+0000","last_active":"2024-09-11T01:12:16.115056+0000","last_peered":"2024-09-11T01:12:16.115056+0000","last_clean":"2024-09-11T01:12:16.115056+0000","last_became_active":"2024-09-11T01:12:15.136222+0000","last_became_peered":"2024-09-11T01:12:15.136222+0000","last_unstale":"2024-09-11T01:12:16.115056+0000","last_undegraded":"2024-09-11T01:12:16.115056+0000","last_fullsized":"2024-09-11T01:12:16.115056+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:07:25.284013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":77,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:16.123820+0000","last_change":"2024-09-11T01:12:16.122836+0000","last_active":"2024-09-11T01:12:16.123820+0000","last_peered":"2024-09-11T01:12:16.123820+0000","last_clean":"2024-09-11T01:12:16.123820+0000","last_became_active":"2024-09-11T01:12:16.122572+0000","last_became_peered":"2024-09-11T01:12:16.122572+0000","last_unstale":"2024-09-11T01:12:16.123820+0000","last_undegraded":"2024-09-11T01:12:16.123820+0000","last_fullsized":"2024-09-11T01:12:16.123820+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:36:29.939442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"36'3139","reported_seq":3247,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-11T01:12:53.750180+0000","last_change":"2024-09-11T01:12:51.740460+0000","last_active":"2024-09-11T01:12:53.750180+0000","last_peered":"2024-09-11T01:12:53.750180+0000","last_clean":"2024-09-11T01:12:53.750180+0000","last_became_active":"2024-09-11T01:12:15.133897+0000","last_became_peered":"2024-09-11T01:12:15.133897+0000","last_unstale":"2024-09-11T01:12:53.750180+0000","last_undegraded":"2024-09-11T01:12:53.750180+0000","last_fullsized":"2024-09-11T01:12:53.750180+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":3139,"log_dups_size":0,"ondisk_log_size":3139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:39:09.423755+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12808809,"num_objects":3128,"num_object_clones":0,"num_object_copies":6256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":209,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":205594453,"num_objects":50205,"num_object_clones":0,"num_object_copies":100410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31326,"num_write_kb":125238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":410406912,"data_stored":410302542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":50274,"ondisk_log_size":50274,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182417,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311200,"kb_used_data":199056,"kb_used_omap":4,"kb_used_meta":112123,"kb_avail":94060640,"statfs":{"total":96636764160,"available":96318095360,"internally_reserved":0,"allocated":203833344,"data_stored":203617929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":114814635},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,3,5,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986189,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301800,"kb_used_data":197976,"kb_used_omap":4,"kb_used_meta":103803,"kb_avail":94070040,"statfs":{"total":96636764160,"available":96327720960,"internally_reserved":0,"allocated":202727424,"data_stored":202512009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":106294954},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":0,"seq":0,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":206839808,"data_stored":206836329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":17276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":203452416,"data_stored":203448937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:12:55.318 INFO:tasks.rep_lost_unfound_delete.ceph_manager:recovered! 2024-09-11T01:12:55.318 INFO:tasks.util.rados:rados -p unfounddel_pool get new_1 - 2024-09-11T01:12:55.318 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_1 - 2024-09-11T01:12:55.347 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 31 20 52336 52316 6.59091 4.42188 0.0178772 0.0118472 2024-09-11T01:12:55.414 INFO:teuthology.orchestra.run.smithi122.stderr:error getting unfounddel_pool/new_1: (2) No such file or directory 2024-09-11T01:12:55.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:12:55.433 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_1 - 2024-09-11T01:12:55.434 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_1 - 2024-09-11T01:12:55.588 INFO:teuthology.orchestra.run.smithi122.stderr:error getting unfounddel_pool/existed_1: (2) No such file or directory 2024-09-11T01:12:55.604 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:12:55.605 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_1 - 2024-09-11T01:12:55.605 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_1 - 2024-09-11T01:12:55.695 INFO:teuthology.orchestra.run.smithi122.stderr:error getting unfounddel_pool/existing_1: (2) No such file or directory 2024-09-11T01:12:55.713 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:12:55.714 INFO:tasks.util.rados:rados -p unfounddel_pool get new_2 - 2024-09-11T01:12:55.714 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_2 - 2024-09-11T01:12:55.812 INFO:teuthology.orchestra.run.smithi122.stderr:error getting unfounddel_pool/new_2: (2) No such file or directory 2024-09-11T01:12:55.816 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:12:55.816 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_2 - 2024-09-11T01:12:55.816 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_2 - 2024-09-11T01:12:55.902 INFO:teuthology.orchestra.run.smithi122.stderr:error getting unfounddel_pool/existed_2: (2) No such file or directory 2024-09-11T01:12:55.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:12:55.913 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_2 - 2024-09-11T01:12:55.913 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_2 - 2024-09-11T01:12:56.001 INFO:teuthology.orchestra.run.smithi122.stderr:error getting unfounddel_pool/existing_2: (2) No such file or directory 2024-09-11T01:12:56.008 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:12:56.008 INFO:tasks.util.rados:rados -p unfounddel_pool get new_3 - 2024-09-11T01:12:56.008 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_3 - 2024-09-11T01:12:56.078 INFO:teuthology.orchestra.run.smithi122.stderr:error getting unfounddel_pool/new_3: (2) No such file or directory 2024-09-11T01:12:56.091 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:12:56.092 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_3 - 2024-09-11T01:12:56.092 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_3 - 2024-09-11T01:12:56.196 INFO:teuthology.orchestra.run.smithi122.stderr:error getting unfounddel_pool/existed_3: (2) No such file or directory 2024-09-11T01:12:56.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:12:56.216 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_3 - 2024-09-11T01:12:56.216 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_3 - 2024-09-11T01:12:56.294 INFO:teuthology.orchestra.run.smithi122.stderr:error getting unfounddel_pool/existing_3: (2) No such file or directory 2024-09-11T01:12:56.315 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:12:56.316 INFO:tasks.util.rados:rados -p unfounddel_pool get new_4 - 2024-09-11T01:12:56.316 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_4 - 2024-09-11T01:12:56.347 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 32 20 53383 53363 6.51275 4.08984 0.0151021 0.0119868 2024-09-11T01:12:56.414 INFO:teuthology.orchestra.run.smithi122.stderr:error getting unfounddel_pool/new_4: (2) No such file or directory 2024-09-11T01:12:56.422 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:12:56.422 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_4 - 2024-09-11T01:12:56.422 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_4 - 2024-09-11T01:12:56.645 INFO:teuthology.orchestra.run.smithi122.stderr:error getting unfounddel_pool/existed_4: (2) No such file or directory 2024-09-11T01:12:56.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:12:56.650 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_4 - 2024-09-11T01:12:56.650 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_4 - 2024-09-11T01:12:56.728 INFO:teuthology.orchestra.run.smithi122.stderr:error getting unfounddel_pool/existing_4: (2) No such file or directory 2024-09-11T01:12:56.733 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:12:56.733 INFO:tasks.util.rados:rados -p unfounddel_pool get new_5 - 2024-09-11T01:12:56.733 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_5 - 2024-09-11T01:12:56.820 INFO:teuthology.orchestra.run.smithi122.stderr:error getting unfounddel_pool/new_5: (2) No such file or directory 2024-09-11T01:12:56.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:12:56.831 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_5 - 2024-09-11T01:12:56.831 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_5 - 2024-09-11T01:12:56.906 INFO:teuthology.orchestra.run.smithi122.stderr:error getting unfounddel_pool/existed_5: (2) No such file or directory 2024-09-11T01:12:56.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:12:56.946 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_5 - 2024-09-11T01:12:56.946 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_5 - 2024-09-11T01:12:57.017 INFO:teuthology.orchestra.run.smithi122.stderr:error getting unfounddel_pool/existing_5: (2) No such file or directory 2024-09-11T01:12:57.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:12:57.055 INFO:tasks.util.rados:rados -p unfounddel_pool get new_6 - 2024-09-11T01:12:57.055 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_6 - 2024-09-11T01:12:57.146 INFO:teuthology.orchestra.run.smithi122.stderr:error getting unfounddel_pool/new_6: (2) No such file or directory 2024-09-11T01:12:57.151 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:12:57.151 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_6 - 2024-09-11T01:12:57.152 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_6 - 2024-09-11T01:12:57.215 INFO:teuthology.orchestra.run.smithi122.stderr:error getting unfounddel_pool/existed_6: (2) No such file or directory 2024-09-11T01:12:57.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:12:57.222 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_6 - 2024-09-11T01:12:57.222 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_6 - 2024-09-11T01:12:57.300 INFO:teuthology.orchestra.run.smithi122.stderr:error getting unfounddel_pool/existing_6: (2) No such file or directory 2024-09-11T01:12:57.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:12:57.312 INFO:tasks.util.rados:rados -p unfounddel_pool get new_7 - 2024-09-11T01:12:57.312 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_7 - 2024-09-11T01:12:57.347 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 33 20 54304 54284 6.42441 3.59766 0.0390738 0.0121538 2024-09-11T01:12:57.390 INFO:teuthology.orchestra.run.smithi122.stderr:error getting unfounddel_pool/new_7: (2) No such file or directory 2024-09-11T01:12:57.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:12:57.398 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_7 - 2024-09-11T01:12:57.398 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_7 - 2024-09-11T01:12:57.535 INFO:teuthology.orchestra.run.smithi122.stderr:error getting unfounddel_pool/existed_7: (2) No such file or directory 2024-09-11T01:12:57.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:12:57.539 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_7 - 2024-09-11T01:12:57.539 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_7 - 2024-09-11T01:12:57.640 INFO:teuthology.orchestra.run.smithi122.stderr:error getting unfounddel_pool/existing_7: (2) No such file or directory 2024-09-11T01:12:57.649 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:12:57.649 INFO:tasks.util.rados:rados -p unfounddel_pool get new_8 - 2024-09-11T01:12:57.650 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_8 - 2024-09-11T01:12:57.745 INFO:teuthology.orchestra.run.smithi122.stderr:error getting unfounddel_pool/new_8: (2) No such file or directory 2024-09-11T01:12:57.748 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:12:57.749 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_8 - 2024-09-11T01:12:57.749 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_8 - 2024-09-11T01:12:57.839 INFO:teuthology.orchestra.run.smithi122.stderr:error getting unfounddel_pool/existed_8: (2) No such file or directory 2024-09-11T01:12:57.857 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:12:57.858 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_8 - 2024-09-11T01:12:57.858 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_8 - 2024-09-11T01:12:57.935 INFO:teuthology.orchestra.run.smithi122.stderr:error getting unfounddel_pool/existing_8: (2) No such file or directory 2024-09-11T01:12:57.938 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:12:57.938 INFO:tasks.util.rados:rados -p unfounddel_pool get new_9 - 2024-09-11T01:12:57.938 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get new_9 - 2024-09-11T01:12:58.034 INFO:teuthology.orchestra.run.smithi122.stderr:error getting unfounddel_pool/new_9: (2) No such file or directory 2024-09-11T01:12:58.049 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:12:58.050 INFO:tasks.util.rados:rados -p unfounddel_pool get existed_9 - 2024-09-11T01:12:58.050 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existed_9 - 2024-09-11T01:12:58.205 INFO:teuthology.orchestra.run.smithi122.stderr:error getting unfounddel_pool/existed_9: (2) No such file or directory 2024-09-11T01:12:58.211 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:12:58.212 INFO:tasks.util.rados:rados -p unfounddel_pool get existing_9 - 2024-09-11T01:12:58.212 DEBUG:teuthology.orchestra.run.smithi122:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados -p unfounddel_pool get existing_9 - 2024-09-11T01:12:58.297 INFO:teuthology.orchestra.run.smithi122.stderr:error getting unfounddel_pool/existing_9: (2) No such file or directory 2024-09-11T01:12:58.301 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:12:58.301 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 1 2024-09-11T01:12:58.347 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 34 20 55357 55337 6.35642 4.11328 0.0161319 0.0122824 2024-09-11T01:12:58.913 INFO:tasks.ceph.mon.a.smithi122.stderr:2024-09-11T01:12:58.908+0000 7f9dc4120640 -1 mon.a@0(leader).osd e36 definitely_dead 0 2024-09-11T01:12:59.348 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 35 20 56478 56458 6.29992 4.37891 0.013815 0.0123945 2024-09-11T01:12:59.849 INFO:teuthology.orchestra.run.smithi122.stderr:marked in osd.1. 2024-09-11T01:12:59.878 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-11T01:12:59.878 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-11T01:12:59.922 INFO:tasks.ceph.osd.1:Started 2024-09-11T01:12:59.922 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T01:13:00.022 INFO:teuthology.orchestra.run.smithi122.stderr:no valid command found; 10 closest matches: 2024-09-11T01:13:00.022 INFO:teuthology.orchestra.run.smithi122.stderr:0 2024-09-11T01:13:00.023 INFO:teuthology.orchestra.run.smithi122.stderr:1 2024-09-11T01:13:00.023 INFO:teuthology.orchestra.run.smithi122.stderr:2 2024-09-11T01:13:00.023 INFO:teuthology.orchestra.run.smithi122.stderr:abort 2024-09-11T01:13:00.023 INFO:teuthology.orchestra.run.smithi122.stderr:assert 2024-09-11T01:13:00.023 INFO:teuthology.orchestra.run.smithi122.stderr:config diff 2024-09-11T01:13:00.023 INFO:teuthology.orchestra.run.smithi122.stderr:config diff get 2024-09-11T01:13:00.023 INFO:teuthology.orchestra.run.smithi122.stderr:config get 2024-09-11T01:13:00.023 INFO:teuthology.orchestra.run.smithi122.stderr:config help [] 2024-09-11T01:13:00.023 INFO:teuthology.orchestra.run.smithi122.stderr:config set ... 2024-09-11T01:13:00.023 INFO:teuthology.orchestra.run.smithi122.stderr:admin_socket: invalid command 2024-09-11T01:13:00.025 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-11T01:13:00.025 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-11T01:13:00.348 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 36 20 57068 57048 6.18894 2.30469 0.0166564 0.0124439 2024-09-11T01:13:01.222 INFO:tasks.ceph.osd.1.smithi122.stderr:2024-09-11T01:13:01.216+0000 7f6dc67267c0 -1 Falling back to public interface 2024-09-11T01:13:01.348 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 37 20 57068 57048 6.02168 0 - 0.0124439 2024-09-11T01:13:02.348 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 38 20 57531 57511 5.91081 0.904297 0.0246556 0.0132104 2024-09-11T01:13:03.348 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 39 20 58642 58622 5.87052 4.33984 0.0216341 0.0133003 2024-09-11T01:13:04.348 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout:2024-09-11T01:13:04.348819+0000 min lat: 0.00219431 max lat: 2.11988 avg lat: 0.0133776 2024-09-11T01:13:04.349 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T01:13:04.349 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 40 20 59800 59780 5.83684 4.52344 0.0200504 0.0133776 2024-09-11T01:13:05.026 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T01:13:05.272 INFO:teuthology.orchestra.run.smithi122.stderr:no valid command found; 10 closest matches: 2024-09-11T01:13:05.272 INFO:teuthology.orchestra.run.smithi122.stderr:0 2024-09-11T01:13:05.272 INFO:teuthology.orchestra.run.smithi122.stderr:1 2024-09-11T01:13:05.272 INFO:teuthology.orchestra.run.smithi122.stderr:2 2024-09-11T01:13:05.272 INFO:teuthology.orchestra.run.smithi122.stderr:abort 2024-09-11T01:13:05.272 INFO:teuthology.orchestra.run.smithi122.stderr:assert 2024-09-11T01:13:05.272 INFO:teuthology.orchestra.run.smithi122.stderr:bluefs debug_inject_read_zeros 2024-09-11T01:13:05.273 INFO:teuthology.orchestra.run.smithi122.stderr:bluefs files list 2024-09-11T01:13:05.273 INFO:teuthology.orchestra.run.smithi122.stderr:bluefs stats 2024-09-11T01:13:05.273 INFO:teuthology.orchestra.run.smithi122.stderr:bluestore allocator dump block 2024-09-11T01:13:05.273 INFO:teuthology.orchestra.run.smithi122.stderr:bluestore allocator fragmentation block 2024-09-11T01:13:05.273 INFO:teuthology.orchestra.run.smithi122.stderr:admin_socket: invalid command 2024-09-11T01:13:05.280 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-11T01:13:05.281 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting on admin_socket for osd-1, ['dump_ops_in_flight'] 2024-09-11T01:13:05.350 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 41 20 60931 60911 5.80193 4.41797 0.01309 0.0134585 2024-09-11T01:13:06.351 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 42 20 62103 62083 5.77279 4.57812 0.0113332 0.0135259 2024-09-11T01:13:06.491 INFO:tasks.ceph.osd.1.smithi122.stderr:2024-09-11T01:13:06.484+0000 7f6dc67267c0 -1 osd.1 26 log_to_monitors true 2024-09-11T01:13:06.637 INFO:tasks.ceph.osd.1.smithi122.stderr:2024-09-11T01:13:06.632+0000 7f6dbe509640 -1 osd.1 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-11T01:13:07.351 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 43 20 63260 63240 5.74364 4.51953 0.0164661 0.0135948 2024-09-11T01:13:08.351 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 44 20 63577 63557 5.64125 1.23828 0.0324019 0.0136201 2024-09-11T01:13:09.351 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 45 20 63577 63557 5.5159 0 - 0.0136201 2024-09-11T01:13:10.282 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-09-11T01:13:10.351 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 46 20 63639 63619 5.40127 0.121094 0.0129281 0.0144514 2024-09-11T01:13:10.589 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-11T01:13:10.589 INFO:teuthology.orchestra.run.smithi122.stdout: "ops": [ 2024-09-11T01:13:10.589 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.590 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(client.5211.0:63754 2.d e43/42 2:bf3849e2:::benchmark_data_smithi122_33997_object63753:head v 43'3908, mlcod=23'3)", 2024-09-11T01:13:10.590 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-11T01:13:10.564504+0000", 2024-09-11T01:13:10.590 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.024016767000000001, 2024-09-11T01:13:10.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.024040458000000001, 2024-09-11T01:13:10.590 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-11T01:13:10.590 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-11T01:13:10.590 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-11T01:13:10.590 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-11T01:13:10.590 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.590 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-11T01:13:10.590 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.564504+0000", 2024-09-11T01:13:10.590 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-11T01:13:10.590 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-11T01:13:10.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.564504+0000", 2024-09-11T01:13:10.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-11T01:13:10.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-11T01:13:10.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.564506+0000", 2024-09-11T01:13:10.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4820000000000001e-06 2024-09-11T01:13:10.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.591 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.591 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-11T01:13:10.591 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.564525+0000", 2024-09-11T01:13:10.591 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8766999999999998e-05 2024-09-11T01:13:10.591 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-11T01:13:10.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.564541+0000", 2024-09-11T01:13:10.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5772000000000001e-05 2024-09-11T01:13:10.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-11T01:13:10.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.564551+0000", 2024-09-11T01:13:10.592 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9080000000000007e-06 2024-09-11T01:13:10.592 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.592 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.592 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-11T01:13:10.592 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.570011+0000", 2024-09-11T01:13:10.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0054608 2024-09-11T01:13:10.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.593 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-11T01:13:10.593 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.570036+0000", 2024-09-11T01:13:10.593 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.5001999999999999e-05 2024-09-11T01:13:10.593 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:13:10.593 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-11T01:13:10.593 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:13:10.593 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.593 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.594 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(client.5211.0:63752 2.f e43/42 2:f85e1507:::benchmark_data_smithi122_33997_object63751:head v 43'4086, mlcod=0'0)", 2024-09-11T01:13:10.594 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-11T01:13:10.565960+0000", 2024-09-11T01:13:10.594 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.022560507, 2024-09-11T01:13:10.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.022641036999999999, 2024-09-11T01:13:10.594 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-11T01:13:10.594 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-11T01:13:10.594 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-11T01:13:10.594 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-11T01:13:10.594 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.594 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-11T01:13:10.594 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.565960+0000", 2024-09-11T01:13:10.594 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-11T01:13:10.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-11T01:13:10.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.565960+0000", 2024-09-11T01:13:10.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-11T01:13:10.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.595 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.595 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-11T01:13:10.595 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.565962+0000", 2024-09-11T01:13:10.595 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7099999999999999e-06 2024-09-11T01:13:10.595 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.596 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-11T01:13:10.596 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.565975+0000", 2024-09-11T01:13:10.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3176999999999999e-05 2024-09-11T01:13:10.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.596 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-11T01:13:10.596 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.565990+0000", 2024-09-11T01:13:10.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5443000000000001e-05 2024-09-11T01:13:10.596 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.596 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.596 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-11T01:13:10.596 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.565998+0000", 2024-09-11T01:13:10.596 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.055e-06 2024-09-11T01:13:10.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.597 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.597 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-11T01:13:10.597 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.579146+0000", 2024-09-11T01:13:10.597 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.013147667 2024-09-11T01:13:10.597 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.597 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.597 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-11T01:13:10.597 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.579174+0000", 2024-09-11T01:13:10.597 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.8436999999999999e-05 2024-09-11T01:13:10.597 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:13:10.597 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-11T01:13:10.597 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:13:10.598 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.598 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(client.5211.0:63753 2.5 e43/42 2:aeac8d24:::benchmark_data_smithi122_33997_object63752:head v 43'4055, mlcod=23'2)", 2024-09-11T01:13:10.598 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-11T01:13:10.566905+0000", 2024-09-11T01:13:10.598 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.021615821, 2024-09-11T01:13:10.598 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.021748506000000001, 2024-09-11T01:13:10.598 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-11T01:13:10.598 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-11T01:13:10.598 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-11T01:13:10.598 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-11T01:13:10.598 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.598 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-11T01:13:10.598 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.566905+0000", 2024-09-11T01:13:10.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-11T01:13:10.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.599 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-11T01:13:10.599 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.566905+0000", 2024-09-11T01:13:10.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-11T01:13:10.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.599 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-11T01:13:10.599 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.566906+0000", 2024-09-11T01:13:10.599 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.096e-06 2024-09-11T01:13:10.599 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.599 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.599 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-11T01:13:10.600 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.566914+0000", 2024-09-11T01:13:10.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.1699999999999997e-06 2024-09-11T01:13:10.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.600 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.600 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-11T01:13:10.600 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.566936+0000", 2024-09-11T01:13:10.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.2016999999999999e-05 2024-09-11T01:13:10.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.600 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.600 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-11T01:13:10.600 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.566944+0000", 2024-09-11T01:13:10.600 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2500000000000006e-06 2024-09-11T01:13:10.600 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.601 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.601 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-11T01:13:10.601 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.568598+0000", 2024-09-11T01:13:10.601 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0016536820000000001 2024-09-11T01:13:10.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.601 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.601 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-11T01:13:10.601 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.568616+0000", 2024-09-11T01:13:10.601 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.8638999999999999e-05 2024-09-11T01:13:10.601 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:13:10.601 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-11T01:13:10.601 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:13:10.601 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.602 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.602 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(client.5211.0:63756 2.a e43/42 2:5389e7ad:::benchmark_data_smithi122_33997_object63755:head v 43'3966, mlcod=23'3)", 2024-09-11T01:13:10.602 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-11T01:13:10.575568+0000", 2024-09-11T01:13:10.602 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.012951944, 2024-09-11T01:13:10.602 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.013134313, 2024-09-11T01:13:10.602 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-11T01:13:10.602 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-11T01:13:10.602 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-11T01:13:10.602 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-11T01:13:10.602 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.602 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-11T01:13:10.602 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.575568+0000", 2024-09-11T01:13:10.602 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-11T01:13:10.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.603 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-11T01:13:10.603 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.575568+0000", 2024-09-11T01:13:10.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-11T01:13:10.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.603 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-11T01:13:10.603 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.575570+0000", 2024-09-11T01:13:10.603 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5230000000000001e-06 2024-09-11T01:13:10.603 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.603 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.603 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-11T01:13:10.603 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.575582+0000", 2024-09-11T01:13:10.604 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.239e-05 2024-09-11T01:13:10.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.604 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-11T01:13:10.604 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.575593+0000", 2024-09-11T01:13:10.604 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0691e-05 2024-09-11T01:13:10.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.604 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-11T01:13:10.604 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.575602+0000", 2024-09-11T01:13:10.604 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.6289999999999999e-06 2024-09-11T01:13:10.604 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.604 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.604 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-11T01:13:10.605 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.575840+0000", 2024-09-11T01:13:10.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.000238667 2024-09-11T01:13:10.605 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.605 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.605 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-11T01:13:10.605 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.575861+0000", 2024-09-11T01:13:10.605 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.109e-05 2024-09-11T01:13:10.605 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:13:10.605 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-11T01:13:10.606 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:13:10.606 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.606 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(client.5211.0:63757 2.1 e43/42 2:8312b3ea:::benchmark_data_smithi122_33997_object63756:head v 43'4045, mlcod=23'3)", 2024-09-11T01:13:10.606 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-11T01:13:10.577917+0000", 2024-09-11T01:13:10.606 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.010603801, 2024-09-11T01:13:10.606 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.010837253, 2024-09-11T01:13:10.606 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-11T01:13:10.606 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-11T01:13:10.606 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-11T01:13:10.606 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-11T01:13:10.606 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.607 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-11T01:13:10.607 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.577917+0000", 2024-09-11T01:13:10.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-11T01:13:10.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.607 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-11T01:13:10.607 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.577917+0000", 2024-09-11T01:13:10.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-11T01:13:10.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.607 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.607 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-11T01:13:10.607 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.577918+0000", 2024-09-11T01:13:10.607 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.7430000000000001e-06 2024-09-11T01:13:10.607 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.608 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-11T01:13:10.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.577930+0000", 2024-09-11T01:13:10.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2149e-05 2024-09-11T01:13:10.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.608 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-11T01:13:10.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.577940+0000", 2024-09-11T01:13:10.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8989999999999996e-06 2024-09-11T01:13:10.608 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.608 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.608 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-11T01:13:10.608 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.577947+0000", 2024-09-11T01:13:10.608 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.7170000000000002e-06 2024-09-11T01:13:10.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.609 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-11T01:13:10.609 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.579204+0000", 2024-09-11T01:13:10.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.001257359 2024-09-11T01:13:10.609 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.609 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.609 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-11T01:13:10.609 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.579229+0000", 2024-09-11T01:13:10.609 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4541000000000001e-05 2024-09-11T01:13:10.609 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:13:10.609 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-11T01:13:10.609 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:13:10.610 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.610 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(client.5211.0:63761 2.1 e43/42)", 2024-09-11T01:13:10.610 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-11T01:13:10.583600+0000", 2024-09-11T01:13:10.610 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0049202079999999997, 2024-09-11T01:13:10.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0051999309999999997, 2024-09-11T01:13:10.610 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-11T01:13:10.610 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-11T01:13:10.610 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-11T01:13:10.610 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-11T01:13:10.610 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.610 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-11T01:13:10.610 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.583600+0000", 2024-09-11T01:13:10.610 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-11T01:13:10.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-11T01:13:10.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.583600+0000", 2024-09-11T01:13:10.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-11T01:13:10.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-11T01:13:10.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.583602+0000", 2024-09-11T01:13:10.611 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3379999999999999e-06 2024-09-11T01:13:10.611 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.611 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.611 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-11T01:13:10.611 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.583612+0000", 2024-09-11T01:13:10.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0271e-05 2024-09-11T01:13:10.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-11T01:13:10.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.583628+0000", 2024-09-11T01:13:10.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5942000000000001e-05 2024-09-11T01:13:10.612 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.612 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.612 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-11T01:13:10.612 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.583636+0000", 2024-09-11T01:13:10.612 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.3469999999999999e-06 2024-09-11T01:13:10.612 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:13:10.612 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-11T01:13:10.613 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:13:10.613 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.613 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(client.5211.0:63766 2.9 e43/42)", 2024-09-11T01:13:10.613 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-11T01:13:10.584397+0000", 2024-09-11T01:13:10.613 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0041229200000000004, 2024-09-11T01:13:10.613 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0044394170000000002, 2024-09-11T01:13:10.613 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-11T01:13:10.613 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-11T01:13:10.613 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-11T01:13:10.613 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-11T01:13:10.613 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.613 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-11T01:13:10.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.584397+0000", 2024-09-11T01:13:10.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-11T01:13:10.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-11T01:13:10.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.584397+0000", 2024-09-11T01:13:10.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-11T01:13:10.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.614 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.614 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-11T01:13:10.614 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.584399+0000", 2024-09-11T01:13:10.614 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3009999999999999e-06 2024-09-11T01:13:10.614 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.615 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-11T01:13:10.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.584410+0000", 2024-09-11T01:13:10.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0746e-05 2024-09-11T01:13:10.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.615 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-11T01:13:10.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.584422+0000", 2024-09-11T01:13:10.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.2865e-05 2024-09-11T01:13:10.615 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.615 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.615 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-11T01:13:10.615 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.584430+0000", 2024-09-11T01:13:10.615 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 7.2350000000000002e-06 2024-09-11T01:13:10.616 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:13:10.616 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-11T01:13:10.616 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:13:10.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.616 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.616 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(client.5211.0:63764 2.4 e43/42 2:23d6ea5b:::benchmark_data_smithi122_33997_object63763:head v 43'3937, mlcod=23'9)", 2024-09-11T01:13:10.616 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-11T01:13:10.584442+0000", 2024-09-11T01:13:10.616 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0040782880000000002, 2024-09-11T01:13:10.616 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0044349510000000003, 2024-09-11T01:13:10.616 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-11T01:13:10.616 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-11T01:13:10.617 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-11T01:13:10.617 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-11T01:13:10.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.617 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-11T01:13:10.617 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.584442+0000", 2024-09-11T01:13:10.617 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-11T01:13:10.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.617 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-11T01:13:10.617 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.584442+0000", 2024-09-11T01:13:10.617 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-11T01:13:10.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.617 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.618 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-11T01:13:10.618 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.584443+0000", 2024-09-11T01:13:10.618 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.0900000000000002e-07 2024-09-11T01:13:10.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.618 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.618 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-11T01:13:10.618 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.584452+0000", 2024-09-11T01:13:10.618 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.0110000000000006e-06 2024-09-11T01:13:10.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.618 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.618 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-11T01:13:10.618 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.584456+0000", 2024-09-11T01:13:10.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.899e-06 2024-09-11T01:13:10.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.619 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.619 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-11T01:13:10.619 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.584462+0000", 2024-09-11T01:13:10.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 5.3449999999999996e-06 2024-09-11T01:13:10.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.619 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.619 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-11T01:13:10.619 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.584558+0000", 2024-09-11T01:13:10.619 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.6082000000000006e-05 2024-09-11T01:13:10.619 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.619 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.619 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-11T01:13:10.619 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.584581+0000", 2024-09-11T01:13:10.620 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.3249000000000001e-05 2024-09-11T01:13:10.620 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:13:10.620 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-11T01:13:10.620 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:13:10.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.620 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.620 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(client.5211.0:63767 2.a e43/42)", 2024-09-11T01:13:10.620 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-11T01:13:10.584475+0000", 2024-09-11T01:13:10.620 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0040452040000000002, 2024-09-11T01:13:10.620 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0044503110000000002, 2024-09-11T01:13:10.620 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-11T01:13:10.620 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-11T01:13:10.620 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-11T01:13:10.620 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-11T01:13:10.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.621 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-11T01:13:10.621 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.584475+0000", 2024-09-11T01:13:10.621 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-11T01:13:10.621 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.621 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-11T01:13:10.621 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.584475+0000", 2024-09-11T01:13:10.621 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-11T01:13:10.621 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.621 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.621 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-11T01:13:10.621 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.584476+0000", 2024-09-11T01:13:10.621 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.6199999999999999e-07 2024-09-11T01:13:10.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.622 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.622 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-11T01:13:10.622 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.584482+0000", 2024-09-11T01:13:10.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.3840000000000002e-06 2024-09-11T01:13:10.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.622 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.622 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-11T01:13:10.622 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.584487+0000", 2024-09-11T01:13:10.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 4.707e-06 2024-09-11T01:13:10.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.622 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.622 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-11T01:13:10.622 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.584490+0000", 2024-09-11T01:13:10.622 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 3.49e-06 2024-09-11T01:13:10.622 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:13:10.623 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-11T01:13:10.623 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:13:10.623 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.623 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(client.5211.0:63771 2.5 e43/42)", 2024-09-11T01:13:10.623 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-11T01:13:10.584735+0000", 2024-09-11T01:13:10.623 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0037851320000000001, 2024-09-11T01:13:10.623 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.0042280039999999996, 2024-09-11T01:13:10.623 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-11T01:13:10.623 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-11T01:13:10.623 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-11T01:13:10.623 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-11T01:13:10.623 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.624 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-11T01:13:10.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.584735+0000", 2024-09-11T01:13:10.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-11T01:13:10.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.624 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.624 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-11T01:13:10.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.584735+0000", 2024-09-11T01:13:10.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-11T01:13:10.624 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.624 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.624 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-11T01:13:10.624 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.584736+0000", 2024-09-11T01:13:10.624 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.217e-06 2024-09-11T01:13:10.625 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.625 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-11T01:13:10.625 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.584745+0000", 2024-09-11T01:13:10.625 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 8.2800000000000003e-06 2024-09-11T01:13:10.625 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.625 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-11T01:13:10.625 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.584755+0000", 2024-09-11T01:13:10.625 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.9049999999999992e-06 2024-09-11T01:13:10.625 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.625 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.625 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-11T01:13:10.626 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.584761+0000", 2024-09-11T01:13:10.626 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 6.2380000000000002e-06 2024-09-11T01:13:10.626 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:13:10.626 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-11T01:13:10.626 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:13:10.626 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.626 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.626 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(client.5211.0:63769 2.1 e43/42)", 2024-09-11T01:13:10.626 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-11T01:13:10.586978+0000", 2024-09-11T01:13:10.626 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.0015425040000000001, 2024-09-11T01:13:10.626 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.002022364, 2024-09-11T01:13:10.626 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-11T01:13:10.626 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-11T01:13:10.626 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "queued for pg", 2024-09-11T01:13:10.626 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-11T01:13:10.627 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.627 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-11T01:13:10.627 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.586978+0000", 2024-09-11T01:13:10.627 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-11T01:13:10.627 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.627 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.627 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-11T01:13:10.627 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.586978+0000", 2024-09-11T01:13:10.627 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-11T01:13:10.627 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.627 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.627 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-11T01:13:10.627 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.586980+0000", 2024-09-11T01:13:10.627 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 2.4540000000000001e-06 2024-09-11T01:13:10.627 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.627 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.628 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-11T01:13:10.628 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.586996+0000", 2024-09-11T01:13:10.628 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.5225000000000001e-05 2024-09-11T01:13:10.628 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.628 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.628 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-11T01:13:10.628 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.587008+0000", 2024-09-11T01:13:10.628 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.1932e-05 2024-09-11T01:13:10.628 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.628 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.628 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-11T01:13:10.628 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.587017+0000", 2024-09-11T01:13:10.628 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.8500000000000006e-06 2024-09-11T01:13:10.628 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:13:10.629 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-11T01:13:10.629 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:13:10.629 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.629 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.629 INFO:teuthology.orchestra.run.smithi122.stdout: "description": "osd_repop(client.5211.0:63740 2.d e43/42 2:baa6e918:::benchmark_data_smithi122_33997_object63739:head v 43'3907, mlcod=23'3)", 2024-09-11T01:13:10.629 INFO:teuthology.orchestra.run.smithi122.stdout: "initiated_at": "2024-09-11T01:13:10.522762+0000", 2024-09-11T01:13:10.629 INFO:teuthology.orchestra.run.smithi122.stdout: "age": 0.065757942999999999, 2024-09-11T01:13:10.629 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.066279827999999999, 2024-09-11T01:13:10.629 INFO:teuthology.orchestra.run.smithi122.stdout: "continuous": false, 2024-09-11T01:13:10.629 INFO:teuthology.orchestra.run.smithi122.stdout: "type_data": { 2024-09-11T01:13:10.629 INFO:teuthology.orchestra.run.smithi122.stdout: "flag_point": "started", 2024-09-11T01:13:10.629 INFO:teuthology.orchestra.run.smithi122.stdout: "events": [ 2024-09-11T01:13:10.629 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.629 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "initiated", 2024-09-11T01:13:10.630 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.522762+0000", 2024-09-11T01:13:10.630 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-11T01:13:10.630 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.630 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.630 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "header_read", 2024-09-11T01:13:10.630 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.522762+0000", 2024-09-11T01:13:10.630 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0 2024-09-11T01:13:10.630 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.630 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.630 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "throttled", 2024-09-11T01:13:10.630 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.522764+0000", 2024-09-11T01:13:10.630 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9309999999999998e-06 2024-09-11T01:13:10.630 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.630 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.631 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "all_read", 2024-09-11T01:13:10.631 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.522775+0000", 2024-09-11T01:13:10.631 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.0485999999999999e-05 2024-09-11T01:13:10.631 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.631 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.631 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "dispatched", 2024-09-11T01:13:10.631 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.522789+0000", 2024-09-11T01:13:10.631 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.3634e-05 2024-09-11T01:13:10.631 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.631 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.631 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "queued_for_pg", 2024-09-11T01:13:10.631 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.522798+0000", 2024-09-11T01:13:10.631 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 9.6819999999999998e-06 2024-09-11T01:13:10.631 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.631 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.631 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "reached_pg", 2024-09-11T01:13:10.632 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.562990+0000", 2024-09-11T01:13:10.632 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 0.040191575 2024-09-11T01:13:10.632 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-11T01:13:10.632 INFO:teuthology.orchestra.run.smithi122.stdout: { 2024-09-11T01:13:10.632 INFO:teuthology.orchestra.run.smithi122.stdout: "event": "started", 2024-09-11T01:13:10.632 INFO:teuthology.orchestra.run.smithi122.stdout: "time": "2024-09-11T01:13:10.563009+0000", 2024-09-11T01:13:10.632 INFO:teuthology.orchestra.run.smithi122.stdout: "duration": 1.9279e-05 2024-09-11T01:13:10.632 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:13:10.632 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-11T01:13:10.632 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:13:10.632 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-11T01:13:10.632 INFO:teuthology.orchestra.run.smithi122.stdout: ], 2024-09-11T01:13:10.633 INFO:teuthology.orchestra.run.smithi122.stdout: "num_ops": 12 2024-09-11T01:13:10.633 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-11T01:13:10.634 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for osd.1 to be up 2024-09-11T01:13:10.634 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T01:13:11.352 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 47 20 64247 64227 5.3368 2.375 0.00846024 0.0146293 2024-09-11T01:13:11.481 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:13:11.481 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":43,"fsid":"6d328773-3914-4412-91b7-393379984de3","created":"2024-09-11T01:11:03.508628+0000","modified":"2024-09-11T01:13:09.637223+0000","last_up_change":"2024-09-11T01:13:07.625340+0000","last_in_change":"2024-09-11T01:12:59.834020+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-11T01:11:18.999872+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfounddel_pool","create_time":"2024-09-11T01:11:30.626364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e160603b-65bb-420f-9130-6d7c6506f2a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":21,"up_from":25,"up_thru":42,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6800","nonce":2613596984}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6801","nonce":2613596984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6803","nonce":2613596984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6802","nonce":2613596984}]},"public_addr":"172.21.15.122:6800/2613596984","cluster_addr":"172.21.15.122:6801/2613596984","heartbeat_back_addr":"172.21.15.122:6803/2613596984","heartbeat_front_addr":"172.21.15.122:6802/2613596984","state":["exists","up"]},{"osd":1,"uuid":"6fbfe9e1-4255-4c44-b619-5fd57486e5fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":26,"up_from":41,"up_thru":42,"down_at":27,"lost_at":27,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6808","nonce":3142337928}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6809","nonce":3142337928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6811","nonce":3142337928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6810","nonce":3142337928}]},"public_addr":"172.21.15.122:6808/3142337928","cluster_addr":"172.21.15.122:6809/3142337928","heartbeat_back_addr":"172.21.15.122:6811/3142337928","heartbeat_front_addr":"172.21.15.122:6810/3142337928","state":["exists","up"]},{"osd":2,"uuid":"e27771fd-ac9d-4a64-a01f-465a5acade91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":17,"up_from":31,"up_thru":42,"down_at":19,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6804","nonce":1506556606}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6805","nonce":1506556606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6807","nonce":1506556606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6806","nonce":1506556606}]},"public_addr":"172.21.15.122:6804/1506556606","cluster_addr":"172.21.15.122:6805/1506556606","heartbeat_back_addr":"172.21.15.122:6807/1506556606","heartbeat_front_addr":"172.21.15.122:6806/1506556606","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:11:12.059865+0000","dead_epoch":22},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:11:11.519118+0000","dead_epoch":27},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:11:11.402743+0000","dead_epoch":19}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[{"pgid":"2.0","osds":[2]},{"pgid":"2.1","osds":[2]},{"pgid":"2.2","osds":[0]},{"pgid":"2.3","osds":[2]},{"pgid":"2.4","osds":[0]},{"pgid":"2.5","osds":[0]},{"pgid":"2.6","osds":[0]},{"pgid":"2.7","osds":[0]},{"pgid":"2.9","osds":[0]},{"pgid":"2.a","osds":[0]},{"pgid":"2.d","osds":[2]},{"pgid":"2.f","osds":[0]}],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T01:13:11.576 INFO:tasks.rep_lost_unfound_delete.ceph_manager:osd.1 is up 2024-09-11T01:13:11.576 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for clean 2024-09-11T01:13:11.576 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:13:12.340 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:13:12.341 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:13:12.354 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 48 20 64851 64831 5.27477 2.35938 0.0778043 0.0147995 2024-09-11T01:13:12.397 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2024-09-11T01:13:10.556011+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":260391548,"num_objects":63575,"num_object_clones":0,"num_object_copies":127150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24506,"num_write_kb":98018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4115,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63657,"ondisk_log_size":63657,"up":45,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":72,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":821060,"kb_used_data":509588,"kb_used_omap":12,"kb_used_meta":311411,"kb_avail":282294460,"statfs":{"total":289910292480,"available":289069527040,"internally_reserved":0,"allocated":521818112,"data_stored":521082415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":318884868},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,7,8,13,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19213102,"num_objects":4689,"num_object_clones":0,"num_object_copies":9378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-44126,"num_objects_unfound":0,"num_objects_dirty":4689,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1034,"num_write_kb":4145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000943"},"pg_stats":[{"pgid":"2.f","version":"40'4072","reported_seq":4138,"reported_epoch":41,"state":"remapped+peering","last_fresh":"2024-09-11T01:13:07.648985+0000","last_change":"2024-09-11T01:13:07.648985+0000","last_active":"2024-09-11T01:13:07.648985+0000","last_peered":"2024-09-11T01:13:07.617385+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:01.915463+0000","last_became_peered":"2024-09-11T01:13:01.915463+0000","last_unstale":"2024-09-11T01:13:07.648985+0000","last_undegraded":"2024-09-11T01:13:07.648985+0000","last_fullsized":"2024-09-11T01:13:07.648985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":4072,"log_dups_size":0,"ondisk_log_size":4072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16678912,"num_objects":4072,"num_object_clones":0,"num_object_copies":8144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4072,"num_write_kb":16288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"40'4095","reported_seq":4187,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:13:07.655439+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:13:07.655439+0000","last_peered":"2024-09-11T01:13:07.655439+0000","last_clean":"2024-09-11T01:13:07.655439+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:13:07.655439+0000","last_undegraded":"2024-09-11T01:13:07.655439+0000","last_fullsized":"2024-09-11T01:13:07.655439+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":4095,"log_dups_size":0,"ondisk_log_size":4095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16748544,"num_objects":4089,"num_object_clones":0,"num_object_copies":8178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2334,"num_write_kb":9336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"40'3900","reported_seq":4014,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-11T01:13:07.633706+0000","last_change":"2024-09-11T01:13:07.651253+0000","last_active":"2024-09-11T01:13:07.651253+0000","last_peered":"2024-09-11T01:13:07.633706+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:01.925595+0000","last_became_peered":"2024-09-11T01:13:01.925595+0000","last_unstale":"2024-09-11T01:13:07.633706+0000","last_undegraded":"2024-09-11T01:13:07.633706+0000","last_fullsized":"2024-09-11T01:13:07.633706+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":3900,"log_dups_size":0,"ondisk_log_size":3900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15958016,"num_objects":3896,"num_object_clones":0,"num_object_copies":7792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2201,"num_write_kb":8804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"40'3922","reported_seq":3982,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:13:07.653838+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:13:07.653838+0000","last_peered":"2024-09-11T01:13:07.653838+0000","last_clean":"2024-09-11T01:13:07.653838+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:13:07.653838+0000","last_undegraded":"2024-09-11T01:13:07.653838+0000","last_fullsized":"2024-09-11T01:13:07.653838+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":3922,"log_dups_size":0,"ondisk_log_size":3922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16064512,"num_objects":3922,"num_object_clones":0,"num_object_copies":7844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3922,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3922,"num_write_kb":15688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"40'3948","reported_seq":4068,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-11T01:13:08.648931+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:13:08.648931+0000","last_peered":"2024-09-11T01:13:08.648931+0000","last_clean":"2024-09-11T01:13:08.648931+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:13:08.648931+0000","last_undegraded":"2024-09-11T01:13:08.648931+0000","last_fullsized":"2024-09-11T01:13:08.648931+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":3948,"log_dups_size":0,"ondisk_log_size":3948,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16142336,"num_objects":3941,"num_object_clones":0,"num_object_copies":7882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":953,"num_write_kb":3812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"40'3956","reported_seq":4115,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-11T01:13:08.647339+0000","last_change":"2024-09-11T01:13:07.651527+0000","last_active":"2024-09-11T01:13:07.651527+0000","last_peered":"2024-09-11T01:13:07.619558+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:01.908550+0000","last_became_peered":"2024-09-11T01:13:01.908550+0000","last_unstale":"2024-09-11T01:13:08.647339+0000","last_undegraded":"2024-09-11T01:13:08.647339+0000","last_fullsized":"2024-09-11T01:13:08.647339+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":3956,"log_dups_size":0,"ondisk_log_size":3956,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16183296,"num_objects":3951,"num_object_clones":0,"num_object_copies":7902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":297,"num_write_kb":1188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"40'3992","reported_seq":4111,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-11T01:13:08.646907+0000","last_change":"2024-09-11T01:13:07.647899+0000","last_active":"2024-09-11T01:13:07.647899+0000","last_peered":"2024-09-11T01:13:07.631917+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:01.889432+0000","last_became_peered":"2024-09-11T01:13:01.889432+0000","last_unstale":"2024-09-11T01:13:08.646907+0000","last_undegraded":"2024-09-11T01:13:08.646907+0000","last_fullsized":"2024-09-11T01:13:08.646907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":3992,"log_dups_size":0,"ondisk_log_size":3992,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16326656,"num_objects":3986,"num_object_clones":0,"num_object_copies":7972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3986,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":849,"num_write_kb":3396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"40'4006","reported_seq":4128,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-11T01:13:08.645694+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:13:08.645694+0000","last_peered":"2024-09-11T01:13:08.645694+0000","last_clean":"2024-09-11T01:13:08.645694+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:13:08.645694+0000","last_undegraded":"2024-09-11T01:13:08.645694+0000","last_fullsized":"2024-09-11T01:13:08.645694+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":4006,"log_dups_size":0,"ondisk_log_size":4006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16400384,"num_objects":4004,"num_object_clones":0,"num_object_copies":8008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":882,"num_write_kb":3528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":86,"reported_epoch":39,"state":"active+clean+remapped","last_fresh":"2024-09-11T01:13:01.873502+0000","last_change":"2024-09-11T01:13:01.873131+0000","last_active":"2024-09-11T01:13:01.873502+0000","last_peered":"2024-09-11T01:13:01.873502+0000","last_clean":"2024-09-11T01:13:01.873502+0000","last_became_active":"2024-09-11T01:13:01.872241+0000","last_became_peered":"2024-09-11T01:13:01.872241+0000","last_unstale":"2024-09-11T01:13:01.873502+0000","last_undegraded":"2024-09-11T01:13:01.873502+0000","last_fullsized":"2024-09-11T01:13:01.873502+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:53:13.046059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"40'3940","reported_seq":4054,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-11T01:13:07.618645+0000","last_change":"2024-09-11T01:13:07.654816+0000","last_active":"2024-09-11T01:13:07.654816+0000","last_peered":"2024-09-11T01:13:07.618645+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:01.883873+0000","last_became_peered":"2024-09-11T01:13:01.883873+0000","last_unstale":"2024-09-11T01:13:07.618645+0000","last_undegraded":"2024-09-11T01:13:07.618645+0000","last_fullsized":"2024-09-11T01:13:07.618645+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":3940,"log_dups_size":0,"ondisk_log_size":3940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16109568,"num_objects":3933,"num_object_clones":0,"num_object_copies":7866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3933,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1911,"num_write_kb":7644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":60,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:13:07.656178+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:07.656178+0000","last_peered":"2024-09-11T01:13:07.656178+0000","last_clean":"2024-09-11T01:13:07.656178+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:07.656178+0000","last_undegraded":"2024-09-11T01:13:07.656178+0000","last_fullsized":"2024-09-11T01:13:07.656178+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"40'3992","reported_seq":4109,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-11T01:13:08.648018+0000","last_change":"2024-09-11T01:13:07.652309+0000","last_active":"2024-09-11T01:13:07.652309+0000","last_peered":"2024-09-11T01:13:07.608035+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:01.889901+0000","last_became_peered":"2024-09-11T01:13:01.889901+0000","last_unstale":"2024-09-11T01:13:08.648018+0000","last_undegraded":"2024-09-11T01:13:08.648018+0000","last_fullsized":"2024-09-11T01:13:08.648018+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":3992,"log_dups_size":0,"ondisk_log_size":3992,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16318464,"num_objects":3984,"num_object_clones":0,"num_object_copies":7968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":579,"num_write_kb":2316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":86,"reported_epoch":41,"state":"peering","last_fresh":"2024-09-11T01:13:07.648396+0000","last_change":"2024-09-11T01:13:08.644294+0000","last_active":"2024-09-11T01:13:07.648396+0000","last_peered":"2024-09-11T01:13:06.627322+0000","last_clean":"2024-09-11T01:13:06.627322+0000","last_became_active":"2024-09-11T01:13:01.889113+0000","last_became_peered":"2024-09-11T01:13:01.889113+0000","last_unstale":"2024-09-11T01:13:07.648396+0000","last_undegraded":"2024-09-11T01:13:07.648396+0000","last_fullsized":"2024-09-11T01:13:07.648396+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"40'4029","reported_seq":4134,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-11T01:13:07.645090+0000","last_change":"2024-09-11T01:13:07.654842+0000","last_active":"2024-09-11T01:13:07.654842+0000","last_peered":"2024-09-11T01:13:07.645090+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:01.883768+0000","last_became_peered":"2024-09-11T01:13:01.883768+0000","last_unstale":"2024-09-11T01:13:07.645090+0000","last_undegraded":"2024-09-11T01:13:07.645090+0000","last_fullsized":"2024-09-11T01:13:07.645090+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":4029,"log_dups_size":0,"ondisk_log_size":4029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16482304,"num_objects":4024,"num_object_clones":0,"num_object_copies":8048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1846,"num_write_kb":7384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":30,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-11T01:13:06.624730+0000","last_change":"2024-09-11T01:13:07.652757+0000","last_active":"2024-09-11T01:13:07.652757+0000","last_peered":"2024-09-11T01:13:06.624730+0000","last_clean":"2024-09-11T01:13:06.624730+0000","last_became_active":"2024-09-11T01:13:01.884701+0000","last_became_peered":"2024-09-11T01:13:01.884701+0000","last_unstale":"2024-09-11T01:13:06.624730+0000","last_undegraded":"2024-09-11T01:13:06.624730+0000","last_fullsized":"2024-09-11T01:13:06.624730+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"40'3931","reported_seq":4036,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-11T01:13:07.638709+0000","last_change":"2024-09-11T01:13:07.651382+0000","last_active":"2024-09-11T01:13:07.651382+0000","last_peered":"2024-09-11T01:13:07.638709+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:01.883991+0000","last_became_peered":"2024-09-11T01:13:01.883991+0000","last_unstale":"2024-09-11T01:13:07.638709+0000","last_undegraded":"2024-09-11T01:13:07.638709+0000","last_fullsized":"2024-09-11T01:13:07.638709+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":3931,"log_dups_size":0,"ondisk_log_size":3931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16076800,"num_objects":3925,"num_object_clones":0,"num_object_copies":7850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1716,"num_write_kb":6864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"40'3952","reported_seq":4072,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-11T01:13:08.649631+0000","last_change":"2024-09-11T01:13:07.647752+0000","last_active":"2024-09-11T01:13:07.647752+0000","last_peered":"2024-09-11T01:13:07.595114+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:01.912973+0000","last_became_peered":"2024-09-11T01:13:01.912973+0000","last_unstale":"2024-09-11T01:13:08.649631+0000","last_undegraded":"2024-09-11T01:13:08.649631+0000","last_fullsized":"2024-09-11T01:13:08.649631+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":3952,"log_dups_size":0,"ondisk_log_size":3952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16179200,"num_objects":3950,"num_object_clones":0,"num_object_copies":7900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":689,"num_write_kb":2756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":86,"reported_epoch":39,"state":"active+clean+remapped","last_fresh":"2024-09-11T01:13:01.909241+0000","last_change":"2024-09-11T01:13:01.908928+0000","last_active":"2024-09-11T01:13:01.909241+0000","last_peered":"2024-09-11T01:13:01.909241+0000","last_clean":"2024-09-11T01:13:01.909241+0000","last_became_active":"2024-09-11T01:13:01.907932+0000","last_became_peered":"2024-09-11T01:13:01.907932+0000","last_unstale":"2024-09-11T01:13:01.909241+0000","last_undegraded":"2024-09-11T01:13:01.909241+0000","last_fullsized":"2024-09-11T01:13:01.909241+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:20:40.632243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"40'3947","reported_seq":4072,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-11T01:13:08.646918+0000","last_change":"2024-09-11T01:13:07.650604+0000","last_active":"2024-09-11T01:13:07.650604+0000","last_peered":"2024-09-11T01:13:07.624565+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:01.908162+0000","last_became_peered":"2024-09-11T01:13:01.908162+0000","last_unstale":"2024-09-11T01:13:08.646918+0000","last_undegraded":"2024-09-11T01:13:08.646918+0000","last_fullsized":"2024-09-11T01:13:08.646918+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":3947,"log_dups_size":0,"ondisk_log_size":3947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16134144,"num_objects":3939,"num_object_clones":0,"num_object_copies":7878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3939,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":602,"num_write_kb":2408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":60,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:13:07.653860+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:07.653860+0000","last_peered":"2024-09-11T01:13:07.653860+0000","last_clean":"2024-09-11T01:13:07.653860+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:07.653860+0000","last_undegraded":"2024-09-11T01:13:07.653860+0000","last_fullsized":"2024-09-11T01:13:07.653860+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"40'4043","reported_seq":4166,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-11T01:13:08.650688+0000","last_change":"2024-09-11T01:13:07.648243+0000","last_active":"2024-09-11T01:13:07.648243+0000","last_peered":"2024-09-11T01:13:07.635968+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:01.912996+0000","last_became_peered":"2024-09-11T01:13:01.912996+0000","last_unstale":"2024-09-11T01:13:08.650688+0000","last_undegraded":"2024-09-11T01:13:08.650688+0000","last_fullsized":"2024-09-11T01:13:08.650688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":4043,"log_dups_size":0,"ondisk_log_size":4043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16543744,"num_objects":4039,"num_object_clones":0,"num_object_copies":8078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":652,"num_write_kb":2608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":86,"reported_epoch":39,"state":"active+clean+remapped","last_fresh":"2024-09-11T01:13:01.910047+0000","last_change":"2024-09-11T01:13:01.909737+0000","last_active":"2024-09-11T01:13:01.910047+0000","last_peered":"2024-09-11T01:13:01.910047+0000","last_clean":"2024-09-11T01:13:01.910047+0000","last_became_active":"2024-09-11T01:13:01.907844+0000","last_became_peered":"2024-09-11T01:13:01.907844+0000","last_unstale":"2024-09-11T01:13:01.910047+0000","last_undegraded":"2024-09-11T01:13:01.910047+0000","last_fullsized":"2024-09-11T01:13:01.910047+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:41:49.801613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":91,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-11T01:13:06.626255+0000","last_change":"2024-09-11T01:13:07.650807+0000","last_active":"2024-09-11T01:13:07.650807+0000","last_peered":"2024-09-11T01:13:06.626255+0000","last_clean":"2024-09-11T01:13:06.626255+0000","last_became_active":"2024-09-11T01:13:01.883627+0000","last_became_peered":"2024-09-11T01:13:01.883627+0000","last_unstale":"2024-09-11T01:13:06.626255+0000","last_undegraded":"2024-09-11T01:13:06.626255+0000","last_fullsized":"2024-09-11T01:13:06.626255+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"40'3929","reported_seq":4051,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-11T01:13:08.651653+0000","last_change":"2024-09-11T01:13:07.652539+0000","last_active":"2024-09-11T01:13:07.652539+0000","last_peered":"2024-09-11T01:13:07.636621+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:01.889393+0000","last_became_peered":"2024-09-11T01:13:01.889393+0000","last_unstale":"2024-09-11T01:13:08.651653+0000","last_undegraded":"2024-09-11T01:13:08.651653+0000","last_fullsized":"2024-09-11T01:13:08.651653+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":3929,"log_dups_size":0,"ondisk_log_size":3929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16044649,"num_objects":3918,"num_object_clones":0,"num_object_copies":7836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":999,"num_write_kb":3996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":260391529,"num_objects":63573,"num_object_clones":0,"num_object_copies":127146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24504,"num_write_kb":98016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":520896512,"data_stored":520792142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63654,"ondisk_log_size":63654,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":13,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182420,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":394544,"kb_used_data":254624,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":93977296,"statfs":{"total":96636764160,"available":96232751104,"internally_reserved":0,"allocated":260734976,"data_stored":260496073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":143257259},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,7,6,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986192,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":398712,"kb_used_data":254632,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93973128,"statfs":{"total":96636764160,"available":96228483072,"internally_reserved":0,"allocated":260743168,"data_stored":260504265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":147517098},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,5,1,7,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":0,"seq":1,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27804,"kb_used_data":332,"kb_used_omap":4,"kb_used_meta":27451,"kb_avail":94344036,"statfs":{"total":96636764160,"available":96608292864,"internally_reserved":0,"allocated":339968,"data_stored":82077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":28110511},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":260386816,"data_stored":260383337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":17276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":260395008,"data_stored":260391529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:13:12.398 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:13:13.339 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:13:13.340 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:13:13.352 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 49 20 65459 65439 5.2156 2.375 0.00691923 0.0149664 2024-09-11T01:13:13.386 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2024-09-11T01:13:12.556405+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":260391548,"num_objects":63575,"num_object_clones":0,"num_object_copies":127150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24506,"num_write_kb":98018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4115,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63657,"ondisk_log_size":63657,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":841388,"kb_used_data":511868,"kb_used_omap":12,"kb_used_meta":329459,"kb_avail":282274132,"statfs":{"total":289910292480,"available":289048711168,"internally_reserved":0,"allocated":524152832,"data_stored":523349669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":337366020},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,7,8,13,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19213102,"num_objects":4689,"num_object_clones":0,"num_object_copies":9378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-44126,"num_objects_unfound":0,"num_objects_dirty":4689,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1034,"num_write_kb":4145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.937150"},"pg_stats":[{"pgid":"2.f","version":"40'4072","reported_seq":4138,"reported_epoch":41,"state":"remapped+peering","last_fresh":"2024-09-11T01:13:07.648985+0000","last_change":"2024-09-11T01:13:07.648985+0000","last_active":"2024-09-11T01:13:07.648985+0000","last_peered":"2024-09-11T01:13:07.617385+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:01.915463+0000","last_became_peered":"2024-09-11T01:13:01.915463+0000","last_unstale":"2024-09-11T01:13:07.648985+0000","last_undegraded":"2024-09-11T01:13:07.648985+0000","last_fullsized":"2024-09-11T01:13:07.648985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":4072,"log_dups_size":0,"ondisk_log_size":4072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16678912,"num_objects":4072,"num_object_clones":0,"num_object_copies":8144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4072,"num_write_kb":16288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"40'4095","reported_seq":4187,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:13:07.655439+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:13:07.655439+0000","last_peered":"2024-09-11T01:13:07.655439+0000","last_clean":"2024-09-11T01:13:07.655439+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:13:07.655439+0000","last_undegraded":"2024-09-11T01:13:07.655439+0000","last_fullsized":"2024-09-11T01:13:07.655439+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":4095,"log_dups_size":0,"ondisk_log_size":4095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16748544,"num_objects":4089,"num_object_clones":0,"num_object_copies":8178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2334,"num_write_kb":9336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"40'3900","reported_seq":4014,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-11T01:13:07.633706+0000","last_change":"2024-09-11T01:13:07.651253+0000","last_active":"2024-09-11T01:13:07.651253+0000","last_peered":"2024-09-11T01:13:07.633706+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:01.925595+0000","last_became_peered":"2024-09-11T01:13:01.925595+0000","last_unstale":"2024-09-11T01:13:07.633706+0000","last_undegraded":"2024-09-11T01:13:07.633706+0000","last_fullsized":"2024-09-11T01:13:07.633706+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":3900,"log_dups_size":0,"ondisk_log_size":3900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15958016,"num_objects":3896,"num_object_clones":0,"num_object_copies":7792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2201,"num_write_kb":8804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"40'3922","reported_seq":3982,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:13:07.653838+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:13:07.653838+0000","last_peered":"2024-09-11T01:13:07.653838+0000","last_clean":"2024-09-11T01:13:07.653838+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:13:07.653838+0000","last_undegraded":"2024-09-11T01:13:07.653838+0000","last_fullsized":"2024-09-11T01:13:07.653838+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":3922,"log_dups_size":0,"ondisk_log_size":3922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16064512,"num_objects":3922,"num_object_clones":0,"num_object_copies":7844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3922,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3922,"num_write_kb":15688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"40'3948","reported_seq":4068,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-11T01:13:08.648931+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:13:08.648931+0000","last_peered":"2024-09-11T01:13:08.648931+0000","last_clean":"2024-09-11T01:13:08.648931+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:13:08.648931+0000","last_undegraded":"2024-09-11T01:13:08.648931+0000","last_fullsized":"2024-09-11T01:13:08.648931+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":3948,"log_dups_size":0,"ondisk_log_size":3948,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16142336,"num_objects":3941,"num_object_clones":0,"num_object_copies":7882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":953,"num_write_kb":3812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"40'3956","reported_seq":4115,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-11T01:13:08.647339+0000","last_change":"2024-09-11T01:13:07.651527+0000","last_active":"2024-09-11T01:13:07.651527+0000","last_peered":"2024-09-11T01:13:07.619558+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:01.908550+0000","last_became_peered":"2024-09-11T01:13:01.908550+0000","last_unstale":"2024-09-11T01:13:08.647339+0000","last_undegraded":"2024-09-11T01:13:08.647339+0000","last_fullsized":"2024-09-11T01:13:08.647339+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":3956,"log_dups_size":0,"ondisk_log_size":3956,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16183296,"num_objects":3951,"num_object_clones":0,"num_object_copies":7902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":297,"num_write_kb":1188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"40'3992","reported_seq":4111,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-11T01:13:08.646907+0000","last_change":"2024-09-11T01:13:07.647899+0000","last_active":"2024-09-11T01:13:07.647899+0000","last_peered":"2024-09-11T01:13:07.631917+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:01.889432+0000","last_became_peered":"2024-09-11T01:13:01.889432+0000","last_unstale":"2024-09-11T01:13:08.646907+0000","last_undegraded":"2024-09-11T01:13:08.646907+0000","last_fullsized":"2024-09-11T01:13:08.646907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":3992,"log_dups_size":0,"ondisk_log_size":3992,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16326656,"num_objects":3986,"num_object_clones":0,"num_object_copies":7972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3986,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":849,"num_write_kb":3396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"40'4006","reported_seq":4128,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-11T01:13:08.645694+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:13:08.645694+0000","last_peered":"2024-09-11T01:13:08.645694+0000","last_clean":"2024-09-11T01:13:08.645694+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:13:08.645694+0000","last_undegraded":"2024-09-11T01:13:08.645694+0000","last_fullsized":"2024-09-11T01:13:08.645694+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":4006,"log_dups_size":0,"ondisk_log_size":4006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16400384,"num_objects":4004,"num_object_clones":0,"num_object_copies":8008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":882,"num_write_kb":3528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"40'3940","reported_seq":4054,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-11T01:13:07.618645+0000","last_change":"2024-09-11T01:13:07.654816+0000","last_active":"2024-09-11T01:13:07.654816+0000","last_peered":"2024-09-11T01:13:07.618645+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:01.883873+0000","last_became_peered":"2024-09-11T01:13:01.883873+0000","last_unstale":"2024-09-11T01:13:07.618645+0000","last_undegraded":"2024-09-11T01:13:07.618645+0000","last_fullsized":"2024-09-11T01:13:07.618645+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":3940,"log_dups_size":0,"ondisk_log_size":3940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16109568,"num_objects":3933,"num_object_clones":0,"num_object_copies":7866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3933,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1911,"num_write_kb":7644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":60,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:13:07.656178+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:07.656178+0000","last_peered":"2024-09-11T01:13:07.656178+0000","last_clean":"2024-09-11T01:13:07.656178+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:07.656178+0000","last_undegraded":"2024-09-11T01:13:07.656178+0000","last_fullsized":"2024-09-11T01:13:07.656178+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"40'3992","reported_seq":4109,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-11T01:13:08.648018+0000","last_change":"2024-09-11T01:13:07.652309+0000","last_active":"2024-09-11T01:13:07.652309+0000","last_peered":"2024-09-11T01:13:07.608035+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:01.889901+0000","last_became_peered":"2024-09-11T01:13:01.889901+0000","last_unstale":"2024-09-11T01:13:08.648018+0000","last_undegraded":"2024-09-11T01:13:08.648018+0000","last_fullsized":"2024-09-11T01:13:08.648018+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":3992,"log_dups_size":0,"ondisk_log_size":3992,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16318464,"num_objects":3984,"num_object_clones":0,"num_object_copies":7968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":579,"num_write_kb":2316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":86,"reported_epoch":41,"state":"peering","last_fresh":"2024-09-11T01:13:07.648396+0000","last_change":"2024-09-11T01:13:08.644294+0000","last_active":"2024-09-11T01:13:07.648396+0000","last_peered":"2024-09-11T01:13:06.627322+0000","last_clean":"2024-09-11T01:13:06.627322+0000","last_became_active":"2024-09-11T01:13:01.889113+0000","last_became_peered":"2024-09-11T01:13:01.889113+0000","last_unstale":"2024-09-11T01:13:07.648396+0000","last_undegraded":"2024-09-11T01:13:07.648396+0000","last_fullsized":"2024-09-11T01:13:07.648396+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"40'4029","reported_seq":4134,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-11T01:13:07.645090+0000","last_change":"2024-09-11T01:13:07.654842+0000","last_active":"2024-09-11T01:13:07.654842+0000","last_peered":"2024-09-11T01:13:07.645090+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:01.883768+0000","last_became_peered":"2024-09-11T01:13:01.883768+0000","last_unstale":"2024-09-11T01:13:07.645090+0000","last_undegraded":"2024-09-11T01:13:07.645090+0000","last_fullsized":"2024-09-11T01:13:07.645090+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":4029,"log_dups_size":0,"ondisk_log_size":4029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16482304,"num_objects":4024,"num_object_clones":0,"num_object_copies":8048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1846,"num_write_kb":7384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"40'3931","reported_seq":4036,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-11T01:13:07.638709+0000","last_change":"2024-09-11T01:13:07.651382+0000","last_active":"2024-09-11T01:13:07.651382+0000","last_peered":"2024-09-11T01:13:07.638709+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:01.883991+0000","last_became_peered":"2024-09-11T01:13:01.883991+0000","last_unstale":"2024-09-11T01:13:07.638709+0000","last_undegraded":"2024-09-11T01:13:07.638709+0000","last_fullsized":"2024-09-11T01:13:07.638709+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":3931,"log_dups_size":0,"ondisk_log_size":3931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16076800,"num_objects":3925,"num_object_clones":0,"num_object_copies":7850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1716,"num_write_kb":6864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"40'3952","reported_seq":4072,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-11T01:13:08.649631+0000","last_change":"2024-09-11T01:13:07.647752+0000","last_active":"2024-09-11T01:13:07.647752+0000","last_peered":"2024-09-11T01:13:07.595114+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:01.912973+0000","last_became_peered":"2024-09-11T01:13:01.912973+0000","last_unstale":"2024-09-11T01:13:08.649631+0000","last_undegraded":"2024-09-11T01:13:08.649631+0000","last_fullsized":"2024-09-11T01:13:08.649631+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":3952,"log_dups_size":0,"ondisk_log_size":3952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16179200,"num_objects":3950,"num_object_clones":0,"num_object_copies":7900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":689,"num_write_kb":2756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"40'3947","reported_seq":4072,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-11T01:13:08.646918+0000","last_change":"2024-09-11T01:13:07.650604+0000","last_active":"2024-09-11T01:13:07.650604+0000","last_peered":"2024-09-11T01:13:07.624565+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:01.908162+0000","last_became_peered":"2024-09-11T01:13:01.908162+0000","last_unstale":"2024-09-11T01:13:08.646918+0000","last_undegraded":"2024-09-11T01:13:08.646918+0000","last_fullsized":"2024-09-11T01:13:08.646918+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":3947,"log_dups_size":0,"ondisk_log_size":3947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16134144,"num_objects":3939,"num_object_clones":0,"num_object_copies":7878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3939,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":602,"num_write_kb":2408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":60,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:13:07.653860+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:07.653860+0000","last_peered":"2024-09-11T01:13:07.653860+0000","last_clean":"2024-09-11T01:13:07.653860+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:07.653860+0000","last_undegraded":"2024-09-11T01:13:07.653860+0000","last_fullsized":"2024-09-11T01:13:07.653860+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"40'4043","reported_seq":4166,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-11T01:13:08.650688+0000","last_change":"2024-09-11T01:13:07.648243+0000","last_active":"2024-09-11T01:13:07.648243+0000","last_peered":"2024-09-11T01:13:07.635968+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:01.912996+0000","last_became_peered":"2024-09-11T01:13:01.912996+0000","last_unstale":"2024-09-11T01:13:08.650688+0000","last_undegraded":"2024-09-11T01:13:08.650688+0000","last_fullsized":"2024-09-11T01:13:08.650688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":4043,"log_dups_size":0,"ondisk_log_size":4043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16543744,"num_objects":4039,"num_object_clones":0,"num_object_copies":8078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":652,"num_write_kb":2608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"40'3929","reported_seq":4051,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-11T01:13:08.651653+0000","last_change":"2024-09-11T01:13:07.652539+0000","last_active":"2024-09-11T01:13:07.652539+0000","last_peered":"2024-09-11T01:13:07.636621+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:01.889393+0000","last_became_peered":"2024-09-11T01:13:01.889393+0000","last_unstale":"2024-09-11T01:13:08.651653+0000","last_undegraded":"2024-09-11T01:13:08.651653+0000","last_fullsized":"2024-09-11T01:13:08.651653+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":3929,"log_dups_size":0,"ondisk_log_size":3929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16044649,"num_objects":3918,"num_object_clones":0,"num_object_copies":7836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":999,"num_write_kb":3996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":260391529,"num_objects":63573,"num_object_clones":0,"num_object_copies":127146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24504,"num_write_kb":98016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":523538432,"data_stored":523451457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63654,"ondisk_log_size":63654,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182420,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":394544,"kb_used_data":254624,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":93977296,"statfs":{"total":96636764160,"available":96232751104,"internally_reserved":0,"allocated":260734976,"data_stored":260496073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":143257259},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,7,6,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986192,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":398712,"kb_used_data":254632,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93973128,"statfs":{"total":96636764160,"available":96228483072,"internally_reserved":0,"allocated":260743168,"data_stored":260504265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":147517098},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,5,1,7,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659138,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48132,"kb_used_data":2612,"kb_used_omap":4,"kb_used_meta":45499,"kb_avail":94323708,"statfs":{"total":96636764160,"available":96587476992,"internally_reserved":0,"allocated":2674688,"data_stored":2349331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":46591663},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":260386816,"data_stored":260383337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2756608,"data_stored":2676591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":260395008,"data_stored":260391529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:13:13.387 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:13:14.276 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:13:14.276 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:13:14.352 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 50 20 66048 66028 5.1573 2.30078 0.0138703 0.015137 2024-09-11T01:13:14.357 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":101,"stamp":"2024-09-11T01:13:12.556405+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":260391548,"num_objects":63575,"num_object_clones":0,"num_object_copies":127150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24506,"num_write_kb":98018,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":4115,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63657,"ondisk_log_size":63657,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":66,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":841388,"kb_used_data":511868,"kb_used_omap":12,"kb_used_meta":329459,"kb_avail":282274132,"statfs":{"total":289910292480,"available":289048711168,"internally_reserved":0,"allocated":524152832,"data_stored":523349669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":337366020},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,7,8,13,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19213102,"num_objects":4689,"num_object_clones":0,"num_object_copies":9378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-44126,"num_objects_unfound":0,"num_objects_dirty":4689,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1034,"num_write_kb":4145,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.937150"},"pg_stats":[{"pgid":"2.f","version":"40'4072","reported_seq":4138,"reported_epoch":41,"state":"remapped+peering","last_fresh":"2024-09-11T01:13:07.648985+0000","last_change":"2024-09-11T01:13:07.648985+0000","last_active":"2024-09-11T01:13:07.648985+0000","last_peered":"2024-09-11T01:13:07.617385+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:01.915463+0000","last_became_peered":"2024-09-11T01:13:01.915463+0000","last_unstale":"2024-09-11T01:13:07.648985+0000","last_undegraded":"2024-09-11T01:13:07.648985+0000","last_fullsized":"2024-09-11T01:13:07.648985+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":4072,"log_dups_size":0,"ondisk_log_size":4072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16678912,"num_objects":4072,"num_object_clones":0,"num_object_copies":8144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4072,"num_write_kb":16288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"40'4095","reported_seq":4187,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:13:07.655439+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:13:07.655439+0000","last_peered":"2024-09-11T01:13:07.655439+0000","last_clean":"2024-09-11T01:13:07.655439+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:13:07.655439+0000","last_undegraded":"2024-09-11T01:13:07.655439+0000","last_fullsized":"2024-09-11T01:13:07.655439+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":4095,"log_dups_size":0,"ondisk_log_size":4095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16748544,"num_objects":4089,"num_object_clones":0,"num_object_copies":8178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2334,"num_write_kb":9336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"40'3900","reported_seq":4014,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-11T01:13:07.633706+0000","last_change":"2024-09-11T01:13:07.651253+0000","last_active":"2024-09-11T01:13:07.651253+0000","last_peered":"2024-09-11T01:13:07.633706+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:01.925595+0000","last_became_peered":"2024-09-11T01:13:01.925595+0000","last_unstale":"2024-09-11T01:13:07.633706+0000","last_undegraded":"2024-09-11T01:13:07.633706+0000","last_fullsized":"2024-09-11T01:13:07.633706+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":3900,"log_dups_size":0,"ondisk_log_size":3900,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15958016,"num_objects":3896,"num_object_clones":0,"num_object_copies":7792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2201,"num_write_kb":8804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"40'3922","reported_seq":3982,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:13:07.653838+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:13:07.653838+0000","last_peered":"2024-09-11T01:13:07.653838+0000","last_clean":"2024-09-11T01:13:07.653838+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:13:07.653838+0000","last_undegraded":"2024-09-11T01:13:07.653838+0000","last_fullsized":"2024-09-11T01:13:07.653838+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":3922,"log_dups_size":0,"ondisk_log_size":3922,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16064512,"num_objects":3922,"num_object_clones":0,"num_object_copies":7844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3922,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3922,"num_write_kb":15688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"40'3948","reported_seq":4068,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-11T01:13:08.648931+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:13:08.648931+0000","last_peered":"2024-09-11T01:13:08.648931+0000","last_clean":"2024-09-11T01:13:08.648931+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:13:08.648931+0000","last_undegraded":"2024-09-11T01:13:08.648931+0000","last_fullsized":"2024-09-11T01:13:08.648931+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":3948,"log_dups_size":0,"ondisk_log_size":3948,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16142336,"num_objects":3941,"num_object_clones":0,"num_object_copies":7882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":953,"num_write_kb":3812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"40'3956","reported_seq":4115,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-11T01:13:08.647339+0000","last_change":"2024-09-11T01:13:07.651527+0000","last_active":"2024-09-11T01:13:07.651527+0000","last_peered":"2024-09-11T01:13:07.619558+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:01.908550+0000","last_became_peered":"2024-09-11T01:13:01.908550+0000","last_unstale":"2024-09-11T01:13:08.647339+0000","last_undegraded":"2024-09-11T01:13:08.647339+0000","last_fullsized":"2024-09-11T01:13:08.647339+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":3956,"log_dups_size":0,"ondisk_log_size":3956,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16183296,"num_objects":3951,"num_object_clones":0,"num_object_copies":7902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":297,"num_write_kb":1188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"40'3992","reported_seq":4111,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-11T01:13:08.646907+0000","last_change":"2024-09-11T01:13:07.647899+0000","last_active":"2024-09-11T01:13:07.647899+0000","last_peered":"2024-09-11T01:13:07.631917+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:01.889432+0000","last_became_peered":"2024-09-11T01:13:01.889432+0000","last_unstale":"2024-09-11T01:13:08.646907+0000","last_undegraded":"2024-09-11T01:13:08.646907+0000","last_fullsized":"2024-09-11T01:13:08.646907+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":3992,"log_dups_size":0,"ondisk_log_size":3992,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16326656,"num_objects":3986,"num_object_clones":0,"num_object_copies":7972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3986,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":849,"num_write_kb":3396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"40'4006","reported_seq":4128,"reported_epoch":42,"state":"active+clean","last_fresh":"2024-09-11T01:13:08.645694+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:13:08.645694+0000","last_peered":"2024-09-11T01:13:08.645694+0000","last_clean":"2024-09-11T01:13:08.645694+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:13:08.645694+0000","last_undegraded":"2024-09-11T01:13:08.645694+0000","last_fullsized":"2024-09-11T01:13:08.645694+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":4006,"log_dups_size":0,"ondisk_log_size":4006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16400384,"num_objects":4004,"num_object_clones":0,"num_object_copies":8008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":882,"num_write_kb":3528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"40'3940","reported_seq":4054,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-11T01:13:07.618645+0000","last_change":"2024-09-11T01:13:07.654816+0000","last_active":"2024-09-11T01:13:07.654816+0000","last_peered":"2024-09-11T01:13:07.618645+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:01.883873+0000","last_became_peered":"2024-09-11T01:13:01.883873+0000","last_unstale":"2024-09-11T01:13:07.618645+0000","last_undegraded":"2024-09-11T01:13:07.618645+0000","last_fullsized":"2024-09-11T01:13:07.618645+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":3940,"log_dups_size":0,"ondisk_log_size":3940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16109568,"num_objects":3933,"num_object_clones":0,"num_object_copies":7866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3933,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1911,"num_write_kb":7644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":60,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:13:07.656178+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:07.656178+0000","last_peered":"2024-09-11T01:13:07.656178+0000","last_clean":"2024-09-11T01:13:07.656178+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:07.656178+0000","last_undegraded":"2024-09-11T01:13:07.656178+0000","last_fullsized":"2024-09-11T01:13:07.656178+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"40'3992","reported_seq":4109,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-11T01:13:08.648018+0000","last_change":"2024-09-11T01:13:07.652309+0000","last_active":"2024-09-11T01:13:07.652309+0000","last_peered":"2024-09-11T01:13:07.608035+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:01.889901+0000","last_became_peered":"2024-09-11T01:13:01.889901+0000","last_unstale":"2024-09-11T01:13:08.648018+0000","last_undegraded":"2024-09-11T01:13:08.648018+0000","last_fullsized":"2024-09-11T01:13:08.648018+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":3992,"log_dups_size":0,"ondisk_log_size":3992,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16318464,"num_objects":3984,"num_object_clones":0,"num_object_copies":7968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":579,"num_write_kb":2316,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":86,"reported_epoch":41,"state":"peering","last_fresh":"2024-09-11T01:13:07.648396+0000","last_change":"2024-09-11T01:13:08.644294+0000","last_active":"2024-09-11T01:13:07.648396+0000","last_peered":"2024-09-11T01:13:06.627322+0000","last_clean":"2024-09-11T01:13:06.627322+0000","last_became_active":"2024-09-11T01:13:01.889113+0000","last_became_peered":"2024-09-11T01:13:01.889113+0000","last_unstale":"2024-09-11T01:13:07.648396+0000","last_undegraded":"2024-09-11T01:13:07.648396+0000","last_fullsized":"2024-09-11T01:13:07.648396+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1,2],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"40'4029","reported_seq":4134,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-11T01:13:07.645090+0000","last_change":"2024-09-11T01:13:07.654842+0000","last_active":"2024-09-11T01:13:07.654842+0000","last_peered":"2024-09-11T01:13:07.645090+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:01.883768+0000","last_became_peered":"2024-09-11T01:13:01.883768+0000","last_unstale":"2024-09-11T01:13:07.645090+0000","last_undegraded":"2024-09-11T01:13:07.645090+0000","last_fullsized":"2024-09-11T01:13:07.645090+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":4029,"log_dups_size":0,"ondisk_log_size":4029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16482304,"num_objects":4024,"num_object_clones":0,"num_object_copies":8048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1846,"num_write_kb":7384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"40'3931","reported_seq":4036,"reported_epoch":40,"state":"remapped+peering","last_fresh":"2024-09-11T01:13:07.638709+0000","last_change":"2024-09-11T01:13:07.651382+0000","last_active":"2024-09-11T01:13:07.651382+0000","last_peered":"2024-09-11T01:13:07.638709+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:01.883991+0000","last_became_peered":"2024-09-11T01:13:01.883991+0000","last_unstale":"2024-09-11T01:13:07.638709+0000","last_undegraded":"2024-09-11T01:13:07.638709+0000","last_fullsized":"2024-09-11T01:13:07.638709+0000","mapping_epoch":41,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":3931,"log_dups_size":0,"ondisk_log_size":3931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16076800,"num_objects":3925,"num_object_clones":0,"num_object_copies":7850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1716,"num_write_kb":6864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"40'3952","reported_seq":4072,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-11T01:13:08.649631+0000","last_change":"2024-09-11T01:13:07.647752+0000","last_active":"2024-09-11T01:13:07.647752+0000","last_peered":"2024-09-11T01:13:07.595114+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:01.912973+0000","last_became_peered":"2024-09-11T01:13:01.912973+0000","last_unstale":"2024-09-11T01:13:08.649631+0000","last_undegraded":"2024-09-11T01:13:08.649631+0000","last_fullsized":"2024-09-11T01:13:08.649631+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":3952,"log_dups_size":0,"ondisk_log_size":3952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16179200,"num_objects":3950,"num_object_clones":0,"num_object_copies":7900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":689,"num_write_kb":2756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"40'3947","reported_seq":4072,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-11T01:13:08.646918+0000","last_change":"2024-09-11T01:13:07.650604+0000","last_active":"2024-09-11T01:13:07.650604+0000","last_peered":"2024-09-11T01:13:07.624565+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:01.908162+0000","last_became_peered":"2024-09-11T01:13:01.908162+0000","last_unstale":"2024-09-11T01:13:08.646918+0000","last_undegraded":"2024-09-11T01:13:08.646918+0000","last_fullsized":"2024-09-11T01:13:08.646918+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":3947,"log_dups_size":0,"ondisk_log_size":3947,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16134144,"num_objects":3939,"num_object_clones":0,"num_object_copies":7878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3939,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":602,"num_write_kb":2408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":60,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-09-11T01:13:07.653860+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:07.653860+0000","last_peered":"2024-09-11T01:13:07.653860+0000","last_clean":"2024-09-11T01:13:07.653860+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:07.653860+0000","last_undegraded":"2024-09-11T01:13:07.653860+0000","last_fullsized":"2024-09-11T01:13:07.653860+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"40'4043","reported_seq":4166,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-11T01:13:08.650688+0000","last_change":"2024-09-11T01:13:07.648243+0000","last_active":"2024-09-11T01:13:07.648243+0000","last_peered":"2024-09-11T01:13:07.635968+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:01.912996+0000","last_became_peered":"2024-09-11T01:13:01.912996+0000","last_unstale":"2024-09-11T01:13:08.650688+0000","last_undegraded":"2024-09-11T01:13:08.650688+0000","last_fullsized":"2024-09-11T01:13:08.650688+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":4043,"log_dups_size":0,"ondisk_log_size":4043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16543744,"num_objects":4039,"num_object_clones":0,"num_object_copies":8078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":652,"num_write_kb":2608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"40'3929","reported_seq":4051,"reported_epoch":42,"state":"remapped+peering","last_fresh":"2024-09-11T01:13:08.651653+0000","last_change":"2024-09-11T01:13:07.652539+0000","last_active":"2024-09-11T01:13:07.652539+0000","last_peered":"2024-09-11T01:13:07.636621+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:01.889393+0000","last_became_peered":"2024-09-11T01:13:01.889393+0000","last_unstale":"2024-09-11T01:13:08.651653+0000","last_undegraded":"2024-09-11T01:13:08.651653+0000","last_fullsized":"2024-09-11T01:13:08.651653+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":3929,"log_dups_size":0,"ondisk_log_size":3929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16044649,"num_objects":3918,"num_object_clones":0,"num_object_copies":7836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":999,"num_write_kb":3996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":260391529,"num_objects":63573,"num_object_clones":0,"num_object_copies":127146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":24504,"num_write_kb":98016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":523538432,"data_stored":523451457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63654,"ondisk_log_size":63654,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":57,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182420,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":394544,"kb_used_data":254624,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":93977296,"statfs":{"total":96636764160,"available":96232751104,"internally_reserved":0,"allocated":260734976,"data_stored":260496073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":143257259},"hb_peers":[2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,7,6,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986192,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":398712,"kb_used_data":254632,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93973128,"statfs":{"total":96636764160,"available":96228483072,"internally_reserved":0,"allocated":260743168,"data_stored":260504265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":147517098},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,5,1,7,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659138,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48132,"kb_used_data":2612,"kb_used_omap":4,"kb_used_meta":45499,"kb_avail":94323708,"statfs":{"total":96636764160,"available":96587476992,"internally_reserved":0,"allocated":2674688,"data_stored":2349331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":46591663},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":260386816,"data_stored":260383337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2756608,"data_stored":2676591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":260395008,"data_stored":260391529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:13:14.358 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:13:14.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "40'4072", 'reported_seq': 4138, 'reported_epoch': 41, 'state': 'remapped+peering', 'last_fresh': '2024-09-11T01:13:07.648985+0000', 'last_change': '2024-09-11T01:13:07.648985+0000', 'last_active': '2024-09-11T01:13:07.648985+0000', 'last_peered': '2024-09-11T01:13:07.617385+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:01.915463+0000', 'last_became_peered': '2024-09-11T01:13:01.915463+0000', 'last_unstale': '2024-09-11T01:13:07.648985+0000', 'last_undegraded': '2024-09-11T01:13:07.648985+0000', 'last_fullsized': '2024-09-11T01:13:07.648985+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 4072, 'log_dups_size': 0, 'ondisk_log_size': 4072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16678912, 'num_objects': 4072, 'num_object_clones': 0, 'num_object_copies': 8144, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4072, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4072, 'num_write_kb': 16288, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1, 2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:14.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:13:14.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "40'3900", 'reported_seq': 4014, 'reported_epoch': 40, 'state': 'remapped+peering', 'last_fresh': '2024-09-11T01:13:07.633706+0000', 'last_change': '2024-09-11T01:13:07.651253+0000', 'last_active': '2024-09-11T01:13:07.651253+0000', 'last_peered': '2024-09-11T01:13:07.633706+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:01.925595+0000', 'last_became_peered': '2024-09-11T01:13:01.925595+0000', 'last_unstale': '2024-09-11T01:13:07.633706+0000', 'last_undegraded': '2024-09-11T01:13:07.633706+0000', 'last_fullsized': '2024-09-11T01:13:07.633706+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 3900, 'log_dups_size': 0, 'ondisk_log_size': 3900, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15958016, 'num_objects': 3896, 'num_object_clones': 0, 'num_object_copies': 7792, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3896, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2201, 'num_write_kb': 8804, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0, 1], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:13:14.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:13:14.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "40'3956", 'reported_seq': 4115, 'reported_epoch': 42, 'state': 'remapped+peering', 'last_fresh': '2024-09-11T01:13:08.647339+0000', 'last_change': '2024-09-11T01:13:07.651527+0000', 'last_active': '2024-09-11T01:13:07.651527+0000', 'last_peered': '2024-09-11T01:13:07.619558+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:01.908550+0000', 'last_became_peered': '2024-09-11T01:13:01.908550+0000', 'last_unstale': '2024-09-11T01:13:08.647339+0000', 'last_undegraded': '2024-09-11T01:13:08.647339+0000', 'last_fullsized': '2024-09-11T01:13:08.647339+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 3956, 'log_dups_size': 0, 'ondisk_log_size': 3956, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16183296, 'num_objects': 3951, 'num_object_clones': 0, 'num_object_copies': 7902, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3951, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 297, 'num_write_kb': 1188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:14.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:13:14.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "40'3992", 'reported_seq': 4111, 'reported_epoch': 42, 'state': 'remapped+peering', 'last_fresh': '2024-09-11T01:13:08.646907+0000', 'last_change': '2024-09-11T01:13:07.647899+0000', 'last_active': '2024-09-11T01:13:07.647899+0000', 'last_peered': '2024-09-11T01:13:07.631917+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:01.889432+0000', 'last_became_peered': '2024-09-11T01:13:01.889432+0000', 'last_unstale': '2024-09-11T01:13:08.646907+0000', 'last_undegraded': '2024-09-11T01:13:08.646907+0000', 'last_fullsized': '2024-09-11T01:13:08.646907+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 3992, 'log_dups_size': 0, 'ondisk_log_size': 3992, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16326656, 'num_objects': 3986, 'num_object_clones': 0, 'num_object_copies': 7972, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3986, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 849, 'num_write_kb': 3396, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:14.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:13:14.360 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "40'3940", 'reported_seq': 4054, 'reported_epoch': 40, 'state': 'remapped+peering', 'last_fresh': '2024-09-11T01:13:07.618645+0000', 'last_change': '2024-09-11T01:13:07.654816+0000', 'last_active': '2024-09-11T01:13:07.654816+0000', 'last_peered': '2024-09-11T01:13:07.618645+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:01.883873+0000', 'last_became_peered': '2024-09-11T01:13:01.883873+0000', 'last_unstale': '2024-09-11T01:13:07.618645+0000', 'last_undegraded': '2024-09-11T01:13:07.618645+0000', 'last_fullsized': '2024-09-11T01:13:07.618645+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 3940, 'log_dups_size': 0, 'ondisk_log_size': 3940, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16109568, 'num_objects': 3933, 'num_object_clones': 0, 'num_object_copies': 7866, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3933, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1911, 'num_write_kb': 7644, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0, 1], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:13:14.360 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:13:14.360 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "40'3992", 'reported_seq': 4109, 'reported_epoch': 42, 'state': 'remapped+peering', 'last_fresh': '2024-09-11T01:13:08.648018+0000', 'last_change': '2024-09-11T01:13:07.652309+0000', 'last_active': '2024-09-11T01:13:07.652309+0000', 'last_peered': '2024-09-11T01:13:07.608035+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:01.889901+0000', 'last_became_peered': '2024-09-11T01:13:01.889901+0000', 'last_unstale': '2024-09-11T01:13:08.648018+0000', 'last_undegraded': '2024-09-11T01:13:08.648018+0000', 'last_fullsized': '2024-09-11T01:13:08.648018+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 3992, 'log_dups_size': 0, 'ondisk_log_size': 3992, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16318464, 'num_objects': 3984, 'num_object_clones': 0, 'num_object_copies': 7968, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3984, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 579, 'num_write_kb': 2316, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:14.360 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 1.2 is not active+clean 2024-09-11T01:13:14.360 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '1.2', 'version': "14'2", 'reported_seq': 86, 'reported_epoch': 41, 'state': 'peering', 'last_fresh': '2024-09-11T01:13:07.648396+0000', 'last_change': '2024-09-11T01:13:08.644294+0000', 'last_active': '2024-09-11T01:13:07.648396+0000', 'last_peered': '2024-09-11T01:13:06.627322+0000', 'last_clean': '2024-09-11T01:13:06.627322+0000', 'last_became_active': '2024-09-11T01:13:01.889113+0000', 'last_became_peered': '2024-09-11T01:13:01.889113+0000', 'last_unstale': '2024-09-11T01:13:07.648396+0000', 'last_undegraded': '2024-09-11T01:13:07.648396+0000', 'last_fullsized': '2024-09-11T01:13:07.648396+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 11, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:19.623673+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:19.623673+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:19.623673+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.001991045, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 19, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1, 2], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:14.360 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:13:14.360 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "40'4029", 'reported_seq': 4134, 'reported_epoch': 40, 'state': 'remapped+peering', 'last_fresh': '2024-09-11T01:13:07.645090+0000', 'last_change': '2024-09-11T01:13:07.654842+0000', 'last_active': '2024-09-11T01:13:07.654842+0000', 'last_peered': '2024-09-11T01:13:07.645090+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:01.883768+0000', 'last_became_peered': '2024-09-11T01:13:01.883768+0000', 'last_unstale': '2024-09-11T01:13:07.645090+0000', 'last_undegraded': '2024-09-11T01:13:07.645090+0000', 'last_fullsized': '2024-09-11T01:13:07.645090+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 4029, 'log_dups_size': 0, 'ondisk_log_size': 4029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16482304, 'num_objects': 4024, 'num_object_clones': 0, 'num_object_copies': 8048, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4024, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1846, 'num_write_kb': 7384, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0, 1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:13:14.360 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:13:14.361 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "40'3931", 'reported_seq': 4036, 'reported_epoch': 40, 'state': 'remapped+peering', 'last_fresh': '2024-09-11T01:13:07.638709+0000', 'last_change': '2024-09-11T01:13:07.651382+0000', 'last_active': '2024-09-11T01:13:07.651382+0000', 'last_peered': '2024-09-11T01:13:07.638709+0000', 'last_clean': '2024-09-11T01:13:07.638709+0000', 'last_became_active': '2024-09-11T01:13:01.883991+0000', 'last_became_peered': '2024-09-11T01:13:01.883991+0000', 'last_unstale': '2024-09-11T01:13:07.638709+0000', 'last_undegraded': '2024-09-11T01:13:07.638709+0000', 'last_fullsized': '2024-09-11T01:13:07.638709+0000', 'mapping_epoch': 41, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2269", 'last_scrub_stamp': '2024-09-11T01:12:42.890920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:42.890920+0000', 'objects_scrubbed': 2209, 'log_size': 3931, 'log_dups_size': 0, 'ondisk_log_size': 3931, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 999, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16076800, 'num_objects': 3925, 'num_object_clones': 0, 'num_object_copies': 7850, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3925, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1716, 'num_write_kb': 6864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0, 1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:13:14.362 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:13:14.362 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "40'3952", 'reported_seq': 4072, 'reported_epoch': 42, 'state': 'remapped+peering', 'last_fresh': '2024-09-11T01:13:08.649631+0000', 'last_change': '2024-09-11T01:13:07.647752+0000', 'last_active': '2024-09-11T01:13:07.647752+0000', 'last_peered': '2024-09-11T01:13:07.595114+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:01.912973+0000', 'last_became_peered': '2024-09-11T01:13:01.912973+0000', 'last_unstale': '2024-09-11T01:13:08.649631+0000', 'last_undegraded': '2024-09-11T01:13:08.649631+0000', 'last_fullsized': '2024-09-11T01:13:08.649631+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 3952, 'log_dups_size': 0, 'ondisk_log_size': 3952, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16179200, 'num_objects': 3950, 'num_object_clones': 0, 'num_object_copies': 7900, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3950, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 689, 'num_write_kb': 2756, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:14.362 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:13:14.362 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "40'3947", 'reported_seq': 4072, 'reported_epoch': 42, 'state': 'remapped+peering', 'last_fresh': '2024-09-11T01:13:08.646918+0000', 'last_change': '2024-09-11T01:13:07.650604+0000', 'last_active': '2024-09-11T01:13:07.650604+0000', 'last_peered': '2024-09-11T01:13:07.624565+0000', 'last_clean': '2024-09-11T01:13:07.624565+0000', 'last_became_active': '2024-09-11T01:13:01.908162+0000', 'last_became_peered': '2024-09-11T01:13:01.908162+0000', 'last_unstale': '2024-09-11T01:13:08.646918+0000', 'last_undegraded': '2024-09-11T01:13:08.646918+0000', 'last_fullsized': '2024-09-11T01:13:08.646918+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3415", 'last_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'objects_scrubbed': 3337, 'log_size': 3947, 'log_dups_size': 0, 'ondisk_log_size': 3947, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1975, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16134144, 'num_objects': 3939, 'num_object_clones': 0, 'num_object_copies': 7878, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3939, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 602, 'num_write_kb': 2408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:14.362 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:13:14.362 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "40'4043", 'reported_seq': 4166, 'reported_epoch': 42, 'state': 'remapped+peering', 'last_fresh': '2024-09-11T01:13:08.650688+0000', 'last_change': '2024-09-11T01:13:07.648243+0000', 'last_active': '2024-09-11T01:13:07.648243+0000', 'last_peered': '2024-09-11T01:13:07.635968+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:01.912996+0000', 'last_became_peered': '2024-09-11T01:13:01.912996+0000', 'last_unstale': '2024-09-11T01:13:08.650688+0000', 'last_undegraded': '2024-09-11T01:13:08.650688+0000', 'last_fullsized': '2024-09-11T01:13:08.650688+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 4043, 'log_dups_size': 0, 'ondisk_log_size': 4043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16543744, 'num_objects': 4039, 'num_object_clones': 0, 'num_object_copies': 8078, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4039, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 652, 'num_write_kb': 2608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:14.362 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:13:14.362 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "40'3929", 'reported_seq': 4051, 'reported_epoch': 42, 'state': 'remapped+peering', 'last_fresh': '2024-09-11T01:13:08.651653+0000', 'last_change': '2024-09-11T01:13:07.652539+0000', 'last_active': '2024-09-11T01:13:07.652539+0000', 'last_peered': '2024-09-11T01:13:07.636621+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:01.889393+0000', 'last_became_peered': '2024-09-11T01:13:01.889393+0000', 'last_unstale': '2024-09-11T01:13:08.651653+0000', 'last_undegraded': '2024-09-11T01:13:08.651653+0000', 'last_fullsized': '2024-09-11T01:13:08.651653+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 3929, 'log_dups_size': 0, 'ondisk_log_size': 3929, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16044649, 'num_objects': 3918, 'num_object_clones': 0, 'num_object_copies': 7836, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3918, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 999, 'num_write_kb': 3996, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:14.364 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:13:15.340 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:13:15.341 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43711/131186 objects degraded (33.320%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":131,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":65593,"data_bytes":268657276,"bytes_used":874811392,"bytes_avail":289035481088,"bytes_total":289910292480,"degraded_objects":43711,"degraded_total":131186,"degraded_ratio":0.33319866449163782,"misplaced_objects":3863,"misplaced_total":131186,"misplaced_ratio":0.029446739743570198,"recovering_objects_per_sec":8,"recovering_bytes_per_sec":36686,"recovering_keys_per_sec":0,"num_objects_recovered":60,"num_bytes_recovered":253952,"num_keys_recovered":0,"write_bytes_sec":1807232,"read_op_per_sec":0,"write_op_per_sec":440},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-11T01:13:08.555583+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T01:13:15.352 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 51 20 66637 66617 5.10128 2.30078 0.0628858 0.015306 2024-09-11T01:13:15.406 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:13:15.406 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:13:16.356 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 52 20 67242 67222 5.0483 2.36328 0.0382067 0.015468 2024-09-11T01:13:16.433 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:13:16.435 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:13:16.503 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":102,"stamp":"2024-09-11T01:13:14.556742+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":268657276,"num_objects":65593,"num_object_clones":0,"num_object_copies":131186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":3863,"num_objects_unfound":0,"num_objects_dirty":65593,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26524,"num_write_kb":106090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":64,"num_bytes_recovered":258067,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65675,"ondisk_log_size":65675,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":64,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":854308,"kb_used_data":520628,"kb_used_omap":12,"kb_used_meta":333619,"kb_avail":282261212,"statfs":{"total":289910292480,"available":289035481088,"internally_reserved":0,"allocated":533123072,"data_stored":532304154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":341625860},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,3,7,1,0,0,0,0,5],"upper_bound":4096},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27478830,"num_objects":6707,"num_object_clones":0,"num_object_copies":13414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":-40263,"num_objects_unfound":0,"num_objects_dirty":6707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3052,"num_write_kb":12217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":60,"num_bytes_recovered":253952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.922300"},"pg_stats":[{"pgid":"2.f","version":"43'4202","reported_seq":4274,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:13.852527+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:13:13.852527+0000","last_peered":"2024-09-11T01:13:13.852527+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:13:13.852527+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":4202,"log_dups_size":0,"ondisk_log_size":4202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17211392,"num_objects":4202,"num_object_clones":0,"num_object_copies":8404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4202,"num_write_kb":16808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":130},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4201","reported_seq":4297,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:13.074491+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:13:13.074491+0000","last_peered":"2024-09-11T01:13:13.074491+0000","last_clean":"2024-09-11T01:13:13.074491+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:13:13.074491+0000","last_undegraded":"2024-09-11T01:13:13.074491+0000","last_fullsized":"2024-09-11T01:13:13.074491+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":4201,"log_dups_size":0,"ondisk_log_size":4201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17182720,"num_objects":4195,"num_object_clones":0,"num_object_copies":8390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2440,"num_write_kb":9760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4009","reported_seq":4131,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:13.031846+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:13:13.031846+0000","last_peered":"2024-09-11T01:13:13.031846+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:13:13.031846+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":4009,"log_dups_size":0,"ondisk_log_size":4009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16404480,"num_objects":4005,"num_object_clones":0,"num_object_copies":8010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2310,"num_write_kb":9240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":109}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4044","reported_seq":4108,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:13.019690+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:13:13.019690+0000","last_peered":"2024-09-11T01:13:13.019690+0000","last_clean":"2024-09-11T01:13:13.019690+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:13:13.019690+0000","last_undegraded":"2024-09-11T01:13:13.019690+0000","last_fullsized":"2024-09-11T01:13:13.019690+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":4044,"log_dups_size":0,"ondisk_log_size":4044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16564224,"num_objects":4044,"num_object_clones":0,"num_object_copies":8088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4044,"num_write_kb":16176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4109","reported_seq":4231,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:13.837436+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:13:13.837436+0000","last_peered":"2024-09-11T01:13:13.837436+0000","last_clean":"2024-09-11T01:13:13.837436+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:13:13.837436+0000","last_undegraded":"2024-09-11T01:13:13.837436+0000","last_fullsized":"2024-09-11T01:13:13.837436+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":4109,"log_dups_size":0,"ondisk_log_size":4109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16801792,"num_objects":4102,"num_object_clones":0,"num_object_copies":8204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1114,"num_write_kb":4456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4096","reported_seq":4259,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:13.803787+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:13:13.803787+0000","last_peered":"2024-09-11T01:13:13.803787+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:13:13.803787+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":4096,"log_dups_size":0,"ondisk_log_size":4096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16756736,"num_objects":4091,"num_object_clones":0,"num_object_copies":8182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4091,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":437,"num_write_kb":1748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":140},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4119","reported_seq":4244,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:13.767318+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:13:13.767318+0000","last_peered":"2024-09-11T01:13:13.767318+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:13:13.767318+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":4119,"log_dups_size":0,"ondisk_log_size":4119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16846848,"num_objects":4113,"num_object_clones":0,"num_object_copies":8226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":976,"num_write_kb":3904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":127},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4143","reported_seq":4267,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:13.798973+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:13:13.798973+0000","last_peered":"2024-09-11T01:13:13.798973+0000","last_clean":"2024-09-11T01:13:13.798973+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:13:13.798973+0000","last_undegraded":"2024-09-11T01:13:13.798973+0000","last_fullsized":"2024-09-11T01:13:13.798973+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":4143,"log_dups_size":0,"ondisk_log_size":4143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16961536,"num_objects":4141,"num_object_clones":0,"num_object_copies":8282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4141,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1019,"num_write_kb":4076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4055","reported_seq":4177,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:13.008734+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:13:13.008734+0000","last_peered":"2024-09-11T01:13:13.008734+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:13:13.008734+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":4055,"log_dups_size":0,"ondisk_log_size":4055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16580608,"num_objects":4048,"num_object_clones":0,"num_object_copies":8096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2026,"num_write_kb":8104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":115}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741858+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741858+0000","last_peered":"2024-09-11T01:13:09.741858+0000","last_clean":"2024-09-11T01:13:09.741858+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741858+0000","last_undegraded":"2024-09-11T01:13:09.741858+0000","last_fullsized":"2024-09-11T01:13:09.741858+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4119","reported_seq":4242,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:13.757116+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:13:13.757116+0000","last_peered":"2024-09-11T01:13:13.757116+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:13:13.757116+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":4119,"log_dups_size":0,"ondisk_log_size":4119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16838656,"num_objects":4111,"num_object_clones":0,"num_object_copies":8222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":706,"num_write_kb":2824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":127},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.837480+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.837480+0000","last_peered":"2024-09-11T01:13:09.837480+0000","last_clean":"2024-09-11T01:13:09.837480+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.837480+0000","last_undegraded":"2024-09-11T01:13:09.837480+0000","last_fullsized":"2024-09-11T01:13:09.837480+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4132","reported_seq":4245,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:13.056230+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:13:13.056230+0000","last_peered":"2024-09-11T01:13:13.056230+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:13:13.056230+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":4132,"log_dups_size":0,"ondisk_log_size":4132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16904192,"num_objects":4127,"num_object_clones":0,"num_object_copies":8254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1949,"num_write_kb":7796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":103}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4018","reported_seq":4256,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:13:13.114731+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:13:13.114731+0000","last_peered":"2024-09-11T01:13:13.114731+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:13:13.114731+0000","last_undegraded":"2024-09-11T01:13:13.114731+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":4018,"log_dups_size":0,"ondisk_log_size":4018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16433152,"num_objects":4012,"num_object_clones":0,"num_object_copies":8024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3863,"num_objects_unfound":0,"num_objects_dirty":4012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1803,"num_write_kb":7212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":62,"num_bytes_recovered":253952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3863},{"shards":"2","objects":149}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4088","reported_seq":4214,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:13.826230+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:13:13.826230+0000","last_peered":"2024-09-11T01:13:13.826230+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:13:13.826230+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":4088,"log_dups_size":0,"ondisk_log_size":4088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16736256,"num_objects":4086,"num_object_clones":0,"num_object_copies":8172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":825,"num_write_kb":3300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":136},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4100","reported_seq":4229,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:13.865677+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:13:13.865677+0000","last_peered":"2024-09-11T01:13:13.865677+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:13:13.865677+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":4100,"log_dups_size":0,"ondisk_log_size":4100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16760832,"num_objects":4092,"num_object_clones":0,"num_object_copies":8184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4092,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":755,"num_write_kb":3020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":153},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658057+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658057+0000","last_peered":"2024-09-11T01:13:09.658057+0000","last_clean":"2024-09-11T01:13:09.658057+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658057+0000","last_undegraded":"2024-09-11T01:13:09.658057+0000","last_fullsized":"2024-09-11T01:13:09.658057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4186","reported_seq":4315,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:13.824909+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:13:13.824909+0000","last_peered":"2024-09-11T01:13:13.824909+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:13:13.824909+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":4186,"log_dups_size":0,"ondisk_log_size":4186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17129472,"num_objects":4182,"num_object_clones":0,"num_object_copies":8364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":795,"num_write_kb":3180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":143},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4051","reported_seq":4179,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:13.826667+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:13:13.826667+0000","last_peered":"2024-09-11T01:13:13.826667+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:13:13.826667+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":4051,"log_dups_size":0,"ondisk_log_size":4051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16544361,"num_objects":4040,"num_object_clones":0,"num_object_copies":8080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1121,"num_write_kb":4484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":123},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":268657257,"num_objects":65591,"num_object_clones":0,"num_object_copies":131182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":3863,"num_objects_unfound":0,"num_objects_dirty":65591,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26522,"num_write_kb":106088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":63,"num_bytes_recovered":258048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":534020096,"data_stored":533933121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65672,"ondisk_log_size":65672,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182421,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":404432,"kb_used_data":260352,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93967408,"statfs":{"total":96636764160,"available":96222625792,"internally_reserved":0,"allocated":266600448,"data_stored":266351629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":147517099},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,3,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986193,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":401744,"kb_used_data":257664,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93970096,"statfs":{"total":96636764160,"available":96225378304,"internally_reserved":0,"allocated":263847936,"data_stored":263603194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":147517098},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,0,0,0,5],"upper_bound":4096},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659138,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48132,"kb_used_data":2612,"kb_used_omap":4,"kb_used_meta":45499,"kb_avail":94323708,"statfs":{"total":96636764160,"available":96587476992,"internally_reserved":0,"allocated":2674688,"data_stored":2349331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":46591663},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":267239424,"data_stored":267235945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":2756608,"data_stored":2676591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":264024064,"data_stored":264020585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:13:17.358 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 53 20 67874 67854 4.99949 2.46875 0.106942 0.0156151 2024-09-11T01:13:18.358 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 54 20 68515 68495 4.95329 2.50391 0.0202347 0.0157635 2024-09-11T01:13:19.360 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 55 20 69128 69108 4.90653 2.39453 0.017712 0.015908 2024-09-11T01:13:19.505 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:13:20.174 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:13:20.175 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:13:20.223 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":104,"stamp":"2024-09-11T01:13:18.557563+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":273236604,"num_objects":66711,"num_object_clones":0,"num_object_copies":133422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":3764,"num_objects_unfound":0,"num_objects_dirty":66711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27642,"num_write_kb":110562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":663571,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66793,"ondisk_log_size":66793,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":886788,"kb_used_data":536468,"kb_used_omap":12,"kb_used_meta":350259,"kb_avail":282228732,"statfs":{"total":289910292480,"available":289002221568,"internally_reserved":0,"allocated":549343232,"data_stored":548524314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":358665220},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,6,4,14,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24124206,"num_objects":5888,"num_object_clones":0,"num_object_copies":11776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":-25881,"num_objects_unfound":0,"num_objects_dirty":5888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2233,"num_write_kb":8941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":159,"num_bytes_recovered":659456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.918578"},"pg_stats":[{"pgid":"2.f","version":"43'4202","reported_seq":4274,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:13.852527+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:13:13.852527+0000","last_peered":"2024-09-11T01:13:13.852527+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:13:13.852527+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":4202,"log_dups_size":0,"ondisk_log_size":4202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17211392,"num_objects":4202,"num_object_clones":0,"num_object_copies":8404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4202,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4202,"num_write_kb":16808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":130},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4384","reported_seq":4480,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:18.136680+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:13:18.136680+0000","last_peered":"2024-09-11T01:13:18.136680+0000","last_clean":"2024-09-11T01:13:18.136680+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:13:18.136680+0000","last_undegraded":"2024-09-11T01:13:18.136680+0000","last_fullsized":"2024-09-11T01:13:18.136680+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":4384,"log_dups_size":0,"ondisk_log_size":4384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17932288,"num_objects":4378,"num_object_clones":0,"num_object_copies":8756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2623,"num_write_kb":10492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4204","reported_seq":4325,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:18.067237+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:13:18.067237+0000","last_peered":"2024-09-11T01:13:18.067237+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:13:18.067237+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":4204,"log_dups_size":0,"ondisk_log_size":4204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17203200,"num_objects":4200,"num_object_clones":0,"num_object_copies":8400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2505,"num_write_kb":10020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":304}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4232","reported_seq":4296,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:18.095127+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:13:18.095127+0000","last_peered":"2024-09-11T01:13:18.095127+0000","last_clean":"2024-09-11T01:13:18.095127+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:13:18.095127+0000","last_undegraded":"2024-09-11T01:13:18.095127+0000","last_fullsized":"2024-09-11T01:13:18.095127+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":4232,"log_dups_size":0,"ondisk_log_size":4232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17334272,"num_objects":4232,"num_object_clones":0,"num_object_copies":8464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4232,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4232,"num_write_kb":16928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4109","reported_seq":4231,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:13.837436+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:13:13.837436+0000","last_peered":"2024-09-11T01:13:13.837436+0000","last_clean":"2024-09-11T01:13:13.837436+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:13:13.837436+0000","last_undegraded":"2024-09-11T01:13:13.837436+0000","last_fullsized":"2024-09-11T01:13:13.837436+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":4109,"log_dups_size":0,"ondisk_log_size":4109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16801792,"num_objects":4102,"num_object_clones":0,"num_object_copies":8204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1114,"num_write_kb":4456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4096","reported_seq":4259,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:13.803787+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:13:13.803787+0000","last_peered":"2024-09-11T01:13:13.803787+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:13:13.803787+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":4096,"log_dups_size":0,"ondisk_log_size":4096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16756736,"num_objects":4091,"num_object_clones":0,"num_object_copies":8182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4091,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":437,"num_write_kb":1748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":140},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4119","reported_seq":4244,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:13.767318+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:13:13.767318+0000","last_peered":"2024-09-11T01:13:13.767318+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:13:13.767318+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":4119,"log_dups_size":0,"ondisk_log_size":4119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16846848,"num_objects":4113,"num_object_clones":0,"num_object_copies":8226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":976,"num_write_kb":3904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":127},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4143","reported_seq":4267,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:13.798973+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:13:13.798973+0000","last_peered":"2024-09-11T01:13:13.798973+0000","last_clean":"2024-09-11T01:13:13.798973+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:13:13.798973+0000","last_undegraded":"2024-09-11T01:13:13.798973+0000","last_fullsized":"2024-09-11T01:13:13.798973+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":4143,"log_dups_size":0,"ondisk_log_size":4143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16961536,"num_objects":4141,"num_object_clones":0,"num_object_copies":8282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4141,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1019,"num_write_kb":4076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4238","reported_seq":4360,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:18.124182+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:13:18.124182+0000","last_peered":"2024-09-11T01:13:18.124182+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:13:18.124182+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":4238,"log_dups_size":0,"ondisk_log_size":4238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17330176,"num_objects":4231,"num_object_clones":0,"num_object_copies":8462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2209,"num_write_kb":8836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":298}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741858+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741858+0000","last_peered":"2024-09-11T01:13:09.741858+0000","last_clean":"2024-09-11T01:13:09.741858+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741858+0000","last_undegraded":"2024-09-11T01:13:09.741858+0000","last_fullsized":"2024-09-11T01:13:09.741858+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4119","reported_seq":4242,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:13.757116+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:13:13.757116+0000","last_peered":"2024-09-11T01:13:13.757116+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:13:13.757116+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":4119,"log_dups_size":0,"ondisk_log_size":4119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16838656,"num_objects":4111,"num_object_clones":0,"num_object_copies":8222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":706,"num_write_kb":2824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":127},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.837480+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.837480+0000","last_peered":"2024-09-11T01:13:09.837480+0000","last_clean":"2024-09-11T01:13:09.837480+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.837480+0000","last_undegraded":"2024-09-11T01:13:09.837480+0000","last_fullsized":"2024-09-11T01:13:09.837480+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4310","reported_seq":4423,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:18.106360+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:13:18.106360+0000","last_peered":"2024-09-11T01:13:18.106360+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:13:18.106360+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":4310,"log_dups_size":0,"ondisk_log_size":4310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17633280,"num_objects":4305,"num_object_clones":0,"num_object_copies":8610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2127,"num_write_kb":8508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":281}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4209","reported_seq":4645,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:13:18.115772+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:13:18.115772+0000","last_peered":"2024-09-11T01:13:18.115772+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:13:18.115772+0000","last_undegraded":"2024-09-11T01:13:18.115772+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":4209,"log_dups_size":0,"ondisk_log_size":4209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17215488,"num_objects":4203,"num_object_clones":0,"num_object_copies":8406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3764,"num_objects_unfound":0,"num_objects_dirty":4203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1994,"num_write_kb":7976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":659456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3764},{"shards":"2","objects":439}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4088","reported_seq":4214,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:13.826230+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:13:13.826230+0000","last_peered":"2024-09-11T01:13:13.826230+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:13:13.826230+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":4088,"log_dups_size":0,"ondisk_log_size":4088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16736256,"num_objects":4086,"num_object_clones":0,"num_object_copies":8172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":825,"num_write_kb":3300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":136},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4100","reported_seq":4229,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:13.865677+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:13:13.865677+0000","last_peered":"2024-09-11T01:13:13.865677+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:13:13.865677+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":4100,"log_dups_size":0,"ondisk_log_size":4100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16760832,"num_objects":4092,"num_object_clones":0,"num_object_copies":8184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4092,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":755,"num_write_kb":3020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":153},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658057+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658057+0000","last_peered":"2024-09-11T01:13:09.658057+0000","last_clean":"2024-09-11T01:13:09.658057+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658057+0000","last_undegraded":"2024-09-11T01:13:09.658057+0000","last_fullsized":"2024-09-11T01:13:09.658057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4186","reported_seq":4315,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:13.824909+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:13:13.824909+0000","last_peered":"2024-09-11T01:13:13.824909+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:13:13.824909+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":4186,"log_dups_size":0,"ondisk_log_size":4186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17129472,"num_objects":4182,"num_object_clones":0,"num_object_copies":8364,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4182,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":795,"num_write_kb":3180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":143},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4051","reported_seq":4179,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:13.826667+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:13:13.826667+0000","last_peered":"2024-09-11T01:13:13.826667+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:13:13.826667+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":4051,"log_dups_size":0,"ondisk_log_size":4051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16544361,"num_objects":4040,"num_object_clones":0,"num_object_copies":8080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1121,"num_write_kb":4484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":123},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":273236585,"num_objects":66709,"num_object_clones":0,"num_object_copies":133418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":3764,"num_objects_unfound":0,"num_objects_dirty":66709,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27640,"num_write_kb":110560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":663552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":549969920,"data_stored":549882945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":66790,"ondisk_log_size":66790,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182421,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":404432,"kb_used_data":260352,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93967408,"statfs":{"total":96636764160,"available":96222625792,"internally_reserved":0,"allocated":266600448,"data_stored":266351629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":147517099},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,3,7,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986194,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":416308,"kb_used_data":263908,"kb_used_omap":4,"kb_used_meta":152379,"kb_avail":93955532,"statfs":{"total":96636764160,"available":96210464768,"internally_reserved":0,"allocated":270241792,"data_stored":269997050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":156036778},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659139,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66048,"kb_used_data":12208,"kb_used_omap":4,"kb_used_meta":53819,"kb_avail":94305792,"statfs":{"total":96636764160,"available":96569131008,"internally_reserved":0,"allocated":12500992,"data_stored":12175635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":55111343},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,0,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":267239424,"data_stored":267235945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12423168,"data_stored":12343151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":270307328,"data_stored":270303849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:13:20.224 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:13:20.360 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 56 20 69706 69686 4.85924 2.25781 0.0598934 0.0160682 2024-09-11T01:13:21.104 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:13:21.104 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:13:21.170 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":105,"stamp":"2024-09-11T01:13:20.558057+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":281338492,"num_objects":68689,"num_object_clones":0,"num_object_copies":137378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":3764,"num_objects_unfound":0,"num_objects_dirty":68689,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29620,"num_write_kb":118474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":663571,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68771,"ondisk_log_size":68771,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":904328,"kb_used_data":545688,"kb_used_omap":12,"kb_used_meta":358579,"kb_avail":282211192,"statfs":{"total":289910292480,"available":288984260608,"internally_reserved":0,"allocated":558784512,"data_stored":557965594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":367184900},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,3,3,15,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20946944,"num_objects":5114,"num_object_clones":0,"num_object_copies":10228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":3764,"num_objects_unfound":0,"num_objects_dirty":5114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5114,"num_write_kb":20456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":659456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.916662"},"pg_stats":[{"pgid":"2.f","version":"43'4393","reported_seq":4465,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:18.858541+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:13:18.858541+0000","last_peered":"2024-09-11T01:13:18.858541+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:13:18.858541+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":4393,"log_dups_size":0,"ondisk_log_size":4393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17993728,"num_objects":4393,"num_object_clones":0,"num_object_copies":8786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4393,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4393,"num_write_kb":17572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":321},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4384","reported_seq":4480,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:18.136680+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:13:18.136680+0000","last_peered":"2024-09-11T01:13:18.136680+0000","last_clean":"2024-09-11T01:13:18.136680+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:13:18.136680+0000","last_undegraded":"2024-09-11T01:13:18.136680+0000","last_fullsized":"2024-09-11T01:13:18.136680+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":4384,"log_dups_size":0,"ondisk_log_size":4384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17932288,"num_objects":4378,"num_object_clones":0,"num_object_copies":8756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2623,"num_write_kb":10492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4204","reported_seq":4325,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:18.067237+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:13:18.067237+0000","last_peered":"2024-09-11T01:13:18.067237+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:13:18.067237+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":4204,"log_dups_size":0,"ondisk_log_size":4204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17203200,"num_objects":4200,"num_object_clones":0,"num_object_copies":8400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2505,"num_write_kb":10020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":304}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4232","reported_seq":4296,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:18.095127+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:13:18.095127+0000","last_peered":"2024-09-11T01:13:18.095127+0000","last_clean":"2024-09-11T01:13:18.095127+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:13:18.095127+0000","last_undegraded":"2024-09-11T01:13:18.095127+0000","last_fullsized":"2024-09-11T01:13:18.095127+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":4232,"log_dups_size":0,"ondisk_log_size":4232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17334272,"num_objects":4232,"num_object_clones":0,"num_object_copies":8464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4232,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4232,"num_write_kb":16928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4307","reported_seq":4429,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:18.839384+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:13:18.839384+0000","last_peered":"2024-09-11T01:13:18.839384+0000","last_clean":"2024-09-11T01:13:18.839384+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:13:18.839384+0000","last_undegraded":"2024-09-11T01:13:18.839384+0000","last_fullsized":"2024-09-11T01:13:18.839384+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":4307,"log_dups_size":0,"ondisk_log_size":4307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17612800,"num_objects":4300,"num_object_clones":0,"num_object_copies":8600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1312,"num_write_kb":5248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4293","reported_seq":4455,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:18.843913+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:13:18.843913+0000","last_peered":"2024-09-11T01:13:18.843913+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:13:18.843913+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":4293,"log_dups_size":0,"ondisk_log_size":4293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17563648,"num_objects":4288,"num_object_clones":0,"num_object_copies":8576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4288,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":634,"num_write_kb":2536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":337},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4322","reported_seq":4446,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:18.886731+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:13:18.886731+0000","last_peered":"2024-09-11T01:13:18.886731+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:13:18.886731+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":4322,"log_dups_size":0,"ondisk_log_size":4322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17678336,"num_objects":4316,"num_object_clones":0,"num_object_copies":8632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1179,"num_write_kb":4716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":330},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4361","reported_seq":4485,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:18.837570+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:13:18.837570+0000","last_peered":"2024-09-11T01:13:18.837570+0000","last_clean":"2024-09-11T01:13:18.837570+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:13:18.837570+0000","last_undegraded":"2024-09-11T01:13:18.837570+0000","last_fullsized":"2024-09-11T01:13:18.837570+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":4361,"log_dups_size":0,"ondisk_log_size":4361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17854464,"num_objects":4359,"num_object_clones":0,"num_object_copies":8718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1237,"num_write_kb":4948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4238","reported_seq":4360,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:18.124182+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:13:18.124182+0000","last_peered":"2024-09-11T01:13:18.124182+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:13:18.124182+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":4238,"log_dups_size":0,"ondisk_log_size":4238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17330176,"num_objects":4231,"num_object_clones":0,"num_object_copies":8462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2209,"num_write_kb":8836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":298}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741858+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741858+0000","last_peered":"2024-09-11T01:13:09.741858+0000","last_clean":"2024-09-11T01:13:09.741858+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741858+0000","last_undegraded":"2024-09-11T01:13:09.741858+0000","last_fullsized":"2024-09-11T01:13:09.741858+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4301","reported_seq":4422,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:18.752600+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:13:18.752600+0000","last_peered":"2024-09-11T01:13:18.752600+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:13:18.752600+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":4301,"log_dups_size":0,"ondisk_log_size":4301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17584128,"num_objects":4293,"num_object_clones":0,"num_object_copies":8586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":888,"num_write_kb":3552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":309},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.837480+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.837480+0000","last_peered":"2024-09-11T01:13:09.837480+0000","last_clean":"2024-09-11T01:13:09.837480+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.837480+0000","last_undegraded":"2024-09-11T01:13:09.837480+0000","last_fullsized":"2024-09-11T01:13:09.837480+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4310","reported_seq":4423,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:18.106360+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:13:18.106360+0000","last_peered":"2024-09-11T01:13:18.106360+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:13:18.106360+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":4310,"log_dups_size":0,"ondisk_log_size":4310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17633280,"num_objects":4305,"num_object_clones":0,"num_object_copies":8610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2127,"num_write_kb":8508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":281}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4209","reported_seq":4645,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:13:18.115772+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:13:18.115772+0000","last_peered":"2024-09-11T01:13:18.115772+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:13:18.115772+0000","last_undegraded":"2024-09-11T01:13:18.115772+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":4209,"log_dups_size":0,"ondisk_log_size":4209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17215488,"num_objects":4203,"num_object_clones":0,"num_object_copies":8406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3764,"num_objects_unfound":0,"num_objects_dirty":4203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1994,"num_write_kb":7976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":659456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3764},{"shards":"2","objects":439}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4275","reported_seq":4401,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:18.853304+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:13:18.853304+0000","last_peered":"2024-09-11T01:13:18.853304+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:13:18.853304+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":4275,"log_dups_size":0,"ondisk_log_size":4275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17502208,"num_objects":4273,"num_object_clones":0,"num_object_copies":8546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1012,"num_write_kb":4048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":323},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4295","reported_seq":4426,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:18.810113+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:13:18.810113+0000","last_peered":"2024-09-11T01:13:18.810113+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:13:18.810113+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":4295,"log_dups_size":0,"ondisk_log_size":4295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17559552,"num_objects":4287,"num_object_clones":0,"num_object_copies":8574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":950,"num_write_kb":3800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":348},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658057+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658057+0000","last_peered":"2024-09-11T01:13:09.658057+0000","last_clean":"2024-09-11T01:13:09.658057+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658057+0000","last_undegraded":"2024-09-11T01:13:09.658057+0000","last_fullsized":"2024-09-11T01:13:09.658057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4369","reported_seq":4498,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:18.866778+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:13:18.866778+0000","last_peered":"2024-09-11T01:13:18.866778+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:13:18.866778+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":4369,"log_dups_size":0,"ondisk_log_size":4369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17879040,"num_objects":4365,"num_object_clones":0,"num_object_copies":8730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":978,"num_write_kb":3912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":326},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4275","reported_seq":4402,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:18.880378+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:13:18.880378+0000","last_peered":"2024-09-11T01:13:18.880378+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:13:18.880378+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":4275,"log_dups_size":0,"ondisk_log_size":4275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17461865,"num_objects":4264,"num_object_clones":0,"num_object_copies":8528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1345,"num_write_kb":5380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":347},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":281338473,"num_objects":68687,"num_object_clones":0,"num_object_copies":137374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":3764,"num_objects_unfound":0,"num_objects_dirty":68687,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29618,"num_write_kb":118472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":663552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":559562752,"data_stored":559475777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68768,"ondisk_log_size":68768,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182422,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":421972,"kb_used_data":269572,"kb_used_omap":4,"kb_used_meta":152379,"kb_avail":93949868,"statfs":{"total":96636764160,"available":96204664832,"internally_reserved":0,"allocated":276041728,"data_stored":275792909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":156036779},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,8],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986194,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":416308,"kb_used_data":263908,"kb_used_omap":4,"kb_used_meta":152379,"kb_avail":93955532,"statfs":{"total":96636764160,"available":96210464768,"internally_reserved":0,"allocated":270241792,"data_stored":269997050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":156036778},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659139,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66048,"kb_used_data":12208,"kb_used_omap":4,"kb_used_meta":53819,"kb_avail":94305792,"statfs":{"total":96636764160,"available":96569131008,"internally_reserved":0,"allocated":12500992,"data_stored":12175635,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":55111343},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,0,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":276832256,"data_stored":276828777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12423168,"data_stored":12343151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":270307328,"data_stored":270303849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:13:21.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:13:21.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'4393", 'reported_seq': 4465, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:18.858541+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:13:18.858541+0000', 'last_peered': '2024-09-11T01:13:18.858541+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:13:18.858541+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 4393, 'log_dups_size': 0, 'ondisk_log_size': 4393, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17993728, 'num_objects': 4393, 'num_object_clones': 0, 'num_object_copies': 8786, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4393, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4393, 'num_write_kb': 17572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 321}, {'shards': '0,2', 'objects': 4072}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:21.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:13:21.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'4204", 'reported_seq': 4325, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:18.067237+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:13:18.067237+0000', 'last_peered': '2024-09-11T01:13:18.067237+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:13:18.067237+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 4204, 'log_dups_size': 0, 'ondisk_log_size': 4204, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17203200, 'num_objects': 4200, 'num_object_clones': 0, 'num_object_copies': 8400, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4200, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2505, 'num_write_kb': 10020, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3896}, {'shards': '2', 'objects': 304}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:13:21.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:13:21.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'4293", 'reported_seq': 4455, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:18.843913+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:13:18.843913+0000', 'last_peered': '2024-09-11T01:13:18.843913+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:13:18.843913+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 4293, 'log_dups_size': 0, 'ondisk_log_size': 4293, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17563648, 'num_objects': 4288, 'num_object_clones': 0, 'num_object_copies': 8576, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4288, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 634, 'num_write_kb': 2536, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 337}, {'shards': '0,2', 'objects': 3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:21.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:13:21.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'4322", 'reported_seq': 4446, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:18.886731+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:13:18.886731+0000', 'last_peered': '2024-09-11T01:13:18.886731+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:13:18.886731+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 4322, 'log_dups_size': 0, 'ondisk_log_size': 4322, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17678336, 'num_objects': 4316, 'num_object_clones': 0, 'num_object_copies': 8632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4316, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1179, 'num_write_kb': 4716, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 330}, {'shards': '0,2', 'objects': 3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:21.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:13:21.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'4238", 'reported_seq': 4360, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:18.124182+0000', 'last_change': '2024-09-11T01:13:10.159069+0000', 'last_active': '2024-09-11T01:13:18.124182+0000', 'last_peered': '2024-09-11T01:13:18.124182+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:13:18.124182+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 4238, 'log_dups_size': 0, 'ondisk_log_size': 4238, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17330176, 'num_objects': 4231, 'num_object_clones': 0, 'num_object_copies': 8462, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4231, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2209, 'num_write_kb': 8836, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3932}, {'shards': '2', 'objects': 298}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:13:21.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:13:21.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'4301", 'reported_seq': 4422, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:18.752600+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:13:18.752600+0000', 'last_peered': '2024-09-11T01:13:18.752600+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:13:18.752600+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 4301, 'log_dups_size': 0, 'ondisk_log_size': 4301, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17584128, 'num_objects': 4293, 'num_object_clones': 0, 'num_object_copies': 8586, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4293, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 888, 'num_write_kb': 3552, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 309}, {'shards': '0,2', 'objects': 3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:21.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:13:21.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'4310", 'reported_seq': 4423, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:18.106360+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:13:18.106360+0000', 'last_peered': '2024-09-11T01:13:18.106360+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:13:18.106360+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 4310, 'log_dups_size': 0, 'ondisk_log_size': 4310, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17633280, 'num_objects': 4305, 'num_object_clones': 0, 'num_object_copies': 8610, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4305, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2127, 'num_write_kb': 8508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4023}, {'shards': '2', 'objects': 281}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:13:21.173 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:13:21.173 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'4209", 'reported_seq': 4645, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:13:18.115772+0000', 'last_change': '2024-09-11T01:13:10.217540+0000', 'last_active': '2024-09-11T01:13:18.115772+0000', 'last_peered': '2024-09-11T01:13:18.115772+0000', 'last_clean': '2024-09-11T01:13:07.638709+0000', 'last_became_active': '2024-09-11T01:13:10.125604+0000', 'last_became_peered': '2024-09-11T01:13:10.125604+0000', 'last_unstale': '2024-09-11T01:13:18.115772+0000', 'last_undegraded': '2024-09-11T01:13:18.115772+0000', 'last_fullsized': '2024-09-11T01:13:09.649604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2269", 'last_scrub_stamp': '2024-09-11T01:12:42.890920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:42.890920+0000', 'objects_scrubbed': 2209, 'log_size': 4209, 'log_dups_size': 0, 'ondisk_log_size': 4209, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 999, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17215488, 'num_objects': 4203, 'num_object_clones': 0, 'num_object_copies': 8406, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3764, 'num_objects_unfound': 0, 'num_objects_dirty': 4203, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1994, 'num_write_kb': 7976, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 161, 'num_bytes_recovered': 659456, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3764}, {'shards': '2', 'objects': 439}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:13:21.173 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:13:21.173 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'4275", 'reported_seq': 4401, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:18.853304+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:13:18.853304+0000', 'last_peered': '2024-09-11T01:13:18.853304+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:13:18.853304+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 4275, 'log_dups_size': 0, 'ondisk_log_size': 4275, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17502208, 'num_objects': 4273, 'num_object_clones': 0, 'num_object_copies': 8546, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4273, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1012, 'num_write_kb': 4048, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 323}, {'shards': '0,2', 'objects': 3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:21.173 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:13:21.173 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'4295", 'reported_seq': 4426, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:18.810113+0000', 'last_change': '2024-09-11T01:13:10.209668+0000', 'last_active': '2024-09-11T01:13:18.810113+0000', 'last_peered': '2024-09-11T01:13:18.810113+0000', 'last_clean': '2024-09-11T01:13:07.624565+0000', 'last_became_active': '2024-09-11T01:13:10.199795+0000', 'last_became_peered': '2024-09-11T01:13:10.199795+0000', 'last_unstale': '2024-09-11T01:13:18.810113+0000', 'last_undegraded': '2024-09-11T01:13:09.791198+0000', 'last_fullsized': '2024-09-11T01:13:09.652311+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3415", 'last_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'objects_scrubbed': 3337, 'log_size': 4295, 'log_dups_size': 0, 'ondisk_log_size': 4295, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1975, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17559552, 'num_objects': 4287, 'num_object_clones': 0, 'num_object_copies': 8574, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3942, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4287, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 950, 'num_write_kb': 3800, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 348}, {'shards': '0,2', 'objects': 3939}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:21.173 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:13:21.173 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'4369", 'reported_seq': 4498, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:18.866778+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:13:18.866778+0000', 'last_peered': '2024-09-11T01:13:18.866778+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:13:18.866778+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 4369, 'log_dups_size': 0, 'ondisk_log_size': 4369, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17879040, 'num_objects': 4365, 'num_object_clones': 0, 'num_object_copies': 8730, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4365, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 978, 'num_write_kb': 3912, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 326}, {'shards': '0,2', 'objects': 4039}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:21.173 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:13:21.173 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'4275", 'reported_seq': 4402, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:18.880378+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:13:18.880378+0000', 'last_peered': '2024-09-11T01:13:18.880378+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:13:18.880378+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 4275, 'log_dups_size': 0, 'ondisk_log_size': 4275, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17461865, 'num_objects': 4264, 'num_object_clones': 0, 'num_object_copies': 8528, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4264, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1345, 'num_write_kb': 5380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 347}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:21.174 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:13:21.361 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 57 20 70294 70274 4.8143 2.29688 0.0334779 0.0162186 2024-09-11T01:13:22.172 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:13:22.172 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43711/137378 objects degraded (31.818%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":138,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":68689,"data_bytes":281338492,"bytes_used":926031872,"bytes_avail":288984260608,"bytes_total":289910292480,"degraded_objects":43711,"degraded_total":137378,"degraded_ratio":0.31818049469347348,"misplaced_objects":3764,"misplaced_total":137378,"misplaced_ratio":0.027398855711977171,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":60408,"recovering_keys_per_sec":0,"num_objects_recovered":161,"num_bytes_recovered":659456,"num_keys_recovered":0,"write_bytes_sec":1918804,"read_op_per_sec":0,"write_op_per_sec":468},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-11T01:13:08.555583+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T01:13:22.252 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:13:22.252 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:13:22.361 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 58 20 70872 70852 4.77023 2.25781 0.0684682 0.0163676 2024-09-11T01:13:23.192 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:13:23.193 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:13:23.260 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":106,"stamp":"2024-09-11T01:13:22.558540+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":281338492,"num_objects":68689,"num_object_clones":0,"num_object_copies":137378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":3764,"num_objects_unfound":0,"num_objects_dirty":68689,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29620,"num_write_kb":118474,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":163,"num_bytes_recovered":663571,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68771,"ondisk_log_size":68771,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":905484,"kb_used_data":555160,"kb_used_omap":12,"kb_used_meta":350259,"kb_avail":282210036,"statfs":{"total":289910292480,"available":288983076864,"internally_reserved":0,"allocated":568483840,"data_stored":567664922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":358665220},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,2,7,14,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20946944,"num_objects":5114,"num_object_clones":0,"num_object_copies":10228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":3764,"num_objects_unfound":0,"num_objects_dirty":5114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5114,"num_write_kb":20456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":659456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002528"},"pg_stats":[{"pgid":"2.f","version":"43'4393","reported_seq":4465,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:18.858541+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:13:18.858541+0000","last_peered":"2024-09-11T01:13:18.858541+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:13:18.858541+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":4393,"log_dups_size":0,"ondisk_log_size":4393,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17993728,"num_objects":4393,"num_object_clones":0,"num_object_copies":8786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4393,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4393,"num_write_kb":17572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":321},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4384","reported_seq":4480,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:18.136680+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:13:18.136680+0000","last_peered":"2024-09-11T01:13:18.136680+0000","last_clean":"2024-09-11T01:13:18.136680+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:13:18.136680+0000","last_undegraded":"2024-09-11T01:13:18.136680+0000","last_fullsized":"2024-09-11T01:13:18.136680+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":4384,"log_dups_size":0,"ondisk_log_size":4384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17932288,"num_objects":4378,"num_object_clones":0,"num_object_copies":8756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2623,"num_write_kb":10492,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4204","reported_seq":4325,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:18.067237+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:13:18.067237+0000","last_peered":"2024-09-11T01:13:18.067237+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:13:18.067237+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":4204,"log_dups_size":0,"ondisk_log_size":4204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17203200,"num_objects":4200,"num_object_clones":0,"num_object_copies":8400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2505,"num_write_kb":10020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":304}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4232","reported_seq":4296,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:18.095127+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:13:18.095127+0000","last_peered":"2024-09-11T01:13:18.095127+0000","last_clean":"2024-09-11T01:13:18.095127+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:13:18.095127+0000","last_undegraded":"2024-09-11T01:13:18.095127+0000","last_fullsized":"2024-09-11T01:13:18.095127+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":4232,"log_dups_size":0,"ondisk_log_size":4232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17334272,"num_objects":4232,"num_object_clones":0,"num_object_copies":8464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4232,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4232,"num_write_kb":16928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4307","reported_seq":4429,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:18.839384+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:13:18.839384+0000","last_peered":"2024-09-11T01:13:18.839384+0000","last_clean":"2024-09-11T01:13:18.839384+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:13:18.839384+0000","last_undegraded":"2024-09-11T01:13:18.839384+0000","last_fullsized":"2024-09-11T01:13:18.839384+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":4307,"log_dups_size":0,"ondisk_log_size":4307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17612800,"num_objects":4300,"num_object_clones":0,"num_object_copies":8600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1312,"num_write_kb":5248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4293","reported_seq":4455,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:18.843913+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:13:18.843913+0000","last_peered":"2024-09-11T01:13:18.843913+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:13:18.843913+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":4293,"log_dups_size":0,"ondisk_log_size":4293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17563648,"num_objects":4288,"num_object_clones":0,"num_object_copies":8576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4288,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":634,"num_write_kb":2536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":337},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4322","reported_seq":4446,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:18.886731+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:13:18.886731+0000","last_peered":"2024-09-11T01:13:18.886731+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:13:18.886731+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":4322,"log_dups_size":0,"ondisk_log_size":4322,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17678336,"num_objects":4316,"num_object_clones":0,"num_object_copies":8632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1179,"num_write_kb":4716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":330},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4361","reported_seq":4485,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:18.837570+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:13:18.837570+0000","last_peered":"2024-09-11T01:13:18.837570+0000","last_clean":"2024-09-11T01:13:18.837570+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:13:18.837570+0000","last_undegraded":"2024-09-11T01:13:18.837570+0000","last_fullsized":"2024-09-11T01:13:18.837570+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":4361,"log_dups_size":0,"ondisk_log_size":4361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17854464,"num_objects":4359,"num_object_clones":0,"num_object_copies":8718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1237,"num_write_kb":4948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4238","reported_seq":4360,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:18.124182+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:13:18.124182+0000","last_peered":"2024-09-11T01:13:18.124182+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:13:18.124182+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":4238,"log_dups_size":0,"ondisk_log_size":4238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17330176,"num_objects":4231,"num_object_clones":0,"num_object_copies":8462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4231,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2209,"num_write_kb":8836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":298}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741858+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741858+0000","last_peered":"2024-09-11T01:13:09.741858+0000","last_clean":"2024-09-11T01:13:09.741858+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741858+0000","last_undegraded":"2024-09-11T01:13:09.741858+0000","last_fullsized":"2024-09-11T01:13:09.741858+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4301","reported_seq":4422,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:18.752600+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:13:18.752600+0000","last_peered":"2024-09-11T01:13:18.752600+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:13:18.752600+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":4301,"log_dups_size":0,"ondisk_log_size":4301,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17584128,"num_objects":4293,"num_object_clones":0,"num_object_copies":8586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":888,"num_write_kb":3552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":309},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.837480+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.837480+0000","last_peered":"2024-09-11T01:13:09.837480+0000","last_clean":"2024-09-11T01:13:09.837480+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.837480+0000","last_undegraded":"2024-09-11T01:13:09.837480+0000","last_fullsized":"2024-09-11T01:13:09.837480+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4310","reported_seq":4423,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:18.106360+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:13:18.106360+0000","last_peered":"2024-09-11T01:13:18.106360+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:13:18.106360+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":4310,"log_dups_size":0,"ondisk_log_size":4310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17633280,"num_objects":4305,"num_object_clones":0,"num_object_copies":8610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2127,"num_write_kb":8508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":281}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4209","reported_seq":4645,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:13:18.115772+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:13:18.115772+0000","last_peered":"2024-09-11T01:13:18.115772+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:13:18.115772+0000","last_undegraded":"2024-09-11T01:13:18.115772+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":4209,"log_dups_size":0,"ondisk_log_size":4209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17215488,"num_objects":4203,"num_object_clones":0,"num_object_copies":8406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3764,"num_objects_unfound":0,"num_objects_dirty":4203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1994,"num_write_kb":7976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":161,"num_bytes_recovered":659456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3764},{"shards":"2","objects":439}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4275","reported_seq":4401,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:18.853304+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:13:18.853304+0000","last_peered":"2024-09-11T01:13:18.853304+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:13:18.853304+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":4275,"log_dups_size":0,"ondisk_log_size":4275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17502208,"num_objects":4273,"num_object_clones":0,"num_object_copies":8546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1012,"num_write_kb":4048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":323},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4295","reported_seq":4426,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:18.810113+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:13:18.810113+0000","last_peered":"2024-09-11T01:13:18.810113+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:13:18.810113+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":4295,"log_dups_size":0,"ondisk_log_size":4295,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17559552,"num_objects":4287,"num_object_clones":0,"num_object_copies":8574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":950,"num_write_kb":3800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":348},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658057+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658057+0000","last_peered":"2024-09-11T01:13:09.658057+0000","last_clean":"2024-09-11T01:13:09.658057+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658057+0000","last_undegraded":"2024-09-11T01:13:09.658057+0000","last_fullsized":"2024-09-11T01:13:09.658057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4369","reported_seq":4498,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:18.866778+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:13:18.866778+0000","last_peered":"2024-09-11T01:13:18.866778+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:13:18.866778+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":4369,"log_dups_size":0,"ondisk_log_size":4369,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17879040,"num_objects":4365,"num_object_clones":0,"num_object_copies":8730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":978,"num_write_kb":3912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":326},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4275","reported_seq":4402,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:18.880378+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:13:18.880378+0000","last_peered":"2024-09-11T01:13:18.880378+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:13:18.880378+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":4275,"log_dups_size":0,"ondisk_log_size":4275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17461865,"num_objects":4264,"num_object_clones":0,"num_object_copies":8528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1345,"num_write_kb":5380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":347},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":281338473,"num_objects":68687,"num_object_clones":0,"num_object_copies":137374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":3764,"num_objects_unfound":0,"num_objects_dirty":68687,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29618,"num_write_kb":118472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":162,"num_bytes_recovered":663552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":569290752,"data_stored":569203777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68768,"ondisk_log_size":68768,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182422,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":421972,"kb_used_data":269572,"kb_used_omap":4,"kb_used_meta":152379,"kb_avail":93949868,"statfs":{"total":96636764160,"available":96204664832,"internally_reserved":0,"allocated":276041728,"data_stored":275792909,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":156036779},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,8],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986194,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":416308,"kb_used_data":263908,"kb_used_omap":4,"kb_used_meta":152379,"kb_avail":93955532,"statfs":{"total":96636764160,"available":96210464768,"internally_reserved":0,"allocated":270241792,"data_stored":269997050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":156036778},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659140,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67204,"kb_used_data":21680,"kb_used_omap":4,"kb_used_meta":45499,"kb_avail":94304636,"statfs":{"total":96636764160,"available":96567947264,"internally_reserved":0,"allocated":22200320,"data_stored":21874963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":46591663},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,4,2,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":276832256,"data_stored":276828777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22151168,"data_stored":22071151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":270307328,"data_stored":270303849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:13:23.363 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 59 20 71466 71446 4.72857 2.32031 0.0737831 0.0165124 2024-09-11T01:13:24.363 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout:2024-09-11T01:13:24.362956+0000 min lat: 0.00219431 max lat: 2.69211 avg lat: 0.0166483 2024-09-11T01:13:24.363 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T01:13:24.363 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 60 20 72083 72063 4.68993 2.41016 0.0412438 0.0166483 2024-09-11T01:13:25.363 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 61 20 72709 72689 4.65314 2.44531 0.0267526 0.0167785 2024-09-11T01:13:26.262 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:13:26.363 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 62 20 73315 73295 4.61628 2.36719 0.00538799 0.0169129 2024-09-11T01:13:27.038 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:13:27.039 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:13:27.097 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":108,"stamp":"2024-09-11T01:13:26.560879+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":293261948,"num_objects":71600,"num_object_clones":0,"num_object_copies":143200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":3649,"num_objects_unfound":0,"num_objects_dirty":71600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32531,"num_write_kb":130118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":1134611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71682,"ondisk_log_size":71682,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":895516,"kb_used_data":570152,"kb_used_omap":12,"kb_used_meta":325299,"kb_avail":282220004,"statfs":{"total":289910292480,"available":288993284096,"internally_reserved":0,"allocated":583835648,"data_stored":583016730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":333106180},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,8,6,6,14,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24604672,"num_objects":6007,"num_object_clones":0,"num_object_copies":12014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-214,"num_objects_unfound":0,"num_objects_dirty":6007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6007,"num_write_kb":24028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":876544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004137"},"pg_stats":[{"pgid":"2.f","version":"43'4577","reported_seq":4649,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:23.674333+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:13:23.674333+0000","last_peered":"2024-09-11T01:13:23.674333+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:13:23.674333+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":4577,"log_dups_size":0,"ondisk_log_size":4577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18747392,"num_objects":4577,"num_object_clones":0,"num_object_copies":9154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4577,"num_write_kb":18308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":505},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4581","reported_seq":4677,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:23.097285+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:13:23.097285+0000","last_peered":"2024-09-11T01:13:23.097285+0000","last_clean":"2024-09-11T01:13:23.097285+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:13:23.097285+0000","last_undegraded":"2024-09-11T01:13:23.097285+0000","last_fullsized":"2024-09-11T01:13:23.097285+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":4581,"log_dups_size":0,"ondisk_log_size":4581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18739200,"num_objects":4575,"num_object_clones":0,"num_object_copies":9150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2820,"num_write_kb":11280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4392","reported_seq":4514,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:23.126697+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:13:23.126697+0000","last_peered":"2024-09-11T01:13:23.126697+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:13:23.126697+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":4392,"log_dups_size":0,"ondisk_log_size":4392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17973248,"num_objects":4388,"num_object_clones":0,"num_object_copies":8776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2693,"num_write_kb":10772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":492}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4418","reported_seq":4482,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:23.118291+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:13:23.118291+0000","last_peered":"2024-09-11T01:13:23.118291+0000","last_clean":"2024-09-11T01:13:23.118291+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:13:23.118291+0000","last_undegraded":"2024-09-11T01:13:23.118291+0000","last_fullsized":"2024-09-11T01:13:23.118291+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":4418,"log_dups_size":0,"ondisk_log_size":4418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18096128,"num_objects":4418,"num_object_clones":0,"num_object_copies":8836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4418,"num_write_kb":17672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4482","reported_seq":4604,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:23.869205+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:13:23.869205+0000","last_peered":"2024-09-11T01:13:23.869205+0000","last_clean":"2024-09-11T01:13:23.869205+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:13:23.869205+0000","last_undegraded":"2024-09-11T01:13:23.869205+0000","last_fullsized":"2024-09-11T01:13:23.869205+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":4482,"log_dups_size":0,"ondisk_log_size":4482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18329600,"num_objects":4475,"num_object_clones":0,"num_object_copies":8950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1487,"num_write_kb":5948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4467","reported_seq":4632,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:23.785217+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:13:23.785217+0000","last_peered":"2024-09-11T01:13:23.785217+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:13:23.785217+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":4467,"log_dups_size":0,"ondisk_log_size":4467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18276352,"num_objects":4462,"num_object_clones":0,"num_object_copies":8924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":808,"num_write_kb":3232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":511},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4470","reported_seq":4595,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:23.815881+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:13:23.815881+0000","last_peered":"2024-09-11T01:13:23.815881+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:13:23.815881+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":4470,"log_dups_size":0,"ondisk_log_size":4470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18284544,"num_objects":4464,"num_object_clones":0,"num_object_copies":8928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1327,"num_write_kb":5308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":478},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4551","reported_seq":4675,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:23.860473+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:13:23.860473+0000","last_peered":"2024-09-11T01:13:23.860473+0000","last_clean":"2024-09-11T01:13:23.860473+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:13:23.860473+0000","last_undegraded":"2024-09-11T01:13:23.860473+0000","last_fullsized":"2024-09-11T01:13:23.860473+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":4551,"log_dups_size":0,"ondisk_log_size":4551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18632704,"num_objects":4549,"num_object_clones":0,"num_object_copies":9098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1427,"num_write_kb":5708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4420","reported_seq":4542,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:23.163737+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:13:23.163737+0000","last_peered":"2024-09-11T01:13:23.163737+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:13:23.163737+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":4420,"log_dups_size":0,"ondisk_log_size":4420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18075648,"num_objects":4413,"num_object_clones":0,"num_object_copies":8826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4413,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2391,"num_write_kb":9564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":480}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741858+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741858+0000","last_peered":"2024-09-11T01:13:09.741858+0000","last_clean":"2024-09-11T01:13:09.741858+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741858+0000","last_undegraded":"2024-09-11T01:13:09.741858+0000","last_fullsized":"2024-09-11T01:13:09.741858+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4481","reported_seq":4603,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:23.822615+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:13:23.822615+0000","last_peered":"2024-09-11T01:13:23.822615+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:13:23.822615+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":4481,"log_dups_size":0,"ondisk_log_size":4481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18321408,"num_objects":4473,"num_object_clones":0,"num_object_copies":8946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1068,"num_write_kb":4272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":489},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.837480+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.837480+0000","last_peered":"2024-09-11T01:13:09.837480+0000","last_clean":"2024-09-11T01:13:09.837480+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.837480+0000","last_undegraded":"2024-09-11T01:13:09.837480+0000","last_fullsized":"2024-09-11T01:13:09.837480+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4483","reported_seq":4596,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:23.028614+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:13:23.028614+0000","last_peered":"2024-09-11T01:13:23.028614+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:13:23.028614+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":4483,"log_dups_size":0,"ondisk_log_size":4483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18341888,"num_objects":4478,"num_object_clones":0,"num_object_copies":8956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2300,"num_write_kb":9200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":454}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4377","reported_seq":5042,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:13:23.149602+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:13:23.149602+0000","last_peered":"2024-09-11T01:13:23.149602+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:13:23.149602+0000","last_undegraded":"2024-09-11T01:13:23.149602+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":4377,"log_dups_size":0,"ondisk_log_size":4377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17903616,"num_objects":4371,"num_object_clones":0,"num_object_copies":8742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3649,"num_objects_unfound":0,"num_objects_dirty":4371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2162,"num_write_kb":8648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":1130496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3649},{"shards":"2","objects":722}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4471","reported_seq":4597,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:23.865697+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:13:23.865697+0000","last_peered":"2024-09-11T01:13:23.865697+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:13:23.865697+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":4471,"log_dups_size":0,"ondisk_log_size":4471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18305024,"num_objects":4469,"num_object_clones":0,"num_object_copies":8938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1208,"num_write_kb":4832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":519},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4495","reported_seq":4626,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:23.862176+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:13:23.862176+0000","last_peered":"2024-09-11T01:13:23.862176+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:13:23.862176+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":4495,"log_dups_size":0,"ondisk_log_size":4495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18378752,"num_objects":4487,"num_object_clones":0,"num_object_copies":8974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1150,"num_write_kb":4600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":548},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658057+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658057+0000","last_peered":"2024-09-11T01:13:09.658057+0000","last_clean":"2024-09-11T01:13:09.658057+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658057+0000","last_undegraded":"2024-09-11T01:13:09.658057+0000","last_fullsized":"2024-09-11T01:13:09.658057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4556","reported_seq":4685,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:23.833845+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:13:23.833845+0000","last_peered":"2024-09-11T01:13:23.833845+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:13:23.833845+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":4556,"log_dups_size":0,"ondisk_log_size":4556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18644992,"num_objects":4552,"num_object_clones":0,"num_object_copies":9104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1165,"num_write_kb":4660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":513},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4458","reported_seq":4586,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:23.855366+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:13:23.855366+0000","last_peered":"2024-09-11T01:13:23.855366+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:13:23.855366+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":4458,"log_dups_size":0,"ondisk_log_size":4458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18211433,"num_objects":4447,"num_object_clones":0,"num_object_copies":8894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1528,"num_write_kb":6112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":530},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":293261929,"num_objects":71598,"num_object_clones":0,"num_object_copies":143196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":3649,"num_objects_unfound":0,"num_objects_dirty":71598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32529,"num_write_kb":130116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":277,"num_bytes_recovered":1134592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":584249344,"data_stored":584162369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71679,"ondisk_log_size":71679,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182423,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418544,"kb_used_data":278624,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":93953296,"statfs":{"total":96636764160,"available":96208175104,"internally_reserved":0,"allocated":285310976,"data_stored":285062157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":143257259},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,1,3,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986195,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409768,"kb_used_data":269848,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":93962072,"statfs":{"total":96636764160,"available":96217161728,"internally_reserved":0,"allocated":276324352,"data_stored":276079610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":143257258},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,1,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659140,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67204,"kb_used_data":21680,"kb_used_omap":4,"kb_used_meta":45499,"kb_avail":94304636,"statfs":{"total":96636764160,"available":96567947264,"internally_reserved":0,"allocated":22200320,"data_stored":21874963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":46591663},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,4,2,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":285831168,"data_stored":285827689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22151168,"data_stored":22071151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":276267008,"data_stored":276263529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:13:27.098 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:13:27.363 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 63 20 73886 73866 4.57842 2.23047 0.0182938 0.0170525 2024-09-11T01:13:27.866 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:13:27.868 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:13:27.907 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":108,"stamp":"2024-09-11T01:13:26.560879+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":293261948,"num_objects":71600,"num_object_clones":0,"num_object_copies":143200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":3649,"num_objects_unfound":0,"num_objects_dirty":71600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32531,"num_write_kb":130118,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":1134611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71682,"ondisk_log_size":71682,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":895516,"kb_used_data":570152,"kb_used_omap":12,"kb_used_meta":325299,"kb_avail":282220004,"statfs":{"total":289910292480,"available":288993284096,"internally_reserved":0,"allocated":583835648,"data_stored":583016730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":333106180},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,8,6,6,14,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24604672,"num_objects":6007,"num_object_clones":0,"num_object_copies":12014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-214,"num_objects_unfound":0,"num_objects_dirty":6007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6007,"num_write_kb":24028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":214,"num_bytes_recovered":876544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004137"},"pg_stats":[{"pgid":"2.f","version":"43'4577","reported_seq":4649,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:23.674333+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:13:23.674333+0000","last_peered":"2024-09-11T01:13:23.674333+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:13:23.674333+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":4577,"log_dups_size":0,"ondisk_log_size":4577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18747392,"num_objects":4577,"num_object_clones":0,"num_object_copies":9154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4577,"num_write_kb":18308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":505},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4581","reported_seq":4677,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:23.097285+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:13:23.097285+0000","last_peered":"2024-09-11T01:13:23.097285+0000","last_clean":"2024-09-11T01:13:23.097285+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:13:23.097285+0000","last_undegraded":"2024-09-11T01:13:23.097285+0000","last_fullsized":"2024-09-11T01:13:23.097285+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":4581,"log_dups_size":0,"ondisk_log_size":4581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18739200,"num_objects":4575,"num_object_clones":0,"num_object_copies":9150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2820,"num_write_kb":11280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4392","reported_seq":4514,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:23.126697+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:13:23.126697+0000","last_peered":"2024-09-11T01:13:23.126697+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:13:23.126697+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":4392,"log_dups_size":0,"ondisk_log_size":4392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17973248,"num_objects":4388,"num_object_clones":0,"num_object_copies":8776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4388,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2693,"num_write_kb":10772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":492}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4418","reported_seq":4482,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:23.118291+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:13:23.118291+0000","last_peered":"2024-09-11T01:13:23.118291+0000","last_clean":"2024-09-11T01:13:23.118291+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:13:23.118291+0000","last_undegraded":"2024-09-11T01:13:23.118291+0000","last_fullsized":"2024-09-11T01:13:23.118291+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":4418,"log_dups_size":0,"ondisk_log_size":4418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18096128,"num_objects":4418,"num_object_clones":0,"num_object_copies":8836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4418,"num_write_kb":17672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4482","reported_seq":4604,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:23.869205+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:13:23.869205+0000","last_peered":"2024-09-11T01:13:23.869205+0000","last_clean":"2024-09-11T01:13:23.869205+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:13:23.869205+0000","last_undegraded":"2024-09-11T01:13:23.869205+0000","last_fullsized":"2024-09-11T01:13:23.869205+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":4482,"log_dups_size":0,"ondisk_log_size":4482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18329600,"num_objects":4475,"num_object_clones":0,"num_object_copies":8950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1487,"num_write_kb":5948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4467","reported_seq":4632,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:23.785217+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:13:23.785217+0000","last_peered":"2024-09-11T01:13:23.785217+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:13:23.785217+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":4467,"log_dups_size":0,"ondisk_log_size":4467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18276352,"num_objects":4462,"num_object_clones":0,"num_object_copies":8924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":808,"num_write_kb":3232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":511},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4470","reported_seq":4595,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:23.815881+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:13:23.815881+0000","last_peered":"2024-09-11T01:13:23.815881+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:13:23.815881+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":4470,"log_dups_size":0,"ondisk_log_size":4470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18284544,"num_objects":4464,"num_object_clones":0,"num_object_copies":8928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1327,"num_write_kb":5308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":478},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4551","reported_seq":4675,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:23.860473+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:13:23.860473+0000","last_peered":"2024-09-11T01:13:23.860473+0000","last_clean":"2024-09-11T01:13:23.860473+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:13:23.860473+0000","last_undegraded":"2024-09-11T01:13:23.860473+0000","last_fullsized":"2024-09-11T01:13:23.860473+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":4551,"log_dups_size":0,"ondisk_log_size":4551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18632704,"num_objects":4549,"num_object_clones":0,"num_object_copies":9098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1427,"num_write_kb":5708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4420","reported_seq":4542,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:23.163737+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:13:23.163737+0000","last_peered":"2024-09-11T01:13:23.163737+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:13:23.163737+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":4420,"log_dups_size":0,"ondisk_log_size":4420,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18075648,"num_objects":4413,"num_object_clones":0,"num_object_copies":8826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4413,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2391,"num_write_kb":9564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":480}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741858+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741858+0000","last_peered":"2024-09-11T01:13:09.741858+0000","last_clean":"2024-09-11T01:13:09.741858+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741858+0000","last_undegraded":"2024-09-11T01:13:09.741858+0000","last_fullsized":"2024-09-11T01:13:09.741858+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4481","reported_seq":4603,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:23.822615+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:13:23.822615+0000","last_peered":"2024-09-11T01:13:23.822615+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:13:23.822615+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":4481,"log_dups_size":0,"ondisk_log_size":4481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18321408,"num_objects":4473,"num_object_clones":0,"num_object_copies":8946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1068,"num_write_kb":4272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":489},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.837480+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.837480+0000","last_peered":"2024-09-11T01:13:09.837480+0000","last_clean":"2024-09-11T01:13:09.837480+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.837480+0000","last_undegraded":"2024-09-11T01:13:09.837480+0000","last_fullsized":"2024-09-11T01:13:09.837480+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4483","reported_seq":4596,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:23.028614+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:13:23.028614+0000","last_peered":"2024-09-11T01:13:23.028614+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:13:23.028614+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":4483,"log_dups_size":0,"ondisk_log_size":4483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18341888,"num_objects":4478,"num_object_clones":0,"num_object_copies":8956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2300,"num_write_kb":9200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":454}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4377","reported_seq":5042,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:13:23.149602+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:13:23.149602+0000","last_peered":"2024-09-11T01:13:23.149602+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:13:23.149602+0000","last_undegraded":"2024-09-11T01:13:23.149602+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":4377,"log_dups_size":0,"ondisk_log_size":4377,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17903616,"num_objects":4371,"num_object_clones":0,"num_object_copies":8742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3649,"num_objects_unfound":0,"num_objects_dirty":4371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2162,"num_write_kb":8648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":276,"num_bytes_recovered":1130496,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3649},{"shards":"2","objects":722}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4471","reported_seq":4597,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:23.865697+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:13:23.865697+0000","last_peered":"2024-09-11T01:13:23.865697+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:13:23.865697+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":4471,"log_dups_size":0,"ondisk_log_size":4471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18305024,"num_objects":4469,"num_object_clones":0,"num_object_copies":8938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1208,"num_write_kb":4832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":519},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4495","reported_seq":4626,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:23.862176+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:13:23.862176+0000","last_peered":"2024-09-11T01:13:23.862176+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:13:23.862176+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":4495,"log_dups_size":0,"ondisk_log_size":4495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18378752,"num_objects":4487,"num_object_clones":0,"num_object_copies":8974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1150,"num_write_kb":4600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":548},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658057+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658057+0000","last_peered":"2024-09-11T01:13:09.658057+0000","last_clean":"2024-09-11T01:13:09.658057+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658057+0000","last_undegraded":"2024-09-11T01:13:09.658057+0000","last_fullsized":"2024-09-11T01:13:09.658057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4556","reported_seq":4685,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:23.833845+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:13:23.833845+0000","last_peered":"2024-09-11T01:13:23.833845+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:13:23.833845+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":4556,"log_dups_size":0,"ondisk_log_size":4556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18644992,"num_objects":4552,"num_object_clones":0,"num_object_copies":9104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1165,"num_write_kb":4660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":513},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4458","reported_seq":4586,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:23.855366+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:13:23.855366+0000","last_peered":"2024-09-11T01:13:23.855366+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:13:23.855366+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":4458,"log_dups_size":0,"ondisk_log_size":4458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18211433,"num_objects":4447,"num_object_clones":0,"num_object_copies":8894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1528,"num_write_kb":6112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":530},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":293261929,"num_objects":71598,"num_object_clones":0,"num_object_copies":143196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":3649,"num_objects_unfound":0,"num_objects_dirty":71598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":32529,"num_write_kb":130116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":277,"num_bytes_recovered":1134592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":584249344,"data_stored":584162369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":71679,"ondisk_log_size":71679,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182423,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418544,"kb_used_data":278624,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":93953296,"statfs":{"total":96636764160,"available":96208175104,"internally_reserved":0,"allocated":285310976,"data_stored":285062157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":143257259},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,1,3,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986195,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":409768,"kb_used_data":269848,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":93962072,"statfs":{"total":96636764160,"available":96217161728,"internally_reserved":0,"allocated":276324352,"data_stored":276079610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":143257258},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,1,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659140,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":67204,"kb_used_data":21680,"kb_used_omap":4,"kb_used_meta":45499,"kb_avail":94304636,"statfs":{"total":96636764160,"available":96567947264,"internally_reserved":0,"allocated":22200320,"data_stored":21874963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":46591663},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,4,2,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":285831168,"data_stored":285827689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22151168,"data_stored":22071151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":276267008,"data_stored":276263529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:13:27.908 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:13:27.908 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'4577", 'reported_seq': 4649, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:23.674333+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:13:23.674333+0000', 'last_peered': '2024-09-11T01:13:23.674333+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:13:23.674333+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 4577, 'log_dups_size': 0, 'ondisk_log_size': 4577, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18747392, 'num_objects': 4577, 'num_object_clones': 0, 'num_object_copies': 9154, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4577, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4577, 'num_write_kb': 18308, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 505}, {'shards': '0,2', 'objects': 4072}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:27.908 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:13:27.909 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'4392", 'reported_seq': 4514, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:23.126697+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:13:23.126697+0000', 'last_peered': '2024-09-11T01:13:23.126697+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:13:23.126697+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 4392, 'log_dups_size': 0, 'ondisk_log_size': 4392, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17973248, 'num_objects': 4388, 'num_object_clones': 0, 'num_object_copies': 8776, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4388, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2693, 'num_write_kb': 10772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3896}, {'shards': '2', 'objects': 492}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:13:27.909 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:13:27.909 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'4467", 'reported_seq': 4632, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:23.785217+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:13:23.785217+0000', 'last_peered': '2024-09-11T01:13:23.785217+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:13:23.785217+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 4467, 'log_dups_size': 0, 'ondisk_log_size': 4467, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18276352, 'num_objects': 4462, 'num_object_clones': 0, 'num_object_copies': 8924, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4462, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 808, 'num_write_kb': 3232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 511}, {'shards': '0,2', 'objects': 3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:27.909 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:13:27.909 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'4470", 'reported_seq': 4595, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:23.815881+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:13:23.815881+0000', 'last_peered': '2024-09-11T01:13:23.815881+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:13:23.815881+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 4470, 'log_dups_size': 0, 'ondisk_log_size': 4470, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18284544, 'num_objects': 4464, 'num_object_clones': 0, 'num_object_copies': 8928, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4464, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1327, 'num_write_kb': 5308, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 478}, {'shards': '0,2', 'objects': 3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:27.909 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:13:27.909 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'4420", 'reported_seq': 4542, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:23.163737+0000', 'last_change': '2024-09-11T01:13:10.159069+0000', 'last_active': '2024-09-11T01:13:23.163737+0000', 'last_peered': '2024-09-11T01:13:23.163737+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:13:23.163737+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 4420, 'log_dups_size': 0, 'ondisk_log_size': 4420, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18075648, 'num_objects': 4413, 'num_object_clones': 0, 'num_object_copies': 8826, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4413, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2391, 'num_write_kb': 9564, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3932}, {'shards': '2', 'objects': 480}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:13:27.909 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:13:27.909 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'4481", 'reported_seq': 4603, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:23.822615+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:13:23.822615+0000', 'last_peered': '2024-09-11T01:13:23.822615+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:13:23.822615+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 4481, 'log_dups_size': 0, 'ondisk_log_size': 4481, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18321408, 'num_objects': 4473, 'num_object_clones': 0, 'num_object_copies': 8946, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4473, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1068, 'num_write_kb': 4272, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 489}, {'shards': '0,2', 'objects': 3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:27.909 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:13:27.910 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'4483", 'reported_seq': 4596, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:23.028614+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:13:23.028614+0000', 'last_peered': '2024-09-11T01:13:23.028614+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:13:23.028614+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 4483, 'log_dups_size': 0, 'ondisk_log_size': 4483, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18341888, 'num_objects': 4478, 'num_object_clones': 0, 'num_object_copies': 8956, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4478, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2300, 'num_write_kb': 9200, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4023}, {'shards': '2', 'objects': 454}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:13:27.910 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:13:27.910 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'4377", 'reported_seq': 5042, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:13:23.149602+0000', 'last_change': '2024-09-11T01:13:10.217540+0000', 'last_active': '2024-09-11T01:13:23.149602+0000', 'last_peered': '2024-09-11T01:13:23.149602+0000', 'last_clean': '2024-09-11T01:13:07.638709+0000', 'last_became_active': '2024-09-11T01:13:10.125604+0000', 'last_became_peered': '2024-09-11T01:13:10.125604+0000', 'last_unstale': '2024-09-11T01:13:23.149602+0000', 'last_undegraded': '2024-09-11T01:13:23.149602+0000', 'last_fullsized': '2024-09-11T01:13:09.649604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2269", 'last_scrub_stamp': '2024-09-11T01:12:42.890920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:42.890920+0000', 'objects_scrubbed': 2209, 'log_size': 4377, 'log_dups_size': 0, 'ondisk_log_size': 4377, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 999, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17903616, 'num_objects': 4371, 'num_object_clones': 0, 'num_object_copies': 8742, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3649, 'num_objects_unfound': 0, 'num_objects_dirty': 4371, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2162, 'num_write_kb': 8648, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 276, 'num_bytes_recovered': 1130496, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3649}, {'shards': '2', 'objects': 722}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:13:27.910 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:13:27.910 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'4471", 'reported_seq': 4597, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:23.865697+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:13:23.865697+0000', 'last_peered': '2024-09-11T01:13:23.865697+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:13:23.865697+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 4471, 'log_dups_size': 0, 'ondisk_log_size': 4471, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18305024, 'num_objects': 4469, 'num_object_clones': 0, 'num_object_copies': 8938, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4469, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1208, 'num_write_kb': 4832, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 519}, {'shards': '0,2', 'objects': 3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:27.910 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:13:27.910 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'4495", 'reported_seq': 4626, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:23.862176+0000', 'last_change': '2024-09-11T01:13:10.209668+0000', 'last_active': '2024-09-11T01:13:23.862176+0000', 'last_peered': '2024-09-11T01:13:23.862176+0000', 'last_clean': '2024-09-11T01:13:07.624565+0000', 'last_became_active': '2024-09-11T01:13:10.199795+0000', 'last_became_peered': '2024-09-11T01:13:10.199795+0000', 'last_unstale': '2024-09-11T01:13:23.862176+0000', 'last_undegraded': '2024-09-11T01:13:09.791198+0000', 'last_fullsized': '2024-09-11T01:13:09.652311+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3415", 'last_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'objects_scrubbed': 3337, 'log_size': 4495, 'log_dups_size': 0, 'ondisk_log_size': 4495, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1975, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18378752, 'num_objects': 4487, 'num_object_clones': 0, 'num_object_copies': 8974, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3942, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4487, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1150, 'num_write_kb': 4600, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 548}, {'shards': '0,2', 'objects': 3939}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:27.910 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:13:27.910 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'4556", 'reported_seq': 4685, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:23.833845+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:13:23.833845+0000', 'last_peered': '2024-09-11T01:13:23.833845+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:13:23.833845+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 4556, 'log_dups_size': 0, 'ondisk_log_size': 4556, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18644992, 'num_objects': 4552, 'num_object_clones': 0, 'num_object_copies': 9104, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4552, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1165, 'num_write_kb': 4660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 513}, {'shards': '0,2', 'objects': 4039}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:27.910 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:13:27.911 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'4458", 'reported_seq': 4586, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:23.855366+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:13:23.855366+0000', 'last_peered': '2024-09-11T01:13:23.855366+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:13:23.855366+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 4458, 'log_dups_size': 0, 'ondisk_log_size': 4458, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18211433, 'num_objects': 4447, 'num_object_clones': 0, 'num_object_copies': 8894, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4447, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1528, 'num_write_kb': 6112, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 530}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:27.911 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:13:28.364 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 64 20 74452 74432 4.54135 2.21094 0.0217498 0.0171892 2024-09-11T01:13:28.939 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:13:28.939 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43711/145570 objects degraded (30.027%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":145,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":72785,"data_bytes":298115708,"bytes_used":982806528,"bytes_avail":288927485952,"bytes_total":289910292480,"degraded_objects":43711,"degraded_total":145570,"degraded_ratio":0.30027478189187334,"misplaced_objects":3531,"misplaced_total":145570,"misplaced_ratio":0.024256371505117814,"recovering_objects_per_sec":27,"recovering_bytes_per_sec":113284,"recovering_keys_per_sec":0,"num_objects_recovered":332,"num_bytes_recovered":1359872,"num_keys_recovered":0,"write_bytes_sec":2454035,"read_op_per_sec":0,"write_op_per_sec":599},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-11T01:13:08.555583+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T01:13:29.016 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:13:29.017 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:13:29.366 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 65 20 75040 75020 4.50671 2.29688 0.0174179 0.0173255 2024-09-11T01:13:29.836 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:13:29.836 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:13:29.910 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2024-09-11T01:13:28.561294+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":298115708,"num_objects":72785,"num_object_clones":0,"num_object_copies":145570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":3531,"num_objects_unfound":0,"num_objects_dirty":72785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33716,"num_write_kb":134858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":396,"num_bytes_recovered":1617939,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72867,"ondisk_log_size":72867,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":959772,"kb_used_data":585836,"kb_used_omap":12,"kb_used_meta":373875,"kb_avail":282155748,"statfs":{"total":289910292480,"available":288927485952,"internally_reserved":0,"allocated":599896064,"data_stored":599077146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":382848004},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,9,7,9,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29458432,"num_objects":7192,"num_object_clones":0,"num_object_copies":14384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-332,"num_objects_unfound":0,"num_objects_dirty":7192,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7192,"num_write_kb":28768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":332,"num_bytes_recovered":1359872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004077"},"pg_stats":[{"pgid":"2.f","version":"43'4577","reported_seq":4649,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:23.674333+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:13:23.674333+0000","last_peered":"2024-09-11T01:13:23.674333+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:13:23.674333+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":4577,"log_dups_size":0,"ondisk_log_size":4577,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18747392,"num_objects":4577,"num_object_clones":0,"num_object_copies":9154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4577,"num_write_kb":18308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":505},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4767","reported_seq":4863,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:28.148886+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:13:28.148886+0000","last_peered":"2024-09-11T01:13:28.148886+0000","last_clean":"2024-09-11T01:13:28.148886+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:13:28.148886+0000","last_undegraded":"2024-09-11T01:13:28.148886+0000","last_fullsized":"2024-09-11T01:13:28.148886+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":4767,"log_dups_size":0,"ondisk_log_size":4767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19501056,"num_objects":4761,"num_object_clones":0,"num_object_copies":9522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3006,"num_write_kb":12024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4606","reported_seq":4727,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:28.181647+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:13:28.181647+0000","last_peered":"2024-09-11T01:13:28.181647+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:13:28.181647+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":4606,"log_dups_size":0,"ondisk_log_size":4606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18849792,"num_objects":4602,"num_object_clones":0,"num_object_copies":9204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2907,"num_write_kb":11628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":706}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4630","reported_seq":4694,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:28.176582+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:13:28.176582+0000","last_peered":"2024-09-11T01:13:28.176582+0000","last_clean":"2024-09-11T01:13:28.176582+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:13:28.176582+0000","last_undegraded":"2024-09-11T01:13:28.176582+0000","last_fullsized":"2024-09-11T01:13:28.176582+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":4630,"log_dups_size":0,"ondisk_log_size":4630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18964480,"num_objects":4630,"num_object_clones":0,"num_object_copies":9260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4630,"num_write_kb":18520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4482","reported_seq":4604,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:23.869205+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:13:23.869205+0000","last_peered":"2024-09-11T01:13:23.869205+0000","last_clean":"2024-09-11T01:13:23.869205+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:13:23.869205+0000","last_undegraded":"2024-09-11T01:13:23.869205+0000","last_fullsized":"2024-09-11T01:13:23.869205+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":4482,"log_dups_size":0,"ondisk_log_size":4482,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18329600,"num_objects":4475,"num_object_clones":0,"num_object_copies":8950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1487,"num_write_kb":5948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4467","reported_seq":4632,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:23.785217+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:13:23.785217+0000","last_peered":"2024-09-11T01:13:23.785217+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:13:23.785217+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":4467,"log_dups_size":0,"ondisk_log_size":4467,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18276352,"num_objects":4462,"num_object_clones":0,"num_object_copies":8924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":808,"num_write_kb":3232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":511},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4470","reported_seq":4595,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:23.815881+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:13:23.815881+0000","last_peered":"2024-09-11T01:13:23.815881+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:13:23.815881+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":4470,"log_dups_size":0,"ondisk_log_size":4470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18284544,"num_objects":4464,"num_object_clones":0,"num_object_copies":8928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1327,"num_write_kb":5308,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":478},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4551","reported_seq":4675,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:23.860473+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:13:23.860473+0000","last_peered":"2024-09-11T01:13:23.860473+0000","last_clean":"2024-09-11T01:13:23.860473+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:13:23.860473+0000","last_undegraded":"2024-09-11T01:13:23.860473+0000","last_fullsized":"2024-09-11T01:13:23.860473+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":4551,"log_dups_size":0,"ondisk_log_size":4551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18632704,"num_objects":4549,"num_object_clones":0,"num_object_copies":9098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1427,"num_write_kb":5708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4618","reported_seq":4740,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:28.157544+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:13:28.157544+0000","last_peered":"2024-09-11T01:13:28.157544+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:13:28.157544+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":4618,"log_dups_size":0,"ondisk_log_size":4618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18886656,"num_objects":4611,"num_object_clones":0,"num_object_copies":9222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2589,"num_write_kb":10356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":678}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741858+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741858+0000","last_peered":"2024-09-11T01:13:09.741858+0000","last_clean":"2024-09-11T01:13:09.741858+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741858+0000","last_undegraded":"2024-09-11T01:13:09.741858+0000","last_fullsized":"2024-09-11T01:13:09.741858+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4481","reported_seq":4603,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:23.822615+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:13:23.822615+0000","last_peered":"2024-09-11T01:13:23.822615+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:13:23.822615+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":4481,"log_dups_size":0,"ondisk_log_size":4481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18321408,"num_objects":4473,"num_object_clones":0,"num_object_copies":8946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1068,"num_write_kb":4272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":489},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.837480+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.837480+0000","last_peered":"2024-09-11T01:13:09.837480+0000","last_clean":"2024-09-11T01:13:09.837480+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.837480+0000","last_undegraded":"2024-09-11T01:13:09.837480+0000","last_fullsized":"2024-09-11T01:13:09.837480+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4682","reported_seq":4795,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:28.115766+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:13:28.115766+0000","last_peered":"2024-09-11T01:13:28.115766+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:13:28.115766+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":4682,"log_dups_size":0,"ondisk_log_size":4682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19156992,"num_objects":4677,"num_object_clones":0,"num_object_copies":9354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2499,"num_write_kb":9996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":653}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4553","reported_seq":5453,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:13:28.163171+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:13:28.163171+0000","last_peered":"2024-09-11T01:13:28.163171+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:13:28.163171+0000","last_undegraded":"2024-09-11T01:13:28.163171+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":4553,"log_dups_size":0,"ondisk_log_size":4553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18624512,"num_objects":4547,"num_object_clones":0,"num_object_copies":9094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3531,"num_objects_unfound":0,"num_objects_dirty":4547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2338,"num_write_kb":9352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":394,"num_bytes_recovered":1613824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3531},{"shards":"2","objects":1016}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4471","reported_seq":4597,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:23.865697+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:13:23.865697+0000","last_peered":"2024-09-11T01:13:23.865697+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:13:23.865697+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":4471,"log_dups_size":0,"ondisk_log_size":4471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18305024,"num_objects":4469,"num_object_clones":0,"num_object_copies":8938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1208,"num_write_kb":4832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":519},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4495","reported_seq":4626,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:23.862176+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:13:23.862176+0000","last_peered":"2024-09-11T01:13:23.862176+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:13:23.862176+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":4495,"log_dups_size":0,"ondisk_log_size":4495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18378752,"num_objects":4487,"num_object_clones":0,"num_object_copies":8974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1150,"num_write_kb":4600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":548},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658057+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658057+0000","last_peered":"2024-09-11T01:13:09.658057+0000","last_clean":"2024-09-11T01:13:09.658057+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658057+0000","last_undegraded":"2024-09-11T01:13:09.658057+0000","last_fullsized":"2024-09-11T01:13:09.658057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4556","reported_seq":4685,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:23.833845+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:13:23.833845+0000","last_peered":"2024-09-11T01:13:23.833845+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:13:23.833845+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":4556,"log_dups_size":0,"ondisk_log_size":4556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18644992,"num_objects":4552,"num_object_clones":0,"num_object_copies":9104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1165,"num_write_kb":4660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":513},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4458","reported_seq":4586,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:23.855366+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:13:23.855366+0000","last_peered":"2024-09-11T01:13:23.855366+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:13:23.855366+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":4458,"log_dups_size":0,"ondisk_log_size":4458,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18211433,"num_objects":4447,"num_object_clones":0,"num_object_copies":8894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1528,"num_write_kb":6112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":530},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":298115689,"num_objects":72783,"num_object_clones":0,"num_object_copies":145566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":3531,"num_objects_unfound":0,"num_objects_dirty":72783,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":33714,"num_write_kb":134856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":395,"num_bytes_recovered":1617920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":600444928,"data_stored":600357953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72864,"ondisk_log_size":72864,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182423,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":418544,"kb_used_data":278624,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":93953296,"statfs":{"total":96636764160,"available":96208175104,"internally_reserved":0,"allocated":285310976,"data_stored":285062157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":143257259},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,1,3,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986196,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":420112,"kb_used_data":276032,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93951728,"statfs":{"total":96636764160,"available":96206569472,"internally_reserved":0,"allocated":282656768,"data_stored":282412026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":147517098},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659141,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":121116,"kb_used_data":31180,"kb_used_omap":4,"kb_used_meta":89915,"kb_avail":94250724,"statfs":{"total":96636764160,"available":96512741376,"internally_reserved":0,"allocated":31928320,"data_stored":31602963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":92073647},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,4,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":285831168,"data_stored":285827689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":31956992,"data_stored":31876975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":282656768,"data_stored":282653289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:13:30.366 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 66 20 75617 75597 4.47259 2.25391 0.0453678 0.0174589 2024-09-11T01:13:31.366 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 67 20 76222 76202 4.44111 2.36328 0.0202229 0.0175817 2024-09-11T01:13:32.366 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 68 20 76834 76814 4.41096 2.39062 0.0682036 0.017704 2024-09-11T01:13:32.912 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:13:33.366 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 69 20 77423 77403 4.38039 2.30078 0.00699702 0.017825 2024-09-11T01:13:33.808 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:13:33.809 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:13:33.880 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":111,"stamp":"2024-09-11T01:13:32.564494+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":305541756,"num_objects":74598,"num_object_clones":0,"num_object_copies":149196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":3531,"num_objects_unfound":0,"num_objects_dirty":74598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35529,"num_write_kb":142110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":396,"num_bytes_recovered":1617939,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74680,"ondisk_log_size":74680,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":994536,"kb_used_data":603960,"kb_used_omap":12,"kb_used_meta":390515,"kb_avail":282120984,"statfs":{"total":289910292480,"available":288891887616,"internally_reserved":0,"allocated":618455040,"data_stored":617636122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":399887364},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,3,14,6,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24203264,"num_objects":5909,"num_object_clones":0,"num_object_copies":11818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-233,"num_objects_unfound":0,"num_objects_dirty":5909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5909,"num_write_kb":23636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":233,"num_bytes_recovered":954368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006437"},"pg_stats":[{"pgid":"2.f","version":"43'4730","reported_seq":4802,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:28.930528+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:13:28.930528+0000","last_peered":"2024-09-11T01:13:28.930528+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:13:28.930528+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":4730,"log_dups_size":0,"ondisk_log_size":4730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19374080,"num_objects":4730,"num_object_clones":0,"num_object_copies":9460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4730,"num_write_kb":18920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":658},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4767","reported_seq":4863,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:28.148886+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:13:28.148886+0000","last_peered":"2024-09-11T01:13:28.148886+0000","last_clean":"2024-09-11T01:13:28.148886+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:13:28.148886+0000","last_undegraded":"2024-09-11T01:13:28.148886+0000","last_fullsized":"2024-09-11T01:13:28.148886+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":4767,"log_dups_size":0,"ondisk_log_size":4767,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19501056,"num_objects":4761,"num_object_clones":0,"num_object_copies":9522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3006,"num_write_kb":12024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4606","reported_seq":4727,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:28.181647+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:13:28.181647+0000","last_peered":"2024-09-11T01:13:28.181647+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:13:28.181647+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":4606,"log_dups_size":0,"ondisk_log_size":4606,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18849792,"num_objects":4602,"num_object_clones":0,"num_object_copies":9204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2907,"num_write_kb":11628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":706}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4630","reported_seq":4694,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:28.176582+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:13:28.176582+0000","last_peered":"2024-09-11T01:13:28.176582+0000","last_clean":"2024-09-11T01:13:28.176582+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:13:28.176582+0000","last_undegraded":"2024-09-11T01:13:28.176582+0000","last_fullsized":"2024-09-11T01:13:28.176582+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":4630,"log_dups_size":0,"ondisk_log_size":4630,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18964480,"num_objects":4630,"num_object_clones":0,"num_object_copies":9260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4630,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4630,"num_write_kb":18520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4691","reported_seq":4813,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:28.900292+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:13:28.900292+0000","last_peered":"2024-09-11T01:13:28.900292+0000","last_clean":"2024-09-11T01:13:28.900292+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:13:28.900292+0000","last_undegraded":"2024-09-11T01:13:28.900292+0000","last_fullsized":"2024-09-11T01:13:28.900292+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":4691,"log_dups_size":0,"ondisk_log_size":4691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19185664,"num_objects":4684,"num_object_clones":0,"num_object_copies":9368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4684,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1696,"num_write_kb":6784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4657","reported_seq":4822,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:28.832775+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:13:28.832775+0000","last_peered":"2024-09-11T01:13:28.832775+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:13:28.832775+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":4657,"log_dups_size":0,"ondisk_log_size":4657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19054592,"num_objects":4652,"num_object_clones":0,"num_object_copies":9304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":998,"num_write_kb":3992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":701},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4642","reported_seq":4767,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:28.885596+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:13:28.885596+0000","last_peered":"2024-09-11T01:13:28.885596+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:13:28.885596+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":4642,"log_dups_size":0,"ondisk_log_size":4642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18989056,"num_objects":4636,"num_object_clones":0,"num_object_copies":9272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1499,"num_write_kb":5996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":650},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4709","reported_seq":4833,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:28.781372+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:13:28.781372+0000","last_peered":"2024-09-11T01:13:28.781372+0000","last_clean":"2024-09-11T01:13:28.781372+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:13:28.781372+0000","last_undegraded":"2024-09-11T01:13:28.781372+0000","last_fullsized":"2024-09-11T01:13:28.781372+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":4709,"log_dups_size":0,"ondisk_log_size":4709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19279872,"num_objects":4707,"num_object_clones":0,"num_object_copies":9414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4707,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1585,"num_write_kb":6340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4618","reported_seq":4740,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:28.157544+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:13:28.157544+0000","last_peered":"2024-09-11T01:13:28.157544+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:13:28.157544+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":4618,"log_dups_size":0,"ondisk_log_size":4618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18886656,"num_objects":4611,"num_object_clones":0,"num_object_copies":9222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4611,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2589,"num_write_kb":10356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":678}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741858+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741858+0000","last_peered":"2024-09-11T01:13:09.741858+0000","last_clean":"2024-09-11T01:13:09.741858+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741858+0000","last_undegraded":"2024-09-11T01:13:09.741858+0000","last_fullsized":"2024-09-11T01:13:09.741858+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4659","reported_seq":4782,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:28.889150+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:13:28.889150+0000","last_peered":"2024-09-11T01:13:28.889150+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:13:28.889150+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":4659,"log_dups_size":0,"ondisk_log_size":4659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19050496,"num_objects":4651,"num_object_clones":0,"num_object_copies":9302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1246,"num_write_kb":4984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":667},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.837480+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.837480+0000","last_peered":"2024-09-11T01:13:09.837480+0000","last_clean":"2024-09-11T01:13:09.837480+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.837480+0000","last_undegraded":"2024-09-11T01:13:09.837480+0000","last_fullsized":"2024-09-11T01:13:09.837480+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4682","reported_seq":4795,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:28.115766+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:13:28.115766+0000","last_peered":"2024-09-11T01:13:28.115766+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:13:28.115766+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":4682,"log_dups_size":0,"ondisk_log_size":4682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19156992,"num_objects":4677,"num_object_clones":0,"num_object_copies":9354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2499,"num_write_kb":9996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":653}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4553","reported_seq":5453,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:13:28.163171+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:13:28.163171+0000","last_peered":"2024-09-11T01:13:28.163171+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:13:28.163171+0000","last_undegraded":"2024-09-11T01:13:28.163171+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":4553,"log_dups_size":0,"ondisk_log_size":4553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18624512,"num_objects":4547,"num_object_clones":0,"num_object_copies":9094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3531,"num_objects_unfound":0,"num_objects_dirty":4547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2338,"num_write_kb":9352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":394,"num_bytes_recovered":1613824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3531},{"shards":"2","objects":1016}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4669","reported_seq":4795,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:28.919615+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:13:28.919615+0000","last_peered":"2024-09-11T01:13:28.919615+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:13:28.919615+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":4669,"log_dups_size":0,"ondisk_log_size":4669,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19116032,"num_objects":4667,"num_object_clones":0,"num_object_copies":9334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4667,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1406,"num_write_kb":5624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":717},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4665","reported_seq":4796,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:28.863247+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:13:28.863247+0000","last_peered":"2024-09-11T01:13:28.863247+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:13:28.863247+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":4665,"log_dups_size":0,"ondisk_log_size":4665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19075072,"num_objects":4657,"num_object_clones":0,"num_object_copies":9314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1320,"num_write_kb":5280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":718},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658057+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658057+0000","last_peered":"2024-09-11T01:13:09.658057+0000","last_clean":"2024-09-11T01:13:09.658057+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658057+0000","last_undegraded":"2024-09-11T01:13:09.658057+0000","last_fullsized":"2024-09-11T01:13:09.658057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4727","reported_seq":4856,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:28.902582+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:13:28.902582+0000","last_peered":"2024-09-11T01:13:28.902582+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:13:28.902582+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":4727,"log_dups_size":0,"ondisk_log_size":4727,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19345408,"num_objects":4723,"num_object_clones":0,"num_object_copies":9446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1336,"num_write_kb":5344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":684},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4672","reported_seq":4800,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:28.884656+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:13:28.884656+0000","last_peered":"2024-09-11T01:13:28.884656+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:13:28.884656+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":4672,"log_dups_size":0,"ondisk_log_size":4672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19087977,"num_objects":4661,"num_object_clones":0,"num_object_copies":9322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1742,"num_write_kb":6968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":744},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":305541737,"num_objects":74596,"num_object_clones":0,"num_object_copies":149192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":3531,"num_objects_unfound":0,"num_objects_dirty":74596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":35527,"num_write_kb":142108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":395,"num_bytes_recovered":1617920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":618946560,"data_stored":618859585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74677,"ondisk_log_size":74677,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182424,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":435780,"kb_used_data":287540,"kb_used_omap":4,"kb_used_meta":148219,"kb_avail":93936060,"statfs":{"total":96636764160,"available":96190525440,"internally_reserved":0,"allocated":294440960,"data_stored":294192141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":151776939},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,7,2,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986196,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":420112,"kb_used_data":276032,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":93951728,"statfs":{"total":96636764160,"available":96206569472,"internally_reserved":0,"allocated":282656768,"data_stored":282412026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":147517098},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659142,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":138644,"kb_used_data":40388,"kb_used_omap":4,"kb_used_meta":98235,"kb_avail":94233196,"statfs":{"total":96636764160,"available":96494792704,"internally_reserved":0,"allocated":41357312,"data_stored":41031955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":100593327},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,5,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":294985728,"data_stored":294982249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":41304064,"data_stored":41224047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":282656768,"data_stored":282653289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:13:33.882 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:13:34.368 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 70 20 78002 77982 4.35003 2.26172 0.051205 0.0179501 2024-09-11T01:13:34.684 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:13:34.684 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:13:34.767 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":112,"stamp":"2024-09-11T01:13:34.564911+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":317895292,"num_objects":77614,"num_object_clones":0,"num_object_copies":155228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":3423,"num_objects_unfound":0,"num_objects_dirty":77614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38545,"num_write_kb":154174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":504,"num_bytes_recovered":2060307,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77696,"ondisk_log_size":77696,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1027672,"kb_used_data":619048,"kb_used_omap":12,"kb_used_meta":408563,"kb_avail":282087848,"statfs":{"total":289910292480,"available":288857956352,"internally_reserved":0,"allocated":633905152,"data_stored":633086234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":418368516},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,3,11,9,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":36556800,"num_objects":8925,"num_object_clones":0,"num_object_copies":17850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-341,"num_objects_unfound":0,"num_objects_dirty":8925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8925,"num_write_kb":35700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":341,"num_bytes_recovered":1396736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006371"},"pg_stats":[{"pgid":"2.f","version":"43'4907","reported_seq":4976,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:33.972333+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:13:33.972333+0000","last_peered":"2024-09-11T01:13:33.972333+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:13:33.972333+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":4907,"log_dups_size":0,"ondisk_log_size":4907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20099072,"num_objects":4907,"num_object_clones":0,"num_object_copies":9814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4907,"num_write_kb":19628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":835},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4967","reported_seq":5060,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:33.161471+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:13:33.161471+0000","last_peered":"2024-09-11T01:13:33.161471+0000","last_clean":"2024-09-11T01:13:33.161471+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:13:33.161471+0000","last_undegraded":"2024-09-11T01:13:33.161471+0000","last_fullsized":"2024-09-11T01:13:33.161471+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":4967,"log_dups_size":0,"ondisk_log_size":4967,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20320256,"num_objects":4961,"num_object_clones":0,"num_object_copies":9922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3206,"num_write_kb":12824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4792","reported_seq":4910,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:33.177113+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:13:33.177113+0000","last_peered":"2024-09-11T01:13:33.177113+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:13:33.177113+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":4792,"log_dups_size":0,"ondisk_log_size":4792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19611648,"num_objects":4788,"num_object_clones":0,"num_object_copies":9576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3093,"num_write_kb":12372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":892}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4820","reported_seq":4884,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:33.153789+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:13:33.153789+0000","last_peered":"2024-09-11T01:13:33.153789+0000","last_clean":"2024-09-11T01:13:33.153789+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:13:33.153789+0000","last_undegraded":"2024-09-11T01:13:33.153789+0000","last_fullsized":"2024-09-11T01:13:33.153789+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":4820,"log_dups_size":0,"ondisk_log_size":4820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19742720,"num_objects":4820,"num_object_clones":0,"num_object_copies":9640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4820,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4820,"num_write_kb":19280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4874","reported_seq":4996,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:33.943850+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:13:33.943850+0000","last_peered":"2024-09-11T01:13:33.943850+0000","last_clean":"2024-09-11T01:13:33.943850+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:13:33.943850+0000","last_undegraded":"2024-09-11T01:13:33.943850+0000","last_fullsized":"2024-09-11T01:13:33.943850+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":4874,"log_dups_size":0,"ondisk_log_size":4874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19935232,"num_objects":4867,"num_object_clones":0,"num_object_copies":9734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1879,"num_write_kb":7516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4812","reported_seq":4977,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:33.957819+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:13:33.957819+0000","last_peered":"2024-09-11T01:13:33.957819+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:13:33.957819+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":4812,"log_dups_size":0,"ondisk_log_size":4812,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19689472,"num_objects":4807,"num_object_clones":0,"num_object_copies":9614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1153,"num_write_kb":4612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":856},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4836","reported_seq":4961,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:33.959989+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:13:33.959989+0000","last_peered":"2024-09-11T01:13:33.959989+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:13:33.959989+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":4836,"log_dups_size":0,"ondisk_log_size":4836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19783680,"num_objects":4830,"num_object_clones":0,"num_object_copies":9660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4830,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1693,"num_write_kb":6772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":844},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4909","reported_seq":5033,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:33.861063+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:13:33.861063+0000","last_peered":"2024-09-11T01:13:33.861063+0000","last_clean":"2024-09-11T01:13:33.861063+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:13:33.861063+0000","last_undegraded":"2024-09-11T01:13:33.861063+0000","last_fullsized":"2024-09-11T01:13:33.861063+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":4909,"log_dups_size":0,"ondisk_log_size":4909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20099072,"num_objects":4907,"num_object_clones":0,"num_object_copies":9814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1785,"num_write_kb":7140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4811","reported_seq":4933,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:33.148640+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:13:33.148640+0000","last_peered":"2024-09-11T01:13:33.148640+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:13:33.148640+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":4811,"log_dups_size":0,"ondisk_log_size":4811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19677184,"num_objects":4804,"num_object_clones":0,"num_object_copies":9608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2782,"num_write_kb":11128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":871}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741858+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741858+0000","last_peered":"2024-09-11T01:13:09.741858+0000","last_clean":"2024-09-11T01:13:09.741858+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741858+0000","last_undegraded":"2024-09-11T01:13:09.741858+0000","last_fullsized":"2024-09-11T01:13:09.741858+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4829","reported_seq":4952,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:33.970877+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:13:33.970877+0000","last_peered":"2024-09-11T01:13:33.970877+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:13:33.970877+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":4829,"log_dups_size":0,"ondisk_log_size":4829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19746816,"num_objects":4821,"num_object_clones":0,"num_object_copies":9642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4821,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1416,"num_write_kb":5664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":837},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.837480+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.837480+0000","last_peered":"2024-09-11T01:13:09.837480+0000","last_clean":"2024-09-11T01:13:09.837480+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.837480+0000","last_undegraded":"2024-09-11T01:13:09.837480+0000","last_fullsized":"2024-09-11T01:13:09.837480+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4870","reported_seq":4983,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:33.121095+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:13:33.121095+0000","last_peered":"2024-09-11T01:13:33.121095+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:13:33.121095+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":4870,"log_dups_size":0,"ondisk_log_size":4870,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19927040,"num_objects":4865,"num_object_clones":0,"num_object_copies":9730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2687,"num_write_kb":10748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":841}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4750","reported_seq":5868,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:13:33.190070+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:13:33.190070+0000","last_peered":"2024-09-11T01:13:33.190070+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:13:33.190070+0000","last_undegraded":"2024-09-11T01:13:33.190070+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":4750,"log_dups_size":0,"ondisk_log_size":4750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19431424,"num_objects":4744,"num_object_clones":0,"num_object_copies":9488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3423,"num_objects_unfound":0,"num_objects_dirty":4744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2535,"num_write_kb":10140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":502,"num_bytes_recovered":2056192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3423},{"shards":"2","objects":1321}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4860","reported_seq":4986,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:33.908305+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:13:33.908305+0000","last_peered":"2024-09-11T01:13:33.908305+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:13:33.908305+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":4860,"log_dups_size":0,"ondisk_log_size":4860,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19898368,"num_objects":4858,"num_object_clones":0,"num_object_copies":9716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1597,"num_write_kb":6388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":908},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4859","reported_seq":4990,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:33.915970+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:13:33.915970+0000","last_peered":"2024-09-11T01:13:33.915970+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:13:33.915970+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":4859,"log_dups_size":0,"ondisk_log_size":4859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19869696,"num_objects":4851,"num_object_clones":0,"num_object_copies":9702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1514,"num_write_kb":6056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":912},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658057+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658057+0000","last_peered":"2024-09-11T01:13:09.658057+0000","last_clean":"2024-09-11T01:13:09.658057+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658057+0000","last_undegraded":"2024-09-11T01:13:09.658057+0000","last_fullsized":"2024-09-11T01:13:09.658057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4919","reported_seq":5048,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:33.952415+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:13:33.952415+0000","last_peered":"2024-09-11T01:13:33.952415+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:13:33.952415+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":4919,"log_dups_size":0,"ondisk_log_size":4919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20131840,"num_objects":4915,"num_object_clones":0,"num_object_copies":9830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1528,"num_write_kb":6112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":876},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4878","reported_seq":5006,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:33.949471+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:13:33.949471+0000","last_peered":"2024-09-11T01:13:33.949471+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:13:33.949471+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":4878,"log_dups_size":0,"ondisk_log_size":4878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19931753,"num_objects":4867,"num_object_clones":0,"num_object_copies":9734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1948,"num_write_kb":7792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":950},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":317895273,"num_objects":77612,"num_object_clones":0,"num_object_copies":155224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":3423,"num_objects_unfound":0,"num_objects_dirty":77612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38543,"num_write_kb":154172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":503,"num_bytes_recovered":2060288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":634396672,"data_stored":634309697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77693,"ondisk_log_size":77693,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182425,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":454604,"kb_used_data":296636,"kb_used_omap":4,"kb_used_meta":157947,"kb_avail":93917236,"statfs":{"total":96636764160,"available":96171249664,"internally_reserved":0,"allocated":303755264,"data_stored":303506445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":161738411},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,5,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986197,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":434424,"kb_used_data":282024,"kb_used_omap":4,"kb_used_meta":152379,"kb_avail":93937416,"statfs":{"total":96636764160,"available":96191913984,"internally_reserved":0,"allocated":288792576,"data_stored":288547834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":156036778},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659142,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":138644,"kb_used_data":40388,"kb_used_omap":4,"kb_used_meta":98235,"kb_avail":94233196,"statfs":{"total":96636764160,"available":96494792704,"internally_reserved":0,"allocated":41357312,"data_stored":41031955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":100593327},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,5,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":304164864,"data_stored":304161385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":41304064,"data_stored":41224047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":288927744,"data_stored":288924265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:13:34.768 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:13:34.768 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'4907", 'reported_seq': 4976, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:33.972333+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:13:33.972333+0000', 'last_peered': '2024-09-11T01:13:33.972333+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:13:33.972333+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 4907, 'log_dups_size': 0, 'ondisk_log_size': 4907, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20099072, 'num_objects': 4907, 'num_object_clones': 0, 'num_object_copies': 9814, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4907, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4907, 'num_write_kb': 19628, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 835}, {'shards': '0,2', 'objects': 4072}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:34.768 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:13:34.768 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'4792", 'reported_seq': 4910, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:33.177113+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:13:33.177113+0000', 'last_peered': '2024-09-11T01:13:33.177113+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:13:33.177113+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 4792, 'log_dups_size': 0, 'ondisk_log_size': 4792, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19611648, 'num_objects': 4788, 'num_object_clones': 0, 'num_object_copies': 9576, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4788, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3093, 'num_write_kb': 12372, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3896}, {'shards': '2', 'objects': 892}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:13:34.768 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:13:34.768 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'4812", 'reported_seq': 4977, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:33.957819+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:13:33.957819+0000', 'last_peered': '2024-09-11T01:13:33.957819+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:13:33.957819+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 4812, 'log_dups_size': 0, 'ondisk_log_size': 4812, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19689472, 'num_objects': 4807, 'num_object_clones': 0, 'num_object_copies': 9614, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4807, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1153, 'num_write_kb': 4612, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 856}, {'shards': '0,2', 'objects': 3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:34.769 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:13:34.769 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'4836", 'reported_seq': 4961, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:33.959989+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:13:33.959989+0000', 'last_peered': '2024-09-11T01:13:33.959989+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:13:33.959989+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 4836, 'log_dups_size': 0, 'ondisk_log_size': 4836, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19783680, 'num_objects': 4830, 'num_object_clones': 0, 'num_object_copies': 9660, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4830, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1693, 'num_write_kb': 6772, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 844}, {'shards': '0,2', 'objects': 3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:34.769 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:13:34.769 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'4811", 'reported_seq': 4933, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:33.148640+0000', 'last_change': '2024-09-11T01:13:10.159069+0000', 'last_active': '2024-09-11T01:13:33.148640+0000', 'last_peered': '2024-09-11T01:13:33.148640+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:13:33.148640+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 4811, 'log_dups_size': 0, 'ondisk_log_size': 4811, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19677184, 'num_objects': 4804, 'num_object_clones': 0, 'num_object_copies': 9608, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4804, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2782, 'num_write_kb': 11128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3932}, {'shards': '2', 'objects': 871}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:13:34.769 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:13:34.769 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'4829", 'reported_seq': 4952, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:33.970877+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:13:33.970877+0000', 'last_peered': '2024-09-11T01:13:33.970877+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:13:33.970877+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 4829, 'log_dups_size': 0, 'ondisk_log_size': 4829, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19746816, 'num_objects': 4821, 'num_object_clones': 0, 'num_object_copies': 9642, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4821, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1416, 'num_write_kb': 5664, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 837}, {'shards': '0,2', 'objects': 3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:34.769 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:13:34.770 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'4870", 'reported_seq': 4983, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:33.121095+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:13:33.121095+0000', 'last_peered': '2024-09-11T01:13:33.121095+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:13:33.121095+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 4870, 'log_dups_size': 0, 'ondisk_log_size': 4870, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19927040, 'num_objects': 4865, 'num_object_clones': 0, 'num_object_copies': 9730, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4865, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2687, 'num_write_kb': 10748, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4023}, {'shards': '2', 'objects': 841}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:13:34.770 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:13:34.770 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'4750", 'reported_seq': 5868, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:13:33.190070+0000', 'last_change': '2024-09-11T01:13:10.217540+0000', 'last_active': '2024-09-11T01:13:33.190070+0000', 'last_peered': '2024-09-11T01:13:33.190070+0000', 'last_clean': '2024-09-11T01:13:07.638709+0000', 'last_became_active': '2024-09-11T01:13:10.125604+0000', 'last_became_peered': '2024-09-11T01:13:10.125604+0000', 'last_unstale': '2024-09-11T01:13:33.190070+0000', 'last_undegraded': '2024-09-11T01:13:33.190070+0000', 'last_fullsized': '2024-09-11T01:13:09.649604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2269", 'last_scrub_stamp': '2024-09-11T01:12:42.890920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:42.890920+0000', 'objects_scrubbed': 2209, 'log_size': 4750, 'log_dups_size': 0, 'ondisk_log_size': 4750, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 999, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19431424, 'num_objects': 4744, 'num_object_clones': 0, 'num_object_copies': 9488, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3423, 'num_objects_unfound': 0, 'num_objects_dirty': 4744, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2535, 'num_write_kb': 10140, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 502, 'num_bytes_recovered': 2056192, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3423}, {'shards': '2', 'objects': 1321}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:13:34.770 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:13:34.770 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'4860", 'reported_seq': 4986, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:33.908305+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:13:33.908305+0000', 'last_peered': '2024-09-11T01:13:33.908305+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:13:33.908305+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 4860, 'log_dups_size': 0, 'ondisk_log_size': 4860, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19898368, 'num_objects': 4858, 'num_object_clones': 0, 'num_object_copies': 9716, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4858, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1597, 'num_write_kb': 6388, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 908}, {'shards': '0,2', 'objects': 3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:34.770 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:13:34.770 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'4859", 'reported_seq': 4990, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:33.915970+0000', 'last_change': '2024-09-11T01:13:10.209668+0000', 'last_active': '2024-09-11T01:13:33.915970+0000', 'last_peered': '2024-09-11T01:13:33.915970+0000', 'last_clean': '2024-09-11T01:13:07.624565+0000', 'last_became_active': '2024-09-11T01:13:10.199795+0000', 'last_became_peered': '2024-09-11T01:13:10.199795+0000', 'last_unstale': '2024-09-11T01:13:33.915970+0000', 'last_undegraded': '2024-09-11T01:13:09.791198+0000', 'last_fullsized': '2024-09-11T01:13:09.652311+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3415", 'last_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'objects_scrubbed': 3337, 'log_size': 4859, 'log_dups_size': 0, 'ondisk_log_size': 4859, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1975, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19869696, 'num_objects': 4851, 'num_object_clones': 0, 'num_object_copies': 9702, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3942, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4851, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1514, 'num_write_kb': 6056, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 912}, {'shards': '0,2', 'objects': 3939}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:34.770 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:13:34.770 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'4919", 'reported_seq': 5048, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:33.952415+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:13:33.952415+0000', 'last_peered': '2024-09-11T01:13:33.952415+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:13:33.952415+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 4919, 'log_dups_size': 0, 'ondisk_log_size': 4919, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20131840, 'num_objects': 4915, 'num_object_clones': 0, 'num_object_copies': 9830, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4915, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1528, 'num_write_kb': 6112, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 876}, {'shards': '0,2', 'objects': 4039}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:34.770 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:13:34.770 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'4878", 'reported_seq': 5006, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:33.949471+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:13:33.949471+0000', 'last_peered': '2024-09-11T01:13:33.949471+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:13:33.949471+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 4878, 'log_dups_size': 0, 'ondisk_log_size': 4878, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19931753, 'num_objects': 4867, 'num_object_clones': 0, 'num_object_copies': 9734, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4867, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1948, 'num_write_kb': 7792, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 950}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:34.771 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:13:35.368 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 71 20 78566 78546 4.3198 2.20312 0.046692 0.0180766 2024-09-11T01:13:35.793 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:13:35.793 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43711/155228 objects degraded (28.159%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":152,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":77614,"data_bytes":317895292,"bytes_used":1052336128,"bytes_avail":288857956352,"bytes_total":289910292480,"degraded_objects":43711,"degraded_total":155228,"degraded_ratio":0.28159223851367021,"misplaced_objects":3423,"misplaced_total":155228,"misplaced_ratio":0.022051434019635628,"recovering_objects_per_sec":28,"recovering_bytes_per_sec":116332,"recovering_keys_per_sec":0,"num_objects_recovered":341,"num_bytes_recovered":1396736,"num_keys_recovered":0,"write_bytes_sec":3044783,"read_op_per_sec":0,"write_op_per_sec":743},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-11T01:13:08.555583+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T01:13:35.843 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:13:35.844 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:13:36.368 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 72 20 79134 79114 4.29062 2.21875 0.00551506 0.0181973 2024-09-11T01:13:36.647 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:13:36.648 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:13:36.732 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2024-09-11T01:13:36.565382+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":317895292,"num_objects":77614,"num_object_clones":0,"num_object_copies":155228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":3423,"num_objects_unfound":0,"num_objects_dirty":77614,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38545,"num_write_kb":154174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":504,"num_bytes_recovered":2060307,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77696,"ondisk_log_size":77696,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1027672,"kb_used_data":619048,"kb_used_omap":12,"kb_used_meta":408563,"kb_avail":282087848,"statfs":{"total":289910292480,"available":288857956352,"internally_reserved":0,"allocated":633905152,"data_stored":633086234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":418368516},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,3,11,9,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24633344,"num_objects":6014,"num_object_clones":0,"num_object_copies":12028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-226,"num_objects_unfound":0,"num_objects_dirty":6014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6014,"num_write_kb":24056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":226,"num_bytes_recovered":925696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004974"},"pg_stats":[{"pgid":"2.f","version":"43'4907","reported_seq":4976,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:33.972333+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:13:33.972333+0000","last_peered":"2024-09-11T01:13:33.972333+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:13:33.972333+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":4907,"log_dups_size":0,"ondisk_log_size":4907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20099072,"num_objects":4907,"num_object_clones":0,"num_object_copies":9814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4907,"num_write_kb":19628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":835},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'4967","reported_seq":5060,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:33.161471+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:13:33.161471+0000","last_peered":"2024-09-11T01:13:33.161471+0000","last_clean":"2024-09-11T01:13:33.161471+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:13:33.161471+0000","last_undegraded":"2024-09-11T01:13:33.161471+0000","last_fullsized":"2024-09-11T01:13:33.161471+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":4967,"log_dups_size":0,"ondisk_log_size":4967,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20320256,"num_objects":4961,"num_object_clones":0,"num_object_copies":9922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3206,"num_write_kb":12824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4792","reported_seq":4910,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:33.177113+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:13:33.177113+0000","last_peered":"2024-09-11T01:13:33.177113+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:13:33.177113+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":4792,"log_dups_size":0,"ondisk_log_size":4792,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19611648,"num_objects":4788,"num_object_clones":0,"num_object_copies":9576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4788,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3093,"num_write_kb":12372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":892}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4820","reported_seq":4884,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:33.153789+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:13:33.153789+0000","last_peered":"2024-09-11T01:13:33.153789+0000","last_clean":"2024-09-11T01:13:33.153789+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:13:33.153789+0000","last_undegraded":"2024-09-11T01:13:33.153789+0000","last_fullsized":"2024-09-11T01:13:33.153789+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":4820,"log_dups_size":0,"ondisk_log_size":4820,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19742720,"num_objects":4820,"num_object_clones":0,"num_object_copies":9640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4820,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4820,"num_write_kb":19280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4874","reported_seq":4996,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:33.943850+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:13:33.943850+0000","last_peered":"2024-09-11T01:13:33.943850+0000","last_clean":"2024-09-11T01:13:33.943850+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:13:33.943850+0000","last_undegraded":"2024-09-11T01:13:33.943850+0000","last_fullsized":"2024-09-11T01:13:33.943850+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":4874,"log_dups_size":0,"ondisk_log_size":4874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19935232,"num_objects":4867,"num_object_clones":0,"num_object_copies":9734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1879,"num_write_kb":7516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4812","reported_seq":4977,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:33.957819+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:13:33.957819+0000","last_peered":"2024-09-11T01:13:33.957819+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:13:33.957819+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":4812,"log_dups_size":0,"ondisk_log_size":4812,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19689472,"num_objects":4807,"num_object_clones":0,"num_object_copies":9614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1153,"num_write_kb":4612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":856},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4836","reported_seq":4961,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:33.959989+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:13:33.959989+0000","last_peered":"2024-09-11T01:13:33.959989+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:13:33.959989+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":4836,"log_dups_size":0,"ondisk_log_size":4836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19783680,"num_objects":4830,"num_object_clones":0,"num_object_copies":9660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4830,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1693,"num_write_kb":6772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":844},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4909","reported_seq":5033,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:33.861063+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:13:33.861063+0000","last_peered":"2024-09-11T01:13:33.861063+0000","last_clean":"2024-09-11T01:13:33.861063+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:13:33.861063+0000","last_undegraded":"2024-09-11T01:13:33.861063+0000","last_fullsized":"2024-09-11T01:13:33.861063+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":4909,"log_dups_size":0,"ondisk_log_size":4909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20099072,"num_objects":4907,"num_object_clones":0,"num_object_copies":9814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1785,"num_write_kb":7140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'4811","reported_seq":4933,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:33.148640+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:13:33.148640+0000","last_peered":"2024-09-11T01:13:33.148640+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:13:33.148640+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":4811,"log_dups_size":0,"ondisk_log_size":4811,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19677184,"num_objects":4804,"num_object_clones":0,"num_object_copies":9608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2782,"num_write_kb":11128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":871}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741858+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741858+0000","last_peered":"2024-09-11T01:13:09.741858+0000","last_clean":"2024-09-11T01:13:09.741858+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741858+0000","last_undegraded":"2024-09-11T01:13:09.741858+0000","last_fullsized":"2024-09-11T01:13:09.741858+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4829","reported_seq":4952,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:33.970877+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:13:33.970877+0000","last_peered":"2024-09-11T01:13:33.970877+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:13:33.970877+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":4829,"log_dups_size":0,"ondisk_log_size":4829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19746816,"num_objects":4821,"num_object_clones":0,"num_object_copies":9642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4821,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1416,"num_write_kb":5664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":837},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.837480+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.837480+0000","last_peered":"2024-09-11T01:13:09.837480+0000","last_clean":"2024-09-11T01:13:09.837480+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.837480+0000","last_undegraded":"2024-09-11T01:13:09.837480+0000","last_fullsized":"2024-09-11T01:13:09.837480+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'4870","reported_seq":4983,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:33.121095+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:13:33.121095+0000","last_peered":"2024-09-11T01:13:33.121095+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:13:33.121095+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":4870,"log_dups_size":0,"ondisk_log_size":4870,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19927040,"num_objects":4865,"num_object_clones":0,"num_object_copies":9730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2687,"num_write_kb":10748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":841}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4750","reported_seq":5868,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:13:33.190070+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:13:33.190070+0000","last_peered":"2024-09-11T01:13:33.190070+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:13:33.190070+0000","last_undegraded":"2024-09-11T01:13:33.190070+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":4750,"log_dups_size":0,"ondisk_log_size":4750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19431424,"num_objects":4744,"num_object_clones":0,"num_object_copies":9488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3423,"num_objects_unfound":0,"num_objects_dirty":4744,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2535,"num_write_kb":10140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":502,"num_bytes_recovered":2056192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3423},{"shards":"2","objects":1321}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4860","reported_seq":4986,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:33.908305+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:13:33.908305+0000","last_peered":"2024-09-11T01:13:33.908305+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:13:33.908305+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":4860,"log_dups_size":0,"ondisk_log_size":4860,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19898368,"num_objects":4858,"num_object_clones":0,"num_object_copies":9716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1597,"num_write_kb":6388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":908},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4859","reported_seq":4990,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:33.915970+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:13:33.915970+0000","last_peered":"2024-09-11T01:13:33.915970+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:13:33.915970+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":4859,"log_dups_size":0,"ondisk_log_size":4859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19869696,"num_objects":4851,"num_object_clones":0,"num_object_copies":9702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1514,"num_write_kb":6056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":912},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658057+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658057+0000","last_peered":"2024-09-11T01:13:09.658057+0000","last_clean":"2024-09-11T01:13:09.658057+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658057+0000","last_undegraded":"2024-09-11T01:13:09.658057+0000","last_fullsized":"2024-09-11T01:13:09.658057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4919","reported_seq":5048,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:33.952415+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:13:33.952415+0000","last_peered":"2024-09-11T01:13:33.952415+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:13:33.952415+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":4919,"log_dups_size":0,"ondisk_log_size":4919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20131840,"num_objects":4915,"num_object_clones":0,"num_object_copies":9830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1528,"num_write_kb":6112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":876},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4878","reported_seq":5006,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:33.949471+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:13:33.949471+0000","last_peered":"2024-09-11T01:13:33.949471+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:13:33.949471+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":4878,"log_dups_size":0,"ondisk_log_size":4878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19931753,"num_objects":4867,"num_object_clones":0,"num_object_copies":9734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1948,"num_write_kb":7792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":950},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":317895273,"num_objects":77612,"num_object_clones":0,"num_object_copies":155224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":3423,"num_objects_unfound":0,"num_objects_dirty":77612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38543,"num_write_kb":154172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":503,"num_bytes_recovered":2060288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":634396672,"data_stored":634309697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":77693,"ondisk_log_size":77693,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182425,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":454604,"kb_used_data":296636,"kb_used_omap":4,"kb_used_meta":157947,"kb_avail":93917236,"statfs":{"total":96636764160,"available":96171249664,"internally_reserved":0,"allocated":303755264,"data_stored":303506445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":161738411},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,5,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986197,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":434424,"kb_used_data":282024,"kb_used_omap":4,"kb_used_meta":152379,"kb_avail":93937416,"statfs":{"total":96636764160,"available":96191913984,"internally_reserved":0,"allocated":288792576,"data_stored":288547834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":156036778},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659142,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":138644,"kb_used_data":40388,"kb_used_omap":4,"kb_used_meta":98235,"kb_avail":94233196,"statfs":{"total":96636764160,"available":96494792704,"internally_reserved":0,"allocated":41357312,"data_stored":41031955,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":100593327},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,5,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":304164864,"data_stored":304161385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":41304064,"data_stored":41224047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":288927744,"data_stored":288924265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:13:37.368 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 73 20 79720 79700 4.26321 2.28906 0.0390137 0.0183173 2024-09-11T01:13:38.368 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 74 20 80342 80322 4.23844 2.42969 0.0314669 0.0184253 2024-09-11T01:13:39.369 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 75 20 80928 80908 4.21245 2.28906 0.0398101 0.0185393 2024-09-11T01:13:39.734 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:13:40.369 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 76 20 81477 81457 4.18524 2.14453 0.0154902 0.0186563 2024-09-11T01:13:40.562 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:13:40.563 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:13:40.605 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":114,"stamp":"2024-09-11T01:13:38.565791+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":322306684,"num_objects":78691,"num_object_clones":0,"num_object_copies":157382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":3316,"num_objects_unfound":0,"num_objects_dirty":78691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39622,"num_write_kb":158482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":611,"num_bytes_recovered":2498579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78773,"ondisk_log_size":78773,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1021912,"kb_used_data":634080,"kb_used_omap":12,"kb_used_meta":387763,"kb_avail":282093608,"statfs":{"total":289910292480,"available":288863854592,"internally_reserved":0,"allocated":649297920,"data_stored":648479002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":397069316},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,3,10,8,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29044736,"num_objects":7091,"num_object_clones":0,"num_object_copies":14182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-333,"num_objects_unfound":0,"num_objects_dirty":7091,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7091,"num_write_kb":28364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":333,"num_bytes_recovered":1363968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004912"},"pg_stats":[{"pgid":"2.f","version":"43'4907","reported_seq":4976,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:33.972333+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:13:33.972333+0000","last_peered":"2024-09-11T01:13:33.972333+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:13:33.972333+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":4907,"log_dups_size":0,"ondisk_log_size":4907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20099072,"num_objects":4907,"num_object_clones":0,"num_object_copies":9814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4907,"num_write_kb":19628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":835},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5152","reported_seq":5247,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:38.186811+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:13:38.186811+0000","last_peered":"2024-09-11T01:13:38.186811+0000","last_clean":"2024-09-11T01:13:38.186811+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:13:38.186811+0000","last_undegraded":"2024-09-11T01:13:38.186811+0000","last_fullsized":"2024-09-11T01:13:38.186811+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":5152,"log_dups_size":0,"ondisk_log_size":5152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21078016,"num_objects":5146,"num_object_clones":0,"num_object_copies":10292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3391,"num_write_kb":13564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4942","reported_seq":5063,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:38.130571+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:13:38.130571+0000","last_peered":"2024-09-11T01:13:38.130571+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:13:38.130571+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":4942,"log_dups_size":0,"ondisk_log_size":4942,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20226048,"num_objects":4938,"num_object_clones":0,"num_object_copies":9876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3243,"num_write_kb":12972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":1042}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4999","reported_seq":5063,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:38.183845+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:13:38.183845+0000","last_peered":"2024-09-11T01:13:38.183845+0000","last_clean":"2024-09-11T01:13:38.183845+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:13:38.183845+0000","last_undegraded":"2024-09-11T01:13:38.183845+0000","last_fullsized":"2024-09-11T01:13:38.183845+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":4999,"log_dups_size":0,"ondisk_log_size":4999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20475904,"num_objects":4999,"num_object_clones":0,"num_object_copies":9998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4999,"num_write_kb":19996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'4874","reported_seq":4996,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:33.943850+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:13:33.943850+0000","last_peered":"2024-09-11T01:13:33.943850+0000","last_clean":"2024-09-11T01:13:33.943850+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:13:33.943850+0000","last_undegraded":"2024-09-11T01:13:33.943850+0000","last_fullsized":"2024-09-11T01:13:33.943850+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":4874,"log_dups_size":0,"ondisk_log_size":4874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19935232,"num_objects":4867,"num_object_clones":0,"num_object_copies":9734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1879,"num_write_kb":7516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'4812","reported_seq":4977,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:33.957819+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:13:33.957819+0000","last_peered":"2024-09-11T01:13:33.957819+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:13:33.957819+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":4812,"log_dups_size":0,"ondisk_log_size":4812,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19689472,"num_objects":4807,"num_object_clones":0,"num_object_copies":9614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1153,"num_write_kb":4612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":856},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'4836","reported_seq":4961,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:33.959989+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:13:33.959989+0000","last_peered":"2024-09-11T01:13:33.959989+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:13:33.959989+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":4836,"log_dups_size":0,"ondisk_log_size":4836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19783680,"num_objects":4830,"num_object_clones":0,"num_object_copies":9660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4830,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1693,"num_write_kb":6772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":844},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'4909","reported_seq":5033,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:33.861063+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:13:33.861063+0000","last_peered":"2024-09-11T01:13:33.861063+0000","last_clean":"2024-09-11T01:13:33.861063+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:13:33.861063+0000","last_undegraded":"2024-09-11T01:13:33.861063+0000","last_fullsized":"2024-09-11T01:13:33.861063+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":4909,"log_dups_size":0,"ondisk_log_size":4909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20099072,"num_objects":4907,"num_object_clones":0,"num_object_copies":9814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1785,"num_write_kb":7140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5004","reported_seq":5126,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:38.205456+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:13:38.205456+0000","last_peered":"2024-09-11T01:13:38.205456+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:13:38.205456+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":5004,"log_dups_size":0,"ondisk_log_size":5004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20467712,"num_objects":4997,"num_object_clones":0,"num_object_copies":9994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4997,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2975,"num_write_kb":11900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":1064}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741858+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741858+0000","last_peered":"2024-09-11T01:13:09.741858+0000","last_clean":"2024-09-11T01:13:09.741858+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741858+0000","last_undegraded":"2024-09-11T01:13:09.741858+0000","last_fullsized":"2024-09-11T01:13:09.741858+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'4829","reported_seq":4952,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:33.970877+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:13:33.970877+0000","last_peered":"2024-09-11T01:13:33.970877+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:13:33.970877+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":4829,"log_dups_size":0,"ondisk_log_size":4829,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19746816,"num_objects":4821,"num_object_clones":0,"num_object_copies":9642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4821,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1416,"num_write_kb":5664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":837},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.837480+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.837480+0000","last_peered":"2024-09-11T01:13:09.837480+0000","last_clean":"2024-09-11T01:13:09.837480+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.837480+0000","last_undegraded":"2024-09-11T01:13:09.837480+0000","last_fullsized":"2024-09-11T01:13:09.837480+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5055","reported_seq":5168,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:38.096628+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:13:38.096628+0000","last_peered":"2024-09-11T01:13:38.096628+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:13:38.096628+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":5055,"log_dups_size":0,"ondisk_log_size":5055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20684800,"num_objects":5050,"num_object_clones":0,"num_object_copies":10100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2872,"num_write_kb":11488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":1026}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4935","reported_seq":6267,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:13:38.211751+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:13:38.211751+0000","last_peered":"2024-09-11T01:13:38.211751+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:13:38.211751+0000","last_undegraded":"2024-09-11T01:13:38.211751+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":4935,"log_dups_size":0,"ondisk_log_size":4935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20189184,"num_objects":4929,"num_object_clones":0,"num_object_copies":9858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3316,"num_objects_unfound":0,"num_objects_dirty":4929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2720,"num_write_kb":10880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":609,"num_bytes_recovered":2494464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3316},{"shards":"2","objects":1613}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'4860","reported_seq":4986,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:33.908305+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:13:33.908305+0000","last_peered":"2024-09-11T01:13:33.908305+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:13:33.908305+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":4860,"log_dups_size":0,"ondisk_log_size":4860,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19898368,"num_objects":4858,"num_object_clones":0,"num_object_copies":9716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1597,"num_write_kb":6388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":908},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'4859","reported_seq":4990,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:33.915970+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:13:33.915970+0000","last_peered":"2024-09-11T01:13:33.915970+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:13:33.915970+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":4859,"log_dups_size":0,"ondisk_log_size":4859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19869696,"num_objects":4851,"num_object_clones":0,"num_object_copies":9702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4851,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1514,"num_write_kb":6056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":912},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658057+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658057+0000","last_peered":"2024-09-11T01:13:09.658057+0000","last_clean":"2024-09-11T01:13:09.658057+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658057+0000","last_undegraded":"2024-09-11T01:13:09.658057+0000","last_fullsized":"2024-09-11T01:13:09.658057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'4919","reported_seq":5048,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:33.952415+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:13:33.952415+0000","last_peered":"2024-09-11T01:13:33.952415+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:13:33.952415+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":4919,"log_dups_size":0,"ondisk_log_size":4919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20131840,"num_objects":4915,"num_object_clones":0,"num_object_copies":9830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1528,"num_write_kb":6112,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":876},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'4878","reported_seq":5006,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:33.949471+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:13:33.949471+0000","last_peered":"2024-09-11T01:13:33.949471+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:13:33.949471+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":4878,"log_dups_size":0,"ondisk_log_size":4878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19931753,"num_objects":4867,"num_object_clones":0,"num_object_copies":9734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4867,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1948,"num_write_kb":7792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":950},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":322306665,"num_objects":78689,"num_object_clones":0,"num_object_copies":157378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":3316,"num_objects_unfound":0,"num_objects_dirty":78689,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":39620,"num_write_kb":158480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":610,"num_bytes_recovered":2498560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":649781248,"data_stored":649694273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78770,"ondisk_log_size":78770,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182425,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":454604,"kb_used_data":296636,"kb_used_omap":4,"kb_used_meta":157947,"kb_avail":93917236,"statfs":{"total":96636764160,"available":96171249664,"internally_reserved":0,"allocated":303755264,"data_stored":303506445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":161738411},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,5,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986198,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":427796,"kb_used_data":287868,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":93944044,"statfs":{"total":96636764160,"available":96198701056,"internally_reserved":0,"allocated":294776832,"data_stored":294532090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":143257258},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659143,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":139512,"kb_used_data":49576,"kb_used_omap":4,"kb_used_meta":89915,"kb_avail":94232328,"statfs":{"total":96636764160,"available":96493903872,"internally_reserved":0,"allocated":50765824,"data_stored":50440467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":92073647},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":304164864,"data_stored":304161385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":50671616,"data_stored":50591599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":294944768,"data_stored":294941289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:13:40.607 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:13:41.323 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:13:41.323 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:13:41.371 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 77 20 82026 82006 4.15875 2.14453 0.0195338 0.0187765 2024-09-11T01:13:41.383 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":115,"stamp":"2024-09-11T01:13:40.566243+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":329876092,"num_objects":80539,"num_object_clones":0,"num_object_copies":161078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":3316,"num_objects_unfound":0,"num_objects_dirty":80539,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41470,"num_write_kb":165874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":611,"num_bytes_recovered":2498579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80621,"ondisk_log_size":80621,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1039148,"kb_used_data":642996,"kb_used_omap":12,"kb_used_meta":396083,"kb_avail":282076372,"statfs":{"total":289910292480,"available":288846204928,"internally_reserved":0,"allocated":658427904,"data_stored":657608986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":405588996},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,6,8,9,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":31760384,"num_objects":7754,"num_object_clones":0,"num_object_copies":15508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-215,"num_objects_unfound":0,"num_objects_dirty":7754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7754,"num_write_kb":31016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":215,"num_bytes_recovered":880640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004948"},"pg_stats":[{"pgid":"2.f","version":"43'5106","reported_seq":5178,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:38.936538+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:13:38.936538+0000","last_peered":"2024-09-11T01:13:38.936538+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:13:38.936538+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":5106,"log_dups_size":0,"ondisk_log_size":5106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20914176,"num_objects":5106,"num_object_clones":0,"num_object_copies":10212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5106,"num_write_kb":20424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1034},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5152","reported_seq":5247,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:38.186811+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:13:38.186811+0000","last_peered":"2024-09-11T01:13:38.186811+0000","last_clean":"2024-09-11T01:13:38.186811+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:13:38.186811+0000","last_undegraded":"2024-09-11T01:13:38.186811+0000","last_fullsized":"2024-09-11T01:13:38.186811+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":5152,"log_dups_size":0,"ondisk_log_size":5152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21078016,"num_objects":5146,"num_object_clones":0,"num_object_copies":10292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3391,"num_write_kb":13564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4942","reported_seq":5063,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:38.130571+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:13:38.130571+0000","last_peered":"2024-09-11T01:13:38.130571+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:13:38.130571+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":4942,"log_dups_size":0,"ondisk_log_size":4942,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20226048,"num_objects":4938,"num_object_clones":0,"num_object_copies":9876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3243,"num_write_kb":12972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":1042}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4999","reported_seq":5063,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:38.183845+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:13:38.183845+0000","last_peered":"2024-09-11T01:13:38.183845+0000","last_clean":"2024-09-11T01:13:38.183845+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:13:38.183845+0000","last_undegraded":"2024-09-11T01:13:38.183845+0000","last_fullsized":"2024-09-11T01:13:38.183845+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":4999,"log_dups_size":0,"ondisk_log_size":4999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20475904,"num_objects":4999,"num_object_clones":0,"num_object_copies":9998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4999,"num_write_kb":19996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5048","reported_seq":5170,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:38.986208+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:13:38.986208+0000","last_peered":"2024-09-11T01:13:38.986208+0000","last_clean":"2024-09-11T01:13:38.986208+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:13:38.986208+0000","last_undegraded":"2024-09-11T01:13:38.986208+0000","last_fullsized":"2024-09-11T01:13:38.986208+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":5048,"log_dups_size":0,"ondisk_log_size":5048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20647936,"num_objects":5041,"num_object_clones":0,"num_object_copies":10082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2053,"num_write_kb":8212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5001","reported_seq":5166,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:38.983270+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:13:38.983270+0000","last_peered":"2024-09-11T01:13:38.983270+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:13:38.983270+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":5001,"log_dups_size":0,"ondisk_log_size":5001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20463616,"num_objects":4996,"num_object_clones":0,"num_object_copies":9992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1342,"num_write_kb":5368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1045},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5013","reported_seq":5138,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:38.969406+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:13:38.969406+0000","last_peered":"2024-09-11T01:13:38.969406+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:13:38.969406+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":5013,"log_dups_size":0,"ondisk_log_size":5013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20508672,"num_objects":5007,"num_object_clones":0,"num_object_copies":10014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1870,"num_write_kb":7480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1021},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5120","reported_seq":5244,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:38.922580+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:13:38.922580+0000","last_peered":"2024-09-11T01:13:38.922580+0000","last_clean":"2024-09-11T01:13:38.922580+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:13:38.922580+0000","last_undegraded":"2024-09-11T01:13:38.922580+0000","last_fullsized":"2024-09-11T01:13:38.922580+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":5120,"log_dups_size":0,"ondisk_log_size":5120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20963328,"num_objects":5118,"num_object_clones":0,"num_object_copies":10236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1996,"num_write_kb":7984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5004","reported_seq":5126,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:38.205456+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:13:38.205456+0000","last_peered":"2024-09-11T01:13:38.205456+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:13:38.205456+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":5004,"log_dups_size":0,"ondisk_log_size":5004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20467712,"num_objects":4997,"num_object_clones":0,"num_object_copies":9994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4997,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2975,"num_write_kb":11900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":1064}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741858+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741858+0000","last_peered":"2024-09-11T01:13:09.741858+0000","last_clean":"2024-09-11T01:13:09.741858+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741858+0000","last_undegraded":"2024-09-11T01:13:09.741858+0000","last_fullsized":"2024-09-11T01:13:09.741858+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5016","reported_seq":5139,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:38.981359+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:13:38.981359+0000","last_peered":"2024-09-11T01:13:38.981359+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:13:38.981359+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":5016,"log_dups_size":0,"ondisk_log_size":5016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20512768,"num_objects":5008,"num_object_clones":0,"num_object_copies":10016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1603,"num_write_kb":6412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1024},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.837480+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.837480+0000","last_peered":"2024-09-11T01:13:09.837480+0000","last_clean":"2024-09-11T01:13:09.837480+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.837480+0000","last_undegraded":"2024-09-11T01:13:09.837480+0000","last_fullsized":"2024-09-11T01:13:09.837480+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5055","reported_seq":5168,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:38.096628+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:13:38.096628+0000","last_peered":"2024-09-11T01:13:38.096628+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:13:38.096628+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":5055,"log_dups_size":0,"ondisk_log_size":5055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20684800,"num_objects":5050,"num_object_clones":0,"num_object_copies":10100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2872,"num_write_kb":11488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":1026}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4935","reported_seq":6267,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:13:38.211751+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:13:38.211751+0000","last_peered":"2024-09-11T01:13:38.211751+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:13:38.211751+0000","last_undegraded":"2024-09-11T01:13:38.211751+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":4935,"log_dups_size":0,"ondisk_log_size":4935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20189184,"num_objects":4929,"num_object_clones":0,"num_object_copies":9858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3316,"num_objects_unfound":0,"num_objects_dirty":4929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2720,"num_write_kb":10880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":609,"num_bytes_recovered":2494464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3316},{"shards":"2","objects":1613}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5036","reported_seq":5162,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:39.010891+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:13:39.010891+0000","last_peered":"2024-09-11T01:13:39.010891+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:13:39.010891+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":5036,"log_dups_size":0,"ondisk_log_size":5036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20619264,"num_objects":5034,"num_object_clones":0,"num_object_copies":10068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1773,"num_write_kb":7092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1084},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5022","reported_seq":5149,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:38.979342+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:13:38.979342+0000","last_peered":"2024-09-11T01:13:38.979342+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:13:38.979342+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":5022,"log_dups_size":0,"ondisk_log_size":5022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20537344,"num_objects":5014,"num_object_clones":0,"num_object_copies":10028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1677,"num_write_kb":6708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1075},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658057+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658057+0000","last_peered":"2024-09-11T01:13:09.658057+0000","last_clean":"2024-09-11T01:13:09.658057+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658057+0000","last_undegraded":"2024-09-11T01:13:09.658057+0000","last_fullsized":"2024-09-11T01:13:09.658057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5112","reported_seq":5241,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:39.002378+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:13:39.002378+0000","last_peered":"2024-09-11T01:13:39.002378+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:13:39.002378+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":5112,"log_dups_size":0,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20922368,"num_objects":5108,"num_object_clones":0,"num_object_copies":10216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5108,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1721,"num_write_kb":6884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1069},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5057","reported_seq":5185,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:38.985080+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:13:38.985080+0000","last_peered":"2024-09-11T01:13:38.985080+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:13:38.985080+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":5057,"log_dups_size":0,"ondisk_log_size":5057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20664937,"num_objects":5046,"num_object_clones":0,"num_object_copies":10092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5046,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2127,"num_write_kb":8508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1129},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":329876073,"num_objects":80537,"num_object_clones":0,"num_object_copies":161074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":3316,"num_objects_unfound":0,"num_objects_dirty":80537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41468,"num_write_kb":165872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":610,"num_bytes_recovered":2498560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":658792448,"data_stored":658705473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80618,"ondisk_log_size":80618,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182426,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":471840,"kb_used_data":305552,"kb_used_omap":4,"kb_used_meta":166267,"kb_avail":93900000,"statfs":{"total":96636764160,"available":96153600000,"internally_reserved":0,"allocated":312885248,"data_stored":312636429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":170258091},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,3,5,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986198,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":427796,"kb_used_data":287868,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":93944044,"statfs":{"total":96636764160,"available":96198701056,"internally_reserved":0,"allocated":294776832,"data_stored":294532090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":143257258},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659143,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":139512,"kb_used_data":49576,"kb_used_omap":4,"kb_used_meta":89915,"kb_avail":94232328,"statfs":{"total":96636764160,"available":96493903872,"internally_reserved":0,"allocated":50765824,"data_stored":50440467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":92073647},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":313176064,"data_stored":313172585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":50671616,"data_stored":50591599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":294944768,"data_stored":294941289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:13:41.385 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:13:41.385 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'5106", 'reported_seq': 5178, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:38.936538+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:13:38.936538+0000', 'last_peered': '2024-09-11T01:13:38.936538+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:13:38.936538+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 5106, 'log_dups_size': 0, 'ondisk_log_size': 5106, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20914176, 'num_objects': 5106, 'num_object_clones': 0, 'num_object_copies': 10212, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5106, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5106, 'num_write_kb': 20424, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1034}, {'shards': '0,2', 'objects': 4072}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:41.385 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:13:41.385 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'4942", 'reported_seq': 5063, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:38.130571+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:13:38.130571+0000', 'last_peered': '2024-09-11T01:13:38.130571+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:13:38.130571+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 4942, 'log_dups_size': 0, 'ondisk_log_size': 4942, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20226048, 'num_objects': 4938, 'num_object_clones': 0, 'num_object_copies': 9876, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4938, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3243, 'num_write_kb': 12972, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3896}, {'shards': '2', 'objects': 1042}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:13:41.385 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:13:41.385 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'5001", 'reported_seq': 5166, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:38.983270+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:13:38.983270+0000', 'last_peered': '2024-09-11T01:13:38.983270+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:13:38.983270+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 5001, 'log_dups_size': 0, 'ondisk_log_size': 5001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20463616, 'num_objects': 4996, 'num_object_clones': 0, 'num_object_copies': 9992, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4996, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1342, 'num_write_kb': 5368, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1045}, {'shards': '0,2', 'objects': 3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:41.385 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:13:41.385 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'5013", 'reported_seq': 5138, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:38.969406+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:13:38.969406+0000', 'last_peered': '2024-09-11T01:13:38.969406+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:13:38.969406+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 5013, 'log_dups_size': 0, 'ondisk_log_size': 5013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20508672, 'num_objects': 5007, 'num_object_clones': 0, 'num_object_copies': 10014, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5007, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1870, 'num_write_kb': 7480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1021}, {'shards': '0,2', 'objects': 3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:41.385 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:13:41.385 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'5004", 'reported_seq': 5126, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:38.205456+0000', 'last_change': '2024-09-11T01:13:10.159069+0000', 'last_active': '2024-09-11T01:13:38.205456+0000', 'last_peered': '2024-09-11T01:13:38.205456+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:13:38.205456+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 5004, 'log_dups_size': 0, 'ondisk_log_size': 5004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20467712, 'num_objects': 4997, 'num_object_clones': 0, 'num_object_copies': 9994, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4997, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2975, 'num_write_kb': 11900, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3932}, {'shards': '2', 'objects': 1064}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:13:41.386 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:13:41.386 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'5016", 'reported_seq': 5139, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:38.981359+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:13:38.981359+0000', 'last_peered': '2024-09-11T01:13:38.981359+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:13:38.981359+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 5016, 'log_dups_size': 0, 'ondisk_log_size': 5016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20512768, 'num_objects': 5008, 'num_object_clones': 0, 'num_object_copies': 10016, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5008, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1603, 'num_write_kb': 6412, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1024}, {'shards': '0,2', 'objects': 3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:41.386 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:13:41.386 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'5055", 'reported_seq': 5168, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:38.096628+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:13:38.096628+0000', 'last_peered': '2024-09-11T01:13:38.096628+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:13:38.096628+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 5055, 'log_dups_size': 0, 'ondisk_log_size': 5055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20684800, 'num_objects': 5050, 'num_object_clones': 0, 'num_object_copies': 10100, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5050, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2872, 'num_write_kb': 11488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4023}, {'shards': '2', 'objects': 1026}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:13:41.386 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:13:41.386 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'4935", 'reported_seq': 6267, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:13:38.211751+0000', 'last_change': '2024-09-11T01:13:10.217540+0000', 'last_active': '2024-09-11T01:13:38.211751+0000', 'last_peered': '2024-09-11T01:13:38.211751+0000', 'last_clean': '2024-09-11T01:13:07.638709+0000', 'last_became_active': '2024-09-11T01:13:10.125604+0000', 'last_became_peered': '2024-09-11T01:13:10.125604+0000', 'last_unstale': '2024-09-11T01:13:38.211751+0000', 'last_undegraded': '2024-09-11T01:13:38.211751+0000', 'last_fullsized': '2024-09-11T01:13:09.649604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2269", 'last_scrub_stamp': '2024-09-11T01:12:42.890920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:42.890920+0000', 'objects_scrubbed': 2209, 'log_size': 4935, 'log_dups_size': 0, 'ondisk_log_size': 4935, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 999, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20189184, 'num_objects': 4929, 'num_object_clones': 0, 'num_object_copies': 9858, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3316, 'num_objects_unfound': 0, 'num_objects_dirty': 4929, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2720, 'num_write_kb': 10880, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 609, 'num_bytes_recovered': 2494464, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3316}, {'shards': '2', 'objects': 1613}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:13:41.386 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:13:41.386 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'5036", 'reported_seq': 5162, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:39.010891+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:13:39.010891+0000', 'last_peered': '2024-09-11T01:13:39.010891+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:13:39.010891+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 5036, 'log_dups_size': 0, 'ondisk_log_size': 5036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20619264, 'num_objects': 5034, 'num_object_clones': 0, 'num_object_copies': 10068, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5034, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1773, 'num_write_kb': 7092, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1084}, {'shards': '0,2', 'objects': 3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:41.386 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:13:41.386 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'5022", 'reported_seq': 5149, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:38.979342+0000', 'last_change': '2024-09-11T01:13:10.209668+0000', 'last_active': '2024-09-11T01:13:38.979342+0000', 'last_peered': '2024-09-11T01:13:38.979342+0000', 'last_clean': '2024-09-11T01:13:07.624565+0000', 'last_became_active': '2024-09-11T01:13:10.199795+0000', 'last_became_peered': '2024-09-11T01:13:10.199795+0000', 'last_unstale': '2024-09-11T01:13:38.979342+0000', 'last_undegraded': '2024-09-11T01:13:09.791198+0000', 'last_fullsized': '2024-09-11T01:13:09.652311+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3415", 'last_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'objects_scrubbed': 3337, 'log_size': 5022, 'log_dups_size': 0, 'ondisk_log_size': 5022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1975, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20537344, 'num_objects': 5014, 'num_object_clones': 0, 'num_object_copies': 10028, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3942, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5014, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1677, 'num_write_kb': 6708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1075}, {'shards': '0,2', 'objects': 3939}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:41.387 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:13:41.387 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'5112", 'reported_seq': 5241, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:39.002378+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:13:39.002378+0000', 'last_peered': '2024-09-11T01:13:39.002378+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:13:39.002378+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 5112, 'log_dups_size': 0, 'ondisk_log_size': 5112, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20922368, 'num_objects': 5108, 'num_object_clones': 0, 'num_object_copies': 10216, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5108, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1721, 'num_write_kb': 6884, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1069}, {'shards': '0,2', 'objects': 4039}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:41.387 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:13:41.387 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'5057", 'reported_seq': 5185, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:38.985080+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:13:38.985080+0000', 'last_peered': '2024-09-11T01:13:38.985080+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:13:38.985080+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 5057, 'log_dups_size': 0, 'ondisk_log_size': 5057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20664937, 'num_objects': 5046, 'num_object_clones': 0, 'num_object_copies': 10092, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5046, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2127, 'num_write_kb': 8508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1129}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:41.387 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:13:42.369 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 78 20 82568 82548 4.13258 2.11719 0.00726175 0.0188947 2024-09-11T01:13:42.398 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:13:42.398 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43711/161078 objects degraded (27.137%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":158,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":80539,"data_bytes":329876092,"bytes_used":1064087552,"bytes_avail":288846204928,"bytes_total":289910292480,"degraded_objects":43711,"degraded_total":161078,"degraded_ratio":0.27136542544605718,"misplaced_objects":3316,"misplaced_total":161078,"misplaced_ratio":0.020586299805063386,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":73356,"recovering_keys_per_sec":0,"num_objects_recovered":215,"num_bytes_recovered":880640,"num_keys_recovered":0,"write_bytes_sec":2645607,"read_op_per_sec":0,"write_op_per_sec":645},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-11T01:13:08.555583+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T01:13:42.449 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:13:42.450 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:13:43.314 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:13:43.315 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:13:43.369 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 79 20 83114 83094 4.10727 2.13281 0.0204907 0.019013 2024-09-11T01:13:43.391 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":116,"stamp":"2024-09-11T01:13:42.568304+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":329876092,"num_objects":80539,"num_object_clones":0,"num_object_copies":161078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":3316,"num_objects_unfound":0,"num_objects_dirty":80539,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41470,"num_write_kb":165874,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":611,"num_bytes_recovered":2498579,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80621,"ondisk_log_size":80621,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1052368,"kb_used_data":652056,"kb_used_omap":12,"kb_used_meta":400243,"kb_avail":282063152,"statfs":{"total":289910292480,"available":288832667648,"internally_reserved":0,"allocated":667705344,"data_stored":666886426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":409848836},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,3,6,9,12,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24334336,"num_objects":5941,"num_object_clones":0,"num_object_copies":11882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-215,"num_objects_unfound":0,"num_objects_dirty":5941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5941,"num_write_kb":23764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":215,"num_bytes_recovered":880640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004288"},"pg_stats":[{"pgid":"2.f","version":"43'5106","reported_seq":5178,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:38.936538+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:13:38.936538+0000","last_peered":"2024-09-11T01:13:38.936538+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:13:38.936538+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":5106,"log_dups_size":0,"ondisk_log_size":5106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20914176,"num_objects":5106,"num_object_clones":0,"num_object_copies":10212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5106,"num_write_kb":20424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1034},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5152","reported_seq":5247,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:38.186811+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:13:38.186811+0000","last_peered":"2024-09-11T01:13:38.186811+0000","last_clean":"2024-09-11T01:13:38.186811+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:13:38.186811+0000","last_undegraded":"2024-09-11T01:13:38.186811+0000","last_fullsized":"2024-09-11T01:13:38.186811+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":5152,"log_dups_size":0,"ondisk_log_size":5152,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21078016,"num_objects":5146,"num_object_clones":0,"num_object_copies":10292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3391,"num_write_kb":13564,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'4942","reported_seq":5063,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:38.130571+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:13:38.130571+0000","last_peered":"2024-09-11T01:13:38.130571+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:13:38.130571+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":4942,"log_dups_size":0,"ondisk_log_size":4942,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20226048,"num_objects":4938,"num_object_clones":0,"num_object_copies":9876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3243,"num_write_kb":12972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":1042}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'4999","reported_seq":5063,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:38.183845+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:13:38.183845+0000","last_peered":"2024-09-11T01:13:38.183845+0000","last_clean":"2024-09-11T01:13:38.183845+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:13:38.183845+0000","last_undegraded":"2024-09-11T01:13:38.183845+0000","last_fullsized":"2024-09-11T01:13:38.183845+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":4999,"log_dups_size":0,"ondisk_log_size":4999,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20475904,"num_objects":4999,"num_object_clones":0,"num_object_copies":9998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4999,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4999,"num_write_kb":19996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5048","reported_seq":5170,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:38.986208+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:13:38.986208+0000","last_peered":"2024-09-11T01:13:38.986208+0000","last_clean":"2024-09-11T01:13:38.986208+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:13:38.986208+0000","last_undegraded":"2024-09-11T01:13:38.986208+0000","last_fullsized":"2024-09-11T01:13:38.986208+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":5048,"log_dups_size":0,"ondisk_log_size":5048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20647936,"num_objects":5041,"num_object_clones":0,"num_object_copies":10082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2053,"num_write_kb":8212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5001","reported_seq":5166,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:38.983270+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:13:38.983270+0000","last_peered":"2024-09-11T01:13:38.983270+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:13:38.983270+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":5001,"log_dups_size":0,"ondisk_log_size":5001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20463616,"num_objects":4996,"num_object_clones":0,"num_object_copies":9992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1342,"num_write_kb":5368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1045},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5013","reported_seq":5138,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:38.969406+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:13:38.969406+0000","last_peered":"2024-09-11T01:13:38.969406+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:13:38.969406+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":5013,"log_dups_size":0,"ondisk_log_size":5013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20508672,"num_objects":5007,"num_object_clones":0,"num_object_copies":10014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5007,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1870,"num_write_kb":7480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1021},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5120","reported_seq":5244,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:38.922580+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:13:38.922580+0000","last_peered":"2024-09-11T01:13:38.922580+0000","last_clean":"2024-09-11T01:13:38.922580+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:13:38.922580+0000","last_undegraded":"2024-09-11T01:13:38.922580+0000","last_fullsized":"2024-09-11T01:13:38.922580+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":5120,"log_dups_size":0,"ondisk_log_size":5120,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20963328,"num_objects":5118,"num_object_clones":0,"num_object_copies":10236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1996,"num_write_kb":7984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5004","reported_seq":5126,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:38.205456+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:13:38.205456+0000","last_peered":"2024-09-11T01:13:38.205456+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:13:38.205456+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":5004,"log_dups_size":0,"ondisk_log_size":5004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20467712,"num_objects":4997,"num_object_clones":0,"num_object_copies":9994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4997,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2975,"num_write_kb":11900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":1064}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741858+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741858+0000","last_peered":"2024-09-11T01:13:09.741858+0000","last_clean":"2024-09-11T01:13:09.741858+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741858+0000","last_undegraded":"2024-09-11T01:13:09.741858+0000","last_fullsized":"2024-09-11T01:13:09.741858+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5016","reported_seq":5139,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:38.981359+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:13:38.981359+0000","last_peered":"2024-09-11T01:13:38.981359+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:13:38.981359+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":5016,"log_dups_size":0,"ondisk_log_size":5016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20512768,"num_objects":5008,"num_object_clones":0,"num_object_copies":10016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1603,"num_write_kb":6412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1024},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.837480+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.837480+0000","last_peered":"2024-09-11T01:13:09.837480+0000","last_clean":"2024-09-11T01:13:09.837480+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.837480+0000","last_undegraded":"2024-09-11T01:13:09.837480+0000","last_fullsized":"2024-09-11T01:13:09.837480+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5055","reported_seq":5168,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:38.096628+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:13:38.096628+0000","last_peered":"2024-09-11T01:13:38.096628+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:13:38.096628+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":5055,"log_dups_size":0,"ondisk_log_size":5055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20684800,"num_objects":5050,"num_object_clones":0,"num_object_copies":10100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2872,"num_write_kb":11488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":1026}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'4935","reported_seq":6267,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:13:38.211751+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:13:38.211751+0000","last_peered":"2024-09-11T01:13:38.211751+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:13:38.211751+0000","last_undegraded":"2024-09-11T01:13:38.211751+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":4935,"log_dups_size":0,"ondisk_log_size":4935,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20189184,"num_objects":4929,"num_object_clones":0,"num_object_copies":9858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3316,"num_objects_unfound":0,"num_objects_dirty":4929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2720,"num_write_kb":10880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":609,"num_bytes_recovered":2494464,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3316},{"shards":"2","objects":1613}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5036","reported_seq":5162,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:39.010891+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:13:39.010891+0000","last_peered":"2024-09-11T01:13:39.010891+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:13:39.010891+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":5036,"log_dups_size":0,"ondisk_log_size":5036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20619264,"num_objects":5034,"num_object_clones":0,"num_object_copies":10068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1773,"num_write_kb":7092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1084},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5022","reported_seq":5149,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:38.979342+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:13:38.979342+0000","last_peered":"2024-09-11T01:13:38.979342+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:13:38.979342+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":5022,"log_dups_size":0,"ondisk_log_size":5022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20537344,"num_objects":5014,"num_object_clones":0,"num_object_copies":10028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1677,"num_write_kb":6708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1075},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658057+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658057+0000","last_peered":"2024-09-11T01:13:09.658057+0000","last_clean":"2024-09-11T01:13:09.658057+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658057+0000","last_undegraded":"2024-09-11T01:13:09.658057+0000","last_fullsized":"2024-09-11T01:13:09.658057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5112","reported_seq":5241,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:39.002378+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:13:39.002378+0000","last_peered":"2024-09-11T01:13:39.002378+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:13:39.002378+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":5112,"log_dups_size":0,"ondisk_log_size":5112,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20922368,"num_objects":5108,"num_object_clones":0,"num_object_copies":10216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5108,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1721,"num_write_kb":6884,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1069},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5057","reported_seq":5185,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:38.985080+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:13:38.985080+0000","last_peered":"2024-09-11T01:13:38.985080+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:13:38.985080+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":5057,"log_dups_size":0,"ondisk_log_size":5057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20664937,"num_objects":5046,"num_object_clones":0,"num_object_copies":10092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5046,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2127,"num_write_kb":8508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1129},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":329876073,"num_objects":80537,"num_object_clones":0,"num_object_copies":161074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":3316,"num_objects_unfound":0,"num_objects_dirty":80537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":41468,"num_write_kb":165872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":610,"num_bytes_recovered":2498560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":668094464,"data_stored":668007489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80618,"ondisk_log_size":80618,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182426,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":471840,"kb_used_data":305552,"kb_used_omap":4,"kb_used_meta":166267,"kb_avail":93900000,"statfs":{"total":96636764160,"available":96153600000,"internally_reserved":0,"allocated":312885248,"data_stored":312636429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":170258091},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,3,5,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986198,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":427796,"kb_used_data":287868,"kb_used_omap":4,"kb_used_meta":139899,"kb_avail":93944044,"statfs":{"total":96636764160,"available":96198701056,"internally_reserved":0,"allocated":294776832,"data_stored":294532090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":143257258},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659144,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":152732,"kb_used_data":58636,"kb_used_omap":4,"kb_used_meta":94075,"kb_avail":94219108,"statfs":{"total":96636764160,"available":96480366592,"internally_reserved":0,"allocated":60043264,"data_stored":59717907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":96333487},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,4,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":313176064,"data_stored":313172585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":59973632,"data_stored":59893615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":294944768,"data_stored":294941289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:13:44.369 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout:2024-09-11T01:13:44.368939+0000 min lat: 0.00219431 max lat: 2.69211 avg lat: 0.0191195 2024-09-11T01:13:44.369 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T01:13:44.369 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 80 20 83693 83673 4.0842 2.26172 0.0255755 0.0191195 2024-09-11T01:13:45.369 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 81 20 84289 84269 4.06252 2.32812 0.090924 0.0192197 2024-09-11T01:13:46.369 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 82 20 84843 84823 4.03938 2.16406 0.0108045 0.0193301 2024-09-11T01:13:46.394 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:13:47.209 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:13:47.209 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:13:47.292 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":118,"stamp":"2024-09-11T01:13:46.569388+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":341344892,"num_objects":83339,"num_object_clones":0,"num_object_copies":166678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":3204,"num_objects_unfound":0,"num_objects_dirty":83339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44270,"num_write_kb":177074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":723,"num_bytes_recovered":2957331,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83421,"ondisk_log_size":83421,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1094032,"kb_used_data":665952,"kb_used_omap":12,"kb_used_meta":428019,"kb_avail":282021488,"statfs":{"total":289910292480,"available":288790003712,"internally_reserved":0,"allocated":681934848,"data_stored":681115930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":438291460},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,6,11,10,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23449600,"num_objects":5725,"num_object_clones":0,"num_object_copies":11450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-219,"num_objects_unfound":0,"num_objects_dirty":5725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5725,"num_write_kb":22900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":219,"num_bytes_recovered":897024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004476"},"pg_stats":[{"pgid":"2.f","version":"43'5282","reported_seq":5354,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:43.996491+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:13:43.996491+0000","last_peered":"2024-09-11T01:13:43.996491+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:13:43.996491+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":5282,"log_dups_size":0,"ondisk_log_size":5282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21635072,"num_objects":5282,"num_object_clones":0,"num_object_copies":10564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5282,"num_write_kb":21128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1210},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5324","reported_seq":5420,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:43.206844+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:13:43.206844+0000","last_peered":"2024-09-11T01:13:43.206844+0000","last_clean":"2024-09-11T01:13:43.206844+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:13:43.206844+0000","last_undegraded":"2024-09-11T01:13:43.206844+0000","last_fullsized":"2024-09-11T01:13:43.206844+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":5324,"log_dups_size":0,"ondisk_log_size":5324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21782528,"num_objects":5318,"num_object_clones":0,"num_object_copies":10636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3563,"num_write_kb":14252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5105","reported_seq":5227,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:43.224486+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:13:43.224486+0000","last_peered":"2024-09-11T01:13:43.224486+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:13:43.224486+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":5105,"log_dups_size":0,"ondisk_log_size":5105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20893696,"num_objects":5101,"num_object_clones":0,"num_object_copies":10202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3406,"num_write_kb":13624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":1205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5173","reported_seq":5237,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:43.180843+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:13:43.180843+0000","last_peered":"2024-09-11T01:13:43.180843+0000","last_clean":"2024-09-11T01:13:43.180843+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:13:43.180843+0000","last_undegraded":"2024-09-11T01:13:43.180843+0000","last_fullsized":"2024-09-11T01:13:43.180843+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":5173,"log_dups_size":0,"ondisk_log_size":5173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21188608,"num_objects":5173,"num_object_clones":0,"num_object_copies":10346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5173,"num_write_kb":20692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5225","reported_seq":5347,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:43.991632+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:13:43.991632+0000","last_peered":"2024-09-11T01:13:43.991632+0000","last_clean":"2024-09-11T01:13:43.991632+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:13:43.991632+0000","last_undegraded":"2024-09-11T01:13:43.991632+0000","last_fullsized":"2024-09-11T01:13:43.991632+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":5225,"log_dups_size":0,"ondisk_log_size":5225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21372928,"num_objects":5218,"num_object_clones":0,"num_object_copies":10436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2230,"num_write_kb":8920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5170","reported_seq":5335,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:43.972045+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:13:43.972045+0000","last_peered":"2024-09-11T01:13:43.972045+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:13:43.972045+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":5170,"log_dups_size":0,"ondisk_log_size":5170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21155840,"num_objects":5165,"num_object_clones":0,"num_object_copies":10330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1511,"num_write_kb":6044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1214},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5183","reported_seq":5308,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:44.007233+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:13:44.007233+0000","last_peered":"2024-09-11T01:13:44.007233+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:13:44.007233+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":5183,"log_dups_size":0,"ondisk_log_size":5183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21204992,"num_objects":5177,"num_object_clones":0,"num_object_copies":10354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2040,"num_write_kb":8160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1191},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5286","reported_seq":5410,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:43.914467+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:13:43.914467+0000","last_peered":"2024-09-11T01:13:43.914467+0000","last_clean":"2024-09-11T01:13:43.914467+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:13:43.914467+0000","last_undegraded":"2024-09-11T01:13:43.914467+0000","last_fullsized":"2024-09-11T01:13:43.914467+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":5286,"log_dups_size":0,"ondisk_log_size":5286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21643264,"num_objects":5284,"num_object_clones":0,"num_object_copies":10568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2162,"num_write_kb":8648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5188","reported_seq":5309,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:43.162822+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:13:43.162822+0000","last_peered":"2024-09-11T01:13:43.162822+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:13:43.162822+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":5188,"log_dups_size":0,"ondisk_log_size":5188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21221376,"num_objects":5181,"num_object_clones":0,"num_object_copies":10362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3159,"num_write_kb":12636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":1248}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741858+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741858+0000","last_peered":"2024-09-11T01:13:09.741858+0000","last_clean":"2024-09-11T01:13:09.741858+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741858+0000","last_undegraded":"2024-09-11T01:13:09.741858+0000","last_fullsized":"2024-09-11T01:13:09.741858+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5186","reported_seq":5306,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:43.951483+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:13:43.951483+0000","last_peered":"2024-09-11T01:13:43.951483+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:13:43.951483+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":5186,"log_dups_size":0,"ondisk_log_size":5186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21209088,"num_objects":5178,"num_object_clones":0,"num_object_copies":10356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1773,"num_write_kb":7092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1194},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.837480+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.837480+0000","last_peered":"2024-09-11T01:13:09.837480+0000","last_clean":"2024-09-11T01:13:09.837480+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.837480+0000","last_undegraded":"2024-09-11T01:13:09.837480+0000","last_fullsized":"2024-09-11T01:13:09.837480+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5240","reported_seq":5353,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:43.205838+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:13:43.205838+0000","last_peered":"2024-09-11T01:13:43.205838+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:13:43.205838+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":5240,"log_dups_size":0,"ondisk_log_size":5240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21442560,"num_objects":5235,"num_object_clones":0,"num_object_copies":10470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3057,"num_write_kb":12228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":1211}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5134","reported_seq":6690,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:13:43.194368+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:13:43.194368+0000","last_peered":"2024-09-11T01:13:43.194368+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:13:43.194368+0000","last_undegraded":"2024-09-11T01:13:43.194368+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":5134,"log_dups_size":0,"ondisk_log_size":5134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21004288,"num_objects":5128,"num_object_clones":0,"num_object_copies":10256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3204,"num_objects_unfound":0,"num_objects_dirty":5128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2919,"num_write_kb":11676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":721,"num_bytes_recovered":2953216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3204},{"shards":"2","objects":1924}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5201","reported_seq":5327,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:43.966392+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:13:43.966392+0000","last_peered":"2024-09-11T01:13:43.966392+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:13:43.966392+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":5201,"log_dups_size":0,"ondisk_log_size":5201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21295104,"num_objects":5199,"num_object_clones":0,"num_object_copies":10398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1938,"num_write_kb":7752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1249},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5203","reported_seq":5334,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:43.964467+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:13:43.964467+0000","last_peered":"2024-09-11T01:13:43.964467+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:13:43.964467+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":5203,"log_dups_size":0,"ondisk_log_size":5203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21278720,"num_objects":5195,"num_object_clones":0,"num_object_copies":10390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1858,"num_write_kb":7432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1256},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658057+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658057+0000","last_peered":"2024-09-11T01:13:09.658057+0000","last_clean":"2024-09-11T01:13:09.658057+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658057+0000","last_undegraded":"2024-09-11T01:13:09.658057+0000","last_fullsized":"2024-09-11T01:13:09.658057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5268","reported_seq":5397,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:43.940921+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:13:43.940921+0000","last_peered":"2024-09-11T01:13:43.940921+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:13:43.940921+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":5268,"log_dups_size":0,"ondisk_log_size":5268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21561344,"num_objects":5264,"num_object_clones":0,"num_object_copies":10528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1877,"num_write_kb":7508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1225},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5250","reported_seq":5378,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:44.007191+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:13:44.007191+0000","last_peered":"2024-09-11T01:13:44.007191+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:13:44.007191+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":5250,"log_dups_size":0,"ondisk_log_size":5250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21455465,"num_objects":5239,"num_object_clones":0,"num_object_copies":10478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2320,"num_write_kb":9280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1322},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":341344873,"num_objects":83337,"num_object_clones":0,"num_object_copies":166674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":3204,"num_objects_unfound":0,"num_objects_dirty":83337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44268,"num_write_kb":177072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":722,"num_bytes_recovered":2957312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":682299392,"data_stored":682212417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83418,"ondisk_log_size":83418,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182427,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485600,"kb_used_data":313744,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93886240,"statfs":{"total":96636764160,"available":96139509760,"internally_reserved":0,"allocated":321273856,"data_stored":321025037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":175959723},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,4,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986199,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":455700,"kb_used_data":293572,"kb_used_omap":4,"kb_used_meta":162107,"kb_avail":93916140,"statfs":{"total":96636764160,"available":96170127360,"internally_reserved":0,"allocated":300617728,"data_stored":300372986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":165998250},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659144,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":152732,"kb_used_data":58636,"kb_used_omap":4,"kb_used_meta":94075,"kb_avail":94219108,"statfs":{"total":96636764160,"available":96480366592,"internally_reserved":0,"allocated":60043264,"data_stored":59717907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":96333487},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,4,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":321613824,"data_stored":321610345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":59973632,"data_stored":59893615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":300711936,"data_stored":300708457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:13:47.293 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:13:47.370 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 83 20 85383 85363 4.01613 2.10938 0.0953566 0.0194434 2024-09-11T01:13:48.103 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:13:48.104 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:13:48.153 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":118,"stamp":"2024-09-11T01:13:46.569388+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":341344892,"num_objects":83339,"num_object_clones":0,"num_object_copies":166678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":3204,"num_objects_unfound":0,"num_objects_dirty":83339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44270,"num_write_kb":177074,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":723,"num_bytes_recovered":2957331,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83421,"ondisk_log_size":83421,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1094032,"kb_used_data":665952,"kb_used_omap":12,"kb_used_meta":428019,"kb_avail":282021488,"statfs":{"total":289910292480,"available":288790003712,"internally_reserved":0,"allocated":681934848,"data_stored":681115930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":438291460},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,6,11,10,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23449600,"num_objects":5725,"num_object_clones":0,"num_object_copies":11450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-219,"num_objects_unfound":0,"num_objects_dirty":5725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5725,"num_write_kb":22900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":219,"num_bytes_recovered":897024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004476"},"pg_stats":[{"pgid":"2.f","version":"43'5282","reported_seq":5354,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:43.996491+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:13:43.996491+0000","last_peered":"2024-09-11T01:13:43.996491+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:13:43.996491+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":5282,"log_dups_size":0,"ondisk_log_size":5282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21635072,"num_objects":5282,"num_object_clones":0,"num_object_copies":10564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5282,"num_write_kb":21128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1210},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5324","reported_seq":5420,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:43.206844+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:13:43.206844+0000","last_peered":"2024-09-11T01:13:43.206844+0000","last_clean":"2024-09-11T01:13:43.206844+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:13:43.206844+0000","last_undegraded":"2024-09-11T01:13:43.206844+0000","last_fullsized":"2024-09-11T01:13:43.206844+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":5324,"log_dups_size":0,"ondisk_log_size":5324,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21782528,"num_objects":5318,"num_object_clones":0,"num_object_copies":10636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5318,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3563,"num_write_kb":14252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5105","reported_seq":5227,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:43.224486+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:13:43.224486+0000","last_peered":"2024-09-11T01:13:43.224486+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:13:43.224486+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":5105,"log_dups_size":0,"ondisk_log_size":5105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20893696,"num_objects":5101,"num_object_clones":0,"num_object_copies":10202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3406,"num_write_kb":13624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":1205}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5173","reported_seq":5237,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:43.180843+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:13:43.180843+0000","last_peered":"2024-09-11T01:13:43.180843+0000","last_clean":"2024-09-11T01:13:43.180843+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:13:43.180843+0000","last_undegraded":"2024-09-11T01:13:43.180843+0000","last_fullsized":"2024-09-11T01:13:43.180843+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":5173,"log_dups_size":0,"ondisk_log_size":5173,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21188608,"num_objects":5173,"num_object_clones":0,"num_object_copies":10346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5173,"num_write_kb":20692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5225","reported_seq":5347,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:43.991632+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:13:43.991632+0000","last_peered":"2024-09-11T01:13:43.991632+0000","last_clean":"2024-09-11T01:13:43.991632+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:13:43.991632+0000","last_undegraded":"2024-09-11T01:13:43.991632+0000","last_fullsized":"2024-09-11T01:13:43.991632+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":5225,"log_dups_size":0,"ondisk_log_size":5225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21372928,"num_objects":5218,"num_object_clones":0,"num_object_copies":10436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2230,"num_write_kb":8920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5170","reported_seq":5335,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:43.972045+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:13:43.972045+0000","last_peered":"2024-09-11T01:13:43.972045+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:13:43.972045+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":5170,"log_dups_size":0,"ondisk_log_size":5170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21155840,"num_objects":5165,"num_object_clones":0,"num_object_copies":10330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1511,"num_write_kb":6044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1214},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5183","reported_seq":5308,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:44.007233+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:13:44.007233+0000","last_peered":"2024-09-11T01:13:44.007233+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:13:44.007233+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":5183,"log_dups_size":0,"ondisk_log_size":5183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21204992,"num_objects":5177,"num_object_clones":0,"num_object_copies":10354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2040,"num_write_kb":8160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1191},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5286","reported_seq":5410,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:43.914467+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:13:43.914467+0000","last_peered":"2024-09-11T01:13:43.914467+0000","last_clean":"2024-09-11T01:13:43.914467+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:13:43.914467+0000","last_undegraded":"2024-09-11T01:13:43.914467+0000","last_fullsized":"2024-09-11T01:13:43.914467+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":5286,"log_dups_size":0,"ondisk_log_size":5286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21643264,"num_objects":5284,"num_object_clones":0,"num_object_copies":10568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2162,"num_write_kb":8648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5188","reported_seq":5309,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:43.162822+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:13:43.162822+0000","last_peered":"2024-09-11T01:13:43.162822+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:13:43.162822+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":5188,"log_dups_size":0,"ondisk_log_size":5188,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21221376,"num_objects":5181,"num_object_clones":0,"num_object_copies":10362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3159,"num_write_kb":12636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":1248}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741858+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741858+0000","last_peered":"2024-09-11T01:13:09.741858+0000","last_clean":"2024-09-11T01:13:09.741858+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741858+0000","last_undegraded":"2024-09-11T01:13:09.741858+0000","last_fullsized":"2024-09-11T01:13:09.741858+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5186","reported_seq":5306,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:43.951483+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:13:43.951483+0000","last_peered":"2024-09-11T01:13:43.951483+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:13:43.951483+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":5186,"log_dups_size":0,"ondisk_log_size":5186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21209088,"num_objects":5178,"num_object_clones":0,"num_object_copies":10356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1773,"num_write_kb":7092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1194},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.837480+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.837480+0000","last_peered":"2024-09-11T01:13:09.837480+0000","last_clean":"2024-09-11T01:13:09.837480+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.837480+0000","last_undegraded":"2024-09-11T01:13:09.837480+0000","last_fullsized":"2024-09-11T01:13:09.837480+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5240","reported_seq":5353,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:43.205838+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:13:43.205838+0000","last_peered":"2024-09-11T01:13:43.205838+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:13:43.205838+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":5240,"log_dups_size":0,"ondisk_log_size":5240,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21442560,"num_objects":5235,"num_object_clones":0,"num_object_copies":10470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5235,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3057,"num_write_kb":12228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":1211}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5134","reported_seq":6690,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:13:43.194368+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:13:43.194368+0000","last_peered":"2024-09-11T01:13:43.194368+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:13:43.194368+0000","last_undegraded":"2024-09-11T01:13:43.194368+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":5134,"log_dups_size":0,"ondisk_log_size":5134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21004288,"num_objects":5128,"num_object_clones":0,"num_object_copies":10256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3204,"num_objects_unfound":0,"num_objects_dirty":5128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2919,"num_write_kb":11676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":721,"num_bytes_recovered":2953216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3204},{"shards":"2","objects":1924}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5201","reported_seq":5327,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:43.966392+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:13:43.966392+0000","last_peered":"2024-09-11T01:13:43.966392+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:13:43.966392+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":5201,"log_dups_size":0,"ondisk_log_size":5201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21295104,"num_objects":5199,"num_object_clones":0,"num_object_copies":10398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1938,"num_write_kb":7752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1249},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5203","reported_seq":5334,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:43.964467+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:13:43.964467+0000","last_peered":"2024-09-11T01:13:43.964467+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:13:43.964467+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":5203,"log_dups_size":0,"ondisk_log_size":5203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21278720,"num_objects":5195,"num_object_clones":0,"num_object_copies":10390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1858,"num_write_kb":7432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1256},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658057+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658057+0000","last_peered":"2024-09-11T01:13:09.658057+0000","last_clean":"2024-09-11T01:13:09.658057+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658057+0000","last_undegraded":"2024-09-11T01:13:09.658057+0000","last_fullsized":"2024-09-11T01:13:09.658057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5268","reported_seq":5397,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:43.940921+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:13:43.940921+0000","last_peered":"2024-09-11T01:13:43.940921+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:13:43.940921+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":5268,"log_dups_size":0,"ondisk_log_size":5268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21561344,"num_objects":5264,"num_object_clones":0,"num_object_copies":10528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1877,"num_write_kb":7508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1225},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5250","reported_seq":5378,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:44.007191+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:13:44.007191+0000","last_peered":"2024-09-11T01:13:44.007191+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:13:44.007191+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":5250,"log_dups_size":0,"ondisk_log_size":5250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21455465,"num_objects":5239,"num_object_clones":0,"num_object_copies":10478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2320,"num_write_kb":9280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1322},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":341344873,"num_objects":83337,"num_object_clones":0,"num_object_copies":166674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":3204,"num_objects_unfound":0,"num_objects_dirty":83337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44268,"num_write_kb":177072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":722,"num_bytes_recovered":2957312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":682299392,"data_stored":682212417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83418,"ondisk_log_size":83418,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182427,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485600,"kb_used_data":313744,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93886240,"statfs":{"total":96636764160,"available":96139509760,"internally_reserved":0,"allocated":321273856,"data_stored":321025037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":175959723},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,4,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986199,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":455700,"kb_used_data":293572,"kb_used_omap":4,"kb_used_meta":162107,"kb_avail":93916140,"statfs":{"total":96636764160,"available":96170127360,"internally_reserved":0,"allocated":300617728,"data_stored":300372986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":165998250},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659144,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":152732,"kb_used_data":58636,"kb_used_omap":4,"kb_used_meta":94075,"kb_avail":94219108,"statfs":{"total":96636764160,"available":96480366592,"internally_reserved":0,"allocated":60043264,"data_stored":59717907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":96333487},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,1,4,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":321613824,"data_stored":321610345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":59973632,"data_stored":59893615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":300711936,"data_stored":300708457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:13:48.154 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:13:48.154 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'5282", 'reported_seq': 5354, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:43.996491+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:13:43.996491+0000', 'last_peered': '2024-09-11T01:13:43.996491+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:13:43.996491+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 5282, 'log_dups_size': 0, 'ondisk_log_size': 5282, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21635072, 'num_objects': 5282, 'num_object_clones': 0, 'num_object_copies': 10564, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5282, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5282, 'num_write_kb': 21128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1210}, {'shards': '0,2', 'objects': 4072}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:48.155 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:13:48.155 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'5105", 'reported_seq': 5227, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:43.224486+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:13:43.224486+0000', 'last_peered': '2024-09-11T01:13:43.224486+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:13:43.224486+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 5105, 'log_dups_size': 0, 'ondisk_log_size': 5105, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20893696, 'num_objects': 5101, 'num_object_clones': 0, 'num_object_copies': 10202, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5101, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3406, 'num_write_kb': 13624, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3896}, {'shards': '2', 'objects': 1205}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:13:48.155 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:13:48.155 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'5170", 'reported_seq': 5335, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:43.972045+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:13:43.972045+0000', 'last_peered': '2024-09-11T01:13:43.972045+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:13:43.972045+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 5170, 'log_dups_size': 0, 'ondisk_log_size': 5170, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21155840, 'num_objects': 5165, 'num_object_clones': 0, 'num_object_copies': 10330, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5165, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1511, 'num_write_kb': 6044, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1214}, {'shards': '0,2', 'objects': 3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:48.155 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:13:48.155 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'5183", 'reported_seq': 5308, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:44.007233+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:13:44.007233+0000', 'last_peered': '2024-09-11T01:13:44.007233+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:13:44.007233+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 5183, 'log_dups_size': 0, 'ondisk_log_size': 5183, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21204992, 'num_objects': 5177, 'num_object_clones': 0, 'num_object_copies': 10354, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5177, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2040, 'num_write_kb': 8160, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1191}, {'shards': '0,2', 'objects': 3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:48.155 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:13:48.155 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'5188", 'reported_seq': 5309, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:43.162822+0000', 'last_change': '2024-09-11T01:13:10.159069+0000', 'last_active': '2024-09-11T01:13:43.162822+0000', 'last_peered': '2024-09-11T01:13:43.162822+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:13:43.162822+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 5188, 'log_dups_size': 0, 'ondisk_log_size': 5188, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21221376, 'num_objects': 5181, 'num_object_clones': 0, 'num_object_copies': 10362, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5181, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3159, 'num_write_kb': 12636, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3932}, {'shards': '2', 'objects': 1248}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:13:48.155 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:13:48.155 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'5186", 'reported_seq': 5306, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:43.951483+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:13:43.951483+0000', 'last_peered': '2024-09-11T01:13:43.951483+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:13:43.951483+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 5186, 'log_dups_size': 0, 'ondisk_log_size': 5186, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21209088, 'num_objects': 5178, 'num_object_clones': 0, 'num_object_copies': 10356, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5178, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1773, 'num_write_kb': 7092, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1194}, {'shards': '0,2', 'objects': 3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:48.156 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:13:48.156 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'5240", 'reported_seq': 5353, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:43.205838+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:13:43.205838+0000', 'last_peered': '2024-09-11T01:13:43.205838+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:13:43.205838+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 5240, 'log_dups_size': 0, 'ondisk_log_size': 5240, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21442560, 'num_objects': 5235, 'num_object_clones': 0, 'num_object_copies': 10470, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5235, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3057, 'num_write_kb': 12228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4023}, {'shards': '2', 'objects': 1211}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:13:48.156 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:13:48.156 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'5134", 'reported_seq': 6690, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:13:43.194368+0000', 'last_change': '2024-09-11T01:13:10.217540+0000', 'last_active': '2024-09-11T01:13:43.194368+0000', 'last_peered': '2024-09-11T01:13:43.194368+0000', 'last_clean': '2024-09-11T01:13:07.638709+0000', 'last_became_active': '2024-09-11T01:13:10.125604+0000', 'last_became_peered': '2024-09-11T01:13:10.125604+0000', 'last_unstale': '2024-09-11T01:13:43.194368+0000', 'last_undegraded': '2024-09-11T01:13:43.194368+0000', 'last_fullsized': '2024-09-11T01:13:09.649604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2269", 'last_scrub_stamp': '2024-09-11T01:12:42.890920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:42.890920+0000', 'objects_scrubbed': 2209, 'log_size': 5134, 'log_dups_size': 0, 'ondisk_log_size': 5134, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 999, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21004288, 'num_objects': 5128, 'num_object_clones': 0, 'num_object_copies': 10256, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3204, 'num_objects_unfound': 0, 'num_objects_dirty': 5128, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2919, 'num_write_kb': 11676, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 721, 'num_bytes_recovered': 2953216, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3204}, {'shards': '2', 'objects': 1924}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:13:48.156 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:13:48.156 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'5201", 'reported_seq': 5327, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:43.966392+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:13:43.966392+0000', 'last_peered': '2024-09-11T01:13:43.966392+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:13:43.966392+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 5201, 'log_dups_size': 0, 'ondisk_log_size': 5201, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21295104, 'num_objects': 5199, 'num_object_clones': 0, 'num_object_copies': 10398, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5199, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1938, 'num_write_kb': 7752, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1249}, {'shards': '0,2', 'objects': 3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:48.156 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:13:48.156 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'5203", 'reported_seq': 5334, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:43.964467+0000', 'last_change': '2024-09-11T01:13:10.209668+0000', 'last_active': '2024-09-11T01:13:43.964467+0000', 'last_peered': '2024-09-11T01:13:43.964467+0000', 'last_clean': '2024-09-11T01:13:07.624565+0000', 'last_became_active': '2024-09-11T01:13:10.199795+0000', 'last_became_peered': '2024-09-11T01:13:10.199795+0000', 'last_unstale': '2024-09-11T01:13:43.964467+0000', 'last_undegraded': '2024-09-11T01:13:09.791198+0000', 'last_fullsized': '2024-09-11T01:13:09.652311+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3415", 'last_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'objects_scrubbed': 3337, 'log_size': 5203, 'log_dups_size': 0, 'ondisk_log_size': 5203, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1975, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21278720, 'num_objects': 5195, 'num_object_clones': 0, 'num_object_copies': 10390, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3942, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5195, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1858, 'num_write_kb': 7432, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1256}, {'shards': '0,2', 'objects': 3939}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:48.156 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:13:48.156 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'5268", 'reported_seq': 5397, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:43.940921+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:13:43.940921+0000', 'last_peered': '2024-09-11T01:13:43.940921+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:13:43.940921+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 5268, 'log_dups_size': 0, 'ondisk_log_size': 5268, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21561344, 'num_objects': 5264, 'num_object_clones': 0, 'num_object_copies': 10528, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5264, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1877, 'num_write_kb': 7508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1225}, {'shards': '0,2', 'objects': 4039}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:48.157 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:13:48.157 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'5250", 'reported_seq': 5378, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:44.007191+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:13:44.007191+0000', 'last_peered': '2024-09-11T01:13:44.007191+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:13:44.007191+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 5250, 'log_dups_size': 0, 'ondisk_log_size': 5250, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21455465, 'num_objects': 5239, 'num_object_clones': 0, 'num_object_copies': 10478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5239, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2320, 'num_write_kb': 9280, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1322}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:48.157 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:13:48.369 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 84 20 85924 85904 3.99347 2.11328 0.00823467 0.0195527 2024-09-11T01:13:49.204 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:13:49.205 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43711/168730 objects degraded (25.906%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":165,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":84365,"data_bytes":345547388,"bytes_used":1131024384,"bytes_avail":288779268096,"bytes_total":289910292480,"degraded_objects":43711,"degraded_total":168730,"degraded_ratio":0.25905885141942747,"misplaced_objects":3094,"misplaced_total":168730,"misplaced_ratio":0.018336988087477035,"recovering_objects_per_sec":27,"recovering_bytes_per_sec":112257,"recovering_keys_per_sec":0,"num_objects_recovered":329,"num_bytes_recovered":1347584,"num_keys_recovered":0,"write_bytes_sec":2303498,"read_op_per_sec":0,"write_op_per_sec":562},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-11T01:13:08.555583+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T01:13:49.249 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:13:49.250 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:13:49.370 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 85 20 86475 86455 3.97181 2.15234 0.0503912 0.0196617 2024-09-11T01:13:50.121 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:13:50.124 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:13:50.211 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2024-09-11T01:13:48.569773+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":345547388,"num_objects":84365,"num_object_clones":0,"num_object_copies":168730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":3094,"num_objects_unfound":0,"num_objects_dirty":84365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45296,"num_write_kb":181178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":833,"num_bytes_recovered":3407891,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":84447,"ondisk_log_size":84447,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1104516,"kb_used_data":680596,"kb_used_omap":12,"kb_used_meta":423859,"kb_avail":282011004,"statfs":{"total":289910292480,"available":288779268096,"internally_reserved":0,"allocated":696930304,"data_stored":696111386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":434031620},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,8,6,10,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27652096,"num_objects":6751,"num_object_clones":0,"num_object_copies":13502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-329,"num_objects_unfound":0,"num_objects_dirty":6751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6751,"num_write_kb":27004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":329,"num_bytes_recovered":1347584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004391"},"pg_stats":[{"pgid":"2.f","version":"43'5282","reported_seq":5354,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:43.996491+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:13:43.996491+0000","last_peered":"2024-09-11T01:13:43.996491+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:13:43.996491+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":5282,"log_dups_size":0,"ondisk_log_size":5282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21635072,"num_objects":5282,"num_object_clones":0,"num_object_copies":10564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5282,"num_write_kb":21128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1210},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5478","reported_seq":5574,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:48.233118+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:13:48.233118+0000","last_peered":"2024-09-11T01:13:48.233118+0000","last_clean":"2024-09-11T01:13:48.233118+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:13:48.233118+0000","last_undegraded":"2024-09-11T01:13:48.233118+0000","last_fullsized":"2024-09-11T01:13:48.233118+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":5478,"log_dups_size":0,"ondisk_log_size":5478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22413312,"num_objects":5472,"num_object_clones":0,"num_object_copies":10944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3717,"num_write_kb":14868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5277","reported_seq":5399,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:48.234003+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:13:48.234003+0000","last_peered":"2024-09-11T01:13:48.234003+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:13:48.234003+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":5277,"log_dups_size":0,"ondisk_log_size":5277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21598208,"num_objects":5273,"num_object_clones":0,"num_object_copies":10546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3578,"num_write_kb":14312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":1377}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5361","reported_seq":5425,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:48.248333+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:13:48.248333+0000","last_peered":"2024-09-11T01:13:48.248333+0000","last_clean":"2024-09-11T01:13:48.248333+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:13:48.248333+0000","last_undegraded":"2024-09-11T01:13:48.248333+0000","last_fullsized":"2024-09-11T01:13:48.248333+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":5361,"log_dups_size":0,"ondisk_log_size":5361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21958656,"num_objects":5361,"num_object_clones":0,"num_object_copies":10722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5361,"num_write_kb":21444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5225","reported_seq":5347,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:43.991632+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:13:43.991632+0000","last_peered":"2024-09-11T01:13:43.991632+0000","last_clean":"2024-09-11T01:13:43.991632+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:13:43.991632+0000","last_undegraded":"2024-09-11T01:13:43.991632+0000","last_fullsized":"2024-09-11T01:13:43.991632+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":5225,"log_dups_size":0,"ondisk_log_size":5225,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21372928,"num_objects":5218,"num_object_clones":0,"num_object_copies":10436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2230,"num_write_kb":8920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5170","reported_seq":5335,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:43.972045+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:13:43.972045+0000","last_peered":"2024-09-11T01:13:43.972045+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:13:43.972045+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":5170,"log_dups_size":0,"ondisk_log_size":5170,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21155840,"num_objects":5165,"num_object_clones":0,"num_object_copies":10330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1511,"num_write_kb":6044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1214},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5183","reported_seq":5308,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:44.007233+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:13:44.007233+0000","last_peered":"2024-09-11T01:13:44.007233+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:13:44.007233+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":5183,"log_dups_size":0,"ondisk_log_size":5183,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21204992,"num_objects":5177,"num_object_clones":0,"num_object_copies":10354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2040,"num_write_kb":8160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1191},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5286","reported_seq":5410,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:43.914467+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:13:43.914467+0000","last_peered":"2024-09-11T01:13:43.914467+0000","last_clean":"2024-09-11T01:13:43.914467+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:13:43.914467+0000","last_undegraded":"2024-09-11T01:13:43.914467+0000","last_fullsized":"2024-09-11T01:13:43.914467+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":5286,"log_dups_size":0,"ondisk_log_size":5286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21643264,"num_objects":5284,"num_object_clones":0,"num_object_copies":10568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5284,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2162,"num_write_kb":8648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5351","reported_seq":5473,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:48.183408+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:13:48.183408+0000","last_peered":"2024-09-11T01:13:48.183408+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:13:48.183408+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":5351,"log_dups_size":0,"ondisk_log_size":5351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21889024,"num_objects":5344,"num_object_clones":0,"num_object_copies":10688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3322,"num_write_kb":13288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":1411}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741858+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741858+0000","last_peered":"2024-09-11T01:13:09.741858+0000","last_clean":"2024-09-11T01:13:09.741858+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741858+0000","last_undegraded":"2024-09-11T01:13:09.741858+0000","last_fullsized":"2024-09-11T01:13:09.741858+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5186","reported_seq":5306,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:43.951483+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:13:43.951483+0000","last_peered":"2024-09-11T01:13:43.951483+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:13:43.951483+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":5186,"log_dups_size":0,"ondisk_log_size":5186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21209088,"num_objects":5178,"num_object_clones":0,"num_object_copies":10356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5178,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1773,"num_write_kb":7092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1194},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.837480+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.837480+0000","last_peered":"2024-09-11T01:13:09.837480+0000","last_clean":"2024-09-11T01:13:09.837480+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.837480+0000","last_undegraded":"2024-09-11T01:13:09.837480+0000","last_fullsized":"2024-09-11T01:13:09.837480+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5416","reported_seq":5529,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:48.137714+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:13:48.137714+0000","last_peered":"2024-09-11T01:13:48.137714+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:13:48.137714+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":5416,"log_dups_size":0,"ondisk_log_size":5416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22163456,"num_objects":5411,"num_object_clones":0,"num_object_copies":10822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3233,"num_write_kb":12932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":1387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5307","reported_seq":7083,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:13:48.216385+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:13:48.216385+0000","last_peered":"2024-09-11T01:13:48.216385+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:13:48.216385+0000","last_undegraded":"2024-09-11T01:13:48.216385+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":5307,"log_dups_size":0,"ondisk_log_size":5307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21712896,"num_objects":5301,"num_object_clones":0,"num_object_copies":10602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3094,"num_objects_unfound":0,"num_objects_dirty":5301,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3092,"num_write_kb":12368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":831,"num_bytes_recovered":3403776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3094},{"shards":"2","objects":2207}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5201","reported_seq":5327,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:43.966392+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:13:43.966392+0000","last_peered":"2024-09-11T01:13:43.966392+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:13:43.966392+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":5201,"log_dups_size":0,"ondisk_log_size":5201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21295104,"num_objects":5199,"num_object_clones":0,"num_object_copies":10398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1938,"num_write_kb":7752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1249},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5203","reported_seq":5334,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:43.964467+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:13:43.964467+0000","last_peered":"2024-09-11T01:13:43.964467+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:13:43.964467+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":5203,"log_dups_size":0,"ondisk_log_size":5203,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21278720,"num_objects":5195,"num_object_clones":0,"num_object_copies":10390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1858,"num_write_kb":7432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1256},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658057+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658057+0000","last_peered":"2024-09-11T01:13:09.658057+0000","last_clean":"2024-09-11T01:13:09.658057+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658057+0000","last_undegraded":"2024-09-11T01:13:09.658057+0000","last_fullsized":"2024-09-11T01:13:09.658057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5268","reported_seq":5397,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:43.940921+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:13:43.940921+0000","last_peered":"2024-09-11T01:13:43.940921+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:13:43.940921+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":5268,"log_dups_size":0,"ondisk_log_size":5268,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21561344,"num_objects":5264,"num_object_clones":0,"num_object_copies":10528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1877,"num_write_kb":7508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1225},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5250","reported_seq":5378,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:44.007191+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:13:44.007191+0000","last_peered":"2024-09-11T01:13:44.007191+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:13:44.007191+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":5250,"log_dups_size":0,"ondisk_log_size":5250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21455465,"num_objects":5239,"num_object_clones":0,"num_object_copies":10478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2320,"num_write_kb":9280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1322},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":345547369,"num_objects":84363,"num_object_clones":0,"num_object_copies":168726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":3094,"num_objects_unfound":0,"num_objects_dirty":84363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45294,"num_write_kb":181176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":832,"num_bytes_recovered":3407872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":697249792,"data_stored":697162817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":84444,"ondisk_log_size":84444,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182427,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485600,"kb_used_data":313744,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93886240,"statfs":{"total":96636764160,"available":96139509760,"internally_reserved":0,"allocated":321273856,"data_stored":321025037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":175959723},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,4,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986200,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465440,"kb_used_data":299152,"kb_used_omap":4,"kb_used_meta":166267,"kb_avail":93906400,"statfs":{"total":96636764160,"available":96160153600,"internally_reserved":0,"allocated":306331648,"data_stored":306086906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":170258090},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,0,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659145,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":153476,"kb_used_data":67700,"kb_used_omap":4,"kb_used_meta":85755,"kb_avail":94218364,"statfs":{"total":96636764160,"available":96479604736,"internally_reserved":0,"allocated":69324800,"data_stored":68999443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":87813807},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,2,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":321613824,"data_stored":321610345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69230592,"data_stored":69150575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":306405376,"data_stored":306401897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:13:50.370 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 86 20 87002 86982 3.94957 2.05859 0.0505957 0.0197728 2024-09-11T01:13:51.370 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 87 20 87577 87557 3.92999 2.24609 0.0207247 0.0198694 2024-09-11T01:13:52.370 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 88 20 88139 88119 3.91028 2.19531 0.0245828 0.0199727 2024-09-11T01:13:53.213 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:13:53.370 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 89 20 88720 88700 3.89185 2.26953 0.0162027 0.0200629 2024-09-11T01:13:54.060 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:13:54.060 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:13:54.109 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":121,"stamp":"2024-09-11T01:13:52.570696+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":352711292,"num_objects":86114,"num_object_clones":0,"num_object_copies":172228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":3094,"num_objects_unfound":0,"num_objects_dirty":86114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47045,"num_write_kb":188174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":833,"num_bytes_recovered":3407891,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86196,"ondisk_log_size":86196,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1138324,"kb_used_data":697764,"kb_used_omap":12,"kb_used_meta":440499,"kb_avail":281977196,"statfs":{"total":289910292480,"available":288744648704,"internally_reserved":0,"allocated":714510336,"data_stored":713691418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":451070980},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,5,3,6,10,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22835200,"num_objects":5575,"num_object_clones":0,"num_object_copies":11150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-222,"num_objects_unfound":0,"num_objects_dirty":5575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5575,"num_write_kb":22300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":909312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004453"},"pg_stats":[{"pgid":"2.f","version":"43'5450","reported_seq":5522,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:49.003081+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:13:49.003081+0000","last_peered":"2024-09-11T01:13:49.003081+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:13:49.003081+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":5450,"log_dups_size":0,"ondisk_log_size":5450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22323200,"num_objects":5450,"num_object_clones":0,"num_object_copies":10900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5450,"num_write_kb":21800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1378},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5478","reported_seq":5574,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:48.233118+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:13:48.233118+0000","last_peered":"2024-09-11T01:13:48.233118+0000","last_clean":"2024-09-11T01:13:48.233118+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:13:48.233118+0000","last_undegraded":"2024-09-11T01:13:48.233118+0000","last_fullsized":"2024-09-11T01:13:48.233118+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":5478,"log_dups_size":0,"ondisk_log_size":5478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22413312,"num_objects":5472,"num_object_clones":0,"num_object_copies":10944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3717,"num_write_kb":14868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5277","reported_seq":5399,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:48.234003+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:13:48.234003+0000","last_peered":"2024-09-11T01:13:48.234003+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:13:48.234003+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":5277,"log_dups_size":0,"ondisk_log_size":5277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21598208,"num_objects":5273,"num_object_clones":0,"num_object_copies":10546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3578,"num_write_kb":14312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":1377}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5361","reported_seq":5425,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:48.248333+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:13:48.248333+0000","last_peered":"2024-09-11T01:13:48.248333+0000","last_clean":"2024-09-11T01:13:48.248333+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:13:48.248333+0000","last_undegraded":"2024-09-11T01:13:48.248333+0000","last_fullsized":"2024-09-11T01:13:48.248333+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":5361,"log_dups_size":0,"ondisk_log_size":5361,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21958656,"num_objects":5361,"num_object_clones":0,"num_object_copies":10722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5361,"num_write_kb":21444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5406","reported_seq":5528,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:49.021802+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:13:49.021802+0000","last_peered":"2024-09-11T01:13:49.021802+0000","last_clean":"2024-09-11T01:13:49.021802+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:13:49.021802+0000","last_undegraded":"2024-09-11T01:13:49.021802+0000","last_fullsized":"2024-09-11T01:13:49.021802+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":5406,"log_dups_size":0,"ondisk_log_size":5406,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22114304,"num_objects":5399,"num_object_clones":0,"num_object_copies":10798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2411,"num_write_kb":9644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5344","reported_seq":5509,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:48.986619+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:13:48.986619+0000","last_peered":"2024-09-11T01:13:48.986619+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:13:48.986619+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":5344,"log_dups_size":0,"ondisk_log_size":5344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21868544,"num_objects":5339,"num_object_clones":0,"num_object_copies":10678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5339,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1685,"num_write_kb":6740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1388},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5354","reported_seq":5479,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:48.961862+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:13:48.961862+0000","last_peered":"2024-09-11T01:13:48.961862+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:13:48.961862+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":5354,"log_dups_size":0,"ondisk_log_size":5354,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21905408,"num_objects":5348,"num_object_clones":0,"num_object_copies":10696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2211,"num_write_kb":8844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1362},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5462","reported_seq":5586,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:48.944576+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:13:48.944576+0000","last_peered":"2024-09-11T01:13:48.944576+0000","last_clean":"2024-09-11T01:13:48.944576+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:13:48.944576+0000","last_undegraded":"2024-09-11T01:13:48.944576+0000","last_fullsized":"2024-09-11T01:13:48.944576+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":5462,"log_dups_size":0,"ondisk_log_size":5462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22364160,"num_objects":5460,"num_object_clones":0,"num_object_copies":10920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2338,"num_write_kb":9352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5351","reported_seq":5473,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:48.183408+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:13:48.183408+0000","last_peered":"2024-09-11T01:13:48.183408+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:13:48.183408+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":5351,"log_dups_size":0,"ondisk_log_size":5351,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21889024,"num_objects":5344,"num_object_clones":0,"num_object_copies":10688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3322,"num_write_kb":13288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":1411}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741858+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741858+0000","last_peered":"2024-09-11T01:13:09.741858+0000","last_clean":"2024-09-11T01:13:09.741858+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741858+0000","last_undegraded":"2024-09-11T01:13:09.741858+0000","last_fullsized":"2024-09-11T01:13:09.741858+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5366","reported_seq":5489,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:48.925776+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:13:48.925776+0000","last_peered":"2024-09-11T01:13:48.925776+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:13:48.925776+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":5366,"log_dups_size":0,"ondisk_log_size":5366,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21946368,"num_objects":5358,"num_object_clones":0,"num_object_copies":10716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1953,"num_write_kb":7812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1374},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.837480+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.837480+0000","last_peered":"2024-09-11T01:13:09.837480+0000","last_clean":"2024-09-11T01:13:09.837480+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.837480+0000","last_undegraded":"2024-09-11T01:13:09.837480+0000","last_fullsized":"2024-09-11T01:13:09.837480+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5416","reported_seq":5529,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:48.137714+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:13:48.137714+0000","last_peered":"2024-09-11T01:13:48.137714+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:13:48.137714+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":5416,"log_dups_size":0,"ondisk_log_size":5416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22163456,"num_objects":5411,"num_object_clones":0,"num_object_copies":10822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3233,"num_write_kb":12932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":1387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5307","reported_seq":7083,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:13:48.216385+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:13:48.216385+0000","last_peered":"2024-09-11T01:13:48.216385+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:13:48.216385+0000","last_undegraded":"2024-09-11T01:13:48.216385+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":5307,"log_dups_size":0,"ondisk_log_size":5307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21712896,"num_objects":5301,"num_object_clones":0,"num_object_copies":10602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3094,"num_objects_unfound":0,"num_objects_dirty":5301,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3092,"num_write_kb":12368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":831,"num_bytes_recovered":3403776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3094},{"shards":"2","objects":2207}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5379","reported_seq":5504,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:48.988036+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:13:48.988036+0000","last_peered":"2024-09-11T01:13:48.988036+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:13:48.988036+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":5379,"log_dups_size":0,"ondisk_log_size":5379,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22024192,"num_objects":5377,"num_object_clones":0,"num_object_copies":10754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2116,"num_write_kb":8464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1427},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5370","reported_seq":5501,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:48.996690+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:13:48.996690+0000","last_peered":"2024-09-11T01:13:48.996690+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:13:48.996690+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":5370,"log_dups_size":0,"ondisk_log_size":5370,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21962752,"num_objects":5362,"num_object_clones":0,"num_object_copies":10724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2025,"num_write_kb":8100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1423},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658057+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658057+0000","last_peered":"2024-09-11T01:13:09.658057+0000","last_clean":"2024-09-11T01:13:09.658057+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658057+0000","last_undegraded":"2024-09-11T01:13:09.658057+0000","last_fullsized":"2024-09-11T01:13:09.658057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5447","reported_seq":5576,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:48.978759+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:13:48.978759+0000","last_peered":"2024-09-11T01:13:48.978759+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:13:48.978759+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":5447,"log_dups_size":0,"ondisk_log_size":5447,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22294528,"num_objects":5443,"num_object_clones":0,"num_object_copies":10886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2056,"num_write_kb":8224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1404},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5425","reported_seq":5553,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:48.960373+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:13:48.960373+0000","last_peered":"2024-09-11T01:13:48.960373+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:13:48.960373+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":5425,"log_dups_size":0,"ondisk_log_size":5425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22172265,"num_objects":5414,"num_object_clones":0,"num_object_copies":10828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2495,"num_write_kb":9980,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1497},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":352711273,"num_objects":86112,"num_object_clones":0,"num_object_copies":172224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":3094,"num_objects_unfound":0,"num_objects_dirty":86112,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47043,"num_write_kb":188172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":832,"num_bytes_recovered":3407872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":714915840,"data_stored":714828865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":86193,"ondisk_log_size":86193,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182428,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":502408,"kb_used_data":322232,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93869432,"statfs":{"total":96636764160,"available":96122298368,"internally_reserved":0,"allocated":329965568,"data_stored":329716749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":184479403},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,4,1,2,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986200,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465440,"kb_used_data":299152,"kb_used_omap":4,"kb_used_meta":166267,"kb_avail":93906400,"statfs":{"total":96636764160,"available":96160153600,"internally_reserved":0,"allocated":306331648,"data_stored":306086906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":170258090},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,0,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659146,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":170476,"kb_used_data":76380,"kb_used_omap":4,"kb_used_meta":94075,"kb_avail":94201364,"statfs":{"total":96636764160,"available":96462196736,"internally_reserved":0,"allocated":78213120,"data_stored":77887763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":96333487},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":330248192,"data_stored":330244713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":78262272,"data_stored":78182255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":306405376,"data_stored":306401897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:13:54.111 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:13:54.370 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 90 20 89255 89235 3.87183 2.08984 0.0180437 0.0201704 2024-09-11T01:13:54.902 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:13:54.903 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:13:54.946 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":122,"stamp":"2024-09-11T01:13:54.571087+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":364192380,"num_objects":88917,"num_object_clones":0,"num_object_copies":177834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":2968,"num_objects_unfound":0,"num_objects_dirty":88917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49848,"num_write_kb":199386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":959,"num_bytes_recovered":3923987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88999,"ondisk_log_size":88999,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1152340,"kb_used_data":711780,"kb_used_omap":12,"kb_used_meta":440499,"kb_avail":281963180,"statfs":{"total":289910292480,"available":288730296320,"internally_reserved":0,"allocated":728862720,"data_stored":728043802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":451070980},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,6,10,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":34316288,"num_objects":8378,"num_object_clones":0,"num_object_copies":16756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-348,"num_objects_unfound":0,"num_objects_dirty":8378,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8378,"num_write_kb":33512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":348,"num_bytes_recovered":1425408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002783"},"pg_stats":[{"pgid":"2.f","version":"43'5617","reported_seq":5689,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:54.019169+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:13:54.019169+0000","last_peered":"2024-09-11T01:13:54.019169+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:13:54.019169+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":5617,"log_dups_size":0,"ondisk_log_size":5617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23007232,"num_objects":5617,"num_object_clones":0,"num_object_copies":11234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5617,"num_write_kb":22468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1545},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5655","reported_seq":5751,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:53.271106+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:13:53.271106+0000","last_peered":"2024-09-11T01:13:53.271106+0000","last_clean":"2024-09-11T01:13:53.271106+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:13:53.271106+0000","last_undegraded":"2024-09-11T01:13:53.271106+0000","last_fullsized":"2024-09-11T01:13:53.271106+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":5655,"log_dups_size":0,"ondisk_log_size":5655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23138304,"num_objects":5649,"num_object_clones":0,"num_object_copies":11298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3894,"num_write_kb":15576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5481","reported_seq":5600,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:53.192332+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:13:53.192332+0000","last_peered":"2024-09-11T01:13:53.192332+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:13:53.192332+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":5481,"log_dups_size":0,"ondisk_log_size":5481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22433792,"num_objects":5477,"num_object_clones":0,"num_object_copies":10954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3782,"num_write_kb":15128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":1581}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5545","reported_seq":5609,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:53.263294+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:13:53.263294+0000","last_peered":"2024-09-11T01:13:53.263294+0000","last_clean":"2024-09-11T01:13:53.263294+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:13:53.263294+0000","last_undegraded":"2024-09-11T01:13:53.263294+0000","last_fullsized":"2024-09-11T01:13:53.263294+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":5545,"log_dups_size":0,"ondisk_log_size":5545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22712320,"num_objects":5545,"num_object_clones":0,"num_object_copies":11090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5545,"num_write_kb":22180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5585","reported_seq":5707,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:53.999810+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:13:53.999810+0000","last_peered":"2024-09-11T01:13:53.999810+0000","last_clean":"2024-09-11T01:13:53.999810+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:13:53.999810+0000","last_undegraded":"2024-09-11T01:13:53.999810+0000","last_fullsized":"2024-09-11T01:13:53.999810+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":5585,"log_dups_size":0,"ondisk_log_size":5585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22847488,"num_objects":5578,"num_object_clones":0,"num_object_copies":11156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2590,"num_write_kb":10360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5529","reported_seq":5694,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:54.027786+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:13:54.027786+0000","last_peered":"2024-09-11T01:13:54.027786+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:13:54.027786+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":5529,"log_dups_size":0,"ondisk_log_size":5529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22626304,"num_objects":5524,"num_object_clones":0,"num_object_copies":11048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1870,"num_write_kb":7480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1573},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5507","reported_seq":5632,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:53.939733+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:13:53.939733+0000","last_peered":"2024-09-11T01:13:53.939733+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:13:53.939733+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":5507,"log_dups_size":0,"ondisk_log_size":5507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22532096,"num_objects":5501,"num_object_clones":0,"num_object_copies":11002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2364,"num_write_kb":9456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1515},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5645","reported_seq":5767,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:54.038666+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:13:54.038666+0000","last_peered":"2024-09-11T01:13:54.038666+0000","last_clean":"2024-09-11T01:13:54.038666+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:13:54.038666+0000","last_undegraded":"2024-09-11T01:13:54.038666+0000","last_fullsized":"2024-09-11T01:13:54.038666+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":5645,"log_dups_size":0,"ondisk_log_size":5645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23113728,"num_objects":5643,"num_object_clones":0,"num_object_copies":11286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2521,"num_write_kb":10084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5532","reported_seq":5654,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:53.243155+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:13:53.243155+0000","last_peered":"2024-09-11T01:13:53.243155+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:13:53.243155+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":5532,"log_dups_size":0,"ondisk_log_size":5532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22630400,"num_objects":5525,"num_object_clones":0,"num_object_copies":11050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3503,"num_write_kb":14012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":1592}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741858+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741858+0000","last_peered":"2024-09-11T01:13:09.741858+0000","last_clean":"2024-09-11T01:13:09.741858+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741858+0000","last_undegraded":"2024-09-11T01:13:09.741858+0000","last_fullsized":"2024-09-11T01:13:09.741858+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5558","reported_seq":5681,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:54.018059+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:13:54.018059+0000","last_peered":"2024-09-11T01:13:54.018059+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:13:54.018059+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":5558,"log_dups_size":0,"ondisk_log_size":5558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22732800,"num_objects":5550,"num_object_clones":0,"num_object_copies":11100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5550,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2145,"num_write_kb":8580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1566},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.837480+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.837480+0000","last_peered":"2024-09-11T01:13:09.837480+0000","last_clean":"2024-09-11T01:13:09.837480+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.837480+0000","last_undegraded":"2024-09-11T01:13:09.837480+0000","last_fullsized":"2024-09-11T01:13:09.837480+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5607","reported_seq":5720,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:53.255943+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:13:53.255943+0000","last_peered":"2024-09-11T01:13:53.255943+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:13:53.255943+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":5607,"log_dups_size":0,"ondisk_log_size":5607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22945792,"num_objects":5602,"num_object_clones":0,"num_object_copies":11204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3424,"num_write_kb":13696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":1578}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5450","reported_seq":7478,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:13:53.273518+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:13:53.273518+0000","last_peered":"2024-09-11T01:13:53.273518+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:13:53.273518+0000","last_undegraded":"2024-09-11T01:13:53.273518+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":5450,"log_dups_size":0,"ondisk_log_size":5450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22298624,"num_objects":5444,"num_object_clones":0,"num_object_copies":10888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2968,"num_objects_unfound":0,"num_objects_dirty":5444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3235,"num_write_kb":12940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":957,"num_bytes_recovered":3919872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2968},{"shards":"2","objects":2476}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5543","reported_seq":5669,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:54.008635+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:13:54.008635+0000","last_peered":"2024-09-11T01:13:54.008635+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:13:54.008635+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":5543,"log_dups_size":0,"ondisk_log_size":5543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22695936,"num_objects":5541,"num_object_clones":0,"num_object_copies":11082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2280,"num_write_kb":9120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1591},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5525","reported_seq":5656,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:53.999671+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:13:53.999671+0000","last_peered":"2024-09-11T01:13:53.999671+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:13:53.999671+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":5525,"log_dups_size":0,"ondisk_log_size":5525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22597632,"num_objects":5517,"num_object_clones":0,"num_object_copies":11034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2180,"num_write_kb":8720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1578},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658057+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658057+0000","last_peered":"2024-09-11T01:13:09.658057+0000","last_clean":"2024-09-11T01:13:09.658057+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658057+0000","last_undegraded":"2024-09-11T01:13:09.658057+0000","last_fullsized":"2024-09-11T01:13:09.658057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5642","reported_seq":5768,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:53.967038+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:13:53.967038+0000","last_peered":"2024-09-11T01:13:53.967038+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:13:53.967038+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":5642,"log_dups_size":0,"ondisk_log_size":5642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23093248,"num_objects":5638,"num_object_clones":0,"num_object_copies":11276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2251,"num_write_kb":9004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1599},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5575","reported_seq":5703,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:53.942241+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:13:53.942241+0000","last_peered":"2024-09-11T01:13:53.942241+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:13:53.942241+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":5575,"log_dups_size":0,"ondisk_log_size":5575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22786665,"num_objects":5564,"num_object_clones":0,"num_object_copies":11128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5564,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2645,"num_write_kb":10580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1647},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":364192361,"num_objects":88915,"num_object_clones":0,"num_object_copies":177830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":2968,"num_objects_unfound":0,"num_objects_dirty":88915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49846,"num_write_kb":199384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":958,"num_bytes_recovered":3923968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":729354240,"data_stored":729267265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88996,"ondisk_log_size":88996,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182429,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519052,"kb_used_data":330556,"kb_used_omap":4,"kb_used_meta":188475,"kb_avail":93852788,"statfs":{"total":96636764160,"available":96105254912,"internally_reserved":0,"allocated":338489344,"data_stored":338240525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":192999083},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986201,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":462812,"kb_used_data":304844,"kb_used_omap":4,"kb_used_meta":157947,"kb_avail":93909028,"statfs":{"total":96636764160,"available":96162844672,"internally_reserved":0,"allocated":312160256,"data_stored":311915514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":161738410},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659146,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":170476,"kb_used_data":76380,"kb_used_omap":4,"kb_used_meta":94075,"kb_avail":94201364,"statfs":{"total":96636764160,"available":96462196736,"internally_reserved":0,"allocated":78213120,"data_stored":77887763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":96333487},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":338767872,"data_stored":338764393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":78262272,"data_stored":78182255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":312324096,"data_stored":312320617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:13:54.947 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:13:54.947 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'5617", 'reported_seq': 5689, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:54.019169+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:13:54.019169+0000', 'last_peered': '2024-09-11T01:13:54.019169+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:13:54.019169+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 5617, 'log_dups_size': 0, 'ondisk_log_size': 5617, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23007232, 'num_objects': 5617, 'num_object_clones': 0, 'num_object_copies': 11234, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5617, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5617, 'num_write_kb': 22468, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1545}, {'shards': '0,2', 'objects': 4072}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:54.947 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:13:54.947 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'5481", 'reported_seq': 5600, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:53.192332+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:13:53.192332+0000', 'last_peered': '2024-09-11T01:13:53.192332+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:13:53.192332+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 5481, 'log_dups_size': 0, 'ondisk_log_size': 5481, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22433792, 'num_objects': 5477, 'num_object_clones': 0, 'num_object_copies': 10954, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5477, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3782, 'num_write_kb': 15128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3896}, {'shards': '2', 'objects': 1581}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:13:54.947 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:13:54.948 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'5529", 'reported_seq': 5694, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:54.027786+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:13:54.027786+0000', 'last_peered': '2024-09-11T01:13:54.027786+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:13:54.027786+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 5529, 'log_dups_size': 0, 'ondisk_log_size': 5529, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22626304, 'num_objects': 5524, 'num_object_clones': 0, 'num_object_copies': 11048, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5524, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 1870, 'num_write_kb': 7480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1573}, {'shards': '0,2', 'objects': 3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:54.948 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:13:54.948 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'5507", 'reported_seq': 5632, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:53.939733+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:13:53.939733+0000', 'last_peered': '2024-09-11T01:13:53.939733+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:13:53.939733+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 5507, 'log_dups_size': 0, 'ondisk_log_size': 5507, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22532096, 'num_objects': 5501, 'num_object_clones': 0, 'num_object_copies': 11002, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5501, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2364, 'num_write_kb': 9456, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1515}, {'shards': '0,2', 'objects': 3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:54.948 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:13:54.948 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'5532", 'reported_seq': 5654, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:53.243155+0000', 'last_change': '2024-09-11T01:13:10.159069+0000', 'last_active': '2024-09-11T01:13:53.243155+0000', 'last_peered': '2024-09-11T01:13:53.243155+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:13:53.243155+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 5532, 'log_dups_size': 0, 'ondisk_log_size': 5532, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22630400, 'num_objects': 5525, 'num_object_clones': 0, 'num_object_copies': 11050, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5525, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3503, 'num_write_kb': 14012, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3932}, {'shards': '2', 'objects': 1592}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:13:54.948 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:13:54.948 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'5558", 'reported_seq': 5681, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:54.018059+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:13:54.018059+0000', 'last_peered': '2024-09-11T01:13:54.018059+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:13:54.018059+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 5558, 'log_dups_size': 0, 'ondisk_log_size': 5558, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22732800, 'num_objects': 5550, 'num_object_clones': 0, 'num_object_copies': 11100, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5550, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2145, 'num_write_kb': 8580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1566}, {'shards': '0,2', 'objects': 3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:54.948 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:13:54.948 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'5607", 'reported_seq': 5720, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:53.255943+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:13:53.255943+0000', 'last_peered': '2024-09-11T01:13:53.255943+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:13:53.255943+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 5607, 'log_dups_size': 0, 'ondisk_log_size': 5607, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22945792, 'num_objects': 5602, 'num_object_clones': 0, 'num_object_copies': 11204, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5602, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3424, 'num_write_kb': 13696, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4023}, {'shards': '2', 'objects': 1578}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:13:54.948 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:13:54.948 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'5450", 'reported_seq': 7478, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:13:53.273518+0000', 'last_change': '2024-09-11T01:13:10.217540+0000', 'last_active': '2024-09-11T01:13:53.273518+0000', 'last_peered': '2024-09-11T01:13:53.273518+0000', 'last_clean': '2024-09-11T01:13:07.638709+0000', 'last_became_active': '2024-09-11T01:13:10.125604+0000', 'last_became_peered': '2024-09-11T01:13:10.125604+0000', 'last_unstale': '2024-09-11T01:13:53.273518+0000', 'last_undegraded': '2024-09-11T01:13:53.273518+0000', 'last_fullsized': '2024-09-11T01:13:09.649604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2269", 'last_scrub_stamp': '2024-09-11T01:12:42.890920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:42.890920+0000', 'objects_scrubbed': 2209, 'log_size': 5450, 'log_dups_size': 0, 'ondisk_log_size': 5450, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 999, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22298624, 'num_objects': 5444, 'num_object_clones': 0, 'num_object_copies': 10888, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2968, 'num_objects_unfound': 0, 'num_objects_dirty': 5444, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3235, 'num_write_kb': 12940, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 957, 'num_bytes_recovered': 3919872, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2968}, {'shards': '2', 'objects': 2476}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:13:54.949 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:13:54.949 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'5543", 'reported_seq': 5669, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:54.008635+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:13:54.008635+0000', 'last_peered': '2024-09-11T01:13:54.008635+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:13:54.008635+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 5543, 'log_dups_size': 0, 'ondisk_log_size': 5543, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22695936, 'num_objects': 5541, 'num_object_clones': 0, 'num_object_copies': 11082, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5541, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2280, 'num_write_kb': 9120, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1591}, {'shards': '0,2', 'objects': 3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:54.949 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:13:54.949 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'5525", 'reported_seq': 5656, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:53.999671+0000', 'last_change': '2024-09-11T01:13:10.209668+0000', 'last_active': '2024-09-11T01:13:53.999671+0000', 'last_peered': '2024-09-11T01:13:53.999671+0000', 'last_clean': '2024-09-11T01:13:07.624565+0000', 'last_became_active': '2024-09-11T01:13:10.199795+0000', 'last_became_peered': '2024-09-11T01:13:10.199795+0000', 'last_unstale': '2024-09-11T01:13:53.999671+0000', 'last_undegraded': '2024-09-11T01:13:09.791198+0000', 'last_fullsized': '2024-09-11T01:13:09.652311+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3415", 'last_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'objects_scrubbed': 3337, 'log_size': 5525, 'log_dups_size': 0, 'ondisk_log_size': 5525, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1975, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22597632, 'num_objects': 5517, 'num_object_clones': 0, 'num_object_copies': 11034, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3942, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5517, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2180, 'num_write_kb': 8720, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1578}, {'shards': '0,2', 'objects': 3939}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:54.949 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:13:54.949 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'5642", 'reported_seq': 5768, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:53.967038+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:13:53.967038+0000', 'last_peered': '2024-09-11T01:13:53.967038+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:13:53.967038+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 5642, 'log_dups_size': 0, 'ondisk_log_size': 5642, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23093248, 'num_objects': 5638, 'num_object_clones': 0, 'num_object_copies': 11276, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5638, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2251, 'num_write_kb': 9004, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1599}, {'shards': '0,2', 'objects': 4039}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:54.949 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:13:54.949 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'5575", 'reported_seq': 5703, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:53.942241+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:13:53.942241+0000', 'last_peered': '2024-09-11T01:13:53.942241+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:13:53.942241+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 5575, 'log_dups_size': 0, 'ondisk_log_size': 5575, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22786665, 'num_objects': 5564, 'num_object_clones': 0, 'num_object_copies': 11128, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5564, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2645, 'num_write_kb': 10580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1647}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:13:54.950 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:13:55.370 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 91 20 89776 89756 3.85165 2.03516 0.0544332 0.0202718 2024-09-11T01:13:55.986 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:13:55.986 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43711/177834 objects degraded (24.580%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":172,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":88917,"data_bytes":364192380,"bytes_used":1179996160,"bytes_avail":288730296320,"bytes_total":289910292480,"degraded_objects":43711,"degraded_total":177834,"degraded_ratio":0.24579664181202696,"misplaced_objects":2968,"misplaced_total":177834,"misplaced_ratio":0.016689721875456885,"recovering_objects_per_sec":28,"recovering_bytes_per_sec":118756,"recovering_keys_per_sec":0,"num_objects_recovered":348,"num_bytes_recovered":1425408,"num_keys_recovered":0,"write_bytes_sec":2859027,"read_op_per_sec":0,"write_op_per_sec":698},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-11T01:13:08.555583+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T01:13:56.060 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:13:56.060 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:13:56.370 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 92 20 90322 90302 3.83297 2.13281 0.00761733 0.0203729 2024-09-11T01:13:56.948 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:13:56.948 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:13:57.029 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2024-09-11T01:13:56.571584+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":364192380,"num_objects":88917,"num_object_clones":0,"num_object_copies":177834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":2968,"num_objects_unfound":0,"num_objects_dirty":88917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49848,"num_write_kb":199386,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":959,"num_bytes_recovered":3923987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88999,"ondisk_log_size":88999,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1152340,"kb_used_data":711780,"kb_used_omap":12,"kb_used_meta":440499,"kb_avail":281963180,"statfs":{"total":289910292480,"available":288730296320,"internally_reserved":0,"allocated":728862720,"data_stored":728043802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":451070980},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,6,10,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22847488,"num_objects":5578,"num_object_clones":0,"num_object_copies":11156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-236,"num_objects_unfound":0,"num_objects_dirty":5578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5578,"num_write_kb":22312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":236,"num_bytes_recovered":966656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002828"},"pg_stats":[{"pgid":"2.f","version":"43'5617","reported_seq":5689,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:54.019169+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:13:54.019169+0000","last_peered":"2024-09-11T01:13:54.019169+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:13:54.019169+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":5617,"log_dups_size":0,"ondisk_log_size":5617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23007232,"num_objects":5617,"num_object_clones":0,"num_object_copies":11234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5617,"num_write_kb":22468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1545},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5655","reported_seq":5751,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:53.271106+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:13:53.271106+0000","last_peered":"2024-09-11T01:13:53.271106+0000","last_clean":"2024-09-11T01:13:53.271106+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:13:53.271106+0000","last_undegraded":"2024-09-11T01:13:53.271106+0000","last_fullsized":"2024-09-11T01:13:53.271106+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":5655,"log_dups_size":0,"ondisk_log_size":5655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23138304,"num_objects":5649,"num_object_clones":0,"num_object_copies":11298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3894,"num_write_kb":15576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5481","reported_seq":5600,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:53.192332+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:13:53.192332+0000","last_peered":"2024-09-11T01:13:53.192332+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:13:53.192332+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":5481,"log_dups_size":0,"ondisk_log_size":5481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22433792,"num_objects":5477,"num_object_clones":0,"num_object_copies":10954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3782,"num_write_kb":15128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":1581}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5545","reported_seq":5609,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:53.263294+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:13:53.263294+0000","last_peered":"2024-09-11T01:13:53.263294+0000","last_clean":"2024-09-11T01:13:53.263294+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:13:53.263294+0000","last_undegraded":"2024-09-11T01:13:53.263294+0000","last_fullsized":"2024-09-11T01:13:53.263294+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":5545,"log_dups_size":0,"ondisk_log_size":5545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22712320,"num_objects":5545,"num_object_clones":0,"num_object_copies":11090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5545,"num_write_kb":22180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5585","reported_seq":5707,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:53.999810+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:13:53.999810+0000","last_peered":"2024-09-11T01:13:53.999810+0000","last_clean":"2024-09-11T01:13:53.999810+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:13:53.999810+0000","last_undegraded":"2024-09-11T01:13:53.999810+0000","last_fullsized":"2024-09-11T01:13:53.999810+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":5585,"log_dups_size":0,"ondisk_log_size":5585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22847488,"num_objects":5578,"num_object_clones":0,"num_object_copies":11156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5578,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2590,"num_write_kb":10360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5529","reported_seq":5694,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:54.027786+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:13:54.027786+0000","last_peered":"2024-09-11T01:13:54.027786+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:13:54.027786+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":5529,"log_dups_size":0,"ondisk_log_size":5529,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22626304,"num_objects":5524,"num_object_clones":0,"num_object_copies":11048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1870,"num_write_kb":7480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1573},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5507","reported_seq":5632,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:53.939733+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:13:53.939733+0000","last_peered":"2024-09-11T01:13:53.939733+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:13:53.939733+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":5507,"log_dups_size":0,"ondisk_log_size":5507,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22532096,"num_objects":5501,"num_object_clones":0,"num_object_copies":11002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2364,"num_write_kb":9456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1515},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5645","reported_seq":5767,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:54.038666+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:13:54.038666+0000","last_peered":"2024-09-11T01:13:54.038666+0000","last_clean":"2024-09-11T01:13:54.038666+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:13:54.038666+0000","last_undegraded":"2024-09-11T01:13:54.038666+0000","last_fullsized":"2024-09-11T01:13:54.038666+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":5645,"log_dups_size":0,"ondisk_log_size":5645,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23113728,"num_objects":5643,"num_object_clones":0,"num_object_copies":11286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5643,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2521,"num_write_kb":10084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5532","reported_seq":5654,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:53.243155+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:13:53.243155+0000","last_peered":"2024-09-11T01:13:53.243155+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:13:53.243155+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":5532,"log_dups_size":0,"ondisk_log_size":5532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22630400,"num_objects":5525,"num_object_clones":0,"num_object_copies":11050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3503,"num_write_kb":14012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":1592}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741858+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741858+0000","last_peered":"2024-09-11T01:13:09.741858+0000","last_clean":"2024-09-11T01:13:09.741858+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741858+0000","last_undegraded":"2024-09-11T01:13:09.741858+0000","last_fullsized":"2024-09-11T01:13:09.741858+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5558","reported_seq":5681,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:54.018059+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:13:54.018059+0000","last_peered":"2024-09-11T01:13:54.018059+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:13:54.018059+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":5558,"log_dups_size":0,"ondisk_log_size":5558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22732800,"num_objects":5550,"num_object_clones":0,"num_object_copies":11100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5550,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2145,"num_write_kb":8580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1566},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":95,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.837480+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.837480+0000","last_peered":"2024-09-11T01:13:09.837480+0000","last_clean":"2024-09-11T01:13:09.837480+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.837480+0000","last_undegraded":"2024-09-11T01:13:09.837480+0000","last_fullsized":"2024-09-11T01:13:09.837480+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5607","reported_seq":5720,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:53.255943+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:13:53.255943+0000","last_peered":"2024-09-11T01:13:53.255943+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:13:53.255943+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":5607,"log_dups_size":0,"ondisk_log_size":5607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22945792,"num_objects":5602,"num_object_clones":0,"num_object_copies":11204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3424,"num_write_kb":13696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":1578}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5450","reported_seq":7478,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:13:53.273518+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:13:53.273518+0000","last_peered":"2024-09-11T01:13:53.273518+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:13:53.273518+0000","last_undegraded":"2024-09-11T01:13:53.273518+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":5450,"log_dups_size":0,"ondisk_log_size":5450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22298624,"num_objects":5444,"num_object_clones":0,"num_object_copies":10888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2968,"num_objects_unfound":0,"num_objects_dirty":5444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3235,"num_write_kb":12940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":957,"num_bytes_recovered":3919872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2968},{"shards":"2","objects":2476}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5543","reported_seq":5669,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:54.008635+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:13:54.008635+0000","last_peered":"2024-09-11T01:13:54.008635+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:13:54.008635+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":5543,"log_dups_size":0,"ondisk_log_size":5543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22695936,"num_objects":5541,"num_object_clones":0,"num_object_copies":11082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2280,"num_write_kb":9120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1591},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5525","reported_seq":5656,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:53.999671+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:13:53.999671+0000","last_peered":"2024-09-11T01:13:53.999671+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:13:53.999671+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":5525,"log_dups_size":0,"ondisk_log_size":5525,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22597632,"num_objects":5517,"num_object_clones":0,"num_object_copies":11034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5517,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2180,"num_write_kb":8720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1578},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658057+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658057+0000","last_peered":"2024-09-11T01:13:09.658057+0000","last_clean":"2024-09-11T01:13:09.658057+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658057+0000","last_undegraded":"2024-09-11T01:13:09.658057+0000","last_fullsized":"2024-09-11T01:13:09.658057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5642","reported_seq":5768,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:53.967038+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:13:53.967038+0000","last_peered":"2024-09-11T01:13:53.967038+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:13:53.967038+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":5642,"log_dups_size":0,"ondisk_log_size":5642,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23093248,"num_objects":5638,"num_object_clones":0,"num_object_copies":11276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2251,"num_write_kb":9004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1599},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5575","reported_seq":5703,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:53.942241+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:13:53.942241+0000","last_peered":"2024-09-11T01:13:53.942241+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:13:53.942241+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":5575,"log_dups_size":0,"ondisk_log_size":5575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22786665,"num_objects":5564,"num_object_clones":0,"num_object_copies":11128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5564,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2645,"num_write_kb":10580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1647},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":364192361,"num_objects":88915,"num_object_clones":0,"num_object_copies":177830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":2968,"num_objects_unfound":0,"num_objects_dirty":88915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49846,"num_write_kb":199384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":958,"num_bytes_recovered":3923968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":729354240,"data_stored":729267265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88996,"ondisk_log_size":88996,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182429,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":519052,"kb_used_data":330556,"kb_used_omap":4,"kb_used_meta":188475,"kb_avail":93852788,"statfs":{"total":96636764160,"available":96105254912,"internally_reserved":0,"allocated":338489344,"data_stored":338240525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":192999083},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986201,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":462812,"kb_used_data":304844,"kb_used_omap":4,"kb_used_meta":157947,"kb_avail":93909028,"statfs":{"total":96636764160,"available":96162844672,"internally_reserved":0,"allocated":312160256,"data_stored":311915514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":161738410},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,4],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659146,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":170476,"kb_used_data":76380,"kb_used_omap":4,"kb_used_meta":94075,"kb_avail":94201364,"statfs":{"total":96636764160,"available":96462196736,"internally_reserved":0,"allocated":78213120,"data_stored":77887763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":96333487},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":338767872,"data_stored":338764393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":78262272,"data_stored":78182255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":312324096,"data_stored":312320617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:13:57.370 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 93 20 90855 90835 3.81414 2.08203 0.0230098 0.0204741 2024-09-11T01:13:58.371 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 94 20 91436 91416 3.79771 2.26953 0.0210647 0.0205631 2024-09-11T01:13:59.370 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 95 20 92022 92002 3.78183 2.28906 0.0153666 0.0206488 2024-09-11T01:14:00.031 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:14:00.371 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 96 20 92580 92560 3.76511 2.17969 0.0389296 0.0207408 2024-09-11T01:14:00.873 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:14:00.876 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:14:00.932 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2024-09-11T01:14:00.572365+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":375485052,"num_objects":91674,"num_object_clones":0,"num_object_copies":183348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":2854,"num_objects_unfound":0,"num_objects_dirty":91674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":52605,"num_write_kb":210414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1073,"num_bytes_recovered":4390931,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91756,"ondisk_log_size":91756,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1175920,"kb_used_data":733952,"kb_used_omap":12,"kb_used_meta":441907,"kb_avail":281939600,"statfs":{"total":289910292480,"available":288706150400,"internally_reserved":0,"allocated":751566848,"data_stored":750747930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":452512772},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,4,5,14,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29937664,"num_objects":7309,"num_object_clones":0,"num_object_copies":14618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-240,"num_objects_unfound":0,"num_objects_dirty":7309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7309,"num_write_kb":29236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":983040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002591"},"pg_stats":[{"pgid":"2.f","version":"43'5784","reported_seq":5856,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:59.049443+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:13:59.049443+0000","last_peered":"2024-09-11T01:13:59.049443+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:13:59.049443+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":5784,"log_dups_size":0,"ondisk_log_size":5784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23691264,"num_objects":5784,"num_object_clones":0,"num_object_copies":11568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5784,"num_write_kb":23136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1712},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5834","reported_seq":5928,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:58.283508+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:13:58.283508+0000","last_peered":"2024-09-11T01:13:58.283508+0000","last_clean":"2024-09-11T01:13:58.283508+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:13:58.283508+0000","last_undegraded":"2024-09-11T01:13:58.283508+0000","last_fullsized":"2024-09-11T01:13:58.283508+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":5834,"log_dups_size":0,"ondisk_log_size":5834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23871488,"num_objects":5828,"num_object_clones":0,"num_object_copies":11656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4073,"num_write_kb":16292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5664","reported_seq":5786,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:58.291592+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:13:58.291592+0000","last_peered":"2024-09-11T01:13:58.291592+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:13:58.291592+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":5664,"log_dups_size":0,"ondisk_log_size":5664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23183360,"num_objects":5660,"num_object_clones":0,"num_object_copies":11320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5660,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3965,"num_write_kb":15860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":1764}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5718","reported_seq":5781,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:58.279692+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:13:58.279692+0000","last_peered":"2024-09-11T01:13:58.279692+0000","last_clean":"2024-09-11T01:13:58.279692+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:13:58.279692+0000","last_undegraded":"2024-09-11T01:13:58.279692+0000","last_fullsized":"2024-09-11T01:13:58.279692+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":5718,"log_dups_size":0,"ondisk_log_size":5718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23420928,"num_objects":5718,"num_object_clones":0,"num_object_copies":11436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5718,"num_write_kb":22872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5763","reported_seq":5885,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:58.993007+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:13:58.993007+0000","last_peered":"2024-09-11T01:13:58.993007+0000","last_clean":"2024-09-11T01:13:58.993007+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:13:58.993007+0000","last_undegraded":"2024-09-11T01:13:58.993007+0000","last_fullsized":"2024-09-11T01:13:58.993007+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":5763,"log_dups_size":0,"ondisk_log_size":5763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23576576,"num_objects":5756,"num_object_clones":0,"num_object_copies":11512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2768,"num_write_kb":11072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5722","reported_seq":5887,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:58.994374+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:13:58.994374+0000","last_peered":"2024-09-11T01:13:58.994374+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:13:58.994374+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":5722,"log_dups_size":0,"ondisk_log_size":5722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23416832,"num_objects":5717,"num_object_clones":0,"num_object_copies":11434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2063,"num_write_kb":8252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1766},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5661","reported_seq":5783,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:59.036314+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:13:59.036314+0000","last_peered":"2024-09-11T01:13:59.036314+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:13:59.036314+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":5661,"log_dups_size":0,"ondisk_log_size":5661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23162880,"num_objects":5655,"num_object_clones":0,"num_object_copies":11310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2518,"num_write_kb":10072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1669},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5841","reported_seq":5965,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:59.027280+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:13:59.027280+0000","last_peered":"2024-09-11T01:13:59.027280+0000","last_clean":"2024-09-11T01:13:59.027280+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:13:59.027280+0000","last_undegraded":"2024-09-11T01:13:59.027280+0000","last_fullsized":"2024-09-11T01:13:59.027280+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":5841,"log_dups_size":0,"ondisk_log_size":5841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23916544,"num_objects":5839,"num_object_clones":0,"num_object_copies":11678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2717,"num_write_kb":10868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5715","reported_seq":5837,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:58.298085+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:13:58.298085+0000","last_peered":"2024-09-11T01:13:58.298085+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:13:58.298085+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":5715,"log_dups_size":0,"ondisk_log_size":5715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23379968,"num_objects":5708,"num_object_clones":0,"num_object_copies":11416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3686,"num_write_kb":14744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":1775}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741858+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741858+0000","last_peered":"2024-09-11T01:13:09.741858+0000","last_clean":"2024-09-11T01:13:09.741858+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741858+0000","last_undegraded":"2024-09-11T01:13:09.741858+0000","last_fullsized":"2024-09-11T01:13:09.741858+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5708","reported_seq":5831,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:59.028987+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:13:59.028987+0000","last_peered":"2024-09-11T01:13:59.028987+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:13:59.028987+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":5708,"log_dups_size":0,"ondisk_log_size":5708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23347200,"num_objects":5700,"num_object_clones":0,"num_object_copies":11400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2295,"num_write_kb":9180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1716},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.910012+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.910012+0000","last_peered":"2024-09-11T01:13:09.910012+0000","last_clean":"2024-09-11T01:13:09.910012+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.910012+0000","last_undegraded":"2024-09-11T01:13:09.910012+0000","last_fullsized":"2024-09-11T01:13:09.910012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5770","reported_seq":5883,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:58.193005+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:13:58.193005+0000","last_peered":"2024-09-11T01:13:58.193005+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:13:58.193005+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":5770,"log_dups_size":0,"ondisk_log_size":5770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23613440,"num_objects":5765,"num_object_clones":0,"num_object_copies":11530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5765,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3587,"num_write_kb":14348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":1741}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5608","reported_seq":7864,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:13:58.293212+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:13:58.293212+0000","last_peered":"2024-09-11T01:13:58.293212+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:13:58.293212+0000","last_undegraded":"2024-09-11T01:13:58.293212+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":5608,"log_dups_size":0,"ondisk_log_size":5608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22945792,"num_objects":5602,"num_object_clones":0,"num_object_copies":11204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2854,"num_objects_unfound":0,"num_objects_dirty":5602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3393,"num_write_kb":13572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1071,"num_bytes_recovered":4386816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2854},{"shards":"2","objects":2748}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5717","reported_seq":5843,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:59.029795+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:13:59.029795+0000","last_peered":"2024-09-11T01:13:59.029795+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:13:59.029795+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":5717,"log_dups_size":0,"ondisk_log_size":5717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23408640,"num_objects":5715,"num_object_clones":0,"num_object_copies":11430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2454,"num_write_kb":9816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1765},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5697","reported_seq":5828,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:59.042220+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:13:59.042220+0000","last_peered":"2024-09-11T01:13:59.042220+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:13:59.042220+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":5697,"log_dups_size":0,"ondisk_log_size":5697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23302144,"num_objects":5689,"num_object_clones":0,"num_object_copies":11378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5689,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2352,"num_write_kb":9408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1750},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658057+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658057+0000","last_peered":"2024-09-11T01:13:09.658057+0000","last_clean":"2024-09-11T01:13:09.658057+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658057+0000","last_undegraded":"2024-09-11T01:13:09.658057+0000","last_fullsized":"2024-09-11T01:13:09.658057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5815","reported_seq":5944,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:58.988009+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:13:58.988009+0000","last_peered":"2024-09-11T01:13:58.988009+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:13:58.988009+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":5815,"log_dups_size":0,"ondisk_log_size":5815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23801856,"num_objects":5811,"num_object_clones":0,"num_object_copies":11622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2424,"num_write_kb":9696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1772},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5736","reported_seq":5864,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:58.996230+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:13:58.996230+0000","last_peered":"2024-09-11T01:13:58.996230+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:13:58.996230+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":5736,"log_dups_size":0,"ondisk_log_size":5736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23446121,"num_objects":5725,"num_object_clones":0,"num_object_copies":11450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2806,"num_write_kb":11224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1808},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":375485033,"num_objects":91672,"num_object_clones":0,"num_object_copies":183344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":2854,"num_objects_unfound":0,"num_objects_dirty":91672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":52603,"num_write_kb":210412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1072,"num_bytes_recovered":4390912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":752345088,"data_stored":752258113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91753,"ondisk_log_size":91753,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182430,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":514712,"kb_used_data":338696,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93857128,"statfs":{"total":96636764160,"available":96109699072,"internally_reserved":0,"allocated":346824704,"data_stored":346575885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":180219563},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2,2,7,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986202,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":472652,"kb_used_data":310524,"kb_used_omap":4,"kb_used_meta":162107,"kb_avail":93899188,"statfs":{"total":96636764160,"available":96152768512,"internally_reserved":0,"allocated":317976576,"data_stored":317731834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":165998250},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,0,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659147,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":188556,"kb_used_data":84732,"kb_used_omap":4,"kb_used_meta":103803,"kb_avail":94183284,"statfs":{"total":96636764160,"available":96443682816,"internally_reserved":0,"allocated":86765568,"data_stored":86440211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":106294959},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,4,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":347271168,"data_stored":347267689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":87011328,"data_stored":86931311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":318062592,"data_stored":318059113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:14:00.934 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:14:01.372 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 97 20 93095 93075 3.74704 2.01172 0.043977 0.0208432 2024-09-11T01:14:01.697 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:14:01.697 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:14:01.794 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2024-09-11T01:14:00.572365+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":375485052,"num_objects":91674,"num_object_clones":0,"num_object_copies":183348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":2854,"num_objects_unfound":0,"num_objects_dirty":91674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":52605,"num_write_kb":210414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1073,"num_bytes_recovered":4390931,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91756,"ondisk_log_size":91756,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1175920,"kb_used_data":733952,"kb_used_omap":12,"kb_used_meta":441907,"kb_avail":281939600,"statfs":{"total":289910292480,"available":288706150400,"internally_reserved":0,"allocated":751566848,"data_stored":750747930,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":452512772},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,4,5,14,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29937664,"num_objects":7309,"num_object_clones":0,"num_object_copies":14618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-240,"num_objects_unfound":0,"num_objects_dirty":7309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7309,"num_write_kb":29236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":983040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002591"},"pg_stats":[{"pgid":"2.f","version":"43'5784","reported_seq":5856,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:59.049443+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:13:59.049443+0000","last_peered":"2024-09-11T01:13:59.049443+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:13:59.049443+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":5784,"log_dups_size":0,"ondisk_log_size":5784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23691264,"num_objects":5784,"num_object_clones":0,"num_object_copies":11568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5784,"num_write_kb":23136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1712},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5834","reported_seq":5928,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:58.283508+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:13:58.283508+0000","last_peered":"2024-09-11T01:13:58.283508+0000","last_clean":"2024-09-11T01:13:58.283508+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:13:58.283508+0000","last_undegraded":"2024-09-11T01:13:58.283508+0000","last_fullsized":"2024-09-11T01:13:58.283508+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":5834,"log_dups_size":0,"ondisk_log_size":5834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23871488,"num_objects":5828,"num_object_clones":0,"num_object_copies":11656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4073,"num_write_kb":16292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5664","reported_seq":5786,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:58.291592+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:13:58.291592+0000","last_peered":"2024-09-11T01:13:58.291592+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:13:58.291592+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":5664,"log_dups_size":0,"ondisk_log_size":5664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23183360,"num_objects":5660,"num_object_clones":0,"num_object_copies":11320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5660,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3965,"num_write_kb":15860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":1764}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5718","reported_seq":5781,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:58.279692+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:13:58.279692+0000","last_peered":"2024-09-11T01:13:58.279692+0000","last_clean":"2024-09-11T01:13:58.279692+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:13:58.279692+0000","last_undegraded":"2024-09-11T01:13:58.279692+0000","last_fullsized":"2024-09-11T01:13:58.279692+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":5718,"log_dups_size":0,"ondisk_log_size":5718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23420928,"num_objects":5718,"num_object_clones":0,"num_object_copies":11436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5718,"num_write_kb":22872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5763","reported_seq":5885,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:58.993007+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:13:58.993007+0000","last_peered":"2024-09-11T01:13:58.993007+0000","last_clean":"2024-09-11T01:13:58.993007+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:13:58.993007+0000","last_undegraded":"2024-09-11T01:13:58.993007+0000","last_fullsized":"2024-09-11T01:13:58.993007+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":5763,"log_dups_size":0,"ondisk_log_size":5763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23576576,"num_objects":5756,"num_object_clones":0,"num_object_copies":11512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2768,"num_write_kb":11072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5722","reported_seq":5887,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:58.994374+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:13:58.994374+0000","last_peered":"2024-09-11T01:13:58.994374+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:13:58.994374+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":5722,"log_dups_size":0,"ondisk_log_size":5722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23416832,"num_objects":5717,"num_object_clones":0,"num_object_copies":11434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2063,"num_write_kb":8252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1766},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5661","reported_seq":5783,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:59.036314+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:13:59.036314+0000","last_peered":"2024-09-11T01:13:59.036314+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:13:59.036314+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":5661,"log_dups_size":0,"ondisk_log_size":5661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23162880,"num_objects":5655,"num_object_clones":0,"num_object_copies":11310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2518,"num_write_kb":10072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1669},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5841","reported_seq":5965,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:59.027280+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:13:59.027280+0000","last_peered":"2024-09-11T01:13:59.027280+0000","last_clean":"2024-09-11T01:13:59.027280+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:13:59.027280+0000","last_undegraded":"2024-09-11T01:13:59.027280+0000","last_fullsized":"2024-09-11T01:13:59.027280+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":5841,"log_dups_size":0,"ondisk_log_size":5841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23916544,"num_objects":5839,"num_object_clones":0,"num_object_copies":11678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2717,"num_write_kb":10868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5715","reported_seq":5837,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:58.298085+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:13:58.298085+0000","last_peered":"2024-09-11T01:13:58.298085+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:13:58.298085+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":5715,"log_dups_size":0,"ondisk_log_size":5715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23379968,"num_objects":5708,"num_object_clones":0,"num_object_copies":11416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3686,"num_write_kb":14744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":1775}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741858+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741858+0000","last_peered":"2024-09-11T01:13:09.741858+0000","last_clean":"2024-09-11T01:13:09.741858+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741858+0000","last_undegraded":"2024-09-11T01:13:09.741858+0000","last_fullsized":"2024-09-11T01:13:09.741858+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5708","reported_seq":5831,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:59.028987+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:13:59.028987+0000","last_peered":"2024-09-11T01:13:59.028987+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:13:59.028987+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":5708,"log_dups_size":0,"ondisk_log_size":5708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23347200,"num_objects":5700,"num_object_clones":0,"num_object_copies":11400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2295,"num_write_kb":9180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1716},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.910012+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.910012+0000","last_peered":"2024-09-11T01:13:09.910012+0000","last_clean":"2024-09-11T01:13:09.910012+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.910012+0000","last_undegraded":"2024-09-11T01:13:09.910012+0000","last_fullsized":"2024-09-11T01:13:09.910012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5770","reported_seq":5883,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:58.193005+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:13:58.193005+0000","last_peered":"2024-09-11T01:13:58.193005+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:13:58.193005+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":5770,"log_dups_size":0,"ondisk_log_size":5770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23613440,"num_objects":5765,"num_object_clones":0,"num_object_copies":11530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5765,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3587,"num_write_kb":14348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":1741}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5608","reported_seq":7864,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:13:58.293212+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:13:58.293212+0000","last_peered":"2024-09-11T01:13:58.293212+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:13:58.293212+0000","last_undegraded":"2024-09-11T01:13:58.293212+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":5608,"log_dups_size":0,"ondisk_log_size":5608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22945792,"num_objects":5602,"num_object_clones":0,"num_object_copies":11204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2854,"num_objects_unfound":0,"num_objects_dirty":5602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3393,"num_write_kb":13572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1071,"num_bytes_recovered":4386816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2854},{"shards":"2","objects":2748}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5717","reported_seq":5843,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:59.029795+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:13:59.029795+0000","last_peered":"2024-09-11T01:13:59.029795+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:13:59.029795+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":5717,"log_dups_size":0,"ondisk_log_size":5717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23408640,"num_objects":5715,"num_object_clones":0,"num_object_copies":11430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2454,"num_write_kb":9816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1765},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5697","reported_seq":5828,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:59.042220+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:13:59.042220+0000","last_peered":"2024-09-11T01:13:59.042220+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:13:59.042220+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":5697,"log_dups_size":0,"ondisk_log_size":5697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23302144,"num_objects":5689,"num_object_clones":0,"num_object_copies":11378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5689,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2352,"num_write_kb":9408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1750},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658057+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658057+0000","last_peered":"2024-09-11T01:13:09.658057+0000","last_clean":"2024-09-11T01:13:09.658057+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658057+0000","last_undegraded":"2024-09-11T01:13:09.658057+0000","last_fullsized":"2024-09-11T01:13:09.658057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5815","reported_seq":5944,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:58.988009+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:13:58.988009+0000","last_peered":"2024-09-11T01:13:58.988009+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:13:58.988009+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":5815,"log_dups_size":0,"ondisk_log_size":5815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23801856,"num_objects":5811,"num_object_clones":0,"num_object_copies":11622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2424,"num_write_kb":9696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1772},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5736","reported_seq":5864,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:58.996230+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:13:58.996230+0000","last_peered":"2024-09-11T01:13:58.996230+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:13:58.996230+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":5736,"log_dups_size":0,"ondisk_log_size":5736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23446121,"num_objects":5725,"num_object_clones":0,"num_object_copies":11450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2806,"num_write_kb":11224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1808},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":375485033,"num_objects":91672,"num_object_clones":0,"num_object_copies":183344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":2854,"num_objects_unfound":0,"num_objects_dirty":91672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":52603,"num_write_kb":210412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1072,"num_bytes_recovered":4390912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":752345088,"data_stored":752258113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91753,"ondisk_log_size":91753,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182430,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":514712,"kb_used_data":338696,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93857128,"statfs":{"total":96636764160,"available":96109699072,"internally_reserved":0,"allocated":346824704,"data_stored":346575885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":180219563},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2,2,7,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986202,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":472652,"kb_used_data":310524,"kb_used_omap":4,"kb_used_meta":162107,"kb_avail":93899188,"statfs":{"total":96636764160,"available":96152768512,"internally_reserved":0,"allocated":317976576,"data_stored":317731834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":165998250},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,0,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659147,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":188556,"kb_used_data":84732,"kb_used_omap":4,"kb_used_meta":103803,"kb_avail":94183284,"statfs":{"total":96636764160,"available":96443682816,"internally_reserved":0,"allocated":86765568,"data_stored":86440211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":106294959},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,4,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":347271168,"data_stored":347267689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":87011328,"data_stored":86931311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":318062592,"data_stored":318059113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:14:01.795 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:14:01.795 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'5784", 'reported_seq': 5856, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:59.049443+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:13:59.049443+0000', 'last_peered': '2024-09-11T01:13:59.049443+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:13:59.049443+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 5784, 'log_dups_size': 0, 'ondisk_log_size': 5784, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23691264, 'num_objects': 5784, 'num_object_clones': 0, 'num_object_copies': 11568, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5784, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5784, 'num_write_kb': 23136, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1712}, {'shards': '0,2', 'objects': 4072}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:01.796 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:14:01.796 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'5664", 'reported_seq': 5786, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:58.291592+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:13:58.291592+0000', 'last_peered': '2024-09-11T01:13:58.291592+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:13:58.291592+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 5664, 'log_dups_size': 0, 'ondisk_log_size': 5664, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23183360, 'num_objects': 5660, 'num_object_clones': 0, 'num_object_copies': 11320, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5660, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3965, 'num_write_kb': 15860, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3896}, {'shards': '2', 'objects': 1764}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:14:01.796 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:14:01.796 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'5722", 'reported_seq': 5887, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:58.994374+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:13:58.994374+0000', 'last_peered': '2024-09-11T01:13:58.994374+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:13:58.994374+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 5722, 'log_dups_size': 0, 'ondisk_log_size': 5722, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23416832, 'num_objects': 5717, 'num_object_clones': 0, 'num_object_copies': 11434, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5717, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2063, 'num_write_kb': 8252, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1766}, {'shards': '0,2', 'objects': 3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:01.796 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:14:01.796 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'5661", 'reported_seq': 5783, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:59.036314+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:13:59.036314+0000', 'last_peered': '2024-09-11T01:13:59.036314+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:13:59.036314+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 5661, 'log_dups_size': 0, 'ondisk_log_size': 5661, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23162880, 'num_objects': 5655, 'num_object_clones': 0, 'num_object_copies': 11310, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5655, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2518, 'num_write_kb': 10072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1669}, {'shards': '0,2', 'objects': 3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:01.796 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:14:01.796 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'5715", 'reported_seq': 5837, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:58.298085+0000', 'last_change': '2024-09-11T01:13:10.159069+0000', 'last_active': '2024-09-11T01:13:58.298085+0000', 'last_peered': '2024-09-11T01:13:58.298085+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:13:58.298085+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 5715, 'log_dups_size': 0, 'ondisk_log_size': 5715, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23379968, 'num_objects': 5708, 'num_object_clones': 0, 'num_object_copies': 11416, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5708, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3686, 'num_write_kb': 14744, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3932}, {'shards': '2', 'objects': 1775}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:14:01.797 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:14:01.797 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'5708", 'reported_seq': 5831, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:59.028987+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:13:59.028987+0000', 'last_peered': '2024-09-11T01:13:59.028987+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:13:59.028987+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 5708, 'log_dups_size': 0, 'ondisk_log_size': 5708, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23347200, 'num_objects': 5700, 'num_object_clones': 0, 'num_object_copies': 11400, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5700, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2295, 'num_write_kb': 9180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1716}, {'shards': '0,2', 'objects': 3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:01.797 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:14:01.797 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'5770", 'reported_seq': 5883, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:58.193005+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:13:58.193005+0000', 'last_peered': '2024-09-11T01:13:58.193005+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:13:58.193005+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 5770, 'log_dups_size': 0, 'ondisk_log_size': 5770, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23613440, 'num_objects': 5765, 'num_object_clones': 0, 'num_object_copies': 11530, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5765, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3587, 'num_write_kb': 14348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4023}, {'shards': '2', 'objects': 1741}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:14:01.797 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:14:01.797 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'5608", 'reported_seq': 7864, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:13:58.293212+0000', 'last_change': '2024-09-11T01:13:10.217540+0000', 'last_active': '2024-09-11T01:13:58.293212+0000', 'last_peered': '2024-09-11T01:13:58.293212+0000', 'last_clean': '2024-09-11T01:13:07.638709+0000', 'last_became_active': '2024-09-11T01:13:10.125604+0000', 'last_became_peered': '2024-09-11T01:13:10.125604+0000', 'last_unstale': '2024-09-11T01:13:58.293212+0000', 'last_undegraded': '2024-09-11T01:13:58.293212+0000', 'last_fullsized': '2024-09-11T01:13:09.649604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2269", 'last_scrub_stamp': '2024-09-11T01:12:42.890920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:42.890920+0000', 'objects_scrubbed': 2209, 'log_size': 5608, 'log_dups_size': 0, 'ondisk_log_size': 5608, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 999, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22945792, 'num_objects': 5602, 'num_object_clones': 0, 'num_object_copies': 11204, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2854, 'num_objects_unfound': 0, 'num_objects_dirty': 5602, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3393, 'num_write_kb': 13572, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1071, 'num_bytes_recovered': 4386816, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2854}, {'shards': '2', 'objects': 2748}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:14:01.797 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:14:01.797 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'5717", 'reported_seq': 5843, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:59.029795+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:13:59.029795+0000', 'last_peered': '2024-09-11T01:13:59.029795+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:13:59.029795+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 5717, 'log_dups_size': 0, 'ondisk_log_size': 5717, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23408640, 'num_objects': 5715, 'num_object_clones': 0, 'num_object_copies': 11430, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5715, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2454, 'num_write_kb': 9816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1765}, {'shards': '0,2', 'objects': 3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:01.797 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:14:01.797 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'5697", 'reported_seq': 5828, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:59.042220+0000', 'last_change': '2024-09-11T01:13:10.209668+0000', 'last_active': '2024-09-11T01:13:59.042220+0000', 'last_peered': '2024-09-11T01:13:59.042220+0000', 'last_clean': '2024-09-11T01:13:07.624565+0000', 'last_became_active': '2024-09-11T01:13:10.199795+0000', 'last_became_peered': '2024-09-11T01:13:10.199795+0000', 'last_unstale': '2024-09-11T01:13:59.042220+0000', 'last_undegraded': '2024-09-11T01:13:09.791198+0000', 'last_fullsized': '2024-09-11T01:13:09.652311+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3415", 'last_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'objects_scrubbed': 3337, 'log_size': 5697, 'log_dups_size': 0, 'ondisk_log_size': 5697, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1975, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23302144, 'num_objects': 5689, 'num_object_clones': 0, 'num_object_copies': 11378, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3942, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5689, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2352, 'num_write_kb': 9408, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1750}, {'shards': '0,2', 'objects': 3939}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:01.798 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:14:01.798 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'5815", 'reported_seq': 5944, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:58.988009+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:13:58.988009+0000', 'last_peered': '2024-09-11T01:13:58.988009+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:13:58.988009+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 5815, 'log_dups_size': 0, 'ondisk_log_size': 5815, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23801856, 'num_objects': 5811, 'num_object_clones': 0, 'num_object_copies': 11622, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5811, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2424, 'num_write_kb': 9696, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1772}, {'shards': '0,2', 'objects': 4039}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:01.798 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:14:01.798 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'5736", 'reported_seq': 5864, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:13:58.996230+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:13:58.996230+0000', 'last_peered': '2024-09-11T01:13:58.996230+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:13:58.996230+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 5736, 'log_dups_size': 0, 'ondisk_log_size': 5736, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23446121, 'num_objects': 5725, 'num_object_clones': 0, 'num_object_copies': 11450, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5725, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2806, 'num_write_kb': 11224, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1808}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:01.798 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:14:02.372 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 98 20 93608 93588 3.72922 2.00391 0.0202066 0.0209373 2024-09-11T01:14:02.795 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:14:02.795 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43711/183348 objects degraded (23.840%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":179,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":91674,"data_bytes":375485052,"bytes_used":1235861504,"bytes_avail":288674430976,"bytes_total":289910292480,"degraded_objects":43711,"degraded_total":183348,"degraded_ratio":0.23840456399851648,"misplaced_objects":2854,"misplaced_total":183348,"misplaced_ratio":0.015566027445077119,"recovering_objects_per_sec":19,"recovering_bytes_per_sec":81902,"recovering_keys_per_sec":0,"num_objects_recovered":240,"num_bytes_recovered":983040,"num_keys_recovered":0,"write_bytes_sec":1897407,"read_op_per_sec":0,"write_op_per_sec":463},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-11T01:13:08.555583+0000","services":{"osd":{"daemons":{"summary":"","1":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-11T01:14:02.852 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:14:02.852 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:14:03.373 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 99 20 94131 94111 3.71219 2.04297 0.0473529 0.0210374 2024-09-11T01:14:03.731 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:14:03.733 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:14:03.781 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2024-09-11T01:14:02.572820+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":375485052,"num_objects":91674,"num_object_clones":0,"num_object_copies":183348,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":2854,"num_objects_unfound":0,"num_objects_dirty":91674,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":52605,"num_write_kb":210414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1073,"num_bytes_recovered":4390931,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91756,"ondisk_log_size":91756,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1206896,"kb_used_data":742720,"kb_used_omap":12,"kb_used_meta":464115,"kb_avail":281908624,"statfs":{"total":289910292480,"available":288674430976,"internally_reserved":0,"allocated":760545280,"data_stored":759726362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":475253764},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,4,6,2,13,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22773760,"num_objects":5560,"num_object_clones":0,"num_object_copies":11120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-240,"num_objects_unfound":0,"num_objects_dirty":5560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5560,"num_write_kb":22240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":240,"num_bytes_recovered":983040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002564"},"pg_stats":[{"pgid":"2.f","version":"43'5784","reported_seq":5856,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:59.049443+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:13:59.049443+0000","last_peered":"2024-09-11T01:13:59.049443+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:13:59.049443+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":5784,"log_dups_size":0,"ondisk_log_size":5784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23691264,"num_objects":5784,"num_object_clones":0,"num_object_copies":11568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5784,"num_write_kb":23136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1712},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5834","reported_seq":5928,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:58.283508+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:13:58.283508+0000","last_peered":"2024-09-11T01:13:58.283508+0000","last_clean":"2024-09-11T01:13:58.283508+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:13:58.283508+0000","last_undegraded":"2024-09-11T01:13:58.283508+0000","last_fullsized":"2024-09-11T01:13:58.283508+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":5834,"log_dups_size":0,"ondisk_log_size":5834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23871488,"num_objects":5828,"num_object_clones":0,"num_object_copies":11656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4073,"num_write_kb":16292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5664","reported_seq":5786,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:58.291592+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:13:58.291592+0000","last_peered":"2024-09-11T01:13:58.291592+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:13:58.291592+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":5664,"log_dups_size":0,"ondisk_log_size":5664,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23183360,"num_objects":5660,"num_object_clones":0,"num_object_copies":11320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5660,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3965,"num_write_kb":15860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":1764}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5718","reported_seq":5781,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:58.279692+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:13:58.279692+0000","last_peered":"2024-09-11T01:13:58.279692+0000","last_clean":"2024-09-11T01:13:58.279692+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:13:58.279692+0000","last_undegraded":"2024-09-11T01:13:58.279692+0000","last_fullsized":"2024-09-11T01:13:58.279692+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":5718,"log_dups_size":0,"ondisk_log_size":5718,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23420928,"num_objects":5718,"num_object_clones":0,"num_object_copies":11436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5718,"num_write_kb":22872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5763","reported_seq":5885,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:58.993007+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:13:58.993007+0000","last_peered":"2024-09-11T01:13:58.993007+0000","last_clean":"2024-09-11T01:13:58.993007+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:13:58.993007+0000","last_undegraded":"2024-09-11T01:13:58.993007+0000","last_fullsized":"2024-09-11T01:13:58.993007+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":5763,"log_dups_size":0,"ondisk_log_size":5763,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23576576,"num_objects":5756,"num_object_clones":0,"num_object_copies":11512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2768,"num_write_kb":11072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5722","reported_seq":5887,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:58.994374+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:13:58.994374+0000","last_peered":"2024-09-11T01:13:58.994374+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:13:58.994374+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":5722,"log_dups_size":0,"ondisk_log_size":5722,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23416832,"num_objects":5717,"num_object_clones":0,"num_object_copies":11434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2063,"num_write_kb":8252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1766},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5661","reported_seq":5783,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:59.036314+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:13:59.036314+0000","last_peered":"2024-09-11T01:13:59.036314+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:13:59.036314+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":5661,"log_dups_size":0,"ondisk_log_size":5661,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23162880,"num_objects":5655,"num_object_clones":0,"num_object_copies":11310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2518,"num_write_kb":10072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1669},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5841","reported_seq":5965,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:59.027280+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:13:59.027280+0000","last_peered":"2024-09-11T01:13:59.027280+0000","last_clean":"2024-09-11T01:13:59.027280+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:13:59.027280+0000","last_undegraded":"2024-09-11T01:13:59.027280+0000","last_fullsized":"2024-09-11T01:13:59.027280+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":5841,"log_dups_size":0,"ondisk_log_size":5841,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23916544,"num_objects":5839,"num_object_clones":0,"num_object_copies":11678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2717,"num_write_kb":10868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5715","reported_seq":5837,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:58.298085+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:13:58.298085+0000","last_peered":"2024-09-11T01:13:58.298085+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:13:58.298085+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":5715,"log_dups_size":0,"ondisk_log_size":5715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23379968,"num_objects":5708,"num_object_clones":0,"num_object_copies":11416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3686,"num_write_kb":14744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":1775}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741858+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741858+0000","last_peered":"2024-09-11T01:13:09.741858+0000","last_clean":"2024-09-11T01:13:09.741858+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741858+0000","last_undegraded":"2024-09-11T01:13:09.741858+0000","last_fullsized":"2024-09-11T01:13:09.741858+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5708","reported_seq":5831,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:59.028987+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:13:59.028987+0000","last_peered":"2024-09-11T01:13:59.028987+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:13:59.028987+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":5708,"log_dups_size":0,"ondisk_log_size":5708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23347200,"num_objects":5700,"num_object_clones":0,"num_object_copies":11400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2295,"num_write_kb":9180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1716},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.910012+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.910012+0000","last_peered":"2024-09-11T01:13:09.910012+0000","last_clean":"2024-09-11T01:13:09.910012+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.910012+0000","last_undegraded":"2024-09-11T01:13:09.910012+0000","last_fullsized":"2024-09-11T01:13:09.910012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5770","reported_seq":5883,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:58.193005+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:13:58.193005+0000","last_peered":"2024-09-11T01:13:58.193005+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:13:58.193005+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":5770,"log_dups_size":0,"ondisk_log_size":5770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23613440,"num_objects":5765,"num_object_clones":0,"num_object_copies":11530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5765,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3587,"num_write_kb":14348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":1741}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5608","reported_seq":7864,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:13:58.293212+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:13:58.293212+0000","last_peered":"2024-09-11T01:13:58.293212+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:13:58.293212+0000","last_undegraded":"2024-09-11T01:13:58.293212+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":5608,"log_dups_size":0,"ondisk_log_size":5608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22945792,"num_objects":5602,"num_object_clones":0,"num_object_copies":11204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2854,"num_objects_unfound":0,"num_objects_dirty":5602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3393,"num_write_kb":13572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1071,"num_bytes_recovered":4386816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2854},{"shards":"2","objects":2748}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5717","reported_seq":5843,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:59.029795+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:13:59.029795+0000","last_peered":"2024-09-11T01:13:59.029795+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:13:59.029795+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":5717,"log_dups_size":0,"ondisk_log_size":5717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23408640,"num_objects":5715,"num_object_clones":0,"num_object_copies":11430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2454,"num_write_kb":9816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1765},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5697","reported_seq":5828,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:59.042220+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:13:59.042220+0000","last_peered":"2024-09-11T01:13:59.042220+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:13:59.042220+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":5697,"log_dups_size":0,"ondisk_log_size":5697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23302144,"num_objects":5689,"num_object_clones":0,"num_object_copies":11378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5689,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2352,"num_write_kb":9408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1750},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658057+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658057+0000","last_peered":"2024-09-11T01:13:09.658057+0000","last_clean":"2024-09-11T01:13:09.658057+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658057+0000","last_undegraded":"2024-09-11T01:13:09.658057+0000","last_fullsized":"2024-09-11T01:13:09.658057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5815","reported_seq":5944,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:58.988009+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:13:58.988009+0000","last_peered":"2024-09-11T01:13:58.988009+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:13:58.988009+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":5815,"log_dups_size":0,"ondisk_log_size":5815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23801856,"num_objects":5811,"num_object_clones":0,"num_object_copies":11622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2424,"num_write_kb":9696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1772},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5736","reported_seq":5864,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:13:58.996230+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:13:58.996230+0000","last_peered":"2024-09-11T01:13:58.996230+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:13:58.996230+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":5736,"log_dups_size":0,"ondisk_log_size":5736,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23446121,"num_objects":5725,"num_object_clones":0,"num_object_copies":11450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2806,"num_write_kb":11224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1808},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":375485033,"num_objects":91672,"num_object_clones":0,"num_object_copies":183344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":2854,"num_objects_unfound":0,"num_objects_dirty":91672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":52603,"num_write_kb":210412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1072,"num_bytes_recovered":4390912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":761368576,"data_stored":761281601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":91753,"ondisk_log_size":91753,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182430,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":514712,"kb_used_data":338696,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93857128,"statfs":{"total":96636764160,"available":96109699072,"internally_reserved":0,"allocated":346824704,"data_stored":346575885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":180219563},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,2,2,7,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986202,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":472652,"kb_used_data":310524,"kb_used_omap":4,"kb_used_meta":162107,"kb_avail":93899188,"statfs":{"total":96636764160,"available":96152768512,"internally_reserved":0,"allocated":317976576,"data_stored":317731834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":165998250},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,0,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659148,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":219532,"kb_used_data":93500,"kb_used_omap":4,"kb_used_meta":126011,"kb_avail":94152308,"statfs":{"total":96636764160,"available":96411963392,"internally_reserved":0,"allocated":95744000,"data_stored":95418643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":129035951},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,0,3,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":347271168,"data_stored":347267689,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":96034816,"data_stored":95954799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":318062592,"data_stored":318059113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:14:04.373 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout:2024-09-11T01:14:04.372535+0000 min lat: 0.00219431 max lat: 2.69211 avg lat: 0.0211305 2024-09-11T01:14:04.373 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T01:14:04.373 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 100 20 94664 94644 3.69589 2.08203 0.0256804 0.0211305 2024-09-11T01:14:05.373 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 101 20 95202 95182 3.68011 2.10156 0.0300293 0.0212198 2024-09-11T01:14:06.373 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 102 20 95755 95735 3.6652 2.16016 0.0108688 0.021305 2024-09-11T01:14:06.784 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:14:07.373 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 103 20 96289 96269 3.64987 2.08594 0.0147177 0.0213965 2024-09-11T01:14:07.446 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:14:07.447 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:14:07.487 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2024-09-11T01:14:06.573987+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":386503292,"num_objects":94364,"num_object_clones":0,"num_object_copies":188728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":2730,"num_objects_unfound":0,"num_objects_dirty":94364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55295,"num_write_kb":221174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1197,"num_bytes_recovered":4898835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94446,"ondisk_log_size":94446,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1233024,"kb_used_data":756368,"kb_used_omap":12,"kb_used_meta":476595,"kb_avail":281882496,"statfs":{"total":289910292480,"available":288647675904,"internally_reserved":0,"allocated":774520832,"data_stored":773701914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":488033284},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,4,7,4,10,8,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22310912,"num_objects":5447,"num_object_clones":0,"num_object_copies":10894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-238,"num_objects_unfound":0,"num_objects_dirty":5447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5447,"num_write_kb":21788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":974848,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002900"},"pg_stats":[{"pgid":"2.f","version":"43'5949","reported_seq":6021,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:03.918502+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:14:03.918502+0000","last_peered":"2024-09-11T01:14:03.918502+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:14:03.918502+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":5949,"log_dups_size":0,"ondisk_log_size":5949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24367104,"num_objects":5949,"num_object_clones":0,"num_object_copies":11898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5949,"num_write_kb":23796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1877},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5998","reported_seq":6094,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:03.311702+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:14:03.311702+0000","last_peered":"2024-09-11T01:14:03.311702+0000","last_clean":"2024-09-11T01:14:03.311702+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:14:03.311702+0000","last_undegraded":"2024-09-11T01:14:03.311702+0000","last_fullsized":"2024-09-11T01:14:03.311702+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":5998,"log_dups_size":0,"ondisk_log_size":5998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24543232,"num_objects":5992,"num_object_clones":0,"num_object_copies":11984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5992,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4237,"num_write_kb":16948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5832","reported_seq":5954,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:03.335792+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:14:03.335792+0000","last_peered":"2024-09-11T01:14:03.335792+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:14:03.335792+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":5832,"log_dups_size":0,"ondisk_log_size":5832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23871488,"num_objects":5828,"num_object_clones":0,"num_object_copies":11656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4133,"num_write_kb":16532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":1932}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5892","reported_seq":5956,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:03.327309+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:14:03.327309+0000","last_peered":"2024-09-11T01:14:03.327309+0000","last_clean":"2024-09-11T01:14:03.327309+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:14:03.327309+0000","last_undegraded":"2024-09-11T01:14:03.327309+0000","last_fullsized":"2024-09-11T01:14:03.327309+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":5892,"log_dups_size":0,"ondisk_log_size":5892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24133632,"num_objects":5892,"num_object_clones":0,"num_object_copies":11784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5892,"num_write_kb":23568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5936","reported_seq":6058,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:04.030403+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:14:04.030403+0000","last_peered":"2024-09-11T01:14:04.030403+0000","last_clean":"2024-09-11T01:14:04.030403+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:14:04.030403+0000","last_undegraded":"2024-09-11T01:14:04.030403+0000","last_fullsized":"2024-09-11T01:14:04.030403+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":5936,"log_dups_size":0,"ondisk_log_size":5936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24285184,"num_objects":5929,"num_object_clones":0,"num_object_copies":11858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2941,"num_write_kb":11764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5893","reported_seq":6058,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:04.010826+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:14:04.010826+0000","last_peered":"2024-09-11T01:14:04.010826+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:14:04.010826+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":5893,"log_dups_size":0,"ondisk_log_size":5893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24117248,"num_objects":5888,"num_object_clones":0,"num_object_copies":11776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2234,"num_write_kb":8936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1937},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5819","reported_seq":5944,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:04.051917+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:14:04.051917+0000","last_peered":"2024-09-11T01:14:04.051917+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:14:04.051917+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":5819,"log_dups_size":0,"ondisk_log_size":5819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23810048,"num_objects":5813,"num_object_clones":0,"num_object_copies":11626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2676,"num_write_kb":10704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1827},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5998","reported_seq":6121,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:04.098066+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:14:04.098066+0000","last_peered":"2024-09-11T01:14:04.098066+0000","last_clean":"2024-09-11T01:14:04.098066+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:14:04.098066+0000","last_undegraded":"2024-09-11T01:14:04.098066+0000","last_fullsized":"2024-09-11T01:14:04.098066+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":5998,"log_dups_size":0,"ondisk_log_size":5998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24559616,"num_objects":5996,"num_object_clones":0,"num_object_copies":11992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2874,"num_write_kb":11496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5878","reported_seq":6000,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:03.350959+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:14:03.350959+0000","last_peered":"2024-09-11T01:14:03.350959+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:14:03.350959+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":5878,"log_dups_size":0,"ondisk_log_size":5878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24047616,"num_objects":5871,"num_object_clones":0,"num_object_copies":11742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5871,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3849,"num_write_kb":15396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":1938}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741858+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741858+0000","last_peered":"2024-09-11T01:13:09.741858+0000","last_clean":"2024-09-11T01:13:09.741858+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741858+0000","last_undegraded":"2024-09-11T01:13:09.741858+0000","last_fullsized":"2024-09-11T01:13:09.741858+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5867","reported_seq":5990,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:03.984612+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:14:03.984612+0000","last_peered":"2024-09-11T01:14:03.984612+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:14:03.984612+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":5867,"log_dups_size":0,"ondisk_log_size":5867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23998464,"num_objects":5859,"num_object_clones":0,"num_object_copies":11718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2454,"num_write_kb":9816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1875},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.910012+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.910012+0000","last_peered":"2024-09-11T01:13:09.910012+0000","last_clean":"2024-09-11T01:13:09.910012+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.910012+0000","last_undegraded":"2024-09-11T01:13:09.910012+0000","last_fullsized":"2024-09-11T01:13:09.910012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5924","reported_seq":6037,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:03.293906+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:14:03.293906+0000","last_peered":"2024-09-11T01:14:03.293906+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:14:03.293906+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":5924,"log_dups_size":0,"ondisk_log_size":5924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24244224,"num_objects":5919,"num_object_clones":0,"num_object_copies":11838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3741,"num_write_kb":14964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":1895}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5784","reported_seq":8288,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:14:03.345417+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:14:03.345417+0000","last_peered":"2024-09-11T01:14:03.345417+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:14:03.345417+0000","last_undegraded":"2024-09-11T01:14:03.345417+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":5784,"log_dups_size":0,"ondisk_log_size":5784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23666688,"num_objects":5778,"num_object_clones":0,"num_object_copies":11556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2730,"num_objects_unfound":0,"num_objects_dirty":5778,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3569,"num_write_kb":14276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1195,"num_bytes_recovered":4894720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2730},{"shards":"2","objects":3048}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5901","reported_seq":6025,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:04.058336+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:14:04.058336+0000","last_peered":"2024-09-11T01:14:04.058336+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:14:04.058336+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":5901,"log_dups_size":0,"ondisk_log_size":5901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24162304,"num_objects":5899,"num_object_clones":0,"num_object_copies":11798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2638,"num_write_kb":10552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1949},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5874","reported_seq":6004,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:04.077981+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:14:04.077981+0000","last_peered":"2024-09-11T01:14:04.077981+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:14:04.077981+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":5874,"log_dups_size":0,"ondisk_log_size":5874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24027136,"num_objects":5866,"num_object_clones":0,"num_object_copies":11732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2529,"num_write_kb":10116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1927},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658057+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658057+0000","last_peered":"2024-09-11T01:13:09.658057+0000","last_clean":"2024-09-11T01:13:09.658057+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658057+0000","last_undegraded":"2024-09-11T01:13:09.658057+0000","last_fullsized":"2024-09-11T01:13:09.658057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5992","reported_seq":6120,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:04.051241+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:14:04.051241+0000","last_peered":"2024-09-11T01:14:04.051241+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:14:04.051241+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":5992,"log_dups_size":0,"ondisk_log_size":5992,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24526848,"num_objects":5988,"num_object_clones":0,"num_object_copies":11976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2601,"num_write_kb":10404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1949},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5906","reported_seq":6034,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:04.067901+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:14:04.067901+0000","last_peered":"2024-09-11T01:14:04.067901+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:14:04.067901+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":5906,"log_dups_size":0,"ondisk_log_size":5906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24142441,"num_objects":5895,"num_object_clones":0,"num_object_copies":11790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2976,"num_write_kb":11904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1978},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":386503273,"num_objects":94362,"num_object_clones":0,"num_object_copies":188724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":2730,"num_objects_unfound":0,"num_objects_dirty":94362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55293,"num_write_kb":221172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1196,"num_bytes_recovered":4898816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":775188480,"data_stored":775101505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94443,"ondisk_log_size":94443,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182431,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":531308,"kb_used_data":346972,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93840532,"statfs":{"total":96636764160,"available":96092704768,"internally_reserved":0,"allocated":355299328,"data_stored":355050509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":188739243},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,3,1,2,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986203,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482184,"kb_used_data":315896,"kb_used_omap":4,"kb_used_meta":166267,"kb_avail":93889656,"statfs":{"total":96636764160,"available":96143007744,"internally_reserved":0,"allocated":323477504,"data_stored":323232762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":170258090},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,3,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659148,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":219532,"kb_used_data":93500,"kb_used_omap":4,"kb_used_meta":126011,"kb_avail":94152308,"statfs":{"total":96636764160,"available":96411963392,"internally_reserved":0,"allocated":95744000,"data_stored":95418643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":129035951},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,0,3,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":355536896,"data_stored":355533417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":96034816,"data_stored":95954799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":323616768,"data_stored":323613289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:14:07.489 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:14:08.376 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 104 20 96810 96790 3.63424 2.03516 0.0147395 0.0214851 2024-09-11T01:14:08.450 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:14:08.452 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:14:08.516 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":128,"stamp":"2024-09-11T01:14:06.573987+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":386503292,"num_objects":94364,"num_object_clones":0,"num_object_copies":188728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":2730,"num_objects_unfound":0,"num_objects_dirty":94364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55295,"num_write_kb":221174,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1197,"num_bytes_recovered":4898835,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94446,"ondisk_log_size":94446,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1233024,"kb_used_data":756368,"kb_used_omap":12,"kb_used_meta":476595,"kb_avail":281882496,"statfs":{"total":289910292480,"available":288647675904,"internally_reserved":0,"allocated":774520832,"data_stored":773701914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":488033284},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,4,7,4,10,8,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22310912,"num_objects":5447,"num_object_clones":0,"num_object_copies":10894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-238,"num_objects_unfound":0,"num_objects_dirty":5447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5447,"num_write_kb":21788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":974848,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002900"},"pg_stats":[{"pgid":"2.f","version":"43'5949","reported_seq":6021,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:03.918502+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:14:03.918502+0000","last_peered":"2024-09-11T01:14:03.918502+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:14:03.918502+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":5949,"log_dups_size":0,"ondisk_log_size":5949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24367104,"num_objects":5949,"num_object_clones":0,"num_object_copies":11898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5949,"num_write_kb":23796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1877},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'5998","reported_seq":6094,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:03.311702+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:14:03.311702+0000","last_peered":"2024-09-11T01:14:03.311702+0000","last_clean":"2024-09-11T01:14:03.311702+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:14:03.311702+0000","last_undegraded":"2024-09-11T01:14:03.311702+0000","last_fullsized":"2024-09-11T01:14:03.311702+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":5998,"log_dups_size":0,"ondisk_log_size":5998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24543232,"num_objects":5992,"num_object_clones":0,"num_object_copies":11984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5992,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4237,"num_write_kb":16948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'5832","reported_seq":5954,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:03.335792+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:14:03.335792+0000","last_peered":"2024-09-11T01:14:03.335792+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:14:03.335792+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":5832,"log_dups_size":0,"ondisk_log_size":5832,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23871488,"num_objects":5828,"num_object_clones":0,"num_object_copies":11656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4133,"num_write_kb":16532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":1932}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'5892","reported_seq":5956,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:03.327309+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:14:03.327309+0000","last_peered":"2024-09-11T01:14:03.327309+0000","last_clean":"2024-09-11T01:14:03.327309+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:14:03.327309+0000","last_undegraded":"2024-09-11T01:14:03.327309+0000","last_fullsized":"2024-09-11T01:14:03.327309+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":5892,"log_dups_size":0,"ondisk_log_size":5892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24133632,"num_objects":5892,"num_object_clones":0,"num_object_copies":11784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5892,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5892,"num_write_kb":23568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5936","reported_seq":6058,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:04.030403+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:14:04.030403+0000","last_peered":"2024-09-11T01:14:04.030403+0000","last_clean":"2024-09-11T01:14:04.030403+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:14:04.030403+0000","last_undegraded":"2024-09-11T01:14:04.030403+0000","last_fullsized":"2024-09-11T01:14:04.030403+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":5936,"log_dups_size":0,"ondisk_log_size":5936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24285184,"num_objects":5929,"num_object_clones":0,"num_object_copies":11858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2941,"num_write_kb":11764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5893","reported_seq":6058,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:04.010826+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:14:04.010826+0000","last_peered":"2024-09-11T01:14:04.010826+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:14:04.010826+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":5893,"log_dups_size":0,"ondisk_log_size":5893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24117248,"num_objects":5888,"num_object_clones":0,"num_object_copies":11776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2234,"num_write_kb":8936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1937},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5819","reported_seq":5944,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:04.051917+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:14:04.051917+0000","last_peered":"2024-09-11T01:14:04.051917+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:14:04.051917+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":5819,"log_dups_size":0,"ondisk_log_size":5819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23810048,"num_objects":5813,"num_object_clones":0,"num_object_copies":11626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2676,"num_write_kb":10704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1827},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5998","reported_seq":6121,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:04.098066+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:14:04.098066+0000","last_peered":"2024-09-11T01:14:04.098066+0000","last_clean":"2024-09-11T01:14:04.098066+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:14:04.098066+0000","last_undegraded":"2024-09-11T01:14:04.098066+0000","last_fullsized":"2024-09-11T01:14:04.098066+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":5998,"log_dups_size":0,"ondisk_log_size":5998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24559616,"num_objects":5996,"num_object_clones":0,"num_object_copies":11992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2874,"num_write_kb":11496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'5878","reported_seq":6000,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:03.350959+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:14:03.350959+0000","last_peered":"2024-09-11T01:14:03.350959+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:14:03.350959+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":5878,"log_dups_size":0,"ondisk_log_size":5878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24047616,"num_objects":5871,"num_object_clones":0,"num_object_copies":11742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5871,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3849,"num_write_kb":15396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":1938}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741858+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741858+0000","last_peered":"2024-09-11T01:13:09.741858+0000","last_clean":"2024-09-11T01:13:09.741858+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741858+0000","last_undegraded":"2024-09-11T01:13:09.741858+0000","last_fullsized":"2024-09-11T01:13:09.741858+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5867","reported_seq":5990,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:03.984612+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:14:03.984612+0000","last_peered":"2024-09-11T01:14:03.984612+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:14:03.984612+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":5867,"log_dups_size":0,"ondisk_log_size":5867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23998464,"num_objects":5859,"num_object_clones":0,"num_object_copies":11718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2454,"num_write_kb":9816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1875},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.910012+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.910012+0000","last_peered":"2024-09-11T01:13:09.910012+0000","last_clean":"2024-09-11T01:13:09.910012+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.910012+0000","last_undegraded":"2024-09-11T01:13:09.910012+0000","last_fullsized":"2024-09-11T01:13:09.910012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'5924","reported_seq":6037,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:03.293906+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:14:03.293906+0000","last_peered":"2024-09-11T01:14:03.293906+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:14:03.293906+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":5924,"log_dups_size":0,"ondisk_log_size":5924,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24244224,"num_objects":5919,"num_object_clones":0,"num_object_copies":11838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3741,"num_write_kb":14964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":1895}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5784","reported_seq":8288,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:14:03.345417+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:14:03.345417+0000","last_peered":"2024-09-11T01:14:03.345417+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:14:03.345417+0000","last_undegraded":"2024-09-11T01:14:03.345417+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":5784,"log_dups_size":0,"ondisk_log_size":5784,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23666688,"num_objects":5778,"num_object_clones":0,"num_object_copies":11556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2730,"num_objects_unfound":0,"num_objects_dirty":5778,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3569,"num_write_kb":14276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1195,"num_bytes_recovered":4894720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2730},{"shards":"2","objects":3048}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5901","reported_seq":6025,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:04.058336+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:14:04.058336+0000","last_peered":"2024-09-11T01:14:04.058336+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:14:04.058336+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":5901,"log_dups_size":0,"ondisk_log_size":5901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24162304,"num_objects":5899,"num_object_clones":0,"num_object_copies":11798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2638,"num_write_kb":10552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1949},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5874","reported_seq":6004,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:04.077981+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:14:04.077981+0000","last_peered":"2024-09-11T01:14:04.077981+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:14:04.077981+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":5874,"log_dups_size":0,"ondisk_log_size":5874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24027136,"num_objects":5866,"num_object_clones":0,"num_object_copies":11732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2529,"num_write_kb":10116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1927},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658057+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658057+0000","last_peered":"2024-09-11T01:13:09.658057+0000","last_clean":"2024-09-11T01:13:09.658057+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658057+0000","last_undegraded":"2024-09-11T01:13:09.658057+0000","last_fullsized":"2024-09-11T01:13:09.658057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5992","reported_seq":6120,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:04.051241+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:14:04.051241+0000","last_peered":"2024-09-11T01:14:04.051241+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:14:04.051241+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":5992,"log_dups_size":0,"ondisk_log_size":5992,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24526848,"num_objects":5988,"num_object_clones":0,"num_object_copies":11976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2601,"num_write_kb":10404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1949},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5906","reported_seq":6034,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:04.067901+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:14:04.067901+0000","last_peered":"2024-09-11T01:14:04.067901+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:14:04.067901+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":5906,"log_dups_size":0,"ondisk_log_size":5906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24142441,"num_objects":5895,"num_object_clones":0,"num_object_copies":11790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2976,"num_write_kb":11904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1978},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":386503273,"num_objects":94362,"num_object_clones":0,"num_object_copies":188724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":2730,"num_objects_unfound":0,"num_objects_dirty":94362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":55293,"num_write_kb":221172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1196,"num_bytes_recovered":4898816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":775188480,"data_stored":775101505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":94443,"ondisk_log_size":94443,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182431,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":531308,"kb_used_data":346972,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93840532,"statfs":{"total":96636764160,"available":96092704768,"internally_reserved":0,"allocated":355299328,"data_stored":355050509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":188739243},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,3,1,2,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986203,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482184,"kb_used_data":315896,"kb_used_omap":4,"kb_used_meta":166267,"kb_avail":93889656,"statfs":{"total":96636764160,"available":96143007744,"internally_reserved":0,"allocated":323477504,"data_stored":323232762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":170258090},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,3,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659148,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":219532,"kb_used_data":93500,"kb_used_omap":4,"kb_used_meta":126011,"kb_avail":94152308,"statfs":{"total":96636764160,"available":96411963392,"internally_reserved":0,"allocated":95744000,"data_stored":95418643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":129035951},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,3,0,3,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":355536896,"data_stored":355533417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":96034816,"data_stored":95954799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":323616768,"data_stored":323613289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:14:08.518 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:14:08.518 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'5949", 'reported_seq': 6021, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:03.918502+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:14:03.918502+0000', 'last_peered': '2024-09-11T01:14:03.918502+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:14:03.918502+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 5949, 'log_dups_size': 0, 'ondisk_log_size': 5949, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24367104, 'num_objects': 5949, 'num_object_clones': 0, 'num_object_copies': 11898, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5949, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5949, 'num_write_kb': 23796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1877}, {'shards': '0,2', 'objects': 4072}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:08.518 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:14:08.518 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'5832", 'reported_seq': 5954, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:03.335792+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:14:03.335792+0000', 'last_peered': '2024-09-11T01:14:03.335792+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:14:03.335792+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 5832, 'log_dups_size': 0, 'ondisk_log_size': 5832, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23871488, 'num_objects': 5828, 'num_object_clones': 0, 'num_object_copies': 11656, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5828, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4133, 'num_write_kb': 16532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3896}, {'shards': '2', 'objects': 1932}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:14:08.518 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:14:08.518 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'5893", 'reported_seq': 6058, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:04.010826+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:14:04.010826+0000', 'last_peered': '2024-09-11T01:14:04.010826+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:14:04.010826+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 5893, 'log_dups_size': 0, 'ondisk_log_size': 5893, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24117248, 'num_objects': 5888, 'num_object_clones': 0, 'num_object_copies': 11776, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5888, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2234, 'num_write_kb': 8936, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1937}, {'shards': '0,2', 'objects': 3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:08.518 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:14:08.518 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'5819", 'reported_seq': 5944, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:04.051917+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:14:04.051917+0000', 'last_peered': '2024-09-11T01:14:04.051917+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:14:04.051917+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 5819, 'log_dups_size': 0, 'ondisk_log_size': 5819, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23810048, 'num_objects': 5813, 'num_object_clones': 0, 'num_object_copies': 11626, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5813, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2676, 'num_write_kb': 10704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1827}, {'shards': '0,2', 'objects': 3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:08.519 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:14:08.519 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'5878", 'reported_seq': 6000, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:03.350959+0000', 'last_change': '2024-09-11T01:13:10.159069+0000', 'last_active': '2024-09-11T01:14:03.350959+0000', 'last_peered': '2024-09-11T01:14:03.350959+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:14:03.350959+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 5878, 'log_dups_size': 0, 'ondisk_log_size': 5878, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24047616, 'num_objects': 5871, 'num_object_clones': 0, 'num_object_copies': 11742, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5871, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3849, 'num_write_kb': 15396, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3932}, {'shards': '2', 'objects': 1938}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:14:08.519 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:14:08.519 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'5867", 'reported_seq': 5990, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:03.984612+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:14:03.984612+0000', 'last_peered': '2024-09-11T01:14:03.984612+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:14:03.984612+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 5867, 'log_dups_size': 0, 'ondisk_log_size': 5867, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23998464, 'num_objects': 5859, 'num_object_clones': 0, 'num_object_copies': 11718, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5859, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2454, 'num_write_kb': 9816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1875}, {'shards': '0,2', 'objects': 3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:08.519 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:14:08.519 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'5924", 'reported_seq': 6037, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:03.293906+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:14:03.293906+0000', 'last_peered': '2024-09-11T01:14:03.293906+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:14:03.293906+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 5924, 'log_dups_size': 0, 'ondisk_log_size': 5924, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24244224, 'num_objects': 5919, 'num_object_clones': 0, 'num_object_copies': 11838, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5919, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3741, 'num_write_kb': 14964, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4023}, {'shards': '2', 'objects': 1895}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:14:08.519 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:14:08.519 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'5784", 'reported_seq': 8288, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:14:03.345417+0000', 'last_change': '2024-09-11T01:13:10.217540+0000', 'last_active': '2024-09-11T01:14:03.345417+0000', 'last_peered': '2024-09-11T01:14:03.345417+0000', 'last_clean': '2024-09-11T01:13:07.638709+0000', 'last_became_active': '2024-09-11T01:13:10.125604+0000', 'last_became_peered': '2024-09-11T01:13:10.125604+0000', 'last_unstale': '2024-09-11T01:14:03.345417+0000', 'last_undegraded': '2024-09-11T01:14:03.345417+0000', 'last_fullsized': '2024-09-11T01:13:09.649604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2269", 'last_scrub_stamp': '2024-09-11T01:12:42.890920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:42.890920+0000', 'objects_scrubbed': 2209, 'log_size': 5784, 'log_dups_size': 0, 'ondisk_log_size': 5784, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 999, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23666688, 'num_objects': 5778, 'num_object_clones': 0, 'num_object_copies': 11556, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2730, 'num_objects_unfound': 0, 'num_objects_dirty': 5778, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3569, 'num_write_kb': 14276, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1195, 'num_bytes_recovered': 4894720, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2730}, {'shards': '2', 'objects': 3048}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:14:08.519 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:14:08.519 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'5901", 'reported_seq': 6025, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:04.058336+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:14:04.058336+0000', 'last_peered': '2024-09-11T01:14:04.058336+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:14:04.058336+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 5901, 'log_dups_size': 0, 'ondisk_log_size': 5901, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24162304, 'num_objects': 5899, 'num_object_clones': 0, 'num_object_copies': 11798, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5899, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2638, 'num_write_kb': 10552, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1949}, {'shards': '0,2', 'objects': 3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:08.520 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:14:08.520 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'5874", 'reported_seq': 6004, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:04.077981+0000', 'last_change': '2024-09-11T01:13:10.209668+0000', 'last_active': '2024-09-11T01:14:04.077981+0000', 'last_peered': '2024-09-11T01:14:04.077981+0000', 'last_clean': '2024-09-11T01:13:07.624565+0000', 'last_became_active': '2024-09-11T01:13:10.199795+0000', 'last_became_peered': '2024-09-11T01:13:10.199795+0000', 'last_unstale': '2024-09-11T01:14:04.077981+0000', 'last_undegraded': '2024-09-11T01:13:09.791198+0000', 'last_fullsized': '2024-09-11T01:13:09.652311+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3415", 'last_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'objects_scrubbed': 3337, 'log_size': 5874, 'log_dups_size': 0, 'ondisk_log_size': 5874, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1975, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24027136, 'num_objects': 5866, 'num_object_clones': 0, 'num_object_copies': 11732, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3942, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5866, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2529, 'num_write_kb': 10116, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1927}, {'shards': '0,2', 'objects': 3939}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:08.520 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:14:08.520 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'5992", 'reported_seq': 6120, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:04.051241+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:14:04.051241+0000', 'last_peered': '2024-09-11T01:14:04.051241+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:14:04.051241+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 5992, 'log_dups_size': 0, 'ondisk_log_size': 5992, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24526848, 'num_objects': 5988, 'num_object_clones': 0, 'num_object_copies': 11976, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5988, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2601, 'num_write_kb': 10404, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1949}, {'shards': '0,2', 'objects': 4039}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:08.520 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:14:08.520 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'5906", 'reported_seq': 6034, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:04.067901+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:14:04.067901+0000', 'last_peered': '2024-09-11T01:14:04.067901+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:14:04.067901+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 5906, 'log_dups_size': 0, 'ondisk_log_size': 5906, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24142441, 'num_objects': 5895, 'num_object_clones': 0, 'num_object_copies': 11790, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5895, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2976, 'num_write_kb': 11904, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1978}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:08.520 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:14:09.376 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 105 20 97334 97314 3.61913 2.04688 0.0218815 0.0215793 2024-09-11T01:14:09.535 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:14:09.535 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43711/190748 objects degraded (22.916%), 11 pgs degraded","count":11},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":186,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":95374,"data_bytes":390640252,"bytes_used":1274155008,"bytes_avail":288636137472,"bytes_total":289910292480,"degraded_objects":43711,"degraded_total":190748,"degraded_ratio":0.22915574475223857,"misplaced_objects":2593,"misplaced_total":190748,"misplaced_ratio":0.013593851573804182,"recovering_objects_per_sec":31,"recovering_bytes_per_sec":127970,"recovering_keys_per_sec":0,"num_objects_recovered":375,"num_bytes_recovered":1536000,"num_keys_recovered":0,"write_bytes_sec":2203486,"read_op_per_sec":0,"write_op_per_sec":537},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:14:09.574 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:14:09.574 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:14:10.324 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:14:10.327 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:14:10.376 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 106 20 97842 97822 3.6037 1.98438 0.0552509 0.021671 2024-09-11T01:14:10.401 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":129,"stamp":"2024-09-11T01:14:08.574321+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":390640252,"num_objects":95374,"num_object_clones":0,"num_object_copies":190748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":2593,"num_objects_unfound":0,"num_objects_dirty":95374,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56305,"num_write_kb":225214,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1334,"num_bytes_recovered":5459987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":95456,"ondisk_log_size":95456,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1244292,"kb_used_data":770388,"kb_used_omap":12,"kb_used_meta":473843,"kb_avail":281871228,"statfs":{"total":289910292480,"available":288636137472,"internally_reserved":0,"allocated":788877312,"data_stored":788058394,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":485215236},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,3,6,6,10,12,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26447872,"num_objects":6457,"num_object_clones":0,"num_object_copies":12914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-375,"num_objects_unfound":0,"num_objects_dirty":6457,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6457,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":375,"num_bytes_recovered":1536000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002736"},"pg_stats":[{"pgid":"2.f","version":"43'5949","reported_seq":6021,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:03.918502+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:14:03.918502+0000","last_peered":"2024-09-11T01:14:03.918502+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:14:03.918502+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":5949,"log_dups_size":0,"ondisk_log_size":5949,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24367104,"num_objects":5949,"num_object_clones":0,"num_object_copies":11898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5949,"num_write_kb":23796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1877},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6164","reported_seq":6260,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:08.324639+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:14:08.324639+0000","last_peered":"2024-09-11T01:14:08.324639+0000","last_clean":"2024-09-11T01:14:08.324639+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:14:08.324639+0000","last_undegraded":"2024-09-11T01:14:08.324639+0000","last_fullsized":"2024-09-11T01:14:08.324639+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":6164,"log_dups_size":0,"ondisk_log_size":6164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25223168,"num_objects":6158,"num_object_clones":0,"num_object_copies":12316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4403,"num_write_kb":17612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6000","reported_seq":6122,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:08.320173+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:14:08.320173+0000","last_peered":"2024-09-11T01:14:08.320173+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:14:08.320173+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":6000,"log_dups_size":0,"ondisk_log_size":6000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24559616,"num_objects":5996,"num_object_clones":0,"num_object_copies":11992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4301,"num_write_kb":17204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":2100}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6066","reported_seq":6130,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:08.333245+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:14:08.333245+0000","last_peered":"2024-09-11T01:14:08.333245+0000","last_clean":"2024-09-11T01:14:08.333245+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:14:08.333245+0000","last_undegraded":"2024-09-11T01:14:08.333245+0000","last_fullsized":"2024-09-11T01:14:08.333245+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":6066,"log_dups_size":0,"ondisk_log_size":6066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24846336,"num_objects":6066,"num_object_clones":0,"num_object_copies":12132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6066,"num_write_kb":24264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'5936","reported_seq":6058,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:04.030403+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:14:04.030403+0000","last_peered":"2024-09-11T01:14:04.030403+0000","last_clean":"2024-09-11T01:14:04.030403+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:14:04.030403+0000","last_undegraded":"2024-09-11T01:14:04.030403+0000","last_fullsized":"2024-09-11T01:14:04.030403+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":5936,"log_dups_size":0,"ondisk_log_size":5936,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24285184,"num_objects":5929,"num_object_clones":0,"num_object_copies":11858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2941,"num_write_kb":11764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'5893","reported_seq":6058,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:04.010826+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:14:04.010826+0000","last_peered":"2024-09-11T01:14:04.010826+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:14:04.010826+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":5893,"log_dups_size":0,"ondisk_log_size":5893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24117248,"num_objects":5888,"num_object_clones":0,"num_object_copies":11776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2234,"num_write_kb":8936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1937},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5819","reported_seq":5944,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:04.051917+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:14:04.051917+0000","last_peered":"2024-09-11T01:14:04.051917+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:14:04.051917+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":5819,"log_dups_size":0,"ondisk_log_size":5819,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23810048,"num_objects":5813,"num_object_clones":0,"num_object_copies":11626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2676,"num_write_kb":10704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1827},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'5998","reported_seq":6121,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:04.098066+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:14:04.098066+0000","last_peered":"2024-09-11T01:14:04.098066+0000","last_clean":"2024-09-11T01:14:04.098066+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:14:04.098066+0000","last_undegraded":"2024-09-11T01:14:04.098066+0000","last_fullsized":"2024-09-11T01:14:04.098066+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":5998,"log_dups_size":0,"ondisk_log_size":5998,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24559616,"num_objects":5996,"num_object_clones":0,"num_object_copies":11992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2874,"num_write_kb":11496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6053","reported_seq":6175,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:08.370225+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:14:08.370225+0000","last_peered":"2024-09-11T01:14:08.370225+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:14:08.370225+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":6053,"log_dups_size":0,"ondisk_log_size":6053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24764416,"num_objects":6046,"num_object_clones":0,"num_object_copies":12092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6046,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4024,"num_write_kb":16096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":2113}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741858+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741858+0000","last_peered":"2024-09-11T01:13:09.741858+0000","last_clean":"2024-09-11T01:13:09.741858+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741858+0000","last_undegraded":"2024-09-11T01:13:09.741858+0000","last_fullsized":"2024-09-11T01:13:09.741858+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'5867","reported_seq":5990,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:03.984612+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:14:03.984612+0000","last_peered":"2024-09-11T01:14:03.984612+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:14:03.984612+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":5867,"log_dups_size":0,"ondisk_log_size":5867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23998464,"num_objects":5859,"num_object_clones":0,"num_object_copies":11718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2454,"num_write_kb":9816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1875},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.910012+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.910012+0000","last_peered":"2024-09-11T01:13:09.910012+0000","last_clean":"2024-09-11T01:13:09.910012+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.910012+0000","last_undegraded":"2024-09-11T01:13:09.910012+0000","last_fullsized":"2024-09-11T01:13:09.910012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6104","reported_seq":6217,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:08.348142+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:14:08.348142+0000","last_peered":"2024-09-11T01:14:08.348142+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:14:08.348142+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":6104,"log_dups_size":0,"ondisk_log_size":6104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24981504,"num_objects":6099,"num_object_clones":0,"num_object_copies":12198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6099,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3921,"num_write_kb":15684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":2075}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5931","reported_seq":8708,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:14:08.348841+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:14:08.348841+0000","last_peered":"2024-09-11T01:14:08.348841+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:14:08.348841+0000","last_undegraded":"2024-09-11T01:14:08.348841+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":5931,"log_dups_size":0,"ondisk_log_size":5931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24268800,"num_objects":5925,"num_object_clones":0,"num_object_copies":11850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2593,"num_objects_unfound":0,"num_objects_dirty":5925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3716,"num_write_kb":14864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1332,"num_bytes_recovered":5455872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2593},{"shards":"2","objects":3332}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'5901","reported_seq":6025,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:04.058336+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:14:04.058336+0000","last_peered":"2024-09-11T01:14:04.058336+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:14:04.058336+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":5901,"log_dups_size":0,"ondisk_log_size":5901,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24162304,"num_objects":5899,"num_object_clones":0,"num_object_copies":11798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5899,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2638,"num_write_kb":10552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1949},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'5874","reported_seq":6004,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:04.077981+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:14:04.077981+0000","last_peered":"2024-09-11T01:14:04.077981+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:14:04.077981+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":5874,"log_dups_size":0,"ondisk_log_size":5874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24027136,"num_objects":5866,"num_object_clones":0,"num_object_copies":11732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2529,"num_write_kb":10116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1927},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658057+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658057+0000","last_peered":"2024-09-11T01:13:09.658057+0000","last_clean":"2024-09-11T01:13:09.658057+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658057+0000","last_undegraded":"2024-09-11T01:13:09.658057+0000","last_fullsized":"2024-09-11T01:13:09.658057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'5992","reported_seq":6120,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:04.051241+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:14:04.051241+0000","last_peered":"2024-09-11T01:14:04.051241+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:14:04.051241+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":5992,"log_dups_size":0,"ondisk_log_size":5992,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24526848,"num_objects":5988,"num_object_clones":0,"num_object_copies":11976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2601,"num_write_kb":10404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1949},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'5906","reported_seq":6034,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:04.067901+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:14:04.067901+0000","last_peered":"2024-09-11T01:14:04.067901+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:14:04.067901+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":5906,"log_dups_size":0,"ondisk_log_size":5906,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24142441,"num_objects":5895,"num_object_clones":0,"num_object_copies":11790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2976,"num_write_kb":11904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1978},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":390640233,"num_objects":95372,"num_object_clones":0,"num_object_copies":190744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":2593,"num_objects_unfound":0,"num_objects_dirty":95372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":56303,"num_write_kb":225212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1333,"num_bytes_recovered":5459968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":789516288,"data_stored":789429313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":95453,"ondisk_log_size":95453,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182431,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":531308,"kb_used_data":346972,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93840532,"statfs":{"total":96636764160,"available":96092704768,"internally_reserved":0,"allocated":355299328,"data_stored":355050509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":188739243},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,3,1,2,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986204,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493016,"kb_used_data":321160,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93878824,"statfs":{"total":96636764160,"available":96131915776,"internally_reserved":0,"allocated":328867840,"data_stored":328623098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":175959722},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,3,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659149,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":219968,"kb_used_data":102256,"kb_used_omap":4,"kb_used_meta":117691,"kb_avail":94151872,"statfs":{"total":96636764160,"available":96411516928,"internally_reserved":0,"allocated":104710144,"data_stored":104384787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":120516271},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,3,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":355536896,"data_stored":355533417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":104964096,"data_stored":104884079,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":329015296,"data_stored":329011817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:14:11.376 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 107 19 98392 98373 3.59014 2.15234 0.0132902 0.0217521 2024-09-11T01:14:12.376 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 108 20 98942 98922 3.57676 2.14453 0.00894361 0.0218322 2024-09-11T01:14:13.377 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 109 20 99493 99473 3.56369 2.15234 0.00892997 0.0219148 2024-09-11T01:14:13.404 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:14:14.179 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:14:14.179 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:14:14.234 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":131,"stamp":"2024-09-11T01:14:12.575464+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":397533820,"num_objects":97057,"num_object_clones":0,"num_object_copies":194114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":2593,"num_objects_unfound":0,"num_objects_dirty":97057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57988,"num_write_kb":231946,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1334,"num_bytes_recovered":5459987,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97139,"ondisk_log_size":97139,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1256628,"kb_used_data":786880,"kb_used_omap":12,"kb_used_meta":469683,"kb_avail":281858892,"statfs":{"total":289910292480,"available":288623505408,"internally_reserved":0,"allocated":805765120,"data_stored":804946202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":480955396},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,3,12,11,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22048768,"num_objects":5383,"num_object_clones":0,"num_object_copies":10766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-261,"num_objects_unfound":0,"num_objects_dirty":5383,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5383,"num_write_kb":21532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":1069056,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003098"},"pg_stats":[{"pgid":"2.f","version":"43'6134","reported_seq":6205,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:09.134522+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:14:09.134522+0000","last_peered":"2024-09-11T01:14:09.134522+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:14:09.134522+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":6134,"log_dups_size":0,"ondisk_log_size":6134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25124864,"num_objects":6134,"num_object_clones":0,"num_object_copies":12268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6134,"num_write_kb":24536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2062},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6164","reported_seq":6260,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:08.324639+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:14:08.324639+0000","last_peered":"2024-09-11T01:14:08.324639+0000","last_clean":"2024-09-11T01:14:08.324639+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:14:08.324639+0000","last_undegraded":"2024-09-11T01:14:08.324639+0000","last_fullsized":"2024-09-11T01:14:08.324639+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":6164,"log_dups_size":0,"ondisk_log_size":6164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25223168,"num_objects":6158,"num_object_clones":0,"num_object_copies":12316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4403,"num_write_kb":17612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6000","reported_seq":6122,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:08.320173+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:14:08.320173+0000","last_peered":"2024-09-11T01:14:08.320173+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:14:08.320173+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":6000,"log_dups_size":0,"ondisk_log_size":6000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24559616,"num_objects":5996,"num_object_clones":0,"num_object_copies":11992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4301,"num_write_kb":17204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":2100}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6066","reported_seq":6130,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:08.333245+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:14:08.333245+0000","last_peered":"2024-09-11T01:14:08.333245+0000","last_clean":"2024-09-11T01:14:08.333245+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:14:08.333245+0000","last_undegraded":"2024-09-11T01:14:08.333245+0000","last_fullsized":"2024-09-11T01:14:08.333245+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":6066,"log_dups_size":0,"ondisk_log_size":6066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24846336,"num_objects":6066,"num_object_clones":0,"num_object_copies":12132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6066,"num_write_kb":24264,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6099","reported_seq":6222,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:09.131045+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:14:09.131045+0000","last_peered":"2024-09-11T01:14:09.131045+0000","last_clean":"2024-09-11T01:14:09.131045+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:14:09.131045+0000","last_undegraded":"2024-09-11T01:14:09.131045+0000","last_fullsized":"2024-09-11T01:14:09.131045+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":6099,"log_dups_size":0,"ondisk_log_size":6099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24952832,"num_objects":6092,"num_object_clones":0,"num_object_copies":12184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6092,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3104,"num_write_kb":12416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6074","reported_seq":6239,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:09.022114+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:14:09.022114+0000","last_peered":"2024-09-11T01:14:09.022114+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:14:09.022114+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":6074,"log_dups_size":0,"ondisk_log_size":6074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24858624,"num_objects":6069,"num_object_clones":0,"num_object_copies":12138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2415,"num_write_kb":9660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2118},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'5976","reported_seq":6101,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:09.101436+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:14:09.101436+0000","last_peered":"2024-09-11T01:14:09.101436+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:14:09.101436+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":5976,"log_dups_size":0,"ondisk_log_size":5976,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24453120,"num_objects":5970,"num_object_clones":0,"num_object_copies":11940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5970,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2833,"num_write_kb":11332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1984},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6161","reported_seq":6286,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:09.095313+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:14:09.095313+0000","last_peered":"2024-09-11T01:14:09.095313+0000","last_clean":"2024-09-11T01:14:09.095313+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:14:09.095313+0000","last_undegraded":"2024-09-11T01:14:09.095313+0000","last_fullsized":"2024-09-11T01:14:09.095313+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":6161,"log_dups_size":0,"ondisk_log_size":6161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25227264,"num_objects":6159,"num_object_clones":0,"num_object_copies":12318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3037,"num_write_kb":12148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6053","reported_seq":6175,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:08.370225+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:14:08.370225+0000","last_peered":"2024-09-11T01:14:08.370225+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:14:08.370225+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":6053,"log_dups_size":0,"ondisk_log_size":6053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24764416,"num_objects":6046,"num_object_clones":0,"num_object_copies":12092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6046,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4024,"num_write_kb":16096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":2113}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741858+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741858+0000","last_peered":"2024-09-11T01:13:09.741858+0000","last_clean":"2024-09-11T01:13:09.741858+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741858+0000","last_undegraded":"2024-09-11T01:13:09.741858+0000","last_fullsized":"2024-09-11T01:13:09.741858+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6027","reported_seq":6150,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:09.125101+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:14:09.125101+0000","last_peered":"2024-09-11T01:14:09.125101+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:14:09.125101+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":6027,"log_dups_size":0,"ondisk_log_size":6027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24653824,"num_objects":6019,"num_object_clones":0,"num_object_copies":12038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2614,"num_write_kb":10456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2035},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.910012+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.910012+0000","last_peered":"2024-09-11T01:13:09.910012+0000","last_clean":"2024-09-11T01:13:09.910012+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.910012+0000","last_undegraded":"2024-09-11T01:13:09.910012+0000","last_fullsized":"2024-09-11T01:13:09.910012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6104","reported_seq":6217,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:08.348142+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:14:08.348142+0000","last_peered":"2024-09-11T01:14:08.348142+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:14:08.348142+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":6104,"log_dups_size":0,"ondisk_log_size":6104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24981504,"num_objects":6099,"num_object_clones":0,"num_object_copies":12198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6099,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3921,"num_write_kb":15684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":2075}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'5931","reported_seq":8708,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:14:08.348841+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:14:08.348841+0000","last_peered":"2024-09-11T01:14:08.348841+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:14:08.348841+0000","last_undegraded":"2024-09-11T01:14:08.348841+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":5931,"log_dups_size":0,"ondisk_log_size":5931,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24268800,"num_objects":5925,"num_object_clones":0,"num_object_copies":11850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2593,"num_objects_unfound":0,"num_objects_dirty":5925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3716,"num_write_kb":14864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1332,"num_bytes_recovered":5455872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2593},{"shards":"2","objects":3332}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6081","reported_seq":6207,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:09.106151+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:14:09.106151+0000","last_peered":"2024-09-11T01:14:09.106151+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:14:09.106151+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":6081,"log_dups_size":0,"ondisk_log_size":6081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24899584,"num_objects":6079,"num_object_clones":0,"num_object_copies":12158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6079,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2818,"num_write_kb":11272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2129},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6024","reported_seq":6155,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:09.085181+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:14:09.085181+0000","last_peered":"2024-09-11T01:14:09.085181+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:14:09.085181+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":6024,"log_dups_size":0,"ondisk_log_size":6024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24641536,"num_objects":6016,"num_object_clones":0,"num_object_copies":12032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2679,"num_write_kb":10716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2077},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658057+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658057+0000","last_peered":"2024-09-11T01:13:09.658057+0000","last_clean":"2024-09-11T01:13:09.658057+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658057+0000","last_undegraded":"2024-09-11T01:13:09.658057+0000","last_fullsized":"2024-09-11T01:13:09.658057+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6156","reported_seq":6283,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:09.095301+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:14:09.095301+0000","last_peered":"2024-09-11T01:14:09.095301+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:14:09.095301+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":6156,"log_dups_size":0,"ondisk_log_size":6156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25198592,"num_objects":6152,"num_object_clones":0,"num_object_copies":12304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2765,"num_write_kb":11060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2113},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6086","reported_seq":6215,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:09.095282+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:14:09.095282+0000","last_peered":"2024-09-11T01:14:09.095282+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:14:09.095282+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":6086,"log_dups_size":0,"ondisk_log_size":6086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24879721,"num_objects":6075,"num_object_clones":0,"num_object_copies":12150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3156,"num_write_kb":12624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2158},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":397533801,"num_objects":97055,"num_object_clones":0,"num_object_copies":194110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":2593,"num_objects_unfound":0,"num_objects_dirty":97055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":57986,"num_write_kb":231944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1333,"num_bytes_recovered":5459968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":806604800,"data_stored":806517825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97136,"ondisk_log_size":97136,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182432,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":526916,"kb_used_data":355060,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93844924,"statfs":{"total":96636764160,"available":96097202176,"internally_reserved":0,"allocated":363581440,"data_stored":363332621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":175959723},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,6,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986204,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":493016,"kb_used_data":321160,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93878824,"statfs":{"total":96636764160,"available":96131915776,"internally_reserved":0,"allocated":328867840,"data_stored":328623098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":175959722},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,3,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659150,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":236696,"kb_used_data":110660,"kb_used_omap":4,"kb_used_meta":126011,"kb_avail":94135144,"statfs":{"total":96636764160,"available":96394387456,"internally_reserved":0,"allocated":113315840,"data_stored":112990483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":129035951},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":363827200,"data_stored":363823721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":113762304,"data_stored":113682287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":329015296,"data_stored":329011817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:14:14.236 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:14:14.377 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 110 20 99996 99976 3.54916 1.96484 0.0876951 0.0220029 2024-09-11T01:14:15.008 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:14:15.008 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:14:15.070 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":132,"stamp":"2024-09-11T01:14:14.575941+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":408617596,"num_objects":99763,"num_object_clones":0,"num_object_copies":199526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":2447,"num_objects_unfound":0,"num_objects_dirty":99763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60694,"num_write_kb":242770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1480,"num_bytes_recovered":6058003,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99845,"ondisk_log_size":99845,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1286592,"kb_used_data":800204,"kb_used_omap":12,"kb_used_meta":486323,"kb_avail":281828928,"statfs":{"total":289910292480,"available":288592822272,"internally_reserved":0,"allocated":819408896,"data_stored":818589978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":497994756},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,6,4,11,13],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":33132544,"num_objects":8089,"num_object_clones":0,"num_object_copies":16178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-407,"num_objects_unfound":0,"num_objects_dirty":8089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8089,"num_write_kb":32356,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":407,"num_bytes_recovered":1667072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003121"},"pg_stats":[{"pgid":"2.f","version":"43'6309","reported_seq":6381,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:14.130481+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:14:14.130481+0000","last_peered":"2024-09-11T01:14:14.130481+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:14:14.130481+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":6309,"log_dups_size":0,"ondisk_log_size":6309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25841664,"num_objects":6309,"num_object_clones":0,"num_object_copies":12618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6309,"num_write_kb":25236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2237},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6343","reported_seq":6439,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:13.338632+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:14:13.338632+0000","last_peered":"2024-09-11T01:14:13.338632+0000","last_clean":"2024-09-11T01:14:13.338632+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:14:13.338632+0000","last_undegraded":"2024-09-11T01:14:13.338632+0000","last_fullsized":"2024-09-11T01:14:13.338632+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":6343,"log_dups_size":0,"ondisk_log_size":6343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25956352,"num_objects":6337,"num_object_clones":0,"num_object_copies":12674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4582,"num_write_kb":18328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6171","reported_seq":6293,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:13.421408+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:14:13.421408+0000","last_peered":"2024-09-11T01:14:13.421408+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:14:13.421408+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":6171,"log_dups_size":0,"ondisk_log_size":6171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25260032,"num_objects":6167,"num_object_clones":0,"num_object_copies":12334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4472,"num_write_kb":17888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":2271}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6223","reported_seq":6287,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:13.416751+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:14:13.416751+0000","last_peered":"2024-09-11T01:14:13.416751+0000","last_clean":"2024-09-11T01:14:13.416751+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:14:13.416751+0000","last_undegraded":"2024-09-11T01:14:13.416751+0000","last_fullsized":"2024-09-11T01:14:13.416751+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":6223,"log_dups_size":0,"ondisk_log_size":6223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25489408,"num_objects":6223,"num_object_clones":0,"num_object_copies":12446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6223,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6223,"num_write_kb":24892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6276","reported_seq":6399,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:14.130712+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:14:14.130712+0000","last_peered":"2024-09-11T01:14:14.130712+0000","last_clean":"2024-09-11T01:14:14.130712+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:14:14.130712+0000","last_undegraded":"2024-09-11T01:14:14.130712+0000","last_fullsized":"2024-09-11T01:14:14.130712+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":6276,"log_dups_size":0,"ondisk_log_size":6276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25677824,"num_objects":6269,"num_object_clones":0,"num_object_copies":12538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3281,"num_write_kb":13124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6251","reported_seq":6416,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:14.095421+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:14:14.095421+0000","last_peered":"2024-09-11T01:14:14.095421+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:14:14.095421+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":6251,"log_dups_size":0,"ondisk_log_size":6251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25583616,"num_objects":6246,"num_object_clones":0,"num_object_copies":12492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6246,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2592,"num_write_kb":10368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2295},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6151","reported_seq":6276,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:14.132346+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:14:14.132346+0000","last_peered":"2024-09-11T01:14:14.132346+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:14:14.132346+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":6151,"log_dups_size":0,"ondisk_log_size":6151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25169920,"num_objects":6145,"num_object_clones":0,"num_object_copies":12290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3008,"num_write_kb":12032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2159},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6315","reported_seq":6440,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:14.065589+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:14:14.065589+0000","last_peered":"2024-09-11T01:14:14.065589+0000","last_clean":"2024-09-11T01:14:14.065589+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:14:14.065589+0000","last_undegraded":"2024-09-11T01:14:14.065589+0000","last_fullsized":"2024-09-11T01:14:14.065589+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":6315,"log_dups_size":0,"ondisk_log_size":6315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25858048,"num_objects":6313,"num_object_clones":0,"num_object_copies":12626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3191,"num_write_kb":12764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6216","reported_seq":6338,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:13.431227+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:14:13.431227+0000","last_peered":"2024-09-11T01:14:13.431227+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:14:13.431227+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":6216,"log_dups_size":0,"ondisk_log_size":6216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25432064,"num_objects":6209,"num_object_clones":0,"num_object_copies":12418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4187,"num_write_kb":16748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":2276}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741922+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741922+0000","last_peered":"2024-09-11T01:13:09.741922+0000","last_clean":"2024-09-11T01:13:09.741922+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741922+0000","last_undegraded":"2024-09-11T01:13:09.741922+0000","last_fullsized":"2024-09-11T01:13:09.741922+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6211","reported_seq":6334,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:14.142448+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:14:14.142448+0000","last_peered":"2024-09-11T01:14:14.142448+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:14:14.142448+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":6211,"log_dups_size":0,"ondisk_log_size":6211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25407488,"num_objects":6203,"num_object_clones":0,"num_object_copies":12406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2798,"num_write_kb":11192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2219},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.910012+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.910012+0000","last_peered":"2024-09-11T01:13:09.910012+0000","last_clean":"2024-09-11T01:13:09.910012+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.910012+0000","last_undegraded":"2024-09-11T01:13:09.910012+0000","last_fullsized":"2024-09-11T01:13:09.910012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6284","reported_seq":6398,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:13.434340+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:14:13.434340+0000","last_peered":"2024-09-11T01:14:13.434340+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:14:13.434340+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":6284,"log_dups_size":0,"ondisk_log_size":6284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25718784,"num_objects":6279,"num_object_clones":0,"num_object_copies":12558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4101,"num_write_kb":16404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":2255}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6098","reported_seq":9168,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:14:13.416532+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:14:13.416532+0000","last_peered":"2024-09-11T01:14:13.416532+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:14:13.416532+0000","last_undegraded":"2024-09-11T01:14:13.416532+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":6098,"log_dups_size":0,"ondisk_log_size":6098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24952832,"num_objects":6092,"num_object_clones":0,"num_object_copies":12184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2447,"num_objects_unfound":0,"num_objects_dirty":6092,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3883,"num_write_kb":15532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1478,"num_bytes_recovered":6053888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2447},{"shards":"2","objects":3645}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6277","reported_seq":6398,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:14.112477+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:14:14.112477+0000","last_peered":"2024-09-11T01:14:14.112477+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:14:14.112477+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":6277,"log_dups_size":0,"ondisk_log_size":6277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25702400,"num_objects":6275,"num_object_clones":0,"num_object_copies":12550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6275,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3014,"num_write_kb":12056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2325},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6169","reported_seq":6301,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:14.150742+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:14:14.150742+0000","last_peered":"2024-09-11T01:14:14.150742+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:14:14.150742+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":6169,"log_dups_size":0,"ondisk_log_size":6169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25235456,"num_objects":6161,"num_object_clones":0,"num_object_copies":12322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6161,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2824,"num_write_kb":11296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2222},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658174+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658174+0000","last_peered":"2024-09-11T01:13:09.658174+0000","last_clean":"2024-09-11T01:13:09.658174+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658174+0000","last_undegraded":"2024-09-11T01:13:09.658174+0000","last_fullsized":"2024-09-11T01:13:09.658174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6328","reported_seq":6457,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:14.169845+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:14:14.169845+0000","last_peered":"2024-09-11T01:14:14.169845+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:14:14.169845+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":6328,"log_dups_size":0,"ondisk_log_size":6328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25903104,"num_objects":6324,"num_object_clones":0,"num_object_copies":12648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2937,"num_write_kb":11748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2285},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6220","reported_seq":6349,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:14.105028+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:14:14.105028+0000","last_peered":"2024-09-11T01:14:14.105028+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:14:14.105028+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":6220,"log_dups_size":0,"ondisk_log_size":6220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25428585,"num_objects":6209,"num_object_clones":0,"num_object_copies":12418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3290,"num_write_kb":13160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2292},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":408617577,"num_objects":99761,"num_object_clones":0,"num_object_copies":199522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":2447,"num_objects_unfound":0,"num_objects_dirty":99761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60692,"num_write_kb":242768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1479,"num_bytes_recovered":6057984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":820416512,"data_stored":820329537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99842,"ondisk_log_size":99842,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182433,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":543204,"kb_used_data":363028,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93828636,"statfs":{"total":96636764160,"available":96080523264,"internally_reserved":0,"allocated":371740672,"data_stored":371491853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":184479403},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,4,1,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986205,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506692,"kb_used_data":326516,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93865148,"statfs":{"total":96636764160,"available":96117911552,"internally_reserved":0,"allocated":334352384,"data_stored":334107642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184479402},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,0,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659150,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":236696,"kb_used_data":110660,"kb_used_omap":4,"kb_used_meta":126011,"kb_avail":94135144,"statfs":{"total":96636764160,"available":96394387456,"internally_reserved":0,"allocated":113315840,"data_stored":112990483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":129035951},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":372068352,"data_stored":372064873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":113762304,"data_stored":113682287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":334585856,"data_stored":334582377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:14:15.072 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:14:15.072 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'6309", 'reported_seq': 6381, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:14.130481+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:14:14.130481+0000', 'last_peered': '2024-09-11T01:14:14.130481+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:14:14.130481+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 6309, 'log_dups_size': 0, 'ondisk_log_size': 6309, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25841664, 'num_objects': 6309, 'num_object_clones': 0, 'num_object_copies': 12618, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6309, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6309, 'num_write_kb': 25236, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2237}, {'shards': '0,2', 'objects': 4072}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:15.072 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:14:15.072 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'6171", 'reported_seq': 6293, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:13.421408+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:14:13.421408+0000', 'last_peered': '2024-09-11T01:14:13.421408+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:14:13.421408+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 6171, 'log_dups_size': 0, 'ondisk_log_size': 6171, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25260032, 'num_objects': 6167, 'num_object_clones': 0, 'num_object_copies': 12334, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6167, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4472, 'num_write_kb': 17888, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3896}, {'shards': '2', 'objects': 2271}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:14:15.072 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:14:15.072 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'6251", 'reported_seq': 6416, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:14.095421+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:14:14.095421+0000', 'last_peered': '2024-09-11T01:14:14.095421+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:14:14.095421+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 6251, 'log_dups_size': 0, 'ondisk_log_size': 6251, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25583616, 'num_objects': 6246, 'num_object_clones': 0, 'num_object_copies': 12492, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6246, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2592, 'num_write_kb': 10368, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2295}, {'shards': '0,2', 'objects': 3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:15.073 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:14:15.073 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'6151", 'reported_seq': 6276, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:14.132346+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:14:14.132346+0000', 'last_peered': '2024-09-11T01:14:14.132346+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:14:14.132346+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 6151, 'log_dups_size': 0, 'ondisk_log_size': 6151, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25169920, 'num_objects': 6145, 'num_object_clones': 0, 'num_object_copies': 12290, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6145, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3008, 'num_write_kb': 12032, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2159}, {'shards': '0,2', 'objects': 3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:15.073 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:14:15.073 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'6216", 'reported_seq': 6338, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:13.431227+0000', 'last_change': '2024-09-11T01:13:10.159069+0000', 'last_active': '2024-09-11T01:14:13.431227+0000', 'last_peered': '2024-09-11T01:14:13.431227+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:14:13.431227+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 6216, 'log_dups_size': 0, 'ondisk_log_size': 6216, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25432064, 'num_objects': 6209, 'num_object_clones': 0, 'num_object_copies': 12418, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6209, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4187, 'num_write_kb': 16748, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3932}, {'shards': '2', 'objects': 2276}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:14:15.073 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:14:15.073 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'6211", 'reported_seq': 6334, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:14.142448+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:14:14.142448+0000', 'last_peered': '2024-09-11T01:14:14.142448+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:14:14.142448+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 6211, 'log_dups_size': 0, 'ondisk_log_size': 6211, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25407488, 'num_objects': 6203, 'num_object_clones': 0, 'num_object_copies': 12406, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6203, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2798, 'num_write_kb': 11192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2219}, {'shards': '0,2', 'objects': 3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:15.073 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:14:15.073 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'6284", 'reported_seq': 6398, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:13.434340+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:14:13.434340+0000', 'last_peered': '2024-09-11T01:14:13.434340+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:14:13.434340+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 6284, 'log_dups_size': 0, 'ondisk_log_size': 6284, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25718784, 'num_objects': 6279, 'num_object_clones': 0, 'num_object_copies': 12558, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6279, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4101, 'num_write_kb': 16404, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4023}, {'shards': '2', 'objects': 2255}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:14:15.073 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:14:15.073 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'6098", 'reported_seq': 9168, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:14:13.416532+0000', 'last_change': '2024-09-11T01:13:10.217540+0000', 'last_active': '2024-09-11T01:14:13.416532+0000', 'last_peered': '2024-09-11T01:14:13.416532+0000', 'last_clean': '2024-09-11T01:13:07.638709+0000', 'last_became_active': '2024-09-11T01:13:10.125604+0000', 'last_became_peered': '2024-09-11T01:13:10.125604+0000', 'last_unstale': '2024-09-11T01:14:13.416532+0000', 'last_undegraded': '2024-09-11T01:14:13.416532+0000', 'last_fullsized': '2024-09-11T01:13:09.649604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2269", 'last_scrub_stamp': '2024-09-11T01:12:42.890920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:42.890920+0000', 'objects_scrubbed': 2209, 'log_size': 6098, 'log_dups_size': 0, 'ondisk_log_size': 6098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 999, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24952832, 'num_objects': 6092, 'num_object_clones': 0, 'num_object_copies': 12184, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2447, 'num_objects_unfound': 0, 'num_objects_dirty': 6092, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3883, 'num_write_kb': 15532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1478, 'num_bytes_recovered': 6053888, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2447}, {'shards': '2', 'objects': 3645}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:14:15.074 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:14:15.074 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'6277", 'reported_seq': 6398, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:14.112477+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:14:14.112477+0000', 'last_peered': '2024-09-11T01:14:14.112477+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:14:14.112477+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 6277, 'log_dups_size': 0, 'ondisk_log_size': 6277, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25702400, 'num_objects': 6275, 'num_object_clones': 0, 'num_object_copies': 12550, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6275, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3014, 'num_write_kb': 12056, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2325}, {'shards': '0,2', 'objects': 3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:15.074 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:14:15.074 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'6169", 'reported_seq': 6301, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:14.150742+0000', 'last_change': '2024-09-11T01:13:10.209668+0000', 'last_active': '2024-09-11T01:14:14.150742+0000', 'last_peered': '2024-09-11T01:14:14.150742+0000', 'last_clean': '2024-09-11T01:13:07.624565+0000', 'last_became_active': '2024-09-11T01:13:10.199795+0000', 'last_became_peered': '2024-09-11T01:13:10.199795+0000', 'last_unstale': '2024-09-11T01:14:14.150742+0000', 'last_undegraded': '2024-09-11T01:13:09.791198+0000', 'last_fullsized': '2024-09-11T01:13:09.652311+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3415", 'last_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'objects_scrubbed': 3337, 'log_size': 6169, 'log_dups_size': 0, 'ondisk_log_size': 6169, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1975, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25235456, 'num_objects': 6161, 'num_object_clones': 0, 'num_object_copies': 12322, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3942, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6161, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2824, 'num_write_kb': 11296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2222}, {'shards': '0,2', 'objects': 3939}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:15.074 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:14:15.074 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'6328", 'reported_seq': 6457, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:14.169845+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:14:14.169845+0000', 'last_peered': '2024-09-11T01:14:14.169845+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:14:14.169845+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 6328, 'log_dups_size': 0, 'ondisk_log_size': 6328, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25903104, 'num_objects': 6324, 'num_object_clones': 0, 'num_object_copies': 12648, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6324, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2937, 'num_write_kb': 11748, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2285}, {'shards': '0,2', 'objects': 4039}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:15.074 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:14:15.074 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'6220", 'reported_seq': 6349, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:14.105028+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:14:14.105028+0000', 'last_peered': '2024-09-11T01:14:14.105028+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:14:14.105028+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 6220, 'log_dups_size': 0, 'ondisk_log_size': 6220, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25428585, 'num_objects': 6209, 'num_object_clones': 0, 'num_object_copies': 12418, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6209, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3290, 'num_write_kb': 13160, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2292}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:15.074 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:14:15.377 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 111 20 100494 100474 3.53471 1.94531 0.00643486 0.0220917 2024-09-11T01:14:16.030 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:14:16.030 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43711/199526 objects degraded (21.907%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":192,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":99763,"data_bytes":408617596,"bytes_used":1317470208,"bytes_avail":288592822272,"bytes_total":289910292480,"degraded_objects":43711,"degraded_total":199526,"degraded_ratio":0.21907420586790693,"misplaced_objects":2447,"misplaced_total":199526,"misplaced_ratio":0.012264065836031394,"recovering_objects_per_sec":33,"recovering_bytes_per_sec":138886,"recovering_keys_per_sec":0,"num_objects_recovered":407,"num_bytes_recovered":1667072,"num_keys_recovered":0,"write_bytes_sec":2760327,"read_op_per_sec":0,"write_op_per_sec":673},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:14:16.088 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:14:16.088 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:14:16.379 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 112 20 100983 100963 3.52017 1.91016 0.0101127 0.022185 2024-09-11T01:14:17.095 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:14:17.096 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:14:17.183 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":133,"stamp":"2024-09-11T01:14:16.576394+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":408617596,"num_objects":99763,"num_object_clones":0,"num_object_copies":199526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":2447,"num_objects_unfound":0,"num_objects_dirty":99763,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60694,"num_write_kb":242770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1480,"num_bytes_recovered":6058003,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99845,"ondisk_log_size":99845,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1286592,"kb_used_data":800204,"kb_used_omap":12,"kb_used_meta":486323,"kb_avail":281828928,"statfs":{"total":289910292480,"available":288592822272,"internally_reserved":0,"allocated":819408896,"data_stored":818589978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":497994756},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,6,4,11,13],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22114304,"num_objects":5399,"num_object_clones":0,"num_object_copies":10798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-283,"num_objects_unfound":0,"num_objects_dirty":5399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5399,"num_write_kb":21596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":1159168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003152"},"pg_stats":[{"pgid":"2.f","version":"43'6309","reported_seq":6381,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:14.130481+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:14:14.130481+0000","last_peered":"2024-09-11T01:14:14.130481+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:14:14.130481+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":6309,"log_dups_size":0,"ondisk_log_size":6309,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25841664,"num_objects":6309,"num_object_clones":0,"num_object_copies":12618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6309,"num_write_kb":25236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2237},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6343","reported_seq":6439,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:13.338632+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:14:13.338632+0000","last_peered":"2024-09-11T01:14:13.338632+0000","last_clean":"2024-09-11T01:14:13.338632+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:14:13.338632+0000","last_undegraded":"2024-09-11T01:14:13.338632+0000","last_fullsized":"2024-09-11T01:14:13.338632+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":6343,"log_dups_size":0,"ondisk_log_size":6343,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25956352,"num_objects":6337,"num_object_clones":0,"num_object_copies":12674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4582,"num_write_kb":18328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6171","reported_seq":6293,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:13.421408+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:14:13.421408+0000","last_peered":"2024-09-11T01:14:13.421408+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:14:13.421408+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":6171,"log_dups_size":0,"ondisk_log_size":6171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25260032,"num_objects":6167,"num_object_clones":0,"num_object_copies":12334,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6167,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4472,"num_write_kb":17888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":2271}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6223","reported_seq":6287,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:13.416751+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:14:13.416751+0000","last_peered":"2024-09-11T01:14:13.416751+0000","last_clean":"2024-09-11T01:14:13.416751+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:14:13.416751+0000","last_undegraded":"2024-09-11T01:14:13.416751+0000","last_fullsized":"2024-09-11T01:14:13.416751+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":6223,"log_dups_size":0,"ondisk_log_size":6223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25489408,"num_objects":6223,"num_object_clones":0,"num_object_copies":12446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6223,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6223,"num_write_kb":24892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6276","reported_seq":6399,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:14.130712+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:14:14.130712+0000","last_peered":"2024-09-11T01:14:14.130712+0000","last_clean":"2024-09-11T01:14:14.130712+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:14:14.130712+0000","last_undegraded":"2024-09-11T01:14:14.130712+0000","last_fullsized":"2024-09-11T01:14:14.130712+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":6276,"log_dups_size":0,"ondisk_log_size":6276,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25677824,"num_objects":6269,"num_object_clones":0,"num_object_copies":12538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3281,"num_write_kb":13124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6251","reported_seq":6416,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:14.095421+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:14:14.095421+0000","last_peered":"2024-09-11T01:14:14.095421+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:14:14.095421+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":6251,"log_dups_size":0,"ondisk_log_size":6251,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25583616,"num_objects":6246,"num_object_clones":0,"num_object_copies":12492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6246,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2592,"num_write_kb":10368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2295},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6151","reported_seq":6276,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:14.132346+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:14:14.132346+0000","last_peered":"2024-09-11T01:14:14.132346+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:14:14.132346+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":6151,"log_dups_size":0,"ondisk_log_size":6151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25169920,"num_objects":6145,"num_object_clones":0,"num_object_copies":12290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3008,"num_write_kb":12032,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2159},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6315","reported_seq":6440,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:14.065589+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:14:14.065589+0000","last_peered":"2024-09-11T01:14:14.065589+0000","last_clean":"2024-09-11T01:14:14.065589+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:14:14.065589+0000","last_undegraded":"2024-09-11T01:14:14.065589+0000","last_fullsized":"2024-09-11T01:14:14.065589+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":6315,"log_dups_size":0,"ondisk_log_size":6315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25858048,"num_objects":6313,"num_object_clones":0,"num_object_copies":12626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6313,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3191,"num_write_kb":12764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6216","reported_seq":6338,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:13.431227+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:14:13.431227+0000","last_peered":"2024-09-11T01:14:13.431227+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:14:13.431227+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":6216,"log_dups_size":0,"ondisk_log_size":6216,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25432064,"num_objects":6209,"num_object_clones":0,"num_object_copies":12418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4187,"num_write_kb":16748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":2276}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741922+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741922+0000","last_peered":"2024-09-11T01:13:09.741922+0000","last_clean":"2024-09-11T01:13:09.741922+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741922+0000","last_undegraded":"2024-09-11T01:13:09.741922+0000","last_fullsized":"2024-09-11T01:13:09.741922+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6211","reported_seq":6334,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:14.142448+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:14:14.142448+0000","last_peered":"2024-09-11T01:14:14.142448+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:14:14.142448+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":6211,"log_dups_size":0,"ondisk_log_size":6211,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25407488,"num_objects":6203,"num_object_clones":0,"num_object_copies":12406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6203,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2798,"num_write_kb":11192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2219},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.910012+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.910012+0000","last_peered":"2024-09-11T01:13:09.910012+0000","last_clean":"2024-09-11T01:13:09.910012+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.910012+0000","last_undegraded":"2024-09-11T01:13:09.910012+0000","last_fullsized":"2024-09-11T01:13:09.910012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6284","reported_seq":6398,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:13.434340+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:14:13.434340+0000","last_peered":"2024-09-11T01:14:13.434340+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:14:13.434340+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":6284,"log_dups_size":0,"ondisk_log_size":6284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25718784,"num_objects":6279,"num_object_clones":0,"num_object_copies":12558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4101,"num_write_kb":16404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":2255}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6098","reported_seq":9168,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:14:13.416532+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:14:13.416532+0000","last_peered":"2024-09-11T01:14:13.416532+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:14:13.416532+0000","last_undegraded":"2024-09-11T01:14:13.416532+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":6098,"log_dups_size":0,"ondisk_log_size":6098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24952832,"num_objects":6092,"num_object_clones":0,"num_object_copies":12184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2447,"num_objects_unfound":0,"num_objects_dirty":6092,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3883,"num_write_kb":15532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1478,"num_bytes_recovered":6053888,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2447},{"shards":"2","objects":3645}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6277","reported_seq":6398,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:14.112477+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:14:14.112477+0000","last_peered":"2024-09-11T01:14:14.112477+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:14:14.112477+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":6277,"log_dups_size":0,"ondisk_log_size":6277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25702400,"num_objects":6275,"num_object_clones":0,"num_object_copies":12550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6275,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3014,"num_write_kb":12056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2325},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6169","reported_seq":6301,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:14.150742+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:14:14.150742+0000","last_peered":"2024-09-11T01:14:14.150742+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:14:14.150742+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":6169,"log_dups_size":0,"ondisk_log_size":6169,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25235456,"num_objects":6161,"num_object_clones":0,"num_object_copies":12322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6161,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2824,"num_write_kb":11296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2222},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658174+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658174+0000","last_peered":"2024-09-11T01:13:09.658174+0000","last_clean":"2024-09-11T01:13:09.658174+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658174+0000","last_undegraded":"2024-09-11T01:13:09.658174+0000","last_fullsized":"2024-09-11T01:13:09.658174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6328","reported_seq":6457,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:14.169845+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:14:14.169845+0000","last_peered":"2024-09-11T01:14:14.169845+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:14:14.169845+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":6328,"log_dups_size":0,"ondisk_log_size":6328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25903104,"num_objects":6324,"num_object_clones":0,"num_object_copies":12648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2937,"num_write_kb":11748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2285},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6220","reported_seq":6349,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:14.105028+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:14:14.105028+0000","last_peered":"2024-09-11T01:14:14.105028+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:14:14.105028+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":6220,"log_dups_size":0,"ondisk_log_size":6220,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25428585,"num_objects":6209,"num_object_clones":0,"num_object_copies":12418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3290,"num_write_kb":13160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2292},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":408617577,"num_objects":99761,"num_object_clones":0,"num_object_copies":199522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":2447,"num_objects_unfound":0,"num_objects_dirty":99761,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":60692,"num_write_kb":242768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1479,"num_bytes_recovered":6057984,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":820416512,"data_stored":820329537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":99842,"ondisk_log_size":99842,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182433,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":543204,"kb_used_data":363028,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93828636,"statfs":{"total":96636764160,"available":96080523264,"internally_reserved":0,"allocated":371740672,"data_stored":371491853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":184479403},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,4,1,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986205,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506692,"kb_used_data":326516,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93865148,"statfs":{"total":96636764160,"available":96117911552,"internally_reserved":0,"allocated":334352384,"data_stored":334107642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184479402},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,0,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659150,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":236696,"kb_used_data":110660,"kb_used_omap":4,"kb_used_meta":126011,"kb_avail":94135144,"statfs":{"total":96636764160,"available":96394387456,"internally_reserved":0,"allocated":113315840,"data_stored":112990483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":129035951},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,3,5,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":372068352,"data_stored":372064873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":113762304,"data_stored":113682287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":334585856,"data_stored":334582377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:14:17.378 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 113 20 101508 101488 3.50717 2.05078 0.0179231 0.0222656 2024-09-11T01:14:18.379 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 114 20 102049 102029 3.49495 2.11328 0.0165512 0.0223441 2024-09-11T01:14:19.378 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 115 20 102585 102565 3.48276 2.09375 0.0182632 0.0224229 2024-09-11T01:14:20.185 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:14:20.378 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 116 20 103100 103080 3.47008 2.01172 0.034366 0.0225062 2024-09-11T01:14:21.006 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:14:21.008 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:14:21.050 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":135,"stamp":"2024-09-11T01:14:20.577319+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":419123836,"num_objects":102328,"num_object_clones":0,"num_object_copies":204656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":2319,"num_objects_unfound":0,"num_objects_dirty":102328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63259,"num_write_kb":253030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1608,"num_bytes_recovered":6582291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102410,"ondisk_log_size":102410,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1338104,"kb_used_data":821192,"kb_used_omap":12,"kb_used_meta":516851,"kb_avail":281777416,"statfs":{"total":289910292480,"available":288540073984,"internally_reserved":0,"allocated":840900608,"data_stored":840081690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":529255428},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3,4,4,10,15,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28483584,"num_objects":6954,"num_object_clones":0,"num_object_copies":13908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-274,"num_objects_unfound":0,"num_objects_dirty":6954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6954,"num_write_kb":27816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":1122304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002998"},"pg_stats":[{"pgid":"2.f","version":"43'6472","reported_seq":6545,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:19.173684+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:14:19.173684+0000","last_peered":"2024-09-11T01:14:19.173684+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:14:19.173684+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":6472,"log_dups_size":0,"ondisk_log_size":6472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26509312,"num_objects":6472,"num_object_clones":0,"num_object_copies":12944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6472,"num_write_kb":25888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2400},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6497","reported_seq":6593,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:18.439245+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:14:18.439245+0000","last_peered":"2024-09-11T01:14:18.439245+0000","last_clean":"2024-09-11T01:14:18.439245+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:14:18.439245+0000","last_undegraded":"2024-09-11T01:14:18.439245+0000","last_fullsized":"2024-09-11T01:14:18.439245+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":6497,"log_dups_size":0,"ondisk_log_size":6497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26587136,"num_objects":6491,"num_object_clones":0,"num_object_copies":12982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4736,"num_write_kb":18944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6338","reported_seq":6460,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:18.387925+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:14:18.387925+0000","last_peered":"2024-09-11T01:14:18.387925+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:14:18.387925+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":6338,"log_dups_size":0,"ondisk_log_size":6338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25944064,"num_objects":6334,"num_object_clones":0,"num_object_copies":12668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4639,"num_write_kb":18556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":2438}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6387","reported_seq":6451,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:18.457922+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:14:18.457922+0000","last_peered":"2024-09-11T01:14:18.457922+0000","last_clean":"2024-09-11T01:14:18.457922+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:14:18.457922+0000","last_undegraded":"2024-09-11T01:14:18.457922+0000","last_fullsized":"2024-09-11T01:14:18.457922+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":6387,"log_dups_size":0,"ondisk_log_size":6387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26161152,"num_objects":6387,"num_object_clones":0,"num_object_copies":12774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6387,"num_write_kb":25548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6444","reported_seq":6567,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:19.035506+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:14:19.035506+0000","last_peered":"2024-09-11T01:14:19.035506+0000","last_clean":"2024-09-11T01:14:19.035506+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:14:19.035506+0000","last_undegraded":"2024-09-11T01:14:19.035506+0000","last_fullsized":"2024-09-11T01:14:19.035506+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":6444,"log_dups_size":0,"ondisk_log_size":6444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26365952,"num_objects":6437,"num_object_clones":0,"num_object_copies":12874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3449,"num_write_kb":13796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6411","reported_seq":6574,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:19.158694+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:14:19.158694+0000","last_peered":"2024-09-11T01:14:19.158694+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:14:19.158694+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":6411,"log_dups_size":0,"ondisk_log_size":6411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26238976,"num_objects":6406,"num_object_clones":0,"num_object_copies":12812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2752,"num_write_kb":11008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2455},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6317","reported_seq":6442,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:19.163466+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:14:19.163466+0000","last_peered":"2024-09-11T01:14:19.163466+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:14:19.163466+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":6317,"log_dups_size":0,"ondisk_log_size":6317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25849856,"num_objects":6311,"num_object_clones":0,"num_object_copies":12622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3174,"num_write_kb":12696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2325},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6450","reported_seq":6575,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:19.156502+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:14:19.156502+0000","last_peered":"2024-09-11T01:14:19.156502+0000","last_clean":"2024-09-11T01:14:19.156502+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:14:19.156502+0000","last_undegraded":"2024-09-11T01:14:19.156502+0000","last_fullsized":"2024-09-11T01:14:19.156502+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":6450,"log_dups_size":0,"ondisk_log_size":6450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26411008,"num_objects":6448,"num_object_clones":0,"num_object_copies":12896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3326,"num_write_kb":13304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6386","reported_seq":6508,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:18.441874+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:14:18.441874+0000","last_peered":"2024-09-11T01:14:18.441874+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:14:18.441874+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":6386,"log_dups_size":0,"ondisk_log_size":6386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26128384,"num_objects":6379,"num_object_clones":0,"num_object_copies":12758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4357,"num_write_kb":17428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":2446}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741922+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741922+0000","last_peered":"2024-09-11T01:13:09.741922+0000","last_clean":"2024-09-11T01:13:09.741922+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741922+0000","last_undegraded":"2024-09-11T01:13:09.741922+0000","last_fullsized":"2024-09-11T01:13:09.741922+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6349","reported_seq":6470,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:19.136106+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:14:19.136106+0000","last_peered":"2024-09-11T01:14:19.136106+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:14:19.136106+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":6349,"log_dups_size":0,"ondisk_log_size":6349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25972736,"num_objects":6341,"num_object_clones":0,"num_object_copies":12682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2936,"num_write_kb":11744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2357},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.910012+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.910012+0000","last_peered":"2024-09-11T01:13:09.910012+0000","last_clean":"2024-09-11T01:13:09.910012+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.910012+0000","last_undegraded":"2024-09-11T01:13:09.910012+0000","last_fullsized":"2024-09-11T01:13:09.910012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6421","reported_seq":6535,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:18.354483+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:14:18.354483+0000","last_peered":"2024-09-11T01:14:18.354483+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:14:18.354483+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":6421,"log_dups_size":0,"ondisk_log_size":6421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26279936,"num_objects":6416,"num_object_clones":0,"num_object_copies":12832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4238,"num_write_kb":16952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":2392}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6284","reported_seq":9611,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:14:18.454501+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:14:18.454501+0000","last_peered":"2024-09-11T01:14:18.454501+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:14:18.454501+0000","last_undegraded":"2024-09-11T01:14:18.454501+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":6284,"log_dups_size":0,"ondisk_log_size":6284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25714688,"num_objects":6278,"num_object_clones":0,"num_object_copies":12556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2319,"num_objects_unfound":0,"num_objects_dirty":6278,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4069,"num_write_kb":16276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1606,"num_bytes_recovered":6578176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2319},{"shards":"2","objects":3959}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6435","reported_seq":6562,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:19.138035+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:14:19.138035+0000","last_peered":"2024-09-11T01:14:19.138035+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:14:19.138035+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":6435,"log_dups_size":0,"ondisk_log_size":6435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26349568,"num_objects":6433,"num_object_clones":0,"num_object_copies":12866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3172,"num_write_kb":12688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2483},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6355","reported_seq":6487,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:19.179633+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:14:19.179633+0000","last_peered":"2024-09-11T01:14:19.179633+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:14:19.179633+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":6355,"log_dups_size":0,"ondisk_log_size":6355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25997312,"num_objects":6347,"num_object_clones":0,"num_object_copies":12694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3010,"num_write_kb":12040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2408},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658174+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658174+0000","last_peered":"2024-09-11T01:13:09.658174+0000","last_clean":"2024-09-11T01:13:09.658174+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658174+0000","last_undegraded":"2024-09-11T01:13:09.658174+0000","last_fullsized":"2024-09-11T01:13:09.658174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6471","reported_seq":6601,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:19.143524+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:14:19.143524+0000","last_peered":"2024-09-11T01:14:19.143524+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:14:19.143524+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":6471,"log_dups_size":0,"ondisk_log_size":6471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26488832,"num_objects":6467,"num_object_clones":0,"num_object_copies":12934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3080,"num_write_kb":12320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2428},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6390","reported_seq":6519,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:19.045085+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:14:19.045085+0000","last_peered":"2024-09-11T01:14:19.045085+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:14:19.045085+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":6390,"log_dups_size":0,"ondisk_log_size":6390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26124905,"num_objects":6379,"num_object_clones":0,"num_object_copies":12758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3460,"num_write_kb":13840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2462},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":419123817,"num_objects":102326,"num_object_clones":0,"num_object_copies":204652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":2319,"num_objects_unfound":0,"num_objects_dirty":102326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63257,"num_write_kb":253028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1607,"num_bytes_recovered":6582272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":842158080,"data_stored":842071105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102407,"ondisk_log_size":102407,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182434,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":572940,"kb_used_data":370556,"kb_used_omap":4,"kb_used_meta":202363,"kb_avail":93798900,"statfs":{"total":96636764160,"available":96050073600,"internally_reserved":0,"allocated":379449344,"data_stored":379200525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":207220395},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,4,1,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986206,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":516012,"kb_used_data":331676,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93855828,"statfs":{"total":96636764160,"available":96108367872,"internally_reserved":0,"allocated":339636224,"data_stored":339391482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188739242},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,4,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659151,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":249152,"kb_used_data":118960,"kb_used_omap":4,"kb_used_meta":130171,"kb_avail":94122688,"statfs":{"total":96636764160,"available":96381632512,"internally_reserved":0,"allocated":121815040,"data_stored":121489683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":133295791},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,1,0,5,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":379895808,"data_stored":379892329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":122396672,"data_stored":122316655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":339865600,"data_stored":339862121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:14:21.051 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:14:21.380 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 117 20 103596 103576 3.45698 1.9375 0.014229 0.0225899 2024-09-11T01:14:21.959 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:14:21.960 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:14:22.036 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":135,"stamp":"2024-09-11T01:14:20.577319+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":419123836,"num_objects":102328,"num_object_clones":0,"num_object_copies":204656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":2319,"num_objects_unfound":0,"num_objects_dirty":102328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63259,"num_write_kb":253030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1608,"num_bytes_recovered":6582291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102410,"ondisk_log_size":102410,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1338104,"kb_used_data":821192,"kb_used_omap":12,"kb_used_meta":516851,"kb_avail":281777416,"statfs":{"total":289910292480,"available":288540073984,"internally_reserved":0,"allocated":840900608,"data_stored":840081690,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":529255428},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3,4,4,10,15,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28483584,"num_objects":6954,"num_object_clones":0,"num_object_copies":13908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-274,"num_objects_unfound":0,"num_objects_dirty":6954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6954,"num_write_kb":27816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":1122304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002998"},"pg_stats":[{"pgid":"2.f","version":"43'6472","reported_seq":6545,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:19.173684+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:14:19.173684+0000","last_peered":"2024-09-11T01:14:19.173684+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:14:19.173684+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":6472,"log_dups_size":0,"ondisk_log_size":6472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26509312,"num_objects":6472,"num_object_clones":0,"num_object_copies":12944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6472,"num_write_kb":25888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2400},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6497","reported_seq":6593,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:18.439245+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:14:18.439245+0000","last_peered":"2024-09-11T01:14:18.439245+0000","last_clean":"2024-09-11T01:14:18.439245+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:14:18.439245+0000","last_undegraded":"2024-09-11T01:14:18.439245+0000","last_fullsized":"2024-09-11T01:14:18.439245+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":6497,"log_dups_size":0,"ondisk_log_size":6497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26587136,"num_objects":6491,"num_object_clones":0,"num_object_copies":12982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4736,"num_write_kb":18944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6338","reported_seq":6460,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:18.387925+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:14:18.387925+0000","last_peered":"2024-09-11T01:14:18.387925+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:14:18.387925+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":6338,"log_dups_size":0,"ondisk_log_size":6338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25944064,"num_objects":6334,"num_object_clones":0,"num_object_copies":12668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4639,"num_write_kb":18556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":2438}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6387","reported_seq":6451,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:18.457922+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:14:18.457922+0000","last_peered":"2024-09-11T01:14:18.457922+0000","last_clean":"2024-09-11T01:14:18.457922+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:14:18.457922+0000","last_undegraded":"2024-09-11T01:14:18.457922+0000","last_fullsized":"2024-09-11T01:14:18.457922+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":6387,"log_dups_size":0,"ondisk_log_size":6387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26161152,"num_objects":6387,"num_object_clones":0,"num_object_copies":12774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6387,"num_write_kb":25548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6444","reported_seq":6567,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:19.035506+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:14:19.035506+0000","last_peered":"2024-09-11T01:14:19.035506+0000","last_clean":"2024-09-11T01:14:19.035506+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:14:19.035506+0000","last_undegraded":"2024-09-11T01:14:19.035506+0000","last_fullsized":"2024-09-11T01:14:19.035506+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":6444,"log_dups_size":0,"ondisk_log_size":6444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26365952,"num_objects":6437,"num_object_clones":0,"num_object_copies":12874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3449,"num_write_kb":13796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6411","reported_seq":6574,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:19.158694+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:14:19.158694+0000","last_peered":"2024-09-11T01:14:19.158694+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:14:19.158694+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":6411,"log_dups_size":0,"ondisk_log_size":6411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26238976,"num_objects":6406,"num_object_clones":0,"num_object_copies":12812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2752,"num_write_kb":11008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2455},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6317","reported_seq":6442,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:19.163466+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:14:19.163466+0000","last_peered":"2024-09-11T01:14:19.163466+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:14:19.163466+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":6317,"log_dups_size":0,"ondisk_log_size":6317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25849856,"num_objects":6311,"num_object_clones":0,"num_object_copies":12622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3174,"num_write_kb":12696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2325},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6450","reported_seq":6575,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:19.156502+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:14:19.156502+0000","last_peered":"2024-09-11T01:14:19.156502+0000","last_clean":"2024-09-11T01:14:19.156502+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:14:19.156502+0000","last_undegraded":"2024-09-11T01:14:19.156502+0000","last_fullsized":"2024-09-11T01:14:19.156502+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":6450,"log_dups_size":0,"ondisk_log_size":6450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26411008,"num_objects":6448,"num_object_clones":0,"num_object_copies":12896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3326,"num_write_kb":13304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6386","reported_seq":6508,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:18.441874+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:14:18.441874+0000","last_peered":"2024-09-11T01:14:18.441874+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:14:18.441874+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":6386,"log_dups_size":0,"ondisk_log_size":6386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26128384,"num_objects":6379,"num_object_clones":0,"num_object_copies":12758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4357,"num_write_kb":17428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":2446}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741922+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741922+0000","last_peered":"2024-09-11T01:13:09.741922+0000","last_clean":"2024-09-11T01:13:09.741922+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741922+0000","last_undegraded":"2024-09-11T01:13:09.741922+0000","last_fullsized":"2024-09-11T01:13:09.741922+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6349","reported_seq":6470,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:19.136106+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:14:19.136106+0000","last_peered":"2024-09-11T01:14:19.136106+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:14:19.136106+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":6349,"log_dups_size":0,"ondisk_log_size":6349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25972736,"num_objects":6341,"num_object_clones":0,"num_object_copies":12682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2936,"num_write_kb":11744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2357},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.910012+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.910012+0000","last_peered":"2024-09-11T01:13:09.910012+0000","last_clean":"2024-09-11T01:13:09.910012+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.910012+0000","last_undegraded":"2024-09-11T01:13:09.910012+0000","last_fullsized":"2024-09-11T01:13:09.910012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6421","reported_seq":6535,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:18.354483+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:14:18.354483+0000","last_peered":"2024-09-11T01:14:18.354483+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:14:18.354483+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":6421,"log_dups_size":0,"ondisk_log_size":6421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26279936,"num_objects":6416,"num_object_clones":0,"num_object_copies":12832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4238,"num_write_kb":16952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":2392}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6284","reported_seq":9611,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:14:18.454501+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:14:18.454501+0000","last_peered":"2024-09-11T01:14:18.454501+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:14:18.454501+0000","last_undegraded":"2024-09-11T01:14:18.454501+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":6284,"log_dups_size":0,"ondisk_log_size":6284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25714688,"num_objects":6278,"num_object_clones":0,"num_object_copies":12556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2319,"num_objects_unfound":0,"num_objects_dirty":6278,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4069,"num_write_kb":16276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1606,"num_bytes_recovered":6578176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2319},{"shards":"2","objects":3959}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6435","reported_seq":6562,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:19.138035+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:14:19.138035+0000","last_peered":"2024-09-11T01:14:19.138035+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:14:19.138035+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":6435,"log_dups_size":0,"ondisk_log_size":6435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26349568,"num_objects":6433,"num_object_clones":0,"num_object_copies":12866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3172,"num_write_kb":12688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2483},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6355","reported_seq":6487,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:19.179633+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:14:19.179633+0000","last_peered":"2024-09-11T01:14:19.179633+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:14:19.179633+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":6355,"log_dups_size":0,"ondisk_log_size":6355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25997312,"num_objects":6347,"num_object_clones":0,"num_object_copies":12694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3010,"num_write_kb":12040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2408},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658174+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658174+0000","last_peered":"2024-09-11T01:13:09.658174+0000","last_clean":"2024-09-11T01:13:09.658174+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658174+0000","last_undegraded":"2024-09-11T01:13:09.658174+0000","last_fullsized":"2024-09-11T01:13:09.658174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6471","reported_seq":6601,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:19.143524+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:14:19.143524+0000","last_peered":"2024-09-11T01:14:19.143524+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:14:19.143524+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":6471,"log_dups_size":0,"ondisk_log_size":6471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26488832,"num_objects":6467,"num_object_clones":0,"num_object_copies":12934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3080,"num_write_kb":12320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2428},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6390","reported_seq":6519,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:19.045085+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:14:19.045085+0000","last_peered":"2024-09-11T01:14:19.045085+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:14:19.045085+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":6390,"log_dups_size":0,"ondisk_log_size":6390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26124905,"num_objects":6379,"num_object_clones":0,"num_object_copies":12758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3460,"num_write_kb":13840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2462},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":419123817,"num_objects":102326,"num_object_clones":0,"num_object_copies":204652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":2319,"num_objects_unfound":0,"num_objects_dirty":102326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63257,"num_write_kb":253028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1607,"num_bytes_recovered":6582272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":842158080,"data_stored":842071105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102407,"ondisk_log_size":102407,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182434,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":572940,"kb_used_data":370556,"kb_used_omap":4,"kb_used_meta":202363,"kb_avail":93798900,"statfs":{"total":96636764160,"available":96050073600,"internally_reserved":0,"allocated":379449344,"data_stored":379200525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":207220395},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,4,1,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986206,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":516012,"kb_used_data":331676,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93855828,"statfs":{"total":96636764160,"available":96108367872,"internally_reserved":0,"allocated":339636224,"data_stored":339391482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188739242},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,4,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659151,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":249152,"kb_used_data":118960,"kb_used_omap":4,"kb_used_meta":130171,"kb_avail":94122688,"statfs":{"total":96636764160,"available":96381632512,"internally_reserved":0,"allocated":121815040,"data_stored":121489683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":133295791},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,1,0,5,5,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":379895808,"data_stored":379892329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":122396672,"data_stored":122316655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":339865600,"data_stored":339862121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:14:22.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:14:22.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'6472", 'reported_seq': 6545, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:19.173684+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:14:19.173684+0000', 'last_peered': '2024-09-11T01:14:19.173684+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:14:19.173684+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 6472, 'log_dups_size': 0, 'ondisk_log_size': 6472, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26509312, 'num_objects': 6472, 'num_object_clones': 0, 'num_object_copies': 12944, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6472, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6472, 'num_write_kb': 25888, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2400}, {'shards': '0,2', 'objects': 4072}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:22.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:14:22.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'6338", 'reported_seq': 6460, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:18.387925+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:14:18.387925+0000', 'last_peered': '2024-09-11T01:14:18.387925+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:14:18.387925+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 6338, 'log_dups_size': 0, 'ondisk_log_size': 6338, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25944064, 'num_objects': 6334, 'num_object_clones': 0, 'num_object_copies': 12668, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6334, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4639, 'num_write_kb': 18556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3896}, {'shards': '2', 'objects': 2438}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:14:22.038 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:14:22.038 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'6411", 'reported_seq': 6574, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:19.158694+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:14:19.158694+0000', 'last_peered': '2024-09-11T01:14:19.158694+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:14:19.158694+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 6411, 'log_dups_size': 0, 'ondisk_log_size': 6411, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26238976, 'num_objects': 6406, 'num_object_clones': 0, 'num_object_copies': 12812, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6406, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2752, 'num_write_kb': 11008, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2455}, {'shards': '0,2', 'objects': 3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:22.038 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:14:22.038 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'6317", 'reported_seq': 6442, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:19.163466+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:14:19.163466+0000', 'last_peered': '2024-09-11T01:14:19.163466+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:14:19.163466+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 6317, 'log_dups_size': 0, 'ondisk_log_size': 6317, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25849856, 'num_objects': 6311, 'num_object_clones': 0, 'num_object_copies': 12622, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6311, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3174, 'num_write_kb': 12696, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2325}, {'shards': '0,2', 'objects': 3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:22.038 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:14:22.038 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'6386", 'reported_seq': 6508, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:18.441874+0000', 'last_change': '2024-09-11T01:13:10.159069+0000', 'last_active': '2024-09-11T01:14:18.441874+0000', 'last_peered': '2024-09-11T01:14:18.441874+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:14:18.441874+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 6386, 'log_dups_size': 0, 'ondisk_log_size': 6386, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26128384, 'num_objects': 6379, 'num_object_clones': 0, 'num_object_copies': 12758, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6379, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4357, 'num_write_kb': 17428, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3932}, {'shards': '2', 'objects': 2446}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:14:22.038 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:14:22.038 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'6349", 'reported_seq': 6470, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:19.136106+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:14:19.136106+0000', 'last_peered': '2024-09-11T01:14:19.136106+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:14:19.136106+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 6349, 'log_dups_size': 0, 'ondisk_log_size': 6349, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25972736, 'num_objects': 6341, 'num_object_clones': 0, 'num_object_copies': 12682, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6341, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2936, 'num_write_kb': 11744, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2357}, {'shards': '0,2', 'objects': 3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:22.038 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:14:22.038 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'6421", 'reported_seq': 6535, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:18.354483+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:14:18.354483+0000', 'last_peered': '2024-09-11T01:14:18.354483+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:14:18.354483+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 6421, 'log_dups_size': 0, 'ondisk_log_size': 6421, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26279936, 'num_objects': 6416, 'num_object_clones': 0, 'num_object_copies': 12832, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6416, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4238, 'num_write_kb': 16952, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4023}, {'shards': '2', 'objects': 2392}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:14:22.039 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:14:22.039 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'6284", 'reported_seq': 9611, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:14:18.454501+0000', 'last_change': '2024-09-11T01:13:10.217540+0000', 'last_active': '2024-09-11T01:14:18.454501+0000', 'last_peered': '2024-09-11T01:14:18.454501+0000', 'last_clean': '2024-09-11T01:13:07.638709+0000', 'last_became_active': '2024-09-11T01:13:10.125604+0000', 'last_became_peered': '2024-09-11T01:13:10.125604+0000', 'last_unstale': '2024-09-11T01:14:18.454501+0000', 'last_undegraded': '2024-09-11T01:14:18.454501+0000', 'last_fullsized': '2024-09-11T01:13:09.649604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2269", 'last_scrub_stamp': '2024-09-11T01:12:42.890920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:42.890920+0000', 'objects_scrubbed': 2209, 'log_size': 6284, 'log_dups_size': 0, 'ondisk_log_size': 6284, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 999, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25714688, 'num_objects': 6278, 'num_object_clones': 0, 'num_object_copies': 12556, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2319, 'num_objects_unfound': 0, 'num_objects_dirty': 6278, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4069, 'num_write_kb': 16276, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1606, 'num_bytes_recovered': 6578176, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2319}, {'shards': '2', 'objects': 3959}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:14:22.039 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:14:22.039 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'6435", 'reported_seq': 6562, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:19.138035+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:14:19.138035+0000', 'last_peered': '2024-09-11T01:14:19.138035+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:14:19.138035+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 6435, 'log_dups_size': 0, 'ondisk_log_size': 6435, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26349568, 'num_objects': 6433, 'num_object_clones': 0, 'num_object_copies': 12866, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6433, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3172, 'num_write_kb': 12688, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2483}, {'shards': '0,2', 'objects': 3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:22.039 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:14:22.039 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'6355", 'reported_seq': 6487, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:19.179633+0000', 'last_change': '2024-09-11T01:13:10.209668+0000', 'last_active': '2024-09-11T01:14:19.179633+0000', 'last_peered': '2024-09-11T01:14:19.179633+0000', 'last_clean': '2024-09-11T01:13:07.624565+0000', 'last_became_active': '2024-09-11T01:13:10.199795+0000', 'last_became_peered': '2024-09-11T01:13:10.199795+0000', 'last_unstale': '2024-09-11T01:14:19.179633+0000', 'last_undegraded': '2024-09-11T01:13:09.791198+0000', 'last_fullsized': '2024-09-11T01:13:09.652311+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3415", 'last_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'objects_scrubbed': 3337, 'log_size': 6355, 'log_dups_size': 0, 'ondisk_log_size': 6355, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1975, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25997312, 'num_objects': 6347, 'num_object_clones': 0, 'num_object_copies': 12694, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3942, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6347, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3010, 'num_write_kb': 12040, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2408}, {'shards': '0,2', 'objects': 3939}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:22.039 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:14:22.039 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'6471", 'reported_seq': 6601, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:19.143524+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:14:19.143524+0000', 'last_peered': '2024-09-11T01:14:19.143524+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:14:19.143524+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 6471, 'log_dups_size': 0, 'ondisk_log_size': 6471, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26488832, 'num_objects': 6467, 'num_object_clones': 0, 'num_object_copies': 12934, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6467, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3080, 'num_write_kb': 12320, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2428}, {'shards': '0,2', 'objects': 4039}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:22.039 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:14:22.040 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'6390", 'reported_seq': 6519, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:19.045085+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:14:19.045085+0000', 'last_peered': '2024-09-11T01:14:19.045085+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:14:19.045085+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 6390, 'log_dups_size': 0, 'ondisk_log_size': 6390, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26124905, 'num_objects': 6379, 'num_object_clones': 0, 'num_object_copies': 12758, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6379, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3460, 'num_write_kb': 13840, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2462}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:22.040 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:14:22.382 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 118 20 104095 104075 3.4442 1.94922 0.0702078 0.0226737 2024-09-11T01:14:23.003 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:14:23.003 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43711/204656 objects degraded (21.358%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":199,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":102328,"data_bytes":419123836,"bytes_used":1370292224,"bytes_avail":288540000256,"bytes_total":289910292480,"degraded_objects":43711,"degraded_total":204656,"degraded_ratio":0.21358279258853885,"misplaced_objects":2319,"misplaced_total":204656,"misplaced_ratio":0.01133120944414041,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":93502,"recovering_keys_per_sec":0,"num_objects_recovered":274,"num_bytes_recovered":1122304,"num_keys_recovered":0,"write_bytes_sec":1798723,"read_op_per_sec":0,"write_op_per_sec":439},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:14:23.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:14:23.053 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:14:23.379 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 119 20 104601 104581 3.43187 1.97656 0.113762 0.0227535 2024-09-11T01:14:23.899 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:14:23.901 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:14:24.000 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2024-09-11T01:14:22.577911+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":419123836,"num_objects":102328,"num_object_clones":0,"num_object_copies":204656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":2319,"num_objects_unfound":0,"num_objects_dirty":102328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63259,"num_write_kb":253030,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1608,"num_bytes_recovered":6582291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102410,"ondisk_log_size":102410,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1338176,"kb_used_data":829584,"kb_used_omap":12,"kb_used_meta":508531,"kb_avail":281777344,"statfs":{"total":289910292480,"available":288540000256,"internally_reserved":0,"allocated":849494016,"data_stored":848675098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":520735748},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,8,4,7,13,11],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21590016,"num_objects":5271,"num_object_clones":0,"num_object_copies":10542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-274,"num_objects_unfound":0,"num_objects_dirty":5271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5271,"num_write_kb":21084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":1122304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002963"},"pg_stats":[{"pgid":"2.f","version":"43'6472","reported_seq":6545,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:19.173684+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:14:19.173684+0000","last_peered":"2024-09-11T01:14:19.173684+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:14:19.173684+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":6472,"log_dups_size":0,"ondisk_log_size":6472,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26509312,"num_objects":6472,"num_object_clones":0,"num_object_copies":12944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6472,"num_write_kb":25888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2400},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6497","reported_seq":6593,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:18.439245+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:14:18.439245+0000","last_peered":"2024-09-11T01:14:18.439245+0000","last_clean":"2024-09-11T01:14:18.439245+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:14:18.439245+0000","last_undegraded":"2024-09-11T01:14:18.439245+0000","last_fullsized":"2024-09-11T01:14:18.439245+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":6497,"log_dups_size":0,"ondisk_log_size":6497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26587136,"num_objects":6491,"num_object_clones":0,"num_object_copies":12982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4736,"num_write_kb":18944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6338","reported_seq":6460,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:18.387925+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:14:18.387925+0000","last_peered":"2024-09-11T01:14:18.387925+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:14:18.387925+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":6338,"log_dups_size":0,"ondisk_log_size":6338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25944064,"num_objects":6334,"num_object_clones":0,"num_object_copies":12668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4639,"num_write_kb":18556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":2438}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6387","reported_seq":6451,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:18.457922+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:14:18.457922+0000","last_peered":"2024-09-11T01:14:18.457922+0000","last_clean":"2024-09-11T01:14:18.457922+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:14:18.457922+0000","last_undegraded":"2024-09-11T01:14:18.457922+0000","last_fullsized":"2024-09-11T01:14:18.457922+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":6387,"log_dups_size":0,"ondisk_log_size":6387,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26161152,"num_objects":6387,"num_object_clones":0,"num_object_copies":12774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6387,"num_write_kb":25548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6444","reported_seq":6567,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:19.035506+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:14:19.035506+0000","last_peered":"2024-09-11T01:14:19.035506+0000","last_clean":"2024-09-11T01:14:19.035506+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:14:19.035506+0000","last_undegraded":"2024-09-11T01:14:19.035506+0000","last_fullsized":"2024-09-11T01:14:19.035506+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":6444,"log_dups_size":0,"ondisk_log_size":6444,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26365952,"num_objects":6437,"num_object_clones":0,"num_object_copies":12874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3449,"num_write_kb":13796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6411","reported_seq":6574,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:19.158694+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:14:19.158694+0000","last_peered":"2024-09-11T01:14:19.158694+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:14:19.158694+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":6411,"log_dups_size":0,"ondisk_log_size":6411,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26238976,"num_objects":6406,"num_object_clones":0,"num_object_copies":12812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2752,"num_write_kb":11008,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2455},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6317","reported_seq":6442,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:19.163466+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:14:19.163466+0000","last_peered":"2024-09-11T01:14:19.163466+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:14:19.163466+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":6317,"log_dups_size":0,"ondisk_log_size":6317,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25849856,"num_objects":6311,"num_object_clones":0,"num_object_copies":12622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3174,"num_write_kb":12696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2325},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6450","reported_seq":6575,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:19.156502+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:14:19.156502+0000","last_peered":"2024-09-11T01:14:19.156502+0000","last_clean":"2024-09-11T01:14:19.156502+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:14:19.156502+0000","last_undegraded":"2024-09-11T01:14:19.156502+0000","last_fullsized":"2024-09-11T01:14:19.156502+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":6450,"log_dups_size":0,"ondisk_log_size":6450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26411008,"num_objects":6448,"num_object_clones":0,"num_object_copies":12896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3326,"num_write_kb":13304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6386","reported_seq":6508,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:18.441874+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:14:18.441874+0000","last_peered":"2024-09-11T01:14:18.441874+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:14:18.441874+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":6386,"log_dups_size":0,"ondisk_log_size":6386,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26128384,"num_objects":6379,"num_object_clones":0,"num_object_copies":12758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4357,"num_write_kb":17428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":2446}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741922+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741922+0000","last_peered":"2024-09-11T01:13:09.741922+0000","last_clean":"2024-09-11T01:13:09.741922+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741922+0000","last_undegraded":"2024-09-11T01:13:09.741922+0000","last_fullsized":"2024-09-11T01:13:09.741922+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6349","reported_seq":6470,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:19.136106+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:14:19.136106+0000","last_peered":"2024-09-11T01:14:19.136106+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:14:19.136106+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":6349,"log_dups_size":0,"ondisk_log_size":6349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25972736,"num_objects":6341,"num_object_clones":0,"num_object_copies":12682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2936,"num_write_kb":11744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2357},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.910012+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.910012+0000","last_peered":"2024-09-11T01:13:09.910012+0000","last_clean":"2024-09-11T01:13:09.910012+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.910012+0000","last_undegraded":"2024-09-11T01:13:09.910012+0000","last_fullsized":"2024-09-11T01:13:09.910012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6421","reported_seq":6535,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:18.354483+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:14:18.354483+0000","last_peered":"2024-09-11T01:14:18.354483+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:14:18.354483+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":6421,"log_dups_size":0,"ondisk_log_size":6421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26279936,"num_objects":6416,"num_object_clones":0,"num_object_copies":12832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4238,"num_write_kb":16952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":2392}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6284","reported_seq":9611,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:14:18.454501+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:14:18.454501+0000","last_peered":"2024-09-11T01:14:18.454501+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:14:18.454501+0000","last_undegraded":"2024-09-11T01:14:18.454501+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":6284,"log_dups_size":0,"ondisk_log_size":6284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25714688,"num_objects":6278,"num_object_clones":0,"num_object_copies":12556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2319,"num_objects_unfound":0,"num_objects_dirty":6278,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4069,"num_write_kb":16276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1606,"num_bytes_recovered":6578176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2319},{"shards":"2","objects":3959}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6435","reported_seq":6562,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:19.138035+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:14:19.138035+0000","last_peered":"2024-09-11T01:14:19.138035+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:14:19.138035+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":6435,"log_dups_size":0,"ondisk_log_size":6435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26349568,"num_objects":6433,"num_object_clones":0,"num_object_copies":12866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3172,"num_write_kb":12688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2483},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6355","reported_seq":6487,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:19.179633+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:14:19.179633+0000","last_peered":"2024-09-11T01:14:19.179633+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:14:19.179633+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":6355,"log_dups_size":0,"ondisk_log_size":6355,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25997312,"num_objects":6347,"num_object_clones":0,"num_object_copies":12694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3010,"num_write_kb":12040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2408},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658174+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658174+0000","last_peered":"2024-09-11T01:13:09.658174+0000","last_clean":"2024-09-11T01:13:09.658174+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658174+0000","last_undegraded":"2024-09-11T01:13:09.658174+0000","last_fullsized":"2024-09-11T01:13:09.658174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6471","reported_seq":6601,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:19.143524+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:14:19.143524+0000","last_peered":"2024-09-11T01:14:19.143524+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:14:19.143524+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":6471,"log_dups_size":0,"ondisk_log_size":6471,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26488832,"num_objects":6467,"num_object_clones":0,"num_object_copies":12934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3080,"num_write_kb":12320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2428},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6390","reported_seq":6519,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:19.045085+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:14:19.045085+0000","last_peered":"2024-09-11T01:14:19.045085+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:14:19.045085+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":6390,"log_dups_size":0,"ondisk_log_size":6390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26124905,"num_objects":6379,"num_object_clones":0,"num_object_copies":12758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3460,"num_write_kb":13840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2462},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":419123817,"num_objects":102326,"num_object_clones":0,"num_object_copies":204652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":2319,"num_objects_unfound":0,"num_objects_dirty":102326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":63257,"num_write_kb":253028,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1607,"num_bytes_recovered":6582272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":850804736,"data_stored":850717761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":102407,"ondisk_log_size":102407,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182434,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":572940,"kb_used_data":370556,"kb_used_omap":4,"kb_used_meta":202363,"kb_avail":93798900,"statfs":{"total":96636764160,"available":96050073600,"internally_reserved":0,"allocated":379449344,"data_stored":379200525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":207220395},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,4,1,5,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986206,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":516012,"kb_used_data":331676,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93855828,"statfs":{"total":96636764160,"available":96108367872,"internally_reserved":0,"allocated":339636224,"data_stored":339391482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188739242},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,0,4,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659152,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":249224,"kb_used_data":127352,"kb_used_omap":4,"kb_used_meta":121851,"kb_avail":94122616,"statfs":{"total":96636764160,"available":96381558784,"internally_reserved":0,"allocated":130408448,"data_stored":130083091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":124776111},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,5,0,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":379895808,"data_stored":379892329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":131043328,"data_stored":130963311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":339865600,"data_stored":339862121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:14:24.379 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout:2024-09-11T01:14:24.378849+0000 min lat: 0.00219431 max lat: 2.69211 avg lat: 0.0228321 2024-09-11T01:14:24.379 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T01:14:24.379 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 120 20 105116 105096 3.42003 2.01172 0.0163304 0.0228321 2024-09-11T01:14:25.379 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 121 20 105651 105631 3.40904 2.08984 0.0246779 0.0229076 2024-09-11T01:14:26.379 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 122 20 106178 106158 3.39797 2.05859 0.0401639 0.0229816 2024-09-11T01:14:27.002 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:14:27.384 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 123 20 106683 106663 3.38639 1.97266 0.0846835 0.0230627 2024-09-11T01:14:27.738 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:14:27.738 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:14:27.811 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":138,"stamp":"2024-09-11T01:14:26.578941+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":429576828,"num_objects":104880,"num_object_clones":0,"num_object_copies":209760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":2181,"num_objects_unfound":0,"num_objects_dirty":104880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65811,"num_write_kb":263238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1746,"num_bytes_recovered":7147539,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104962,"ondisk_log_size":104962,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1330148,"kb_used_data":842352,"kb_used_omap":12,"kb_used_meta":487731,"kb_avail":281785372,"statfs":{"total":289910292480,"available":288548220928,"internally_reserved":0,"allocated":862568448,"data_stored":861749530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":499436548},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,6,0,13,7,5,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20959232,"num_objects":5117,"num_object_clones":0,"num_object_copies":10234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-266,"num_objects_unfound":0,"num_objects_dirty":5117,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5117,"num_write_kb":20468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":266,"num_bytes_recovered":1089536,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002999"},"pg_stats":[{"pgid":"2.f","version":"43'6652","reported_seq":6722,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:24.165968+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:14:24.165968+0000","last_peered":"2024-09-11T01:14:24.165968+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:14:24.165968+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":6652,"log_dups_size":0,"ondisk_log_size":6652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27246592,"num_objects":6652,"num_object_clones":0,"num_object_copies":13304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6652,"num_write_kb":26608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2580},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6646","reported_seq":6743,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:23.449472+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:14:23.449472+0000","last_peered":"2024-09-11T01:14:23.449472+0000","last_clean":"2024-09-11T01:14:23.449472+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:14:23.449472+0000","last_undegraded":"2024-09-11T01:14:23.449472+0000","last_fullsized":"2024-09-11T01:14:23.449472+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":6646,"log_dups_size":0,"ondisk_log_size":6646,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27197440,"num_objects":6640,"num_object_clones":0,"num_object_copies":13280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4885,"num_write_kb":19540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6499","reported_seq":6621,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:23.398118+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:14:23.398118+0000","last_peered":"2024-09-11T01:14:23.398118+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:14:23.398118+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":6499,"log_dups_size":0,"ondisk_log_size":6499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26603520,"num_objects":6495,"num_object_clones":0,"num_object_copies":12990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4800,"num_write_kb":19200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":2599}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6538","reported_seq":6599,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:23.436705+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:14:23.436705+0000","last_peered":"2024-09-11T01:14:23.436705+0000","last_clean":"2024-09-11T01:14:23.436705+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:14:23.436705+0000","last_undegraded":"2024-09-11T01:14:23.436705+0000","last_fullsized":"2024-09-11T01:14:23.436705+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":6538,"log_dups_size":0,"ondisk_log_size":6538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26779648,"num_objects":6538,"num_object_clones":0,"num_object_copies":13076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6538,"num_write_kb":26152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6593","reported_seq":6716,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:24.146908+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:14:24.146908+0000","last_peered":"2024-09-11T01:14:24.146908+0000","last_clean":"2024-09-11T01:14:24.146908+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:14:24.146908+0000","last_undegraded":"2024-09-11T01:14:24.146908+0000","last_fullsized":"2024-09-11T01:14:24.146908+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":6593,"log_dups_size":0,"ondisk_log_size":6593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26976256,"num_objects":6586,"num_object_clones":0,"num_object_copies":13172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3598,"num_write_kb":14392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6579","reported_seq":6742,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:24.158177+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:14:24.158177+0000","last_peered":"2024-09-11T01:14:24.158177+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:14:24.158177+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":6579,"log_dups_size":0,"ondisk_log_size":6579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26927104,"num_objects":6574,"num_object_clones":0,"num_object_copies":13148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2920,"num_write_kb":11680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2623},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6483","reported_seq":6608,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:24.091042+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:14:24.091042+0000","last_peered":"2024-09-11T01:14:24.091042+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:14:24.091042+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":6483,"log_dups_size":0,"ondisk_log_size":6483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26529792,"num_objects":6477,"num_object_clones":0,"num_object_copies":12954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3340,"num_write_kb":13360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2491},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6621","reported_seq":6746,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:24.124350+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:14:24.124350+0000","last_peered":"2024-09-11T01:14:24.124350+0000","last_clean":"2024-09-11T01:14:24.124350+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:14:24.124350+0000","last_undegraded":"2024-09-11T01:14:24.124350+0000","last_fullsized":"2024-09-11T01:14:24.124350+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":6621,"log_dups_size":0,"ondisk_log_size":6621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27111424,"num_objects":6619,"num_object_clones":0,"num_object_copies":13238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3497,"num_write_kb":13988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6544","reported_seq":6667,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:23.480519+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:14:23.480519+0000","last_peered":"2024-09-11T01:14:23.480519+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:14:23.480519+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":6544,"log_dups_size":0,"ondisk_log_size":6544,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26775552,"num_objects":6537,"num_object_clones":0,"num_object_copies":13074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4515,"num_write_kb":18060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":2604}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741922+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741922+0000","last_peered":"2024-09-11T01:13:09.741922+0000","last_clean":"2024-09-11T01:13:09.741922+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741922+0000","last_undegraded":"2024-09-11T01:13:09.741922+0000","last_fullsized":"2024-09-11T01:13:09.741922+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6512","reported_seq":6635,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:24.118252+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:14:24.118252+0000","last_peered":"2024-09-11T01:14:24.118252+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:14:24.118252+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":6512,"log_dups_size":0,"ondisk_log_size":6512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26640384,"num_objects":6504,"num_object_clones":0,"num_object_copies":13008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6504,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3099,"num_write_kb":12396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2520},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.910012+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.910012+0000","last_peered":"2024-09-11T01:13:09.910012+0000","last_clean":"2024-09-11T01:13:09.910012+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.910012+0000","last_undegraded":"2024-09-11T01:13:09.910012+0000","last_fullsized":"2024-09-11T01:13:09.910012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6568","reported_seq":6682,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:23.439652+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:14:23.439652+0000","last_peered":"2024-09-11T01:14:23.439652+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:14:23.439652+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":6568,"log_dups_size":0,"ondisk_log_size":6568,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26882048,"num_objects":6563,"num_object_clones":0,"num_object_copies":13126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4385,"num_write_kb":17540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":2539}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6442","reported_seq":10045,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:14:23.473302+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:14:23.473302+0000","last_peered":"2024-09-11T01:14:23.473302+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:14:23.473302+0000","last_undegraded":"2024-09-11T01:14:23.473302+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":6442,"log_dups_size":0,"ondisk_log_size":6442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26361856,"num_objects":6436,"num_object_clones":0,"num_object_copies":12872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2181,"num_objects_unfound":0,"num_objects_dirty":6436,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4227,"num_write_kb":16908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1744,"num_bytes_recovered":7143424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2181},{"shards":"2","objects":4255}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6618","reported_seq":6745,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:24.158007+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:14:24.158007+0000","last_peered":"2024-09-11T01:14:24.158007+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:14:24.158007+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":6618,"log_dups_size":0,"ondisk_log_size":6618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27099136,"num_objects":6616,"num_object_clones":0,"num_object_copies":13232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3355,"num_write_kb":13420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2666},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6495","reported_seq":6627,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:24.172537+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:14:24.172537+0000","last_peered":"2024-09-11T01:14:24.172537+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:14:24.172537+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":6495,"log_dups_size":0,"ondisk_log_size":6495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26570752,"num_objects":6487,"num_object_clones":0,"num_object_copies":12974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3150,"num_write_kb":12600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2548},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658174+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658174+0000","last_peered":"2024-09-11T01:13:09.658174+0000","last_clean":"2024-09-11T01:13:09.658174+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658174+0000","last_undegraded":"2024-09-11T01:13:09.658174+0000","last_fullsized":"2024-09-11T01:13:09.658174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6628","reported_seq":6758,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:24.066514+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:14:24.066514+0000","last_peered":"2024-09-11T01:14:24.066514+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:14:24.066514+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":6628,"log_dups_size":0,"ondisk_log_size":6628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27131904,"num_objects":6624,"num_object_clones":0,"num_object_copies":13248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3237,"num_write_kb":12948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2585},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6541","reported_seq":6670,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:24.158749+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:14:24.158749+0000","last_peered":"2024-09-11T01:14:24.158749+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:14:24.158749+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":6541,"log_dups_size":0,"ondisk_log_size":6541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26743401,"num_objects":6530,"num_object_clones":0,"num_object_copies":13060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3611,"num_write_kb":14444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2613},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":429576809,"num_objects":104878,"num_object_clones":0,"num_object_copies":209756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":2181,"num_objects_unfound":0,"num_objects_dirty":104878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65809,"num_write_kb":263236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1745,"num_bytes_recovered":7147520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":863756288,"data_stored":863669313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":104959,"ondisk_log_size":104959,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182435,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":572464,"kb_used_data":378400,"kb_used_omap":4,"kb_used_meta":194043,"kb_avail":93799376,"statfs":{"total":96636764160,"available":96050561024,"internally_reserved":0,"allocated":387481600,"data_stored":387232781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":198700715},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,8,3,2,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986207,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":508460,"kb_used_data":336600,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93863380,"statfs":{"total":96636764160,"available":96116101120,"internally_reserved":0,"allocated":344678400,"data_stored":344433658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":175959722},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659152,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":249224,"kb_used_data":127352,"kb_used_omap":4,"kb_used_meta":121851,"kb_avail":94122616,"statfs":{"total":96636764160,"available":96381558784,"internally_reserved":0,"allocated":130408448,"data_stored":130083091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":124776111},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,5,0,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":387796992,"data_stored":387793513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":131043328,"data_stored":130963311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":344915968,"data_stored":344912489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:14:27.813 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:14:28.379 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 124 20 107170 107150 3.37442 1.90234 0.0757772 0.0231431 2024-09-11T01:14:28.600 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:14:28.601 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:14:28.681 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":139,"stamp":"2024-09-11T01:14:28.579292+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":433406588,"num_objects":105815,"num_object_clones":0,"num_object_copies":211630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":2027,"num_objects_unfound":0,"num_objects_dirty":105815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66746,"num_write_kb":266978,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1900,"num_bytes_recovered":7778323,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105897,"ondisk_log_size":105897,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1356124,"kb_used_data":855852,"kb_used_omap":12,"kb_used_meta":500211,"kb_avail":281759396,"statfs":{"total":289910292480,"available":288521621504,"internally_reserved":0,"allocated":876392448,"data_stored":875573530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":512216068},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,7,3,9,9,7,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24788992,"num_objects":6052,"num_object_clones":0,"num_object_copies":12104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-420,"num_objects_unfound":0,"num_objects_dirty":6052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6052,"num_write_kb":24208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":420,"num_bytes_recovered":1720320,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002897"},"pg_stats":[{"pgid":"2.f","version":"43'6652","reported_seq":6722,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:24.165968+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:14:24.165968+0000","last_peered":"2024-09-11T01:14:24.165968+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:14:24.165968+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":6652,"log_dups_size":0,"ondisk_log_size":6652,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27246592,"num_objects":6652,"num_object_clones":0,"num_object_copies":13304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6652,"num_write_kb":26608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2580},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6801","reported_seq":6898,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:28.500695+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:14:28.500695+0000","last_peered":"2024-09-11T01:14:28.500695+0000","last_clean":"2024-09-11T01:14:28.500695+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:14:28.500695+0000","last_undegraded":"2024-09-11T01:14:28.500695+0000","last_fullsized":"2024-09-11T01:14:28.500695+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":6801,"log_dups_size":0,"ondisk_log_size":6801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27832320,"num_objects":6795,"num_object_clones":0,"num_object_copies":13590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5040,"num_write_kb":20160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6665","reported_seq":6787,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:28.480253+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:14:28.480253+0000","last_peered":"2024-09-11T01:14:28.480253+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:14:28.480253+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":6665,"log_dups_size":0,"ondisk_log_size":6665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27283456,"num_objects":6661,"num_object_clones":0,"num_object_copies":13322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4966,"num_write_kb":19864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":2765}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6708","reported_seq":6773,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:28.500313+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:14:28.500313+0000","last_peered":"2024-09-11T01:14:28.500313+0000","last_clean":"2024-09-11T01:14:28.500313+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:14:28.500313+0000","last_undegraded":"2024-09-11T01:14:28.500313+0000","last_fullsized":"2024-09-11T01:14:28.500313+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":6708,"log_dups_size":0,"ondisk_log_size":6708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27475968,"num_objects":6708,"num_object_clones":0,"num_object_copies":13416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6708,"num_write_kb":26832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6593","reported_seq":6716,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:24.146908+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:14:24.146908+0000","last_peered":"2024-09-11T01:14:24.146908+0000","last_clean":"2024-09-11T01:14:24.146908+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:14:24.146908+0000","last_undegraded":"2024-09-11T01:14:24.146908+0000","last_fullsized":"2024-09-11T01:14:24.146908+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":6593,"log_dups_size":0,"ondisk_log_size":6593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26976256,"num_objects":6586,"num_object_clones":0,"num_object_copies":13172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3598,"num_write_kb":14392,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6579","reported_seq":6742,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:24.158177+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:14:24.158177+0000","last_peered":"2024-09-11T01:14:24.158177+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:14:24.158177+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":6579,"log_dups_size":0,"ondisk_log_size":6579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26927104,"num_objects":6574,"num_object_clones":0,"num_object_copies":13148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2920,"num_write_kb":11680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2623},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6483","reported_seq":6608,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:24.091042+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:14:24.091042+0000","last_peered":"2024-09-11T01:14:24.091042+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:14:24.091042+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":6483,"log_dups_size":0,"ondisk_log_size":6483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26529792,"num_objects":6477,"num_object_clones":0,"num_object_copies":12954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6477,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3340,"num_write_kb":13360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2491},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6621","reported_seq":6746,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:24.124350+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:14:24.124350+0000","last_peered":"2024-09-11T01:14:24.124350+0000","last_clean":"2024-09-11T01:14:24.124350+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:14:24.124350+0000","last_undegraded":"2024-09-11T01:14:24.124350+0000","last_fullsized":"2024-09-11T01:14:24.124350+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":6621,"log_dups_size":0,"ondisk_log_size":6621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27111424,"num_objects":6619,"num_object_clones":0,"num_object_copies":13238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3497,"num_write_kb":13988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6678","reported_seq":6800,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:28.457861+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:14:28.457861+0000","last_peered":"2024-09-11T01:14:28.457861+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:14:28.457861+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":6678,"log_dups_size":0,"ondisk_log_size":6678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27324416,"num_objects":6671,"num_object_clones":0,"num_object_copies":13342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4649,"num_write_kb":18596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":2738}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741922+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741922+0000","last_peered":"2024-09-11T01:13:09.741922+0000","last_clean":"2024-09-11T01:13:09.741922+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741922+0000","last_undegraded":"2024-09-11T01:13:09.741922+0000","last_fullsized":"2024-09-11T01:13:09.741922+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6512","reported_seq":6635,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:24.118252+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:14:24.118252+0000","last_peered":"2024-09-11T01:14:24.118252+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:14:24.118252+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":6512,"log_dups_size":0,"ondisk_log_size":6512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26640384,"num_objects":6504,"num_object_clones":0,"num_object_copies":13008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6504,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3099,"num_write_kb":12396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2520},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.910012+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.910012+0000","last_peered":"2024-09-11T01:13:09.910012+0000","last_clean":"2024-09-11T01:13:09.910012+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.910012+0000","last_undegraded":"2024-09-11T01:13:09.910012+0000","last_fullsized":"2024-09-11T01:13:09.910012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6716","reported_seq":6830,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:28.439763+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:14:28.439763+0000","last_peered":"2024-09-11T01:14:28.439763+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:14:28.439763+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":6716,"log_dups_size":0,"ondisk_log_size":6716,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27488256,"num_objects":6711,"num_object_clones":0,"num_object_copies":13422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4533,"num_write_kb":18132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":2687}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6604","reported_seq":10511,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:14:28.500693+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:14:28.500693+0000","last_peered":"2024-09-11T01:14:28.500693+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:14:28.500693+0000","last_undegraded":"2024-09-11T01:14:28.500693+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":6604,"log_dups_size":0,"ondisk_log_size":6604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27025408,"num_objects":6598,"num_object_clones":0,"num_object_copies":13196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2027,"num_objects_unfound":0,"num_objects_dirty":6598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4389,"num_write_kb":17556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1898,"num_bytes_recovered":7774208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2027},{"shards":"2","objects":4571}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6618","reported_seq":6745,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:24.158007+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:14:24.158007+0000","last_peered":"2024-09-11T01:14:24.158007+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:14:24.158007+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":6618,"log_dups_size":0,"ondisk_log_size":6618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27099136,"num_objects":6616,"num_object_clones":0,"num_object_copies":13232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3355,"num_write_kb":13420,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2666},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6495","reported_seq":6627,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:24.172537+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:14:24.172537+0000","last_peered":"2024-09-11T01:14:24.172537+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:14:24.172537+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":6495,"log_dups_size":0,"ondisk_log_size":6495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26570752,"num_objects":6487,"num_object_clones":0,"num_object_copies":12974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3150,"num_write_kb":12600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2548},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658174+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658174+0000","last_peered":"2024-09-11T01:13:09.658174+0000","last_clean":"2024-09-11T01:13:09.658174+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658174+0000","last_undegraded":"2024-09-11T01:13:09.658174+0000","last_fullsized":"2024-09-11T01:13:09.658174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6628","reported_seq":6758,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:24.066514+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:14:24.066514+0000","last_peered":"2024-09-11T01:14:24.066514+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:14:24.066514+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":6628,"log_dups_size":0,"ondisk_log_size":6628,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27131904,"num_objects":6624,"num_object_clones":0,"num_object_copies":13248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3237,"num_write_kb":12948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2585},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6541","reported_seq":6670,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:24.158749+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:14:24.158749+0000","last_peered":"2024-09-11T01:14:24.158749+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:14:24.158749+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":6541,"log_dups_size":0,"ondisk_log_size":6541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26743401,"num_objects":6530,"num_object_clones":0,"num_object_copies":13060,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6530,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3611,"num_write_kb":14444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2613},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":433406569,"num_objects":105813,"num_object_clones":0,"num_object_copies":211626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":2027,"num_objects_unfound":0,"num_objects_dirty":105813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":66744,"num_write_kb":266976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1899,"num_bytes_recovered":7778304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":877551616,"data_stored":877464641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":105894,"ondisk_log_size":105894,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182435,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":572464,"kb_used_data":378400,"kb_used_omap":4,"kb_used_meta":194043,"kb_avail":93799376,"statfs":{"total":96636764160,"available":96050561024,"internally_reserved":0,"allocated":387481600,"data_stored":387232781,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":198700715},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,8,3,2,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986208,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517744,"kb_used_data":341728,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93854096,"statfs":{"total":96636764160,"available":96106594304,"internally_reserved":0,"allocated":349929472,"data_stored":349684730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180219562},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,0,2,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659153,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":265916,"kb_used_data":135724,"kb_used_omap":4,"kb_used_meta":130171,"kb_avail":94105924,"statfs":{"total":96636764160,"available":96364466176,"internally_reserved":0,"allocated":138981376,"data_stored":138656019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":133295791},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,1,1,4,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":387796992,"data_stored":387793513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139608064,"data_stored":139528047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":350146560,"data_stored":350143081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:14:28.683 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:14:28.683 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'6652", 'reported_seq': 6722, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:24.165968+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:14:24.165968+0000', 'last_peered': '2024-09-11T01:14:24.165968+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:14:24.165968+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 6652, 'log_dups_size': 0, 'ondisk_log_size': 6652, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27246592, 'num_objects': 6652, 'num_object_clones': 0, 'num_object_copies': 13304, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6652, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6652, 'num_write_kb': 26608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2580}, {'shards': '0,2', 'objects': 4072}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:28.683 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:14:28.683 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'6665", 'reported_seq': 6787, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:28.480253+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:14:28.480253+0000', 'last_peered': '2024-09-11T01:14:28.480253+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:14:28.480253+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 6665, 'log_dups_size': 0, 'ondisk_log_size': 6665, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27283456, 'num_objects': 6661, 'num_object_clones': 0, 'num_object_copies': 13322, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6661, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4966, 'num_write_kb': 19864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3896}, {'shards': '2', 'objects': 2765}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:14:28.683 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:14:28.683 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'6579", 'reported_seq': 6742, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:24.158177+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:14:24.158177+0000', 'last_peered': '2024-09-11T01:14:24.158177+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:14:24.158177+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 6579, 'log_dups_size': 0, 'ondisk_log_size': 6579, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26927104, 'num_objects': 6574, 'num_object_clones': 0, 'num_object_copies': 13148, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6574, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2920, 'num_write_kb': 11680, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2623}, {'shards': '0,2', 'objects': 3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:28.683 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:14:28.683 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'6483", 'reported_seq': 6608, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:24.091042+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:14:24.091042+0000', 'last_peered': '2024-09-11T01:14:24.091042+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:14:24.091042+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 6483, 'log_dups_size': 0, 'ondisk_log_size': 6483, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26529792, 'num_objects': 6477, 'num_object_clones': 0, 'num_object_copies': 12954, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6477, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3340, 'num_write_kb': 13360, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2491}, {'shards': '0,2', 'objects': 3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:28.683 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:14:28.683 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'6678", 'reported_seq': 6800, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:28.457861+0000', 'last_change': '2024-09-11T01:13:10.159069+0000', 'last_active': '2024-09-11T01:14:28.457861+0000', 'last_peered': '2024-09-11T01:14:28.457861+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:14:28.457861+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 6678, 'log_dups_size': 0, 'ondisk_log_size': 6678, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27324416, 'num_objects': 6671, 'num_object_clones': 0, 'num_object_copies': 13342, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6671, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4649, 'num_write_kb': 18596, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3932}, {'shards': '2', 'objects': 2738}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:14:28.684 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:14:28.684 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'6512", 'reported_seq': 6635, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:24.118252+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:14:24.118252+0000', 'last_peered': '2024-09-11T01:14:24.118252+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:14:24.118252+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 6512, 'log_dups_size': 0, 'ondisk_log_size': 6512, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26640384, 'num_objects': 6504, 'num_object_clones': 0, 'num_object_copies': 13008, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6504, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3099, 'num_write_kb': 12396, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2520}, {'shards': '0,2', 'objects': 3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:28.684 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:14:28.684 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'6716", 'reported_seq': 6830, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:28.439763+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:14:28.439763+0000', 'last_peered': '2024-09-11T01:14:28.439763+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:14:28.439763+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 6716, 'log_dups_size': 0, 'ondisk_log_size': 6716, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27488256, 'num_objects': 6711, 'num_object_clones': 0, 'num_object_copies': 13422, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6711, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4533, 'num_write_kb': 18132, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4023}, {'shards': '2', 'objects': 2687}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:14:28.684 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:14:28.684 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'6604", 'reported_seq': 10511, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:14:28.500693+0000', 'last_change': '2024-09-11T01:13:10.217540+0000', 'last_active': '2024-09-11T01:14:28.500693+0000', 'last_peered': '2024-09-11T01:14:28.500693+0000', 'last_clean': '2024-09-11T01:13:07.638709+0000', 'last_became_active': '2024-09-11T01:13:10.125604+0000', 'last_became_peered': '2024-09-11T01:13:10.125604+0000', 'last_unstale': '2024-09-11T01:14:28.500693+0000', 'last_undegraded': '2024-09-11T01:14:28.500693+0000', 'last_fullsized': '2024-09-11T01:13:09.649604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2269", 'last_scrub_stamp': '2024-09-11T01:12:42.890920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:42.890920+0000', 'objects_scrubbed': 2209, 'log_size': 6604, 'log_dups_size': 0, 'ondisk_log_size': 6604, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 999, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27025408, 'num_objects': 6598, 'num_object_clones': 0, 'num_object_copies': 13196, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2027, 'num_objects_unfound': 0, 'num_objects_dirty': 6598, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4389, 'num_write_kb': 17556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1898, 'num_bytes_recovered': 7774208, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2027}, {'shards': '2', 'objects': 4571}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:14:28.684 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:14:28.684 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'6618", 'reported_seq': 6745, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:24.158007+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:14:24.158007+0000', 'last_peered': '2024-09-11T01:14:24.158007+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:14:24.158007+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 6618, 'log_dups_size': 0, 'ondisk_log_size': 6618, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27099136, 'num_objects': 6616, 'num_object_clones': 0, 'num_object_copies': 13232, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6616, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3355, 'num_write_kb': 13420, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2666}, {'shards': '0,2', 'objects': 3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:28.684 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:14:28.684 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'6495", 'reported_seq': 6627, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:24.172537+0000', 'last_change': '2024-09-11T01:13:10.209668+0000', 'last_active': '2024-09-11T01:14:24.172537+0000', 'last_peered': '2024-09-11T01:14:24.172537+0000', 'last_clean': '2024-09-11T01:13:07.624565+0000', 'last_became_active': '2024-09-11T01:13:10.199795+0000', 'last_became_peered': '2024-09-11T01:13:10.199795+0000', 'last_unstale': '2024-09-11T01:14:24.172537+0000', 'last_undegraded': '2024-09-11T01:13:09.791198+0000', 'last_fullsized': '2024-09-11T01:13:09.652311+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3415", 'last_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'objects_scrubbed': 3337, 'log_size': 6495, 'log_dups_size': 0, 'ondisk_log_size': 6495, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1975, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26570752, 'num_objects': 6487, 'num_object_clones': 0, 'num_object_copies': 12974, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3942, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6487, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3150, 'num_write_kb': 12600, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2548}, {'shards': '0,2', 'objects': 3939}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:28.684 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:14:28.685 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'6628", 'reported_seq': 6758, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:24.066514+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:14:24.066514+0000', 'last_peered': '2024-09-11T01:14:24.066514+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:14:24.066514+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 6628, 'log_dups_size': 0, 'ondisk_log_size': 6628, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27131904, 'num_objects': 6624, 'num_object_clones': 0, 'num_object_copies': 13248, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6624, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3237, 'num_write_kb': 12948, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2585}, {'shards': '0,2', 'objects': 4039}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:28.685 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:14:28.685 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'6541", 'reported_seq': 6670, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:24.158749+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:14:24.158749+0000', 'last_peered': '2024-09-11T01:14:24.158749+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:14:24.158749+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 6541, 'log_dups_size': 0, 'ondisk_log_size': 6541, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26743401, 'num_objects': 6530, 'num_object_clones': 0, 'num_object_copies': 13060, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6530, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3611, 'num_write_kb': 14444, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2613}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:28.685 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:14:29.379 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 125 20 107654 107634 3.36255 1.89062 0.0184907 0.0232248 2024-09-11T01:14:29.709 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:14:29.709 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43711/211630 objects degraded (20.654%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":206,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":105815,"data_bytes":433406588,"bytes_used":1388670976,"bytes_avail":288521621504,"bytes_total":289910292480,"degraded_objects":43711,"degraded_total":211630,"degraded_ratio":0.20654444076926712,"misplaced_objects":2027,"misplaced_total":211630,"misplaced_ratio":0.0095780371402920186,"recovering_objects_per_sec":34,"recovering_bytes_per_sec":143325,"recovering_keys_per_sec":0,"num_objects_recovered":420,"num_bytes_recovered":1720320,"num_keys_recovered":0,"write_bytes_sec":2065250,"read_op_per_sec":0,"write_op_per_sec":504},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:14:29.791 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:14:29.791 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:14:30.380 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 126 20 108141 108121 3.35096 1.90234 0.0371966 0.0233053 2024-09-11T01:14:30.734 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:14:30.735 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:14:30.793 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":140,"stamp":"2024-09-11T01:14:30.580407+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":440156796,"num_objects":107463,"num_object_clones":0,"num_object_copies":214926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":2027,"num_objects_unfound":0,"num_objects_dirty":107463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68394,"num_write_kb":273570,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1900,"num_bytes_recovered":7778323,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107545,"ondisk_log_size":107545,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1390356,"kb_used_data":863716,"kb_used_omap":12,"kb_used_meta":526579,"kb_avail":281725164,"statfs":{"total":289910292480,"available":288486567936,"internally_reserved":0,"allocated":884445184,"data_stored":883626266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":539216900},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,8,3,4,9,11,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27533312,"num_objects":6722,"num_object_clones":0,"num_object_copies":13444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-292,"num_objects_unfound":0,"num_objects_dirty":6722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6722,"num_write_kb":26888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":292,"num_bytes_recovered":1196032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003588"},"pg_stats":[{"pgid":"2.f","version":"43'6824","reported_seq":6895,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:29.073777+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:14:29.073777+0000","last_peered":"2024-09-11T01:14:29.073777+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:14:29.073777+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":6824,"log_dups_size":0,"ondisk_log_size":6824,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27951104,"num_objects":6824,"num_object_clones":0,"num_object_copies":13648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6824,"num_write_kb":27296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2752},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6801","reported_seq":6898,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:28.500695+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:14:28.500695+0000","last_peered":"2024-09-11T01:14:28.500695+0000","last_clean":"2024-09-11T01:14:28.500695+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:14:28.500695+0000","last_undegraded":"2024-09-11T01:14:28.500695+0000","last_fullsized":"2024-09-11T01:14:28.500695+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":6801,"log_dups_size":0,"ondisk_log_size":6801,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27832320,"num_objects":6795,"num_object_clones":0,"num_object_copies":13590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5040,"num_write_kb":20160,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6665","reported_seq":6787,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:28.480253+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:14:28.480253+0000","last_peered":"2024-09-11T01:14:28.480253+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:14:28.480253+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":6665,"log_dups_size":0,"ondisk_log_size":6665,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27283456,"num_objects":6661,"num_object_clones":0,"num_object_copies":13322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4966,"num_write_kb":19864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":2765}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6708","reported_seq":6773,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:28.500313+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:14:28.500313+0000","last_peered":"2024-09-11T01:14:28.500313+0000","last_clean":"2024-09-11T01:14:28.500313+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:14:28.500313+0000","last_undegraded":"2024-09-11T01:14:28.500313+0000","last_fullsized":"2024-09-11T01:14:28.500313+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":6708,"log_dups_size":0,"ondisk_log_size":6708,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27475968,"num_objects":6708,"num_object_clones":0,"num_object_copies":13416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6708,"num_write_kb":26832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6758","reported_seq":6881,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:29.177613+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:14:29.177613+0000","last_peered":"2024-09-11T01:14:29.177613+0000","last_clean":"2024-09-11T01:14:29.177613+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:14:29.177613+0000","last_undegraded":"2024-09-11T01:14:29.177613+0000","last_fullsized":"2024-09-11T01:14:29.177613+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":6758,"log_dups_size":0,"ondisk_log_size":6758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27652096,"num_objects":6751,"num_object_clones":0,"num_object_copies":13502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3763,"num_write_kb":15052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6730","reported_seq":6895,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:29.167102+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:14:29.167102+0000","last_peered":"2024-09-11T01:14:29.167102+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:14:29.167102+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":6730,"log_dups_size":0,"ondisk_log_size":6730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27545600,"num_objects":6725,"num_object_clones":0,"num_object_copies":13450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3071,"num_write_kb":12284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2774},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6648","reported_seq":6773,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:29.257560+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:14:29.257560+0000","last_peered":"2024-09-11T01:14:29.257560+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:14:29.257560+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":6648,"log_dups_size":0,"ondisk_log_size":6648,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27205632,"num_objects":6642,"num_object_clones":0,"num_object_copies":13284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3505,"num_write_kb":14020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2656},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6795","reported_seq":6920,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:29.210509+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:14:29.210509+0000","last_peered":"2024-09-11T01:14:29.210509+0000","last_clean":"2024-09-11T01:14:29.210509+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:14:29.210509+0000","last_undegraded":"2024-09-11T01:14:29.210509+0000","last_fullsized":"2024-09-11T01:14:29.210509+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":6795,"log_dups_size":0,"ondisk_log_size":6795,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27824128,"num_objects":6793,"num_object_clones":0,"num_object_copies":13586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3671,"num_write_kb":14684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6678","reported_seq":6800,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:28.457861+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:14:28.457861+0000","last_peered":"2024-09-11T01:14:28.457861+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:14:28.457861+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":6678,"log_dups_size":0,"ondisk_log_size":6678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27324416,"num_objects":6671,"num_object_clones":0,"num_object_copies":13342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4649,"num_write_kb":18596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":2738}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741922+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741922+0000","last_peered":"2024-09-11T01:13:09.741922+0000","last_clean":"2024-09-11T01:13:09.741922+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741922+0000","last_undegraded":"2024-09-11T01:13:09.741922+0000","last_fullsized":"2024-09-11T01:13:09.741922+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6677","reported_seq":6800,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:29.228039+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:14:29.228039+0000","last_peered":"2024-09-11T01:14:29.228039+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:14:29.228039+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":6677,"log_dups_size":0,"ondisk_log_size":6677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27316224,"num_objects":6669,"num_object_clones":0,"num_object_copies":13338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3264,"num_write_kb":13056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2685},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.910012+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.910012+0000","last_peered":"2024-09-11T01:13:09.910012+0000","last_clean":"2024-09-11T01:13:09.910012+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.910012+0000","last_undegraded":"2024-09-11T01:13:09.910012+0000","last_fullsized":"2024-09-11T01:13:09.910012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6716","reported_seq":6830,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:28.439763+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:14:28.439763+0000","last_peered":"2024-09-11T01:14:28.439763+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:14:28.439763+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":6716,"log_dups_size":0,"ondisk_log_size":6716,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27488256,"num_objects":6711,"num_object_clones":0,"num_object_copies":13422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4533,"num_write_kb":18132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":2687}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6604","reported_seq":10511,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:14:28.500693+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:14:28.500693+0000","last_peered":"2024-09-11T01:14:28.500693+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:14:28.500693+0000","last_undegraded":"2024-09-11T01:14:28.500693+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":6604,"log_dups_size":0,"ondisk_log_size":6604,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27025408,"num_objects":6598,"num_object_clones":0,"num_object_copies":13196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2027,"num_objects_unfound":0,"num_objects_dirty":6598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4389,"num_write_kb":17556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1898,"num_bytes_recovered":7774208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2027},{"shards":"2","objects":4571}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6794","reported_seq":6921,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:29.234456+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:14:29.234456+0000","last_peered":"2024-09-11T01:14:29.234456+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:14:29.234456+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":6794,"log_dups_size":0,"ondisk_log_size":6794,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27820032,"num_objects":6792,"num_object_clones":0,"num_object_copies":13584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3531,"num_write_kb":14124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2842},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6644","reported_seq":6776,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:29.217038+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:14:29.217038+0000","last_peered":"2024-09-11T01:14:29.217038+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:14:29.217038+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":6644,"log_dups_size":0,"ondisk_log_size":6644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27181056,"num_objects":6636,"num_object_clones":0,"num_object_copies":13272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3299,"num_write_kb":13196,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2697},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658174+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658174+0000","last_peered":"2024-09-11T01:13:09.658174+0000","last_clean":"2024-09-11T01:13:09.658174+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658174+0000","last_undegraded":"2024-09-11T01:13:09.658174+0000","last_fullsized":"2024-09-11T01:13:09.658174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6787","reported_seq":6917,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:29.216126+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:14:29.216126+0000","last_peered":"2024-09-11T01:14:29.216126+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:14:29.216126+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":6787,"log_dups_size":0,"ondisk_log_size":6787,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27783168,"num_objects":6783,"num_object_clones":0,"num_object_copies":13566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6783,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3396,"num_write_kb":13584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2744},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6713","reported_seq":6842,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:29.262020+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:14:29.262020+0000","last_peered":"2024-09-11T01:14:29.262020+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:14:29.262020+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":6713,"log_dups_size":0,"ondisk_log_size":6713,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27447913,"num_objects":6702,"num_object_clones":0,"num_object_copies":13404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3783,"num_write_kb":15132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2785},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":440156777,"num_objects":107461,"num_object_clones":0,"num_object_copies":214922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":2027,"num_objects_unfound":0,"num_objects_dirty":107461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":68392,"num_write_kb":273568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1899,"num_bytes_recovered":7778304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":885628928,"data_stored":885541953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":107542,"ondisk_log_size":107542,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182436,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":606696,"kb_used_data":386264,"kb_used_omap":4,"kb_used_meta":220411,"kb_avail":93765144,"statfs":{"total":96636764160,"available":96015507456,"internally_reserved":0,"allocated":395534336,"data_stored":395285517,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":225701547},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,3,3,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986208,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517744,"kb_used_data":341728,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93854096,"statfs":{"total":96636764160,"available":96106594304,"internally_reserved":0,"allocated":349929472,"data_stored":349684730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180219562},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,0,2,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659153,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":265916,"kb_used_data":135724,"kb_used_omap":4,"kb_used_meta":130171,"kb_avail":94105924,"statfs":{"total":96636764160,"available":96364466176,"internally_reserved":0,"allocated":138981376,"data_stored":138656019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":133295791},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,5,1,1,4,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":395874304,"data_stored":395870825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139608064,"data_stored":139528047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":350146560,"data_stored":350143081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:14:31.382 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 127 20 108659 108639 3.34045 2.02344 0.0278674 0.0233802 2024-09-11T01:14:32.381 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 128 20 109188 109168 3.3305 2.06641 0.0342332 0.0234476 2024-09-11T01:14:33.382 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 129 20 109719 109699 3.32076 2.07422 0.0163109 0.0235183 2024-09-11T01:14:33.796 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:14:34.382 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 130 20 110229 110209 3.31054 1.99219 0.0262201 0.0235907 2024-09-11T01:14:34.639 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:14:34.640 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:14:34.694 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-09-11T01:14:34.582350+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":450675324,"num_objects":110031,"num_object_clones":0,"num_object_copies":220062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":1869,"num_objects_unfound":0,"num_objects_dirty":110031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":70962,"num_write_kb":283842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2058,"num_bytes_recovered":8425491,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110113,"ondisk_log_size":110113,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1462644,"kb_used_data":884612,"kb_used_omap":12,"kb_used_meta":577971,"kb_avail":281652876,"statfs":{"total":289910292480,"available":288412545024,"internally_reserved":0,"allocated":905842688,"data_stored":905023770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":591842308},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,4,4,15,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":31551488,"num_objects":7703,"num_object_clones":0,"num_object_copies":15406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-450,"num_objects_unfound":0,"num_objects_dirty":7703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7703,"num_write_kb":30812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":450,"num_bytes_recovered":1843200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004439"},"pg_stats":[{"pgid":"2.f","version":"43'6988","reported_seq":7058,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:34.106623+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:14:34.106623+0000","last_peered":"2024-09-11T01:14:34.106623+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:14:34.106623+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":6988,"log_dups_size":0,"ondisk_log_size":6988,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28622848,"num_objects":6988,"num_object_clones":0,"num_object_copies":13976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6988,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2916},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6958","reported_seq":7055,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:33.530031+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:14:33.530031+0000","last_peered":"2024-09-11T01:14:33.530031+0000","last_clean":"2024-09-11T01:14:33.530031+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:14:33.530031+0000","last_undegraded":"2024-09-11T01:14:33.530031+0000","last_fullsized":"2024-09-11T01:14:33.530031+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":6958,"log_dups_size":0,"ondisk_log_size":6958,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28475392,"num_objects":6952,"num_object_clones":0,"num_object_copies":13904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5197,"num_write_kb":20788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6815","reported_seq":6937,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:33.525934+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:14:33.525934+0000","last_peered":"2024-09-11T01:14:33.525934+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:14:33.525934+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":6815,"log_dups_size":0,"ondisk_log_size":6815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27897856,"num_objects":6811,"num_object_clones":0,"num_object_copies":13622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5116,"num_write_kb":20464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":2915}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6842","reported_seq":6907,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:33.434320+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:14:33.434320+0000","last_peered":"2024-09-11T01:14:33.434320+0000","last_clean":"2024-09-11T01:14:33.434320+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:14:33.434320+0000","last_undegraded":"2024-09-11T01:14:33.434320+0000","last_fullsized":"2024-09-11T01:14:33.434320+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":6842,"log_dups_size":0,"ondisk_log_size":6842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28024832,"num_objects":6842,"num_object_clones":0,"num_object_copies":13684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6842,"num_write_kb":27368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6943","reported_seq":7066,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:34.175691+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:14:34.175691+0000","last_peered":"2024-09-11T01:14:34.175691+0000","last_clean":"2024-09-11T01:14:34.175691+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:14:34.175691+0000","last_undegraded":"2024-09-11T01:14:34.175691+0000","last_fullsized":"2024-09-11T01:14:34.175691+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":6943,"log_dups_size":0,"ondisk_log_size":6943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28409856,"num_objects":6936,"num_object_clones":0,"num_object_copies":13872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3948,"num_write_kb":15792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6890","reported_seq":7055,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:34.206470+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:14:34.206470+0000","last_peered":"2024-09-11T01:14:34.206470+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:14:34.206470+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":6890,"log_dups_size":0,"ondisk_log_size":6890,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28200960,"num_objects":6885,"num_object_clones":0,"num_object_copies":13770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3231,"num_write_kb":12924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2934},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6814","reported_seq":6939,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:34.169096+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:14:34.169096+0000","last_peered":"2024-09-11T01:14:34.169096+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:14:34.169096+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":6814,"log_dups_size":0,"ondisk_log_size":6814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27885568,"num_objects":6808,"num_object_clones":0,"num_object_copies":13616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6808,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3671,"num_write_kb":14684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2822},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6958","reported_seq":7083,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:34.226947+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:14:34.226947+0000","last_peered":"2024-09-11T01:14:34.226947+0000","last_clean":"2024-09-11T01:14:34.226947+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:14:34.226947+0000","last_undegraded":"2024-09-11T01:14:34.226947+0000","last_fullsized":"2024-09-11T01:14:34.226947+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":6958,"log_dups_size":0,"ondisk_log_size":6958,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28491776,"num_objects":6956,"num_object_clones":0,"num_object_copies":13912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6956,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3834,"num_write_kb":15336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6834","reported_seq":6957,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:33.446462+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:14:33.446462+0000","last_peered":"2024-09-11T01:14:33.446462+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:14:33.446462+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":6834,"log_dups_size":0,"ondisk_log_size":6834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27963392,"num_objects":6827,"num_object_clones":0,"num_object_copies":13654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4805,"num_write_kb":19220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":2894}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741922+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741922+0000","last_peered":"2024-09-11T01:13:09.741922+0000","last_clean":"2024-09-11T01:13:09.741922+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741922+0000","last_undegraded":"2024-09-11T01:13:09.741922+0000","last_fullsized":"2024-09-11T01:13:09.741922+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6847","reported_seq":6969,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:34.247237+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:14:34.247237+0000","last_peered":"2024-09-11T01:14:34.247237+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:14:34.247237+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":6847,"log_dups_size":0,"ondisk_log_size":6847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28012544,"num_objects":6839,"num_object_clones":0,"num_object_copies":13678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3434,"num_write_kb":13736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2855},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.910012+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.910012+0000","last_peered":"2024-09-11T01:13:09.910012+0000","last_clean":"2024-09-11T01:13:09.910012+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.910012+0000","last_undegraded":"2024-09-11T01:13:09.910012+0000","last_fullsized":"2024-09-11T01:13:09.910012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6873","reported_seq":6987,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:33.527687+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:14:33.527687+0000","last_peered":"2024-09-11T01:14:33.527687+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:14:33.527687+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":6873,"log_dups_size":0,"ondisk_log_size":6873,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28131328,"num_objects":6868,"num_object_clones":0,"num_object_copies":13736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4690,"num_write_kb":18760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":2844}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6762","reported_seq":10989,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:14:33.514940+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:14:33.514940+0000","last_peered":"2024-09-11T01:14:33.514940+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:14:33.514940+0000","last_undegraded":"2024-09-11T01:14:33.514940+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":6762,"log_dups_size":0,"ondisk_log_size":6762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27672576,"num_objects":6756,"num_object_clones":0,"num_object_copies":13512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1869,"num_objects_unfound":0,"num_objects_dirty":6756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4547,"num_write_kb":18188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2056,"num_bytes_recovered":8421376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1869},{"shards":"2","objects":4887}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6953","reported_seq":7080,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:34.227835+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:14:34.227835+0000","last_peered":"2024-09-11T01:14:34.227835+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:14:34.227835+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":6953,"log_dups_size":0,"ondisk_log_size":6953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28471296,"num_objects":6951,"num_object_clones":0,"num_object_copies":13902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3690,"num_write_kb":14760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3001},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6812","reported_seq":6944,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:34.231257+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:14:34.231257+0000","last_peered":"2024-09-11T01:14:34.231257+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:14:34.231257+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":6812,"log_dups_size":0,"ondisk_log_size":6812,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27869184,"num_objects":6804,"num_object_clones":0,"num_object_copies":13608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3467,"num_write_kb":13868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2865},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658174+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658174+0000","last_peered":"2024-09-11T01:13:09.658174+0000","last_clean":"2024-09-11T01:13:09.658174+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658174+0000","last_undegraded":"2024-09-11T01:13:09.658174+0000","last_fullsized":"2024-09-11T01:13:09.658174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6951","reported_seq":7079,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:34.288075+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:14:34.288075+0000","last_peered":"2024-09-11T01:14:34.288075+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:14:34.288075+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":6951,"log_dups_size":0,"ondisk_log_size":6951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28454912,"num_objects":6947,"num_object_clones":0,"num_object_copies":13894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3560,"num_write_kb":14240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2908},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6870","reported_seq":6999,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:34.227052+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:14:34.227052+0000","last_peered":"2024-09-11T01:14:34.227052+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:14:34.227052+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":6870,"log_dups_size":0,"ondisk_log_size":6870,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28090985,"num_objects":6859,"num_object_clones":0,"num_object_copies":13718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3940,"num_write_kb":15760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2942},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":450675305,"num_objects":110029,"num_object_clones":0,"num_object_copies":220058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":1869,"num_objects_unfound":0,"num_objects_dirty":110029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":70960,"num_write_kb":283840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2057,"num_bytes_recovered":8425472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":907169792,"data_stored":907082817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110110,"ondisk_log_size":110110,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182437,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":656132,"kb_used_data":394036,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93715708,"statfs":{"total":96636764160,"available":95964884992,"internally_reserved":0,"allocated":403492864,"data_stored":403244045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":268365483},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,1,7,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986209,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":526892,"kb_used_data":346716,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93844948,"statfs":{"total":96636764160,"available":96097226752,"internally_reserved":0,"allocated":355037184,"data_stored":354792442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184479402},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659154,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":279620,"kb_used_data":143860,"kb_used_omap":4,"kb_used_meta":135739,"kb_avail":94092220,"statfs":{"total":96636764160,"available":96350433280,"internally_reserved":0,"allocated":147312640,"data_stored":146987283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":138997423},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":403881984,"data_stored":403878505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":147976192,"data_stored":147896175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":355311616,"data_stored":355308137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:14:34.696 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:14:35.382 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 131 20 110726 110706 3.30009 1.94141 0.0162816 0.0236665 2024-09-11T01:14:35.514 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:14:35.520 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:14:35.598 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":142,"stamp":"2024-09-11T01:14:34.582350+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":450675324,"num_objects":110031,"num_object_clones":0,"num_object_copies":220062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":1869,"num_objects_unfound":0,"num_objects_dirty":110031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":70962,"num_write_kb":283842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2058,"num_bytes_recovered":8425491,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110113,"ondisk_log_size":110113,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1462644,"kb_used_data":884612,"kb_used_omap":12,"kb_used_meta":577971,"kb_avail":281652876,"statfs":{"total":289910292480,"available":288412545024,"internally_reserved":0,"allocated":905842688,"data_stored":905023770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":591842308},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,4,4,15,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":31551488,"num_objects":7703,"num_object_clones":0,"num_object_copies":15406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-450,"num_objects_unfound":0,"num_objects_dirty":7703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7703,"num_write_kb":30812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":450,"num_bytes_recovered":1843200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004439"},"pg_stats":[{"pgid":"2.f","version":"43'6988","reported_seq":7058,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:34.106623+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:14:34.106623+0000","last_peered":"2024-09-11T01:14:34.106623+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:14:34.106623+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":6988,"log_dups_size":0,"ondisk_log_size":6988,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28622848,"num_objects":6988,"num_object_clones":0,"num_object_copies":13976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6988,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2916},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6958","reported_seq":7055,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:33.530031+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:14:33.530031+0000","last_peered":"2024-09-11T01:14:33.530031+0000","last_clean":"2024-09-11T01:14:33.530031+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:14:33.530031+0000","last_undegraded":"2024-09-11T01:14:33.530031+0000","last_fullsized":"2024-09-11T01:14:33.530031+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":6958,"log_dups_size":0,"ondisk_log_size":6958,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28475392,"num_objects":6952,"num_object_clones":0,"num_object_copies":13904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5197,"num_write_kb":20788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6815","reported_seq":6937,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:33.525934+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:14:33.525934+0000","last_peered":"2024-09-11T01:14:33.525934+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:14:33.525934+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":6815,"log_dups_size":0,"ondisk_log_size":6815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27897856,"num_objects":6811,"num_object_clones":0,"num_object_copies":13622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5116,"num_write_kb":20464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":2915}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6842","reported_seq":6907,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:33.434320+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:14:33.434320+0000","last_peered":"2024-09-11T01:14:33.434320+0000","last_clean":"2024-09-11T01:14:33.434320+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:14:33.434320+0000","last_undegraded":"2024-09-11T01:14:33.434320+0000","last_fullsized":"2024-09-11T01:14:33.434320+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":6842,"log_dups_size":0,"ondisk_log_size":6842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28024832,"num_objects":6842,"num_object_clones":0,"num_object_copies":13684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6842,"num_write_kb":27368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6943","reported_seq":7066,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:34.175691+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:14:34.175691+0000","last_peered":"2024-09-11T01:14:34.175691+0000","last_clean":"2024-09-11T01:14:34.175691+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:14:34.175691+0000","last_undegraded":"2024-09-11T01:14:34.175691+0000","last_fullsized":"2024-09-11T01:14:34.175691+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":6943,"log_dups_size":0,"ondisk_log_size":6943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28409856,"num_objects":6936,"num_object_clones":0,"num_object_copies":13872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3948,"num_write_kb":15792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6890","reported_seq":7055,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:34.206470+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:14:34.206470+0000","last_peered":"2024-09-11T01:14:34.206470+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:14:34.206470+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":6890,"log_dups_size":0,"ondisk_log_size":6890,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28200960,"num_objects":6885,"num_object_clones":0,"num_object_copies":13770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3231,"num_write_kb":12924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2934},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6814","reported_seq":6939,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:34.169096+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:14:34.169096+0000","last_peered":"2024-09-11T01:14:34.169096+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:14:34.169096+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":6814,"log_dups_size":0,"ondisk_log_size":6814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27885568,"num_objects":6808,"num_object_clones":0,"num_object_copies":13616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6808,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3671,"num_write_kb":14684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2822},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6958","reported_seq":7083,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:34.226947+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:14:34.226947+0000","last_peered":"2024-09-11T01:14:34.226947+0000","last_clean":"2024-09-11T01:14:34.226947+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:14:34.226947+0000","last_undegraded":"2024-09-11T01:14:34.226947+0000","last_fullsized":"2024-09-11T01:14:34.226947+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":6958,"log_dups_size":0,"ondisk_log_size":6958,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28491776,"num_objects":6956,"num_object_clones":0,"num_object_copies":13912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6956,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3834,"num_write_kb":15336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6834","reported_seq":6957,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:33.446462+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:14:33.446462+0000","last_peered":"2024-09-11T01:14:33.446462+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:14:33.446462+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":6834,"log_dups_size":0,"ondisk_log_size":6834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27963392,"num_objects":6827,"num_object_clones":0,"num_object_copies":13654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4805,"num_write_kb":19220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":2894}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741922+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741922+0000","last_peered":"2024-09-11T01:13:09.741922+0000","last_clean":"2024-09-11T01:13:09.741922+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741922+0000","last_undegraded":"2024-09-11T01:13:09.741922+0000","last_fullsized":"2024-09-11T01:13:09.741922+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6847","reported_seq":6969,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:34.247237+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:14:34.247237+0000","last_peered":"2024-09-11T01:14:34.247237+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:14:34.247237+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":6847,"log_dups_size":0,"ondisk_log_size":6847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28012544,"num_objects":6839,"num_object_clones":0,"num_object_copies":13678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3434,"num_write_kb":13736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2855},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.910012+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.910012+0000","last_peered":"2024-09-11T01:13:09.910012+0000","last_clean":"2024-09-11T01:13:09.910012+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.910012+0000","last_undegraded":"2024-09-11T01:13:09.910012+0000","last_fullsized":"2024-09-11T01:13:09.910012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6873","reported_seq":6987,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:33.527687+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:14:33.527687+0000","last_peered":"2024-09-11T01:14:33.527687+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:14:33.527687+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":6873,"log_dups_size":0,"ondisk_log_size":6873,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28131328,"num_objects":6868,"num_object_clones":0,"num_object_copies":13736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4690,"num_write_kb":18760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":2844}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6762","reported_seq":10989,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:14:33.514940+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:14:33.514940+0000","last_peered":"2024-09-11T01:14:33.514940+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:14:33.514940+0000","last_undegraded":"2024-09-11T01:14:33.514940+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":6762,"log_dups_size":0,"ondisk_log_size":6762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27672576,"num_objects":6756,"num_object_clones":0,"num_object_copies":13512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1869,"num_objects_unfound":0,"num_objects_dirty":6756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4547,"num_write_kb":18188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2056,"num_bytes_recovered":8421376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1869},{"shards":"2","objects":4887}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6953","reported_seq":7080,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:34.227835+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:14:34.227835+0000","last_peered":"2024-09-11T01:14:34.227835+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:14:34.227835+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":6953,"log_dups_size":0,"ondisk_log_size":6953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28471296,"num_objects":6951,"num_object_clones":0,"num_object_copies":13902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3690,"num_write_kb":14760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3001},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6812","reported_seq":6944,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:34.231257+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:14:34.231257+0000","last_peered":"2024-09-11T01:14:34.231257+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:14:34.231257+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":6812,"log_dups_size":0,"ondisk_log_size":6812,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27869184,"num_objects":6804,"num_object_clones":0,"num_object_copies":13608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3467,"num_write_kb":13868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2865},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658174+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658174+0000","last_peered":"2024-09-11T01:13:09.658174+0000","last_clean":"2024-09-11T01:13:09.658174+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658174+0000","last_undegraded":"2024-09-11T01:13:09.658174+0000","last_fullsized":"2024-09-11T01:13:09.658174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6951","reported_seq":7079,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:34.288075+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:14:34.288075+0000","last_peered":"2024-09-11T01:14:34.288075+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:14:34.288075+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":6951,"log_dups_size":0,"ondisk_log_size":6951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28454912,"num_objects":6947,"num_object_clones":0,"num_object_copies":13894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3560,"num_write_kb":14240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2908},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6870","reported_seq":6999,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:34.227052+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:14:34.227052+0000","last_peered":"2024-09-11T01:14:34.227052+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:14:34.227052+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":6870,"log_dups_size":0,"ondisk_log_size":6870,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28090985,"num_objects":6859,"num_object_clones":0,"num_object_copies":13718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3940,"num_write_kb":15760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2942},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":450675305,"num_objects":110029,"num_object_clones":0,"num_object_copies":220058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":1869,"num_objects_unfound":0,"num_objects_dirty":110029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":70960,"num_write_kb":283840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2057,"num_bytes_recovered":8425472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":907169792,"data_stored":907082817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110110,"ondisk_log_size":110110,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182437,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":656132,"kb_used_data":394036,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93715708,"statfs":{"total":96636764160,"available":95964884992,"internally_reserved":0,"allocated":403492864,"data_stored":403244045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":268365483},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,1,7,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986209,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":526892,"kb_used_data":346716,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93844948,"statfs":{"total":96636764160,"available":96097226752,"internally_reserved":0,"allocated":355037184,"data_stored":354792442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184479402},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659154,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":279620,"kb_used_data":143860,"kb_used_omap":4,"kb_used_meta":135739,"kb_avail":94092220,"statfs":{"total":96636764160,"available":96350433280,"internally_reserved":0,"allocated":147312640,"data_stored":146987283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":138997423},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":403881984,"data_stored":403878505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":147976192,"data_stored":147896175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":355311616,"data_stored":355308137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:14:35.599 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:14:35.600 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'6988", 'reported_seq': 7058, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:34.106623+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:14:34.106623+0000', 'last_peered': '2024-09-11T01:14:34.106623+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:14:34.106623+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 6988, 'log_dups_size': 0, 'ondisk_log_size': 6988, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28622848, 'num_objects': 6988, 'num_object_clones': 0, 'num_object_copies': 13976, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6988, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6988, 'num_write_kb': 27952, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2916}, {'shards': '0,2', 'objects': 4072}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:35.600 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:14:35.600 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'6815", 'reported_seq': 6937, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:33.525934+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:14:33.525934+0000', 'last_peered': '2024-09-11T01:14:33.525934+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:14:33.525934+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 6815, 'log_dups_size': 0, 'ondisk_log_size': 6815, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27897856, 'num_objects': 6811, 'num_object_clones': 0, 'num_object_copies': 13622, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6811, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5116, 'num_write_kb': 20464, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3896}, {'shards': '2', 'objects': 2915}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:14:35.600 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:14:35.600 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'6890", 'reported_seq': 7055, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:34.206470+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:14:34.206470+0000', 'last_peered': '2024-09-11T01:14:34.206470+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:14:34.206470+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 6890, 'log_dups_size': 0, 'ondisk_log_size': 6890, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28200960, 'num_objects': 6885, 'num_object_clones': 0, 'num_object_copies': 13770, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6885, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3231, 'num_write_kb': 12924, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2934}, {'shards': '0,2', 'objects': 3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:35.600 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:14:35.600 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'6814", 'reported_seq': 6939, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:34.169096+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:14:34.169096+0000', 'last_peered': '2024-09-11T01:14:34.169096+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:14:34.169096+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 6814, 'log_dups_size': 0, 'ondisk_log_size': 6814, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27885568, 'num_objects': 6808, 'num_object_clones': 0, 'num_object_copies': 13616, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6808, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3671, 'num_write_kb': 14684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2822}, {'shards': '0,2', 'objects': 3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:35.600 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:14:35.600 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'6834", 'reported_seq': 6957, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:33.446462+0000', 'last_change': '2024-09-11T01:13:10.159069+0000', 'last_active': '2024-09-11T01:14:33.446462+0000', 'last_peered': '2024-09-11T01:14:33.446462+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:14:33.446462+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 6834, 'log_dups_size': 0, 'ondisk_log_size': 6834, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27963392, 'num_objects': 6827, 'num_object_clones': 0, 'num_object_copies': 13654, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6827, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4805, 'num_write_kb': 19220, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3932}, {'shards': '2', 'objects': 2894}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:14:35.601 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:14:35.601 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'6847", 'reported_seq': 6969, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:34.247237+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:14:34.247237+0000', 'last_peered': '2024-09-11T01:14:34.247237+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:14:34.247237+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 6847, 'log_dups_size': 0, 'ondisk_log_size': 6847, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28012544, 'num_objects': 6839, 'num_object_clones': 0, 'num_object_copies': 13678, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6839, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3434, 'num_write_kb': 13736, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2855}, {'shards': '0,2', 'objects': 3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:35.601 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:14:35.601 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'6873", 'reported_seq': 6987, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:33.527687+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:14:33.527687+0000', 'last_peered': '2024-09-11T01:14:33.527687+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:14:33.527687+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 6873, 'log_dups_size': 0, 'ondisk_log_size': 6873, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28131328, 'num_objects': 6868, 'num_object_clones': 0, 'num_object_copies': 13736, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6868, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4690, 'num_write_kb': 18760, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4023}, {'shards': '2', 'objects': 2844}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:14:35.601 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:14:35.601 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'6762", 'reported_seq': 10989, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:14:33.514940+0000', 'last_change': '2024-09-11T01:13:10.217540+0000', 'last_active': '2024-09-11T01:14:33.514940+0000', 'last_peered': '2024-09-11T01:14:33.514940+0000', 'last_clean': '2024-09-11T01:13:07.638709+0000', 'last_became_active': '2024-09-11T01:13:10.125604+0000', 'last_became_peered': '2024-09-11T01:13:10.125604+0000', 'last_unstale': '2024-09-11T01:14:33.514940+0000', 'last_undegraded': '2024-09-11T01:14:33.514940+0000', 'last_fullsized': '2024-09-11T01:13:09.649604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2269", 'last_scrub_stamp': '2024-09-11T01:12:42.890920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:42.890920+0000', 'objects_scrubbed': 2209, 'log_size': 6762, 'log_dups_size': 0, 'ondisk_log_size': 6762, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 999, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27672576, 'num_objects': 6756, 'num_object_clones': 0, 'num_object_copies': 13512, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1869, 'num_objects_unfound': 0, 'num_objects_dirty': 6756, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4547, 'num_write_kb': 18188, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2056, 'num_bytes_recovered': 8421376, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1869}, {'shards': '2', 'objects': 4887}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:14:35.601 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:14:35.601 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'6953", 'reported_seq': 7080, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:34.227835+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:14:34.227835+0000', 'last_peered': '2024-09-11T01:14:34.227835+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:14:34.227835+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 6953, 'log_dups_size': 0, 'ondisk_log_size': 6953, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28471296, 'num_objects': 6951, 'num_object_clones': 0, 'num_object_copies': 13902, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6951, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3690, 'num_write_kb': 14760, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3001}, {'shards': '0,2', 'objects': 3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:35.601 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:14:35.601 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'6812", 'reported_seq': 6944, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:34.231257+0000', 'last_change': '2024-09-11T01:13:10.209668+0000', 'last_active': '2024-09-11T01:14:34.231257+0000', 'last_peered': '2024-09-11T01:14:34.231257+0000', 'last_clean': '2024-09-11T01:13:07.624565+0000', 'last_became_active': '2024-09-11T01:13:10.199795+0000', 'last_became_peered': '2024-09-11T01:13:10.199795+0000', 'last_unstale': '2024-09-11T01:14:34.231257+0000', 'last_undegraded': '2024-09-11T01:13:09.791198+0000', 'last_fullsized': '2024-09-11T01:13:09.652311+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3415", 'last_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'objects_scrubbed': 3337, 'log_size': 6812, 'log_dups_size': 0, 'ondisk_log_size': 6812, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1975, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27869184, 'num_objects': 6804, 'num_object_clones': 0, 'num_object_copies': 13608, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3942, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6804, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3467, 'num_write_kb': 13868, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2865}, {'shards': '0,2', 'objects': 3939}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:35.601 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:14:35.602 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'6951", 'reported_seq': 7079, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:34.288075+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:14:34.288075+0000', 'last_peered': '2024-09-11T01:14:34.288075+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:14:34.288075+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 6951, 'log_dups_size': 0, 'ondisk_log_size': 6951, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28454912, 'num_objects': 6947, 'num_object_clones': 0, 'num_object_copies': 13894, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6947, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3560, 'num_write_kb': 14240, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2908}, {'shards': '0,2', 'objects': 4039}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:35.602 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:14:35.602 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'6870", 'reported_seq': 6999, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:34.227052+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:14:34.227052+0000', 'last_peered': '2024-09-11T01:14:34.227052+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:14:34.227052+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 6870, 'log_dups_size': 0, 'ondisk_log_size': 6870, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28090985, 'num_objects': 6859, 'num_object_clones': 0, 'num_object_copies': 13718, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6859, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3940, 'num_write_kb': 15760, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2942}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:35.602 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:14:36.382 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 132 20 111215 111195 3.28956 1.91016 0.0949031 0.0237407 2024-09-11T01:14:36.578 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:14:36.578 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43711/220062 objects degraded (19.863%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":213,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":110031,"data_bytes":450675324,"bytes_used":1497747456,"bytes_avail":288412545024,"bytes_total":289910292480,"degraded_objects":43711,"degraded_total":220062,"degraded_ratio":0.19863038598213228,"misplaced_objects":1869,"misplaced_total":220062,"misplaced_ratio":0.0084930610464323698,"recovering_objects_per_sec":37,"recovering_bytes_per_sec":153543,"recovering_keys_per_sec":0,"num_objects_recovered":450,"num_bytes_recovered":1843200,"num_keys_recovered":0,"write_bytes_sec":2628318,"read_op_per_sec":0,"write_op_per_sec":641},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:14:36.652 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:14:36.652 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:14:37.385 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 133 20 111720 111700 3.27958 1.97266 0.0196307 0.0238136 2024-09-11T01:14:37.523 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:14:37.525 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:14:37.598 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":143,"stamp":"2024-09-11T01:14:36.582885+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":450675324,"num_objects":110031,"num_object_clones":0,"num_object_copies":220062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":1869,"num_objects_unfound":0,"num_objects_dirty":110031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":70962,"num_write_kb":283842,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2058,"num_bytes_recovered":8425491,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110113,"ondisk_log_size":110113,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1462644,"kb_used_data":884612,"kb_used_omap":12,"kb_used_meta":577971,"kb_avail":281652876,"statfs":{"total":289910292480,"available":288412545024,"internally_reserved":0,"allocated":905842688,"data_stored":905023770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":591842308},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,4,4,15,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21098496,"num_objects":5151,"num_object_clones":0,"num_object_copies":10302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-312,"num_objects_unfound":0,"num_objects_dirty":5151,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5151,"num_write_kb":20604,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":1277952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004501"},"pg_stats":[{"pgid":"2.f","version":"43'6988","reported_seq":7058,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:34.106623+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:14:34.106623+0000","last_peered":"2024-09-11T01:14:34.106623+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:14:34.106623+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":6988,"log_dups_size":0,"ondisk_log_size":6988,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28622848,"num_objects":6988,"num_object_clones":0,"num_object_copies":13976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6988,"num_write_kb":27952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2916},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'6958","reported_seq":7055,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:33.530031+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:14:33.530031+0000","last_peered":"2024-09-11T01:14:33.530031+0000","last_clean":"2024-09-11T01:14:33.530031+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:14:33.530031+0000","last_undegraded":"2024-09-11T01:14:33.530031+0000","last_fullsized":"2024-09-11T01:14:33.530031+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":6958,"log_dups_size":0,"ondisk_log_size":6958,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28475392,"num_objects":6952,"num_object_clones":0,"num_object_copies":13904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5197,"num_write_kb":20788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6815","reported_seq":6937,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:33.525934+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:14:33.525934+0000","last_peered":"2024-09-11T01:14:33.525934+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:14:33.525934+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":6815,"log_dups_size":0,"ondisk_log_size":6815,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27897856,"num_objects":6811,"num_object_clones":0,"num_object_copies":13622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5116,"num_write_kb":20464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":2915}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'6842","reported_seq":6907,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:33.434320+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:14:33.434320+0000","last_peered":"2024-09-11T01:14:33.434320+0000","last_clean":"2024-09-11T01:14:33.434320+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:14:33.434320+0000","last_undegraded":"2024-09-11T01:14:33.434320+0000","last_fullsized":"2024-09-11T01:14:33.434320+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":6842,"log_dups_size":0,"ondisk_log_size":6842,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28024832,"num_objects":6842,"num_object_clones":0,"num_object_copies":13684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6842,"num_write_kb":27368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'6943","reported_seq":7066,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:34.175691+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:14:34.175691+0000","last_peered":"2024-09-11T01:14:34.175691+0000","last_clean":"2024-09-11T01:14:34.175691+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:14:34.175691+0000","last_undegraded":"2024-09-11T01:14:34.175691+0000","last_fullsized":"2024-09-11T01:14:34.175691+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":6943,"log_dups_size":0,"ondisk_log_size":6943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28409856,"num_objects":6936,"num_object_clones":0,"num_object_copies":13872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3948,"num_write_kb":15792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'6890","reported_seq":7055,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:34.206470+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:14:34.206470+0000","last_peered":"2024-09-11T01:14:34.206470+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:14:34.206470+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":6890,"log_dups_size":0,"ondisk_log_size":6890,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28200960,"num_objects":6885,"num_object_clones":0,"num_object_copies":13770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3231,"num_write_kb":12924,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2934},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6814","reported_seq":6939,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:34.169096+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:14:34.169096+0000","last_peered":"2024-09-11T01:14:34.169096+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:14:34.169096+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":6814,"log_dups_size":0,"ondisk_log_size":6814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27885568,"num_objects":6808,"num_object_clones":0,"num_object_copies":13616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6808,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3671,"num_write_kb":14684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2822},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'6958","reported_seq":7083,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:34.226947+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:14:34.226947+0000","last_peered":"2024-09-11T01:14:34.226947+0000","last_clean":"2024-09-11T01:14:34.226947+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:14:34.226947+0000","last_undegraded":"2024-09-11T01:14:34.226947+0000","last_fullsized":"2024-09-11T01:14:34.226947+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":6958,"log_dups_size":0,"ondisk_log_size":6958,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28491776,"num_objects":6956,"num_object_clones":0,"num_object_copies":13912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6956,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3834,"num_write_kb":15336,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6834","reported_seq":6957,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:33.446462+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:14:33.446462+0000","last_peered":"2024-09-11T01:14:33.446462+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:14:33.446462+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":6834,"log_dups_size":0,"ondisk_log_size":6834,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27963392,"num_objects":6827,"num_object_clones":0,"num_object_copies":13654,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6827,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4805,"num_write_kb":19220,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":2894}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741922+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741922+0000","last_peered":"2024-09-11T01:13:09.741922+0000","last_clean":"2024-09-11T01:13:09.741922+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741922+0000","last_undegraded":"2024-09-11T01:13:09.741922+0000","last_fullsized":"2024-09-11T01:13:09.741922+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'6847","reported_seq":6969,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:34.247237+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:14:34.247237+0000","last_peered":"2024-09-11T01:14:34.247237+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:14:34.247237+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":6847,"log_dups_size":0,"ondisk_log_size":6847,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28012544,"num_objects":6839,"num_object_clones":0,"num_object_copies":13678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3434,"num_write_kb":13736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2855},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.910012+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.910012+0000","last_peered":"2024-09-11T01:13:09.910012+0000","last_clean":"2024-09-11T01:13:09.910012+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.910012+0000","last_undegraded":"2024-09-11T01:13:09.910012+0000","last_fullsized":"2024-09-11T01:13:09.910012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'6873","reported_seq":6987,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:33.527687+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:14:33.527687+0000","last_peered":"2024-09-11T01:14:33.527687+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:14:33.527687+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":6873,"log_dups_size":0,"ondisk_log_size":6873,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28131328,"num_objects":6868,"num_object_clones":0,"num_object_copies":13736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4690,"num_write_kb":18760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":2844}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6762","reported_seq":10989,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:14:33.514940+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:14:33.514940+0000","last_peered":"2024-09-11T01:14:33.514940+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:14:33.514940+0000","last_undegraded":"2024-09-11T01:14:33.514940+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":6762,"log_dups_size":0,"ondisk_log_size":6762,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27672576,"num_objects":6756,"num_object_clones":0,"num_object_copies":13512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1869,"num_objects_unfound":0,"num_objects_dirty":6756,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4547,"num_write_kb":18188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2056,"num_bytes_recovered":8421376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1869},{"shards":"2","objects":4887}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'6953","reported_seq":7080,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:34.227835+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:14:34.227835+0000","last_peered":"2024-09-11T01:14:34.227835+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:14:34.227835+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":6953,"log_dups_size":0,"ondisk_log_size":6953,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28471296,"num_objects":6951,"num_object_clones":0,"num_object_copies":13902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3690,"num_write_kb":14760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3001},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6812","reported_seq":6944,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:34.231257+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:14:34.231257+0000","last_peered":"2024-09-11T01:14:34.231257+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:14:34.231257+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":6812,"log_dups_size":0,"ondisk_log_size":6812,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27869184,"num_objects":6804,"num_object_clones":0,"num_object_copies":13608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3467,"num_write_kb":13868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2865},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658174+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658174+0000","last_peered":"2024-09-11T01:13:09.658174+0000","last_clean":"2024-09-11T01:13:09.658174+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658174+0000","last_undegraded":"2024-09-11T01:13:09.658174+0000","last_fullsized":"2024-09-11T01:13:09.658174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'6951","reported_seq":7079,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:34.288075+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:14:34.288075+0000","last_peered":"2024-09-11T01:14:34.288075+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:14:34.288075+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":6951,"log_dups_size":0,"ondisk_log_size":6951,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28454912,"num_objects":6947,"num_object_clones":0,"num_object_copies":13894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6947,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3560,"num_write_kb":14240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2908},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'6870","reported_seq":6999,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:34.227052+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:14:34.227052+0000","last_peered":"2024-09-11T01:14:34.227052+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:14:34.227052+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":6870,"log_dups_size":0,"ondisk_log_size":6870,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28090985,"num_objects":6859,"num_object_clones":0,"num_object_copies":13718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6859,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3940,"num_write_kb":15760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2942},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":450675305,"num_objects":110029,"num_object_clones":0,"num_object_copies":220058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":1869,"num_objects_unfound":0,"num_objects_dirty":110029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":70960,"num_write_kb":283840,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2057,"num_bytes_recovered":8425472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":907169792,"data_stored":907082817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110110,"ondisk_log_size":110110,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182437,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":656132,"kb_used_data":394036,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93715708,"statfs":{"total":96636764160,"available":95964884992,"internally_reserved":0,"allocated":403492864,"data_stored":403244045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":268365483},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,1,7,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986209,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":526892,"kb_used_data":346716,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93844948,"statfs":{"total":96636764160,"available":96097226752,"internally_reserved":0,"allocated":355037184,"data_stored":354792442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184479402},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659154,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":279620,"kb_used_data":143860,"kb_used_omap":4,"kb_used_meta":135739,"kb_avail":94092220,"statfs":{"total":96636764160,"available":96350433280,"internally_reserved":0,"allocated":147312640,"data_stored":146987283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":138997423},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":403881984,"data_stored":403878505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":147976192,"data_stored":147896175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":355311616,"data_stored":355308137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:14:38.392 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 134 20 112224 112204 3.26973 1.96875 0.0690134 0.0238839 2024-09-11T01:14:39.388 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 135 20 112748 112728 3.26067 2.04688 0.0461297 0.023951 2024-09-11T01:14:40.388 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 136 20 113249 113229 3.25109 1.95703 0.0625491 0.0240209 2024-09-11T01:14:40.600 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:14:41.388 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 137 20 113731 113711 3.2411 1.88281 0.0295478 0.024095 2024-09-11T01:14:41.415 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:14:41.416 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:14:41.460 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2024-09-11T01:14:40.583685+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":460972668,"num_objects":112545,"num_object_clones":0,"num_object_copies":225090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":1715,"num_objects_unfound":0,"num_objects_dirty":112545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73476,"num_write_kb":293898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2214,"num_bytes_recovered":9056275,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112627,"ondisk_log_size":112627,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1504184,"kb_used_data":905352,"kb_used_omap":12,"kb_used_meta":598771,"kb_avail":281611336,"statfs":{"total":289910292480,"available":288370008064,"internally_reserved":0,"allocated":927080448,"data_stored":926268488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":613141508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,6,3,6,7,12,10,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27566080,"num_objects":6730,"num_object_clones":0,"num_object_copies":13460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-312,"num_objects_unfound":0,"num_objects_dirty":6730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6730,"num_write_kb":26920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":314,"num_bytes_recovered":1277952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004392"},"pg_stats":[{"pgid":"2.f","version":"43'7118","reported_seq":7190,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:39.267670+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:14:39.267670+0000","last_peered":"2024-09-11T01:14:39.267670+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:14:39.267670+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":7118,"log_dups_size":0,"ondisk_log_size":7118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29155328,"num_objects":7118,"num_object_clones":0,"num_object_copies":14236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7118,"num_write_kb":28472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3046},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7111","reported_seq":7208,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:38.217917+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:14:38.217917+0000","last_peered":"2024-09-11T01:14:38.217917+0000","last_clean":"2024-09-11T01:14:38.217917+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:14:38.217917+0000","last_undegraded":"2024-09-11T01:14:38.217917+0000","last_fullsized":"2024-09-11T01:14:38.217917+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":7111,"log_dups_size":0,"ondisk_log_size":7111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29102080,"num_objects":7105,"num_object_clones":0,"num_object_copies":14210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5350,"num_write_kb":21400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6983","reported_seq":7106,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:38.531946+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:14:38.531946+0000","last_peered":"2024-09-11T01:14:38.531946+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:14:38.531946+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":6983,"log_dups_size":0,"ondisk_log_size":6983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28585984,"num_objects":6979,"num_object_clones":0,"num_object_copies":13958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5284,"num_write_kb":21136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":3083}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7009","reported_seq":7074,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:38.550552+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:14:38.550552+0000","last_peered":"2024-09-11T01:14:38.550552+0000","last_clean":"2024-09-11T01:14:38.550552+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:14:38.550552+0000","last_undegraded":"2024-09-11T01:14:38.550552+0000","last_fullsized":"2024-09-11T01:14:38.550552+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":7009,"log_dups_size":0,"ondisk_log_size":7009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28708864,"num_objects":7009,"num_object_clones":0,"num_object_copies":14018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7009,"num_write_kb":28036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7082","reported_seq":7205,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:39.254791+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:14:39.254791+0000","last_peered":"2024-09-11T01:14:39.254791+0000","last_clean":"2024-09-11T01:14:39.254791+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:14:39.254791+0000","last_undegraded":"2024-09-11T01:14:39.254791+0000","last_fullsized":"2024-09-11T01:14:39.254791+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":7082,"log_dups_size":0,"ondisk_log_size":7082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28979200,"num_objects":7075,"num_object_clones":0,"num_object_copies":14150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4087,"num_write_kb":16348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7043","reported_seq":7208,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:39.276315+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:14:39.276315+0000","last_peered":"2024-09-11T01:14:39.276315+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:14:39.276315+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":7043,"log_dups_size":0,"ondisk_log_size":7043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28827648,"num_objects":7038,"num_object_clones":0,"num_object_copies":14076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3384,"num_write_kb":13536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3087},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6969","reported_seq":7094,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:39.233828+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:14:39.233828+0000","last_peered":"2024-09-11T01:14:39.233828+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:14:39.233828+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":6969,"log_dups_size":0,"ondisk_log_size":6969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28520448,"num_objects":6963,"num_object_clones":0,"num_object_copies":13926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3826,"num_write_kb":15304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2977},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7121","reported_seq":7245,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:39.326805+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:14:39.326805+0000","last_peered":"2024-09-11T01:14:39.326805+0000","last_clean":"2024-09-11T01:14:39.326805+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:14:39.326805+0000","last_undegraded":"2024-09-11T01:14:39.326805+0000","last_fullsized":"2024-09-11T01:14:39.326805+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":7121,"log_dups_size":0,"ondisk_log_size":7121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29159424,"num_objects":7119,"num_object_clones":0,"num_object_copies":14238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3997,"num_write_kb":15988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6975","reported_seq":7098,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:38.522990+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:14:38.522990+0000","last_peered":"2024-09-11T01:14:38.522990+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:14:38.522990+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":6975,"log_dups_size":0,"ondisk_log_size":6975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28540928,"num_objects":6968,"num_object_clones":0,"num_object_copies":13936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4946,"num_write_kb":19784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":3035}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741922+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741922+0000","last_peered":"2024-09-11T01:13:09.741922+0000","last_clean":"2024-09-11T01:13:09.741922+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741922+0000","last_undegraded":"2024-09-11T01:13:09.741922+0000","last_fullsized":"2024-09-11T01:13:09.741922+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7000","reported_seq":7123,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:39.242406+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:14:39.242406+0000","last_peered":"2024-09-11T01:14:39.242406+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:14:39.242406+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":7000,"log_dups_size":0,"ondisk_log_size":7000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28639232,"num_objects":6992,"num_object_clones":0,"num_object_copies":13984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6992,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3587,"num_write_kb":14348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3008},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.910012+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.910012+0000","last_peered":"2024-09-11T01:13:09.910012+0000","last_clean":"2024-09-11T01:13:09.910012+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.910012+0000","last_undegraded":"2024-09-11T01:13:09.910012+0000","last_fullsized":"2024-09-11T01:13:09.910012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7029","reported_seq":7143,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:38.495839+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:14:38.495839+0000","last_peered":"2024-09-11T01:14:38.495839+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:14:38.495839+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":7029,"log_dups_size":0,"ondisk_log_size":7029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28770304,"num_objects":7024,"num_object_clones":0,"num_object_copies":14048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4846,"num_write_kb":19384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":3000}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6913","reported_seq":11452,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:14:38.544748+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:14:38.544748+0000","last_peered":"2024-09-11T01:14:38.544748+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:14:38.544748+0000","last_undegraded":"2024-09-11T01:14:38.544748+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":6913,"log_dups_size":0,"ondisk_log_size":6913,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28291072,"num_objects":6907,"num_object_clones":0,"num_object_copies":13814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1715,"num_objects_unfound":0,"num_objects_dirty":6907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4698,"num_write_kb":18792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2212,"num_bytes_recovered":9052160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1715},{"shards":"2","objects":5192}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7129","reported_seq":7256,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:39.304818+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:14:39.304818+0000","last_peered":"2024-09-11T01:14:39.304818+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:14:39.304818+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":7129,"log_dups_size":0,"ondisk_log_size":7129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29192192,"num_objects":7127,"num_object_clones":0,"num_object_copies":14254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3866,"num_write_kb":15464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3177},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6989","reported_seq":7121,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:39.201110+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:14:39.201110+0000","last_peered":"2024-09-11T01:14:39.201110+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:14:39.201110+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":6989,"log_dups_size":0,"ondisk_log_size":6989,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28594176,"num_objects":6981,"num_object_clones":0,"num_object_copies":13962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3644,"num_write_kb":14576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3042},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658174+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658174+0000","last_peered":"2024-09-11T01:13:09.658174+0000","last_clean":"2024-09-11T01:13:09.658174+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658174+0000","last_undegraded":"2024-09-11T01:13:09.658174+0000","last_fullsized":"2024-09-11T01:13:09.658174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7113","reported_seq":7243,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:39.302417+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:14:39.302417+0000","last_peered":"2024-09-11T01:14:39.302417+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:14:39.302417+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":7113,"log_dups_size":0,"ondisk_log_size":7113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29118464,"num_objects":7109,"num_object_clones":0,"num_object_copies":14218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3722,"num_write_kb":14888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3070},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7040","reported_seq":7169,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:39.260977+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:14:39.260977+0000","last_peered":"2024-09-11T01:14:39.260977+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:14:39.260977+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":7040,"log_dups_size":0,"ondisk_log_size":7040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28787305,"num_objects":7029,"num_object_clones":0,"num_object_copies":14058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4110,"num_write_kb":16440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3112},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":460972649,"num_objects":112543,"num_object_clones":0,"num_object_copies":225086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":1715,"num_objects_unfound":0,"num_objects_dirty":112543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73474,"num_write_kb":293896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2213,"num_bytes_recovered":9056256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":928489472,"data_stored":928409455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112624,"ondisk_log_size":112624,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182438,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":672028,"kb_used_data":401612,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93699812,"statfs":{"total":96636764160,"available":95948607488,"internally_reserved":0,"allocated":411250688,"data_stored":411001869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":276885163},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,3,3,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986210,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":536048,"kb_used_data":351712,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93835792,"statfs":{"total":96636764160,"available":96087851008,"internally_reserved":0,"allocated":360153088,"data_stored":359908346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188739242},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,2,2,2,1,3,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659155,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":296108,"kb_used_data":152028,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":94075732,"statfs":{"total":96636764160,"available":96333549568,"internally_reserved":0,"allocated":155676672,"data_stored":155358273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":147517103},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,2,7,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":411672576,"data_stored":411669097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":156438528,"data_stored":156365469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360378368,"data_stored":360374889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:14:41.461 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:14:42.246 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:14:42.246 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:14:42.292 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":145,"stamp":"2024-09-11T01:14:40.583685+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":460972668,"num_objects":112545,"num_object_clones":0,"num_object_copies":225090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":1715,"num_objects_unfound":0,"num_objects_dirty":112545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73476,"num_write_kb":293898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2214,"num_bytes_recovered":9056275,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112627,"ondisk_log_size":112627,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1504184,"kb_used_data":905352,"kb_used_omap":12,"kb_used_meta":598771,"kb_avail":281611336,"statfs":{"total":289910292480,"available":288370008064,"internally_reserved":0,"allocated":927080448,"data_stored":926268488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":613141508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,6,3,6,7,12,10,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27566080,"num_objects":6730,"num_object_clones":0,"num_object_copies":13460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-312,"num_objects_unfound":0,"num_objects_dirty":6730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6730,"num_write_kb":26920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":314,"num_bytes_recovered":1277952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004392"},"pg_stats":[{"pgid":"2.f","version":"43'7118","reported_seq":7190,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:39.267670+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:14:39.267670+0000","last_peered":"2024-09-11T01:14:39.267670+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:14:39.267670+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":7118,"log_dups_size":0,"ondisk_log_size":7118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29155328,"num_objects":7118,"num_object_clones":0,"num_object_copies":14236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7118,"num_write_kb":28472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3046},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7111","reported_seq":7208,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:38.217917+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:14:38.217917+0000","last_peered":"2024-09-11T01:14:38.217917+0000","last_clean":"2024-09-11T01:14:38.217917+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:14:38.217917+0000","last_undegraded":"2024-09-11T01:14:38.217917+0000","last_fullsized":"2024-09-11T01:14:38.217917+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":7111,"log_dups_size":0,"ondisk_log_size":7111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29102080,"num_objects":7105,"num_object_clones":0,"num_object_copies":14210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5350,"num_write_kb":21400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6983","reported_seq":7106,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:38.531946+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:14:38.531946+0000","last_peered":"2024-09-11T01:14:38.531946+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:14:38.531946+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":6983,"log_dups_size":0,"ondisk_log_size":6983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28585984,"num_objects":6979,"num_object_clones":0,"num_object_copies":13958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5284,"num_write_kb":21136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":3083}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7009","reported_seq":7074,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:38.550552+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:14:38.550552+0000","last_peered":"2024-09-11T01:14:38.550552+0000","last_clean":"2024-09-11T01:14:38.550552+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:14:38.550552+0000","last_undegraded":"2024-09-11T01:14:38.550552+0000","last_fullsized":"2024-09-11T01:14:38.550552+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":7009,"log_dups_size":0,"ondisk_log_size":7009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28708864,"num_objects":7009,"num_object_clones":0,"num_object_copies":14018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7009,"num_write_kb":28036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7082","reported_seq":7205,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:39.254791+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:14:39.254791+0000","last_peered":"2024-09-11T01:14:39.254791+0000","last_clean":"2024-09-11T01:14:39.254791+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:14:39.254791+0000","last_undegraded":"2024-09-11T01:14:39.254791+0000","last_fullsized":"2024-09-11T01:14:39.254791+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":7082,"log_dups_size":0,"ondisk_log_size":7082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28979200,"num_objects":7075,"num_object_clones":0,"num_object_copies":14150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4087,"num_write_kb":16348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7043","reported_seq":7208,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:39.276315+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:14:39.276315+0000","last_peered":"2024-09-11T01:14:39.276315+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:14:39.276315+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":7043,"log_dups_size":0,"ondisk_log_size":7043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28827648,"num_objects":7038,"num_object_clones":0,"num_object_copies":14076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3384,"num_write_kb":13536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3087},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6969","reported_seq":7094,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:39.233828+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:14:39.233828+0000","last_peered":"2024-09-11T01:14:39.233828+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:14:39.233828+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":6969,"log_dups_size":0,"ondisk_log_size":6969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28520448,"num_objects":6963,"num_object_clones":0,"num_object_copies":13926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3826,"num_write_kb":15304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2977},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7121","reported_seq":7245,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:39.326805+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:14:39.326805+0000","last_peered":"2024-09-11T01:14:39.326805+0000","last_clean":"2024-09-11T01:14:39.326805+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:14:39.326805+0000","last_undegraded":"2024-09-11T01:14:39.326805+0000","last_fullsized":"2024-09-11T01:14:39.326805+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":7121,"log_dups_size":0,"ondisk_log_size":7121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29159424,"num_objects":7119,"num_object_clones":0,"num_object_copies":14238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3997,"num_write_kb":15988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6975","reported_seq":7098,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:38.522990+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:14:38.522990+0000","last_peered":"2024-09-11T01:14:38.522990+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:14:38.522990+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":6975,"log_dups_size":0,"ondisk_log_size":6975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28540928,"num_objects":6968,"num_object_clones":0,"num_object_copies":13936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4946,"num_write_kb":19784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":3035}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741922+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741922+0000","last_peered":"2024-09-11T01:13:09.741922+0000","last_clean":"2024-09-11T01:13:09.741922+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741922+0000","last_undegraded":"2024-09-11T01:13:09.741922+0000","last_fullsized":"2024-09-11T01:13:09.741922+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7000","reported_seq":7123,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:39.242406+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:14:39.242406+0000","last_peered":"2024-09-11T01:14:39.242406+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:14:39.242406+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":7000,"log_dups_size":0,"ondisk_log_size":7000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28639232,"num_objects":6992,"num_object_clones":0,"num_object_copies":13984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6992,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3587,"num_write_kb":14348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3008},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.910012+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.910012+0000","last_peered":"2024-09-11T01:13:09.910012+0000","last_clean":"2024-09-11T01:13:09.910012+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.910012+0000","last_undegraded":"2024-09-11T01:13:09.910012+0000","last_fullsized":"2024-09-11T01:13:09.910012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7029","reported_seq":7143,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:38.495839+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:14:38.495839+0000","last_peered":"2024-09-11T01:14:38.495839+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:14:38.495839+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":7029,"log_dups_size":0,"ondisk_log_size":7029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28770304,"num_objects":7024,"num_object_clones":0,"num_object_copies":14048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4846,"num_write_kb":19384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":3000}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6913","reported_seq":11452,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:14:38.544748+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:14:38.544748+0000","last_peered":"2024-09-11T01:14:38.544748+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:14:38.544748+0000","last_undegraded":"2024-09-11T01:14:38.544748+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":6913,"log_dups_size":0,"ondisk_log_size":6913,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28291072,"num_objects":6907,"num_object_clones":0,"num_object_copies":13814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1715,"num_objects_unfound":0,"num_objects_dirty":6907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4698,"num_write_kb":18792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2212,"num_bytes_recovered":9052160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1715},{"shards":"2","objects":5192}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7129","reported_seq":7256,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:39.304818+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:14:39.304818+0000","last_peered":"2024-09-11T01:14:39.304818+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:14:39.304818+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":7129,"log_dups_size":0,"ondisk_log_size":7129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29192192,"num_objects":7127,"num_object_clones":0,"num_object_copies":14254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3866,"num_write_kb":15464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3177},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6989","reported_seq":7121,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:39.201110+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:14:39.201110+0000","last_peered":"2024-09-11T01:14:39.201110+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:14:39.201110+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":6989,"log_dups_size":0,"ondisk_log_size":6989,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28594176,"num_objects":6981,"num_object_clones":0,"num_object_copies":13962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3644,"num_write_kb":14576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3042},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658174+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658174+0000","last_peered":"2024-09-11T01:13:09.658174+0000","last_clean":"2024-09-11T01:13:09.658174+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658174+0000","last_undegraded":"2024-09-11T01:13:09.658174+0000","last_fullsized":"2024-09-11T01:13:09.658174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7113","reported_seq":7243,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:39.302417+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:14:39.302417+0000","last_peered":"2024-09-11T01:14:39.302417+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:14:39.302417+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":7113,"log_dups_size":0,"ondisk_log_size":7113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29118464,"num_objects":7109,"num_object_clones":0,"num_object_copies":14218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3722,"num_write_kb":14888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3070},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7040","reported_seq":7169,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:39.260977+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:14:39.260977+0000","last_peered":"2024-09-11T01:14:39.260977+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:14:39.260977+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":7040,"log_dups_size":0,"ondisk_log_size":7040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28787305,"num_objects":7029,"num_object_clones":0,"num_object_copies":14058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4110,"num_write_kb":16440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3112},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":460972649,"num_objects":112543,"num_object_clones":0,"num_object_copies":225086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":1715,"num_objects_unfound":0,"num_objects_dirty":112543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73474,"num_write_kb":293896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2213,"num_bytes_recovered":9056256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":928489472,"data_stored":928409455,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112624,"ondisk_log_size":112624,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182438,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":672028,"kb_used_data":401612,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93699812,"statfs":{"total":96636764160,"available":95948607488,"internally_reserved":0,"allocated":411250688,"data_stored":411001869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":276885163},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,3,3,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986210,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":536048,"kb_used_data":351712,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93835792,"statfs":{"total":96636764160,"available":96087851008,"internally_reserved":0,"allocated":360153088,"data_stored":359908346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188739242},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,2,2,2,1,3,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659155,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":296108,"kb_used_data":152028,"kb_used_omap":4,"kb_used_meta":144059,"kb_avail":94075732,"statfs":{"total":96636764160,"available":96333549568,"internally_reserved":0,"allocated":155676672,"data_stored":155358273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":147517103},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,2,7,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":411672576,"data_stored":411669097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":156438528,"data_stored":156365469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360378368,"data_stored":360374889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:14:42.293 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:14:42.293 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'7118", 'reported_seq': 7190, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:39.267670+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:14:39.267670+0000', 'last_peered': '2024-09-11T01:14:39.267670+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:14:39.267670+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 7118, 'log_dups_size': 0, 'ondisk_log_size': 7118, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29155328, 'num_objects': 7118, 'num_object_clones': 0, 'num_object_copies': 14236, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7118, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7118, 'num_write_kb': 28472, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3046}, {'shards': '0,2', 'objects': 4072}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:42.293 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:14:42.293 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'6983", 'reported_seq': 7106, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:38.531946+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:14:38.531946+0000', 'last_peered': '2024-09-11T01:14:38.531946+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:14:38.531946+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 6983, 'log_dups_size': 0, 'ondisk_log_size': 6983, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28585984, 'num_objects': 6979, 'num_object_clones': 0, 'num_object_copies': 13958, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6979, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5284, 'num_write_kb': 21136, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3896}, {'shards': '2', 'objects': 3083}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:14:42.294 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:14:42.294 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'7043", 'reported_seq': 7208, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:39.276315+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:14:39.276315+0000', 'last_peered': '2024-09-11T01:14:39.276315+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:14:39.276315+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 7043, 'log_dups_size': 0, 'ondisk_log_size': 7043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28827648, 'num_objects': 7038, 'num_object_clones': 0, 'num_object_copies': 14076, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7038, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3384, 'num_write_kb': 13536, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3087}, {'shards': '0,2', 'objects': 3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:42.294 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:14:42.294 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'6969", 'reported_seq': 7094, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:39.233828+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:14:39.233828+0000', 'last_peered': '2024-09-11T01:14:39.233828+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:14:39.233828+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 6969, 'log_dups_size': 0, 'ondisk_log_size': 6969, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28520448, 'num_objects': 6963, 'num_object_clones': 0, 'num_object_copies': 13926, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6963, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3826, 'num_write_kb': 15304, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2977}, {'shards': '0,2', 'objects': 3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:42.294 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:14:42.294 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'6975", 'reported_seq': 7098, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:38.522990+0000', 'last_change': '2024-09-11T01:13:10.159069+0000', 'last_active': '2024-09-11T01:14:38.522990+0000', 'last_peered': '2024-09-11T01:14:38.522990+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:14:38.522990+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 6975, 'log_dups_size': 0, 'ondisk_log_size': 6975, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28540928, 'num_objects': 6968, 'num_object_clones': 0, 'num_object_copies': 13936, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6968, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4946, 'num_write_kb': 19784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3932}, {'shards': '2', 'objects': 3035}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:14:42.294 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:14:42.294 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'7000", 'reported_seq': 7123, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:39.242406+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:14:39.242406+0000', 'last_peered': '2024-09-11T01:14:39.242406+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:14:39.242406+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 7000, 'log_dups_size': 0, 'ondisk_log_size': 7000, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28639232, 'num_objects': 6992, 'num_object_clones': 0, 'num_object_copies': 13984, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6992, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3587, 'num_write_kb': 14348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3008}, {'shards': '0,2', 'objects': 3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:42.294 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:14:42.294 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'7029", 'reported_seq': 7143, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:38.495839+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:14:38.495839+0000', 'last_peered': '2024-09-11T01:14:38.495839+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:14:38.495839+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 7029, 'log_dups_size': 0, 'ondisk_log_size': 7029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28770304, 'num_objects': 7024, 'num_object_clones': 0, 'num_object_copies': 14048, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7024, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4846, 'num_write_kb': 19384, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4023}, {'shards': '2', 'objects': 3000}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:14:42.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:14:42.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'6913", 'reported_seq': 11452, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:14:38.544748+0000', 'last_change': '2024-09-11T01:13:10.217540+0000', 'last_active': '2024-09-11T01:14:38.544748+0000', 'last_peered': '2024-09-11T01:14:38.544748+0000', 'last_clean': '2024-09-11T01:13:07.638709+0000', 'last_became_active': '2024-09-11T01:13:10.125604+0000', 'last_became_peered': '2024-09-11T01:13:10.125604+0000', 'last_unstale': '2024-09-11T01:14:38.544748+0000', 'last_undegraded': '2024-09-11T01:14:38.544748+0000', 'last_fullsized': '2024-09-11T01:13:09.649604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2269", 'last_scrub_stamp': '2024-09-11T01:12:42.890920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:42.890920+0000', 'objects_scrubbed': 2209, 'log_size': 6913, 'log_dups_size': 0, 'ondisk_log_size': 6913, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 999, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28291072, 'num_objects': 6907, 'num_object_clones': 0, 'num_object_copies': 13814, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1715, 'num_objects_unfound': 0, 'num_objects_dirty': 6907, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4698, 'num_write_kb': 18792, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2212, 'num_bytes_recovered': 9052160, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1715}, {'shards': '2', 'objects': 5192}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:14:42.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:14:42.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'7129", 'reported_seq': 7256, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:39.304818+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:14:39.304818+0000', 'last_peered': '2024-09-11T01:14:39.304818+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:14:39.304818+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 7129, 'log_dups_size': 0, 'ondisk_log_size': 7129, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29192192, 'num_objects': 7127, 'num_object_clones': 0, 'num_object_copies': 14254, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7127, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3866, 'num_write_kb': 15464, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3177}, {'shards': '0,2', 'objects': 3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:42.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:14:42.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'6989", 'reported_seq': 7121, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:39.201110+0000', 'last_change': '2024-09-11T01:13:10.209668+0000', 'last_active': '2024-09-11T01:14:39.201110+0000', 'last_peered': '2024-09-11T01:14:39.201110+0000', 'last_clean': '2024-09-11T01:13:07.624565+0000', 'last_became_active': '2024-09-11T01:13:10.199795+0000', 'last_became_peered': '2024-09-11T01:13:10.199795+0000', 'last_unstale': '2024-09-11T01:14:39.201110+0000', 'last_undegraded': '2024-09-11T01:13:09.791198+0000', 'last_fullsized': '2024-09-11T01:13:09.652311+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3415", 'last_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'objects_scrubbed': 3337, 'log_size': 6989, 'log_dups_size': 0, 'ondisk_log_size': 6989, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1975, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28594176, 'num_objects': 6981, 'num_object_clones': 0, 'num_object_copies': 13962, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3942, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6981, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3644, 'num_write_kb': 14576, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3042}, {'shards': '0,2', 'objects': 3939}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:42.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:14:42.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'7113", 'reported_seq': 7243, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:39.302417+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:14:39.302417+0000', 'last_peered': '2024-09-11T01:14:39.302417+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:14:39.302417+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 7113, 'log_dups_size': 0, 'ondisk_log_size': 7113, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29118464, 'num_objects': 7109, 'num_object_clones': 0, 'num_object_copies': 14218, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7109, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3722, 'num_write_kb': 14888, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3070}, {'shards': '0,2', 'objects': 4039}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:42.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:14:42.295 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'7040", 'reported_seq': 7169, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:39.260977+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:14:39.260977+0000', 'last_peered': '2024-09-11T01:14:39.260977+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:14:39.260977+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 7040, 'log_dups_size': 0, 'ondisk_log_size': 7040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28787305, 'num_objects': 7029, 'num_object_clones': 0, 'num_object_copies': 14058, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7029, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4110, 'num_write_kb': 16440, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3112}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:42.296 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:14:42.389 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 138 20 114217 114197 3.23138 1.89844 0.0289529 0.0241707 2024-09-11T01:14:43.389 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 139 20 114691 114671 3.22145 1.85156 0.0705164 0.0242418 2024-09-11T01:14:43.431 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:14:43.432 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43711/225090 objects degraded (19.419%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":219,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":112545,"data_bytes":460972668,"bytes_used":1558716416,"bytes_avail":288351576064,"bytes_total":289910292480,"degraded_objects":43711,"degraded_total":225090,"degraded_ratio":0.19419343373761605,"misplaced_objects":1715,"misplaced_total":225090,"misplaced_ratio":0.0076191745524012621,"recovering_objects_per_sec":26,"recovering_bytes_per_sec":106462,"recovering_keys_per_sec":0,"num_objects_recovered":314,"num_bytes_recovered":1277952,"num_keys_recovered":0,"write_bytes_sec":1734112,"read_op_per_sec":0,"write_op_per_sec":423},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:14:43.479 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:14:43.479 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:14:44.339 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:14:44.340 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:14:44.392 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout:2024-09-11T01:14:44.392360+0000 min lat: 0.00219431 max lat: 2.69211 avg lat: 0.0243134 2024-09-11T01:14:44.393 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T01:14:44.393 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 140 20 115177 115157 3.21192 1.89844 0.00945373 0.0243134 2024-09-11T01:14:44.418 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":146,"stamp":"2024-09-11T01:14:42.584170+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":460972668,"num_objects":112545,"num_object_clones":0,"num_object_copies":225090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":1715,"num_objects_unfound":0,"num_objects_dirty":112545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73476,"num_write_kb":293898,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2214,"num_bytes_recovered":9056275,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112627,"ondisk_log_size":112627,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1522184,"kb_used_data":913620,"kb_used_omap":12,"kb_used_meta":608499,"kb_avail":281593336,"statfs":{"total":289910292480,"available":288351576064,"internally_reserved":0,"allocated":935546880,"data_stored":934734920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":623102980},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,6,4,7,9,11,10,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20815872,"num_objects":5082,"num_object_clones":0,"num_object_copies":10164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-312,"num_objects_unfound":0,"num_objects_dirty":5082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5082,"num_write_kb":20328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":314,"num_bytes_recovered":1277952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003762"},"pg_stats":[{"pgid":"2.f","version":"43'7118","reported_seq":7190,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:39.267670+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:14:39.267670+0000","last_peered":"2024-09-11T01:14:39.267670+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:14:39.267670+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":7118,"log_dups_size":0,"ondisk_log_size":7118,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29155328,"num_objects":7118,"num_object_clones":0,"num_object_copies":14236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7118,"num_write_kb":28472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3046},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7111","reported_seq":7208,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:38.217917+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:14:38.217917+0000","last_peered":"2024-09-11T01:14:38.217917+0000","last_clean":"2024-09-11T01:14:38.217917+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:14:38.217917+0000","last_undegraded":"2024-09-11T01:14:38.217917+0000","last_fullsized":"2024-09-11T01:14:38.217917+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":7111,"log_dups_size":0,"ondisk_log_size":7111,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29102080,"num_objects":7105,"num_object_clones":0,"num_object_copies":14210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5350,"num_write_kb":21400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'6983","reported_seq":7106,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:38.531946+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:14:38.531946+0000","last_peered":"2024-09-11T01:14:38.531946+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:14:38.531946+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":6983,"log_dups_size":0,"ondisk_log_size":6983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28585984,"num_objects":6979,"num_object_clones":0,"num_object_copies":13958,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6979,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5284,"num_write_kb":21136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":3083}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7009","reported_seq":7074,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:38.550552+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:14:38.550552+0000","last_peered":"2024-09-11T01:14:38.550552+0000","last_clean":"2024-09-11T01:14:38.550552+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:14:38.550552+0000","last_undegraded":"2024-09-11T01:14:38.550552+0000","last_fullsized":"2024-09-11T01:14:38.550552+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":7009,"log_dups_size":0,"ondisk_log_size":7009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28708864,"num_objects":7009,"num_object_clones":0,"num_object_copies":14018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7009,"num_write_kb":28036,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7082","reported_seq":7205,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:39.254791+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:14:39.254791+0000","last_peered":"2024-09-11T01:14:39.254791+0000","last_clean":"2024-09-11T01:14:39.254791+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:14:39.254791+0000","last_undegraded":"2024-09-11T01:14:39.254791+0000","last_fullsized":"2024-09-11T01:14:39.254791+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":7082,"log_dups_size":0,"ondisk_log_size":7082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28979200,"num_objects":7075,"num_object_clones":0,"num_object_copies":14150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7075,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4087,"num_write_kb":16348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7043","reported_seq":7208,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:39.276315+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:14:39.276315+0000","last_peered":"2024-09-11T01:14:39.276315+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:14:39.276315+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":7043,"log_dups_size":0,"ondisk_log_size":7043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28827648,"num_objects":7038,"num_object_clones":0,"num_object_copies":14076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3384,"num_write_kb":13536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3087},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'6969","reported_seq":7094,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:39.233828+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:14:39.233828+0000","last_peered":"2024-09-11T01:14:39.233828+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:14:39.233828+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":6969,"log_dups_size":0,"ondisk_log_size":6969,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28520448,"num_objects":6963,"num_object_clones":0,"num_object_copies":13926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6963,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3826,"num_write_kb":15304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2977},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7121","reported_seq":7245,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:39.326805+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:14:39.326805+0000","last_peered":"2024-09-11T01:14:39.326805+0000","last_clean":"2024-09-11T01:14:39.326805+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:14:39.326805+0000","last_undegraded":"2024-09-11T01:14:39.326805+0000","last_fullsized":"2024-09-11T01:14:39.326805+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":7121,"log_dups_size":0,"ondisk_log_size":7121,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29159424,"num_objects":7119,"num_object_clones":0,"num_object_copies":14238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3997,"num_write_kb":15988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'6975","reported_seq":7098,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:38.522990+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:14:38.522990+0000","last_peered":"2024-09-11T01:14:38.522990+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:14:38.522990+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":6975,"log_dups_size":0,"ondisk_log_size":6975,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28540928,"num_objects":6968,"num_object_clones":0,"num_object_copies":13936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4946,"num_write_kb":19784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":3035}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741922+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741922+0000","last_peered":"2024-09-11T01:13:09.741922+0000","last_clean":"2024-09-11T01:13:09.741922+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741922+0000","last_undegraded":"2024-09-11T01:13:09.741922+0000","last_fullsized":"2024-09-11T01:13:09.741922+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7000","reported_seq":7123,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:39.242406+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:14:39.242406+0000","last_peered":"2024-09-11T01:14:39.242406+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:14:39.242406+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":7000,"log_dups_size":0,"ondisk_log_size":7000,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28639232,"num_objects":6992,"num_object_clones":0,"num_object_copies":13984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6992,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3587,"num_write_kb":14348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3008},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.910012+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.910012+0000","last_peered":"2024-09-11T01:13:09.910012+0000","last_clean":"2024-09-11T01:13:09.910012+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.910012+0000","last_undegraded":"2024-09-11T01:13:09.910012+0000","last_fullsized":"2024-09-11T01:13:09.910012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7029","reported_seq":7143,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:38.495839+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:14:38.495839+0000","last_peered":"2024-09-11T01:14:38.495839+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:14:38.495839+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":7029,"log_dups_size":0,"ondisk_log_size":7029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28770304,"num_objects":7024,"num_object_clones":0,"num_object_copies":14048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4846,"num_write_kb":19384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":3000}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'6913","reported_seq":11452,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:14:38.544748+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:14:38.544748+0000","last_peered":"2024-09-11T01:14:38.544748+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:14:38.544748+0000","last_undegraded":"2024-09-11T01:14:38.544748+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":6913,"log_dups_size":0,"ondisk_log_size":6913,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28291072,"num_objects":6907,"num_object_clones":0,"num_object_copies":13814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1715,"num_objects_unfound":0,"num_objects_dirty":6907,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4698,"num_write_kb":18792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2212,"num_bytes_recovered":9052160,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1715},{"shards":"2","objects":5192}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7129","reported_seq":7256,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:39.304818+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:14:39.304818+0000","last_peered":"2024-09-11T01:14:39.304818+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:14:39.304818+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":7129,"log_dups_size":0,"ondisk_log_size":7129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29192192,"num_objects":7127,"num_object_clones":0,"num_object_copies":14254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3866,"num_write_kb":15464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3177},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'6989","reported_seq":7121,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:39.201110+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:14:39.201110+0000","last_peered":"2024-09-11T01:14:39.201110+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:14:39.201110+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":6989,"log_dups_size":0,"ondisk_log_size":6989,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28594176,"num_objects":6981,"num_object_clones":0,"num_object_copies":13962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3644,"num_write_kb":14576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3042},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658174+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658174+0000","last_peered":"2024-09-11T01:13:09.658174+0000","last_clean":"2024-09-11T01:13:09.658174+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658174+0000","last_undegraded":"2024-09-11T01:13:09.658174+0000","last_fullsized":"2024-09-11T01:13:09.658174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7113","reported_seq":7243,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:39.302417+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:14:39.302417+0000","last_peered":"2024-09-11T01:14:39.302417+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:14:39.302417+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":7113,"log_dups_size":0,"ondisk_log_size":7113,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29118464,"num_objects":7109,"num_object_clones":0,"num_object_copies":14218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7109,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3722,"num_write_kb":14888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3070},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7040","reported_seq":7169,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:39.260977+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:14:39.260977+0000","last_peered":"2024-09-11T01:14:39.260977+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:14:39.260977+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":7040,"log_dups_size":0,"ondisk_log_size":7040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28787305,"num_objects":7029,"num_object_clones":0,"num_object_copies":14058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4110,"num_write_kb":16440,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3112},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":460972649,"num_objects":112543,"num_object_clones":0,"num_object_copies":225086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":1715,"num_objects_unfound":0,"num_objects_dirty":112543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":73474,"num_write_kb":293896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2213,"num_bytes_recovered":9056256,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":936943616,"data_stored":936863599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":112624,"ondisk_log_size":112624,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182438,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":672028,"kb_used_data":401612,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93699812,"statfs":{"total":96636764160,"available":95948607488,"internally_reserved":0,"allocated":411250688,"data_stored":411001869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":276885163},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,3,3,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986210,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":536048,"kb_used_data":351712,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93835792,"statfs":{"total":96636764160,"available":96087851008,"internally_reserved":0,"allocated":360153088,"data_stored":359908346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188739242},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,2,2,2,1,3,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659156,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":314108,"kb_used_data":160296,"kb_used_omap":4,"kb_used_meta":153787,"kb_avail":94057732,"statfs":{"total":96636764160,"available":96315117568,"internally_reserved":0,"allocated":164143104,"data_stored":163824705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":157478575},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,4,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":411672576,"data_stored":411669097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":164892672,"data_stored":164819613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":360378368,"data_stored":360374889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:14:45.392 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 141 20 115693 115673 3.20344 2.01562 0.0253654 0.0243801 2024-09-11T01:14:46.394 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 142 20 116205 116185 3.19492 2 0.0395005 0.0244437 2024-09-11T01:14:47.394 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 143 20 116690 116670 3.18582 1.89453 0.0216947 0.0245134 2024-09-11T01:14:47.422 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:14:48.373 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:14:48.375 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:14:48.395 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 144 20 117193 117173 3.17735 1.96484 0.0652849 0.0245804 2024-09-11T01:14:48.457 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":148,"stamp":"2024-09-11T01:14:46.585298+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":471028348,"num_objects":115000,"num_object_clones":0,"num_object_copies":230000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":1554,"num_objects_unfound":0,"num_objects_dirty":115000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":75931,"num_write_kb":303718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2375,"num_bytes_recovered":9715731,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115082,"ondisk_log_size":115082,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1526160,"kb_used_data":925916,"kb_used_omap":12,"kb_used_meta":600179,"kb_avail":281589360,"statfs":{"total":289910292480,"available":288347504640,"internally_reserved":0,"allocated":948137984,"data_stored":947326024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":614583300},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,7,6,9,14,6,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20353024,"num_objects":4969,"num_object_clones":0,"num_object_copies":9938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-315,"num_objects_unfound":0,"num_objects_dirty":4969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4969,"num_write_kb":19876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":317,"num_bytes_recovered":1290240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002948"},"pg_stats":[{"pgid":"2.f","version":"43'7286","reported_seq":7358,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:44.306826+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:14:44.306826+0000","last_peered":"2024-09-11T01:14:44.306826+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:14:44.306826+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":7286,"log_dups_size":0,"ondisk_log_size":7286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29843456,"num_objects":7286,"num_object_clones":0,"num_object_copies":14572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7286,"num_write_kb":29144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3214},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7264","reported_seq":7360,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:43.552485+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:14:43.552485+0000","last_peered":"2024-09-11T01:14:43.552485+0000","last_clean":"2024-09-11T01:14:43.552485+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:14:43.552485+0000","last_undegraded":"2024-09-11T01:14:43.552485+0000","last_fullsized":"2024-09-11T01:14:43.552485+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":7264,"log_dups_size":0,"ondisk_log_size":7264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29728768,"num_objects":7258,"num_object_clones":0,"num_object_copies":14516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5503,"num_write_kb":22012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7132","reported_seq":7255,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:43.539863+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:14:43.539863+0000","last_peered":"2024-09-11T01:14:43.539863+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:14:43.539863+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":7132,"log_dups_size":0,"ondisk_log_size":7132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29196288,"num_objects":7128,"num_object_clones":0,"num_object_copies":14256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5433,"num_write_kb":21732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":3232}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7162","reported_seq":7227,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:43.493647+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:14:43.493647+0000","last_peered":"2024-09-11T01:14:43.493647+0000","last_clean":"2024-09-11T01:14:43.493647+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:14:43.493647+0000","last_undegraded":"2024-09-11T01:14:43.493647+0000","last_fullsized":"2024-09-11T01:14:43.493647+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":7162,"log_dups_size":0,"ondisk_log_size":7162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29335552,"num_objects":7162,"num_object_clones":0,"num_object_copies":14324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7162,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7228","reported_seq":7351,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:44.281203+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:14:44.281203+0000","last_peered":"2024-09-11T01:14:44.281203+0000","last_clean":"2024-09-11T01:14:44.281203+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:14:44.281203+0000","last_undegraded":"2024-09-11T01:14:44.281203+0000","last_fullsized":"2024-09-11T01:14:44.281203+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":7228,"log_dups_size":0,"ondisk_log_size":7228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29577216,"num_objects":7221,"num_object_clones":0,"num_object_copies":14442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4233,"num_write_kb":16932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7204","reported_seq":7369,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:44.318212+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:14:44.318212+0000","last_peered":"2024-09-11T01:14:44.318212+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:14:44.318212+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":7204,"log_dups_size":0,"ondisk_log_size":7204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29487104,"num_objects":7199,"num_object_clones":0,"num_object_copies":14398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3545,"num_write_kb":14180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3248},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7130","reported_seq":7255,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:44.300369+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:14:44.300369+0000","last_peered":"2024-09-11T01:14:44.300369+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:14:44.300369+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":7130,"log_dups_size":0,"ondisk_log_size":7130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29179904,"num_objects":7124,"num_object_clones":0,"num_object_copies":14248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3987,"num_write_kb":15948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3138},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7272","reported_seq":7397,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:44.289215+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:14:44.289215+0000","last_peered":"2024-09-11T01:14:44.289215+0000","last_clean":"2024-09-11T01:14:44.289215+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:14:44.289215+0000","last_undegraded":"2024-09-11T01:14:44.289215+0000","last_fullsized":"2024-09-11T01:14:44.289215+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":7272,"log_dups_size":0,"ondisk_log_size":7272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29777920,"num_objects":7270,"num_object_clones":0,"num_object_copies":14540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4148,"num_write_kb":16592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7137","reported_seq":7260,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:43.577903+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:14:43.577903+0000","last_peered":"2024-09-11T01:14:43.577903+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:14:43.577903+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29204480,"num_objects":7130,"num_object_clones":0,"num_object_copies":14260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5108,"num_write_kb":20432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":3197}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741922+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741922+0000","last_peered":"2024-09-11T01:13:09.741922+0000","last_clean":"2024-09-11T01:13:09.741922+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741922+0000","last_undegraded":"2024-09-11T01:13:09.741922+0000","last_fullsized":"2024-09-11T01:13:09.741922+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7136","reported_seq":7259,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:44.158058+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:14:44.158058+0000","last_peered":"2024-09-11T01:14:44.158058+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:14:44.158058+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":7136,"log_dups_size":0,"ondisk_log_size":7136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29196288,"num_objects":7128,"num_object_clones":0,"num_object_copies":14256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3723,"num_write_kb":14892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3144},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.910012+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.910012+0000","last_peered":"2024-09-11T01:13:09.910012+0000","last_clean":"2024-09-11T01:13:09.910012+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.910012+0000","last_undegraded":"2024-09-11T01:13:09.910012+0000","last_fullsized":"2024-09-11T01:13:09.910012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7182","reported_seq":7296,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:43.587515+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:14:43.587515+0000","last_peered":"2024-09-11T01:14:43.587515+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:14:43.587515+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":7182,"log_dups_size":0,"ondisk_log_size":7182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29396992,"num_objects":7177,"num_object_clones":0,"num_object_copies":14354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4999,"num_write_kb":19996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":3153}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7080","reported_seq":11939,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:14:43.604944+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:14:43.604944+0000","last_peered":"2024-09-11T01:14:43.604944+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:14:43.604944+0000","last_undegraded":"2024-09-11T01:14:43.604944+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":7080,"log_dups_size":0,"ondisk_log_size":7080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28975104,"num_objects":7074,"num_object_clones":0,"num_object_copies":14148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1554,"num_objects_unfound":0,"num_objects_dirty":7074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4865,"num_write_kb":19460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2373,"num_bytes_recovered":9711616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1554},{"shards":"2","objects":5520}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7287","reported_seq":7413,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:44.304721+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:14:44.304721+0000","last_peered":"2024-09-11T01:14:44.304721+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:14:44.304721+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":7287,"log_dups_size":0,"ondisk_log_size":7287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29839360,"num_objects":7285,"num_object_clones":0,"num_object_copies":14570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4024,"num_write_kb":16096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3335},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7139","reported_seq":7271,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:44.120987+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:14:44.120987+0000","last_peered":"2024-09-11T01:14:44.120987+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:14:44.120987+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":7139,"log_dups_size":0,"ondisk_log_size":7139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29208576,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3794,"num_write_kb":15176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3192},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658174+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658174+0000","last_peered":"2024-09-11T01:13:09.658174+0000","last_clean":"2024-09-11T01:13:09.658174+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658174+0000","last_undegraded":"2024-09-11T01:13:09.658174+0000","last_fullsized":"2024-09-11T01:13:09.658174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7244","reported_seq":7374,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:44.302430+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:14:44.302430+0000","last_peered":"2024-09-11T01:14:44.302430+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:14:44.302430+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":7244,"log_dups_size":0,"ondisk_log_size":7244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29655040,"num_objects":7240,"num_object_clones":0,"num_object_copies":14480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3853,"num_write_kb":15412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3201},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7196","reported_seq":7325,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:44.329562+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:14:44.329562+0000","last_peered":"2024-09-11T01:14:44.329562+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:14:44.329562+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":7196,"log_dups_size":0,"ondisk_log_size":7196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29426281,"num_objects":7185,"num_object_clones":0,"num_object_copies":14370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4266,"num_write_kb":17064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3268},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":471028329,"num_objects":114998,"num_object_clones":0,"num_object_copies":229996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":1554,"num_objects_unfound":0,"num_objects_dirty":114998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":75929,"num_write_kb":303716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2374,"num_bytes_recovered":9715712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":949493760,"data_stored":949413743,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115079,"ondisk_log_size":115079,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182439,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":683508,"kb_used_data":408932,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93688332,"statfs":{"total":96636764160,"available":95936851968,"internally_reserved":0,"allocated":418746368,"data_stored":418497549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":281145003},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,1,3,6,3,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986211,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":528544,"kb_used_data":356688,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93843296,"statfs":{"total":96636764160,"available":96095535104,"internally_reserved":0,"allocated":365248512,"data_stored":365003770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":175959722},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659156,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":314108,"kb_used_data":160296,"kb_used_omap":4,"kb_used_meta":153787,"kb_avail":94057732,"statfs":{"total":96636764160,"available":96315117568,"internally_reserved":0,"allocated":164143104,"data_stored":163824705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":157478575},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,4,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":419143680,"data_stored":419140201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":164892672,"data_stored":164819613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":365457408,"data_stored":365453929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:14:48.459 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:14:49.395 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 145 20 117680 117660 3.16855 1.90234 0.0322142 0.024648 2024-09-11T01:14:49.509 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:14:49.509 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:14:49.595 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":149,"stamp":"2024-09-11T01:14:48.585804+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":471028348,"num_objects":115000,"num_object_clones":0,"num_object_copies":230000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":1554,"num_objects_unfound":0,"num_objects_dirty":115000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":75931,"num_write_kb":303718,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2375,"num_bytes_recovered":9715731,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115082,"ondisk_log_size":115082,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1538148,"kb_used_data":933748,"kb_used_omap":12,"kb_used_meta":604339,"kb_avail":281577372,"statfs":{"total":289910292480,"available":288335228928,"internally_reserved":0,"allocated":956157952,"data_stored":955345992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":618843140},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,6,5,10,13,4,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20353024,"num_objects":4969,"num_object_clones":0,"num_object_copies":9938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-315,"num_objects_unfound":0,"num_objects_dirty":4969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4969,"num_write_kb":19876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":317,"num_bytes_recovered":1290240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002918"},"pg_stats":[{"pgid":"2.f","version":"43'7286","reported_seq":7358,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:44.306826+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:14:44.306826+0000","last_peered":"2024-09-11T01:14:44.306826+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:14:44.306826+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":7286,"log_dups_size":0,"ondisk_log_size":7286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29843456,"num_objects":7286,"num_object_clones":0,"num_object_copies":14572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7286,"num_write_kb":29144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3214},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7264","reported_seq":7360,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:43.552485+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:14:43.552485+0000","last_peered":"2024-09-11T01:14:43.552485+0000","last_clean":"2024-09-11T01:14:43.552485+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:14:43.552485+0000","last_undegraded":"2024-09-11T01:14:43.552485+0000","last_fullsized":"2024-09-11T01:14:43.552485+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":7264,"log_dups_size":0,"ondisk_log_size":7264,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29728768,"num_objects":7258,"num_object_clones":0,"num_object_copies":14516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7258,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5503,"num_write_kb":22012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7132","reported_seq":7255,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:43.539863+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:14:43.539863+0000","last_peered":"2024-09-11T01:14:43.539863+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:14:43.539863+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":7132,"log_dups_size":0,"ondisk_log_size":7132,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29196288,"num_objects":7128,"num_object_clones":0,"num_object_copies":14256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5433,"num_write_kb":21732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":3232}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7162","reported_seq":7227,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:43.493647+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:14:43.493647+0000","last_peered":"2024-09-11T01:14:43.493647+0000","last_clean":"2024-09-11T01:14:43.493647+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:14:43.493647+0000","last_undegraded":"2024-09-11T01:14:43.493647+0000","last_fullsized":"2024-09-11T01:14:43.493647+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":7162,"log_dups_size":0,"ondisk_log_size":7162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29335552,"num_objects":7162,"num_object_clones":0,"num_object_copies":14324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7162,"num_write_kb":28648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7228","reported_seq":7351,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:44.281203+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:14:44.281203+0000","last_peered":"2024-09-11T01:14:44.281203+0000","last_clean":"2024-09-11T01:14:44.281203+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:14:44.281203+0000","last_undegraded":"2024-09-11T01:14:44.281203+0000","last_fullsized":"2024-09-11T01:14:44.281203+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":7228,"log_dups_size":0,"ondisk_log_size":7228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29577216,"num_objects":7221,"num_object_clones":0,"num_object_copies":14442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4233,"num_write_kb":16932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7204","reported_seq":7369,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:44.318212+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:14:44.318212+0000","last_peered":"2024-09-11T01:14:44.318212+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:14:44.318212+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":7204,"log_dups_size":0,"ondisk_log_size":7204,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29487104,"num_objects":7199,"num_object_clones":0,"num_object_copies":14398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3545,"num_write_kb":14180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3248},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7130","reported_seq":7255,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:44.300369+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:14:44.300369+0000","last_peered":"2024-09-11T01:14:44.300369+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:14:44.300369+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":7130,"log_dups_size":0,"ondisk_log_size":7130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29179904,"num_objects":7124,"num_object_clones":0,"num_object_copies":14248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3987,"num_write_kb":15948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3138},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7272","reported_seq":7397,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:44.289215+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:14:44.289215+0000","last_peered":"2024-09-11T01:14:44.289215+0000","last_clean":"2024-09-11T01:14:44.289215+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:14:44.289215+0000","last_undegraded":"2024-09-11T01:14:44.289215+0000","last_fullsized":"2024-09-11T01:14:44.289215+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":7272,"log_dups_size":0,"ondisk_log_size":7272,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29777920,"num_objects":7270,"num_object_clones":0,"num_object_copies":14540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4148,"num_write_kb":16592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7137","reported_seq":7260,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:43.577903+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:14:43.577903+0000","last_peered":"2024-09-11T01:14:43.577903+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:14:43.577903+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":7137,"log_dups_size":0,"ondisk_log_size":7137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29204480,"num_objects":7130,"num_object_clones":0,"num_object_copies":14260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7130,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5108,"num_write_kb":20432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":3197}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741922+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741922+0000","last_peered":"2024-09-11T01:13:09.741922+0000","last_clean":"2024-09-11T01:13:09.741922+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741922+0000","last_undegraded":"2024-09-11T01:13:09.741922+0000","last_fullsized":"2024-09-11T01:13:09.741922+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7136","reported_seq":7259,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:44.158058+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:14:44.158058+0000","last_peered":"2024-09-11T01:14:44.158058+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:14:44.158058+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":7136,"log_dups_size":0,"ondisk_log_size":7136,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29196288,"num_objects":7128,"num_object_clones":0,"num_object_copies":14256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7128,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3723,"num_write_kb":14892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3144},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.910012+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.910012+0000","last_peered":"2024-09-11T01:13:09.910012+0000","last_clean":"2024-09-11T01:13:09.910012+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.910012+0000","last_undegraded":"2024-09-11T01:13:09.910012+0000","last_fullsized":"2024-09-11T01:13:09.910012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7182","reported_seq":7296,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:43.587515+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:14:43.587515+0000","last_peered":"2024-09-11T01:14:43.587515+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:14:43.587515+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":7182,"log_dups_size":0,"ondisk_log_size":7182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29396992,"num_objects":7177,"num_object_clones":0,"num_object_copies":14354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4999,"num_write_kb":19996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":3153}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7080","reported_seq":11939,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:14:43.604944+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:14:43.604944+0000","last_peered":"2024-09-11T01:14:43.604944+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:14:43.604944+0000","last_undegraded":"2024-09-11T01:14:43.604944+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":7080,"log_dups_size":0,"ondisk_log_size":7080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28975104,"num_objects":7074,"num_object_clones":0,"num_object_copies":14148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1554,"num_objects_unfound":0,"num_objects_dirty":7074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4865,"num_write_kb":19460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2373,"num_bytes_recovered":9711616,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1554},{"shards":"2","objects":5520}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7287","reported_seq":7413,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:44.304721+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:14:44.304721+0000","last_peered":"2024-09-11T01:14:44.304721+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:14:44.304721+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":7287,"log_dups_size":0,"ondisk_log_size":7287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29839360,"num_objects":7285,"num_object_clones":0,"num_object_copies":14570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7285,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4024,"num_write_kb":16096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3335},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7139","reported_seq":7271,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:44.120987+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:14:44.120987+0000","last_peered":"2024-09-11T01:14:44.120987+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:14:44.120987+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":7139,"log_dups_size":0,"ondisk_log_size":7139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29208576,"num_objects":7131,"num_object_clones":0,"num_object_copies":14262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7131,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3794,"num_write_kb":15176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3192},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658174+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658174+0000","last_peered":"2024-09-11T01:13:09.658174+0000","last_clean":"2024-09-11T01:13:09.658174+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658174+0000","last_undegraded":"2024-09-11T01:13:09.658174+0000","last_fullsized":"2024-09-11T01:13:09.658174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7244","reported_seq":7374,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:44.302430+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:14:44.302430+0000","last_peered":"2024-09-11T01:14:44.302430+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:14:44.302430+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":7244,"log_dups_size":0,"ondisk_log_size":7244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29655040,"num_objects":7240,"num_object_clones":0,"num_object_copies":14480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3853,"num_write_kb":15412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3201},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7196","reported_seq":7325,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:44.329562+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:14:44.329562+0000","last_peered":"2024-09-11T01:14:44.329562+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:14:44.329562+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":7196,"log_dups_size":0,"ondisk_log_size":7196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29426281,"num_objects":7185,"num_object_clones":0,"num_object_copies":14370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4266,"num_write_kb":17064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3268},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":471028329,"num_objects":114998,"num_object_clones":0,"num_object_copies":229996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":1554,"num_objects_unfound":0,"num_objects_dirty":114998,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":75929,"num_write_kb":303716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2374,"num_bytes_recovered":9715712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":957878272,"data_stored":957798255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":115079,"ondisk_log_size":115079,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182439,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":683508,"kb_used_data":408932,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93688332,"statfs":{"total":96636764160,"available":95936851968,"internally_reserved":0,"allocated":418746368,"data_stored":418497549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":281145003},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,1,3,6,3,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986211,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":528544,"kb_used_data":356688,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93843296,"statfs":{"total":96636764160,"available":96095535104,"internally_reserved":0,"allocated":365248512,"data_stored":365003770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":175959722},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659157,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":326096,"kb_used_data":168128,"kb_used_omap":4,"kb_used_meta":157947,"kb_avail":94045744,"statfs":{"total":96636764160,"available":96302841856,"internally_reserved":0,"allocated":172163072,"data_stored":171844673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":161738415},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,1,5,5,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":419143680,"data_stored":419140201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":173277184,"data_stored":173204125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":365457408,"data_stored":365453929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:14:49.596 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:14:49.596 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'7286", 'reported_seq': 7358, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:44.306826+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:14:44.306826+0000', 'last_peered': '2024-09-11T01:14:44.306826+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:14:44.306826+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 7286, 'log_dups_size': 0, 'ondisk_log_size': 7286, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29843456, 'num_objects': 7286, 'num_object_clones': 0, 'num_object_copies': 14572, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7286, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7286, 'num_write_kb': 29144, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3214}, {'shards': '0,2', 'objects': 4072}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:49.596 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:14:49.596 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'7132", 'reported_seq': 7255, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:43.539863+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:14:43.539863+0000', 'last_peered': '2024-09-11T01:14:43.539863+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:14:43.539863+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 7132, 'log_dups_size': 0, 'ondisk_log_size': 7132, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29196288, 'num_objects': 7128, 'num_object_clones': 0, 'num_object_copies': 14256, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7128, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5433, 'num_write_kb': 21732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3896}, {'shards': '2', 'objects': 3232}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:14:49.597 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:14:49.597 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'7204", 'reported_seq': 7369, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:44.318212+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:14:44.318212+0000', 'last_peered': '2024-09-11T01:14:44.318212+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:14:44.318212+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 7204, 'log_dups_size': 0, 'ondisk_log_size': 7204, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29487104, 'num_objects': 7199, 'num_object_clones': 0, 'num_object_copies': 14398, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7199, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3545, 'num_write_kb': 14180, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3248}, {'shards': '0,2', 'objects': 3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:49.597 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:14:49.597 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'7130", 'reported_seq': 7255, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:44.300369+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:14:44.300369+0000', 'last_peered': '2024-09-11T01:14:44.300369+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:14:44.300369+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 7130, 'log_dups_size': 0, 'ondisk_log_size': 7130, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29179904, 'num_objects': 7124, 'num_object_clones': 0, 'num_object_copies': 14248, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7124, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3987, 'num_write_kb': 15948, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3138}, {'shards': '0,2', 'objects': 3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:49.597 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:14:49.597 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'7137", 'reported_seq': 7260, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:43.577903+0000', 'last_change': '2024-09-11T01:13:10.159069+0000', 'last_active': '2024-09-11T01:14:43.577903+0000', 'last_peered': '2024-09-11T01:14:43.577903+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:14:43.577903+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 7137, 'log_dups_size': 0, 'ondisk_log_size': 7137, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29204480, 'num_objects': 7130, 'num_object_clones': 0, 'num_object_copies': 14260, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7130, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5108, 'num_write_kb': 20432, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3932}, {'shards': '2', 'objects': 3197}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:14:49.597 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:14:49.597 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'7136", 'reported_seq': 7259, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:44.158058+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:14:44.158058+0000', 'last_peered': '2024-09-11T01:14:44.158058+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:14:44.158058+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 7136, 'log_dups_size': 0, 'ondisk_log_size': 7136, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29196288, 'num_objects': 7128, 'num_object_clones': 0, 'num_object_copies': 14256, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7128, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3723, 'num_write_kb': 14892, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3144}, {'shards': '0,2', 'objects': 3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:49.597 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:14:49.597 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'7182", 'reported_seq': 7296, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:43.587515+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:14:43.587515+0000', 'last_peered': '2024-09-11T01:14:43.587515+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:14:43.587515+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 7182, 'log_dups_size': 0, 'ondisk_log_size': 7182, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29396992, 'num_objects': 7177, 'num_object_clones': 0, 'num_object_copies': 14354, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7177, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4999, 'num_write_kb': 19996, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4023}, {'shards': '2', 'objects': 3153}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:14:49.598 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:14:49.598 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'7080", 'reported_seq': 11939, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:14:43.604944+0000', 'last_change': '2024-09-11T01:13:10.217540+0000', 'last_active': '2024-09-11T01:14:43.604944+0000', 'last_peered': '2024-09-11T01:14:43.604944+0000', 'last_clean': '2024-09-11T01:13:07.638709+0000', 'last_became_active': '2024-09-11T01:13:10.125604+0000', 'last_became_peered': '2024-09-11T01:13:10.125604+0000', 'last_unstale': '2024-09-11T01:14:43.604944+0000', 'last_undegraded': '2024-09-11T01:14:43.604944+0000', 'last_fullsized': '2024-09-11T01:13:09.649604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2269", 'last_scrub_stamp': '2024-09-11T01:12:42.890920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:42.890920+0000', 'objects_scrubbed': 2209, 'log_size': 7080, 'log_dups_size': 0, 'ondisk_log_size': 7080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 999, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28975104, 'num_objects': 7074, 'num_object_clones': 0, 'num_object_copies': 14148, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1554, 'num_objects_unfound': 0, 'num_objects_dirty': 7074, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4865, 'num_write_kb': 19460, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2373, 'num_bytes_recovered': 9711616, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1554}, {'shards': '2', 'objects': 5520}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:14:49.598 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:14:49.598 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'7287", 'reported_seq': 7413, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:44.304721+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:14:44.304721+0000', 'last_peered': '2024-09-11T01:14:44.304721+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:14:44.304721+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 7287, 'log_dups_size': 0, 'ondisk_log_size': 7287, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29839360, 'num_objects': 7285, 'num_object_clones': 0, 'num_object_copies': 14570, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7285, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4024, 'num_write_kb': 16096, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3335}, {'shards': '0,2', 'objects': 3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:49.598 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:14:49.598 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'7139", 'reported_seq': 7271, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:44.120987+0000', 'last_change': '2024-09-11T01:13:10.209668+0000', 'last_active': '2024-09-11T01:14:44.120987+0000', 'last_peered': '2024-09-11T01:14:44.120987+0000', 'last_clean': '2024-09-11T01:13:07.624565+0000', 'last_became_active': '2024-09-11T01:13:10.199795+0000', 'last_became_peered': '2024-09-11T01:13:10.199795+0000', 'last_unstale': '2024-09-11T01:14:44.120987+0000', 'last_undegraded': '2024-09-11T01:13:09.791198+0000', 'last_fullsized': '2024-09-11T01:13:09.652311+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3415", 'last_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'objects_scrubbed': 3337, 'log_size': 7139, 'log_dups_size': 0, 'ondisk_log_size': 7139, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1975, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29208576, 'num_objects': 7131, 'num_object_clones': 0, 'num_object_copies': 14262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3942, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7131, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3794, 'num_write_kb': 15176, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3192}, {'shards': '0,2', 'objects': 3939}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:49.598 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:14:49.598 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'7244", 'reported_seq': 7374, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:44.302430+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:14:44.302430+0000', 'last_peered': '2024-09-11T01:14:44.302430+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:14:44.302430+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 7244, 'log_dups_size': 0, 'ondisk_log_size': 7244, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29655040, 'num_objects': 7240, 'num_object_clones': 0, 'num_object_copies': 14480, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7240, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3853, 'num_write_kb': 15412, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3201}, {'shards': '0,2', 'objects': 4039}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:49.598 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:14:49.598 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'7196", 'reported_seq': 7325, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:44.329562+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:14:44.329562+0000', 'last_peered': '2024-09-11T01:14:44.329562+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:14:44.329562+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 7196, 'log_dups_size': 0, 'ondisk_log_size': 7196, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29426281, 'num_objects': 7185, 'num_object_clones': 0, 'num_object_copies': 14370, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7185, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4266, 'num_write_kb': 17064, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3268}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:49.599 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:14:50.395 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 146 20 118126 118106 3.15879 1.74219 0.0148006 0.0247211 2024-09-11T01:14:50.507 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:14:50.507 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43711/230000 objects degraded (19.005%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":226,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":115000,"data_bytes":471028348,"bytes_used":1575063552,"bytes_avail":288335228928,"bytes_total":289910292480,"degraded_objects":43711,"degraded_total":230000,"degraded_ratio":0.19004782608695653,"misplaced_objects":1554,"misplaced_total":230000,"misplaced_ratio":0.0067565217391304345,"recovering_objects_per_sec":26,"recovering_bytes_per_sec":107493,"recovering_keys_per_sec":0,"num_objects_recovered":317,"num_bytes_recovered":1290240,"num_keys_recovered":0,"write_bytes_sec":1695672,"read_op_per_sec":0,"write_op_per_sec":413},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:14:50.561 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:14:50.562 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:14:51.326 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:14:51.327 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:14:51.395 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 147 20 118603 118583 3.14998 1.86328 0.0503583 0.024794 2024-09-11T01:14:51.415 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":150,"stamp":"2024-09-11T01:14:50.586290+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":481251964,"num_objects":117496,"num_object_clones":0,"num_object_copies":234992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":1393,"num_objects_unfound":0,"num_objects_dirty":117496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":78427,"num_write_kb":313702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2536,"num_bytes_recovered":10375187,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117578,"ondisk_log_size":117578,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1546296,"kb_used_data":946056,"kb_used_omap":12,"kb_used_meta":600179,"kb_avail":281569224,"statfs":{"total":289910292480,"available":288326885376,"internally_reserved":0,"allocated":968761344,"data_stored":967949384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":614583300},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,6,3,12,13,4,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":26742784,"num_objects":6529,"num_object_clones":0,"num_object_copies":13058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-322,"num_objects_unfound":0,"num_objects_dirty":6529,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6529,"num_write_kb":26116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":322,"num_bytes_recovered":1318912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002977"},"pg_stats":[{"pgid":"2.f","version":"43'7440","reported_seq":7513,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:49.313774+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:14:49.313774+0000","last_peered":"2024-09-11T01:14:49.313774+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:14:49.313774+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":7440,"log_dups_size":0,"ondisk_log_size":7440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30474240,"num_objects":7440,"num_object_clones":0,"num_object_copies":14880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7440,"num_write_kb":29760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3368},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7423","reported_seq":7520,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:48.574987+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:14:48.574987+0000","last_peered":"2024-09-11T01:14:48.574987+0000","last_clean":"2024-09-11T01:14:48.574987+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:14:48.574987+0000","last_undegraded":"2024-09-11T01:14:48.574987+0000","last_fullsized":"2024-09-11T01:14:48.574987+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":7423,"log_dups_size":0,"ondisk_log_size":7423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30380032,"num_objects":7417,"num_object_clones":0,"num_object_copies":14834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7417,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5662,"num_write_kb":22648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7287","reported_seq":7410,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:48.531697+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:14:48.531697+0000","last_peered":"2024-09-11T01:14:48.531697+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:14:48.531697+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":7287,"log_dups_size":0,"ondisk_log_size":7287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29831168,"num_objects":7283,"num_object_clones":0,"num_object_copies":14566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5588,"num_write_kb":22352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":3387}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7323","reported_seq":7388,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:48.586166+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:14:48.586166+0000","last_peered":"2024-09-11T01:14:48.586166+0000","last_clean":"2024-09-11T01:14:48.586166+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:14:48.586166+0000","last_undegraded":"2024-09-11T01:14:48.586166+0000","last_fullsized":"2024-09-11T01:14:48.586166+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":7323,"log_dups_size":0,"ondisk_log_size":7323,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29995008,"num_objects":7323,"num_object_clones":0,"num_object_copies":14646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7323,"num_write_kb":29292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7384","reported_seq":7507,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:49.277288+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:14:49.277288+0000","last_peered":"2024-09-11T01:14:49.277288+0000","last_clean":"2024-09-11T01:14:49.277288+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:14:49.277288+0000","last_undegraded":"2024-09-11T01:14:49.277288+0000","last_fullsized":"2024-09-11T01:14:49.277288+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":7384,"log_dups_size":0,"ondisk_log_size":7384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30216192,"num_objects":7377,"num_object_clones":0,"num_object_copies":14754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4389,"num_write_kb":17556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7367","reported_seq":7532,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:49.260607+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:14:49.260607+0000","last_peered":"2024-09-11T01:14:49.260607+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:14:49.260607+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":7367,"log_dups_size":0,"ondisk_log_size":7367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30154752,"num_objects":7362,"num_object_clones":0,"num_object_copies":14724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3708,"num_write_kb":14832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3411},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7277","reported_seq":7400,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:49.341341+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:14:49.341341+0000","last_peered":"2024-09-11T01:14:49.341341+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:14:49.341341+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":7277,"log_dups_size":0,"ondisk_log_size":7277,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29782016,"num_objects":7271,"num_object_clones":0,"num_object_copies":14542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4134,"num_write_kb":16536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3285},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7415","reported_seq":7540,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:49.249684+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:14:49.249684+0000","last_peered":"2024-09-11T01:14:49.249684+0000","last_clean":"2024-09-11T01:14:49.249684+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:14:49.249684+0000","last_undegraded":"2024-09-11T01:14:49.249684+0000","last_fullsized":"2024-09-11T01:14:49.249684+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":7415,"log_dups_size":0,"ondisk_log_size":7415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30363648,"num_objects":7413,"num_object_clones":0,"num_object_copies":14826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7413,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4291,"num_write_kb":17164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7293","reported_seq":7416,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:48.482624+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:14:48.482624+0000","last_peered":"2024-09-11T01:14:48.482624+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:14:48.482624+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":7293,"log_dups_size":0,"ondisk_log_size":7293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29843456,"num_objects":7286,"num_object_clones":0,"num_object_copies":14572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5264,"num_write_kb":21056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":3353}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741922+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741922+0000","last_peered":"2024-09-11T01:13:09.741922+0000","last_clean":"2024-09-11T01:13:09.741922+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741922+0000","last_undegraded":"2024-09-11T01:13:09.741922+0000","last_fullsized":"2024-09-11T01:13:09.741922+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7299","reported_seq":7422,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:49.282613+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:14:49.282613+0000","last_peered":"2024-09-11T01:14:49.282613+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:14:49.282613+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":7299,"log_dups_size":0,"ondisk_log_size":7299,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29863936,"num_objects":7291,"num_object_clones":0,"num_object_copies":14582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3886,"num_write_kb":15544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3307},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.910012+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.910012+0000","last_peered":"2024-09-11T01:13:09.910012+0000","last_clean":"2024-09-11T01:13:09.910012+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.910012+0000","last_undegraded":"2024-09-11T01:13:09.910012+0000","last_fullsized":"2024-09-11T01:13:09.910012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7331","reported_seq":7445,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:48.549109+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:14:48.549109+0000","last_peered":"2024-09-11T01:14:48.549109+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:14:48.549109+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":7331,"log_dups_size":0,"ondisk_log_size":7331,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30007296,"num_objects":7326,"num_object_clones":0,"num_object_copies":14652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7326,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5148,"num_write_kb":20592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":3302}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7258","reported_seq":12441,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:14:48.616014+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:14:48.616014+0000","last_peered":"2024-09-11T01:14:48.616014+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:14:48.616014+0000","last_undegraded":"2024-09-11T01:14:48.616014+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":7258,"log_dups_size":0,"ondisk_log_size":7258,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29704192,"num_objects":7252,"num_object_clones":0,"num_object_copies":14504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1393,"num_objects_unfound":0,"num_objects_dirty":7252,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5043,"num_write_kb":20172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2534,"num_bytes_recovered":10371072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1393},{"shards":"2","objects":5859}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7433","reported_seq":7560,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:49.307531+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:14:49.307531+0000","last_peered":"2024-09-11T01:14:49.307531+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:14:49.307531+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":7433,"log_dups_size":0,"ondisk_log_size":7433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30437376,"num_objects":7431,"num_object_clones":0,"num_object_copies":14862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7431,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4170,"num_write_kb":16680,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3481},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7281","reported_seq":7413,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:49.338756+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:14:49.338756+0000","last_peered":"2024-09-11T01:14:49.338756+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:14:49.338756+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":7281,"log_dups_size":0,"ondisk_log_size":7281,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29790208,"num_objects":7273,"num_object_clones":0,"num_object_copies":14546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3936,"num_write_kb":15744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3334},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658174+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658174+0000","last_peered":"2024-09-11T01:13:09.658174+0000","last_clean":"2024-09-11T01:13:09.658174+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658174+0000","last_undegraded":"2024-09-11T01:13:09.658174+0000","last_fullsized":"2024-09-11T01:13:09.658174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7407","reported_seq":7537,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:49.328534+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:14:49.328534+0000","last_peered":"2024-09-11T01:14:49.328534+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:14:49.328534+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":7407,"log_dups_size":0,"ondisk_log_size":7407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30322688,"num_objects":7403,"num_object_clones":0,"num_object_copies":14806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4016,"num_write_kb":16064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3364},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7357","reported_seq":7486,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:49.299863+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:14:49.299863+0000","last_peered":"2024-09-11T01:14:49.299863+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:14:49.299863+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":7357,"log_dups_size":0,"ondisk_log_size":7357,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30085737,"num_objects":7346,"num_object_clones":0,"num_object_copies":14692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4427,"num_write_kb":17708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3429},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":481251945,"num_objects":117494,"num_object_clones":0,"num_object_copies":234988,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":1393,"num_objects_unfound":0,"num_objects_dirty":117494,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":78425,"num_write_kb":313700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2535,"num_bytes_recovered":10375168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":970645504,"data_stored":970565487,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117575,"ondisk_log_size":117575,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182440,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":682552,"kb_used_data":416296,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93689288,"statfs":{"total":96636764160,"available":95937830912,"internally_reserved":0,"allocated":426287104,"data_stored":426038285,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":272625323},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,1,3,6,3,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986212,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537648,"kb_used_data":361632,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93834192,"statfs":{"total":96636764160,"available":96086212608,"internally_reserved":0,"allocated":370311168,"data_stored":370066426,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":180219562},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,4,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659157,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":326096,"kb_used_data":168128,"kb_used_omap":4,"kb_used_meta":157947,"kb_avail":94045744,"statfs":{"total":96636764160,"available":96302841856,"internally_reserved":0,"allocated":172163072,"data_stored":171844673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":161738415},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,1,5,5,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":426737664,"data_stored":426734185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":173277184,"data_stored":173204125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":370630656,"data_stored":370627177,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:14:52.396 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 148 20 119115 119095 3.14219 2 0.0168319 0.0248559 2024-09-11T01:14:53.396 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 149 20 119615 119595 3.13421 1.95312 0.0278658 0.0249167 2024-09-11T01:14:54.398 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 150 20 120104 120084 3.12602 1.91016 0.0798011 0.0249833 2024-09-11T01:14:54.417 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:14:55.323 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:14:55.325 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:14:55.359 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":152,"stamp":"2024-09-11T01:14:54.587261+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":491332220,"num_objects":119957,"num_object_clones":0,"num_object_copies":239914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":1227,"num_objects_unfound":0,"num_objects_dirty":119957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80888,"num_write_kb":323546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2702,"num_bytes_recovered":11055123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120039,"ondisk_log_size":120039,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1582776,"kb_used_data":965896,"kb_used_omap":12,"kb_used_meta":616819,"kb_avail":281532744,"statfs":{"total":289910292480,"available":288289529856,"internally_reserved":0,"allocated":989077504,"data_stored":988265544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":631622660},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,6,7,9,14,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30359552,"num_objects":7412,"num_object_clones":0,"num_object_copies":14824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-488,"num_objects_unfound":0,"num_objects_dirty":7412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7412,"num_write_kb":29648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":488,"num_bytes_recovered":1998848,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003090"},"pg_stats":[{"pgid":"2.f","version":"43'7600","reported_seq":7673,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:54.371039+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:14:54.371039+0000","last_peered":"2024-09-11T01:14:54.371039+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:14:54.371039+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":7600,"log_dups_size":0,"ondisk_log_size":7600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31129600,"num_objects":7600,"num_object_clones":0,"num_object_copies":15200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7600,"num_write_kb":30400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3528},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7579","reported_seq":7676,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:53.612756+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:14:53.612756+0000","last_peered":"2024-09-11T01:14:53.612756+0000","last_clean":"2024-09-11T01:14:53.612756+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:14:53.612756+0000","last_undegraded":"2024-09-11T01:14:53.612756+0000","last_fullsized":"2024-09-11T01:14:53.612756+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":7579,"log_dups_size":0,"ondisk_log_size":7579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31019008,"num_objects":7573,"num_object_clones":0,"num_object_copies":15146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5818,"num_write_kb":23272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7441","reported_seq":7564,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:53.618115+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:14:53.618115+0000","last_peered":"2024-09-11T01:14:53.618115+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:14:53.618115+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":7441,"log_dups_size":0,"ondisk_log_size":7441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30461952,"num_objects":7437,"num_object_clones":0,"num_object_copies":14874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5742,"num_write_kb":22968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":3541}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7459","reported_seq":7524,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:53.567397+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:14:53.567397+0000","last_peered":"2024-09-11T01:14:53.567397+0000","last_clean":"2024-09-11T01:14:53.567397+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:14:53.567397+0000","last_undegraded":"2024-09-11T01:14:53.567397+0000","last_fullsized":"2024-09-11T01:14:53.567397+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":7459,"log_dups_size":0,"ondisk_log_size":7459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30552064,"num_objects":7459,"num_object_clones":0,"num_object_copies":14918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7459,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7459,"num_write_kb":29836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7541","reported_seq":7664,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:54.267162+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:14:54.267162+0000","last_peered":"2024-09-11T01:14:54.267162+0000","last_clean":"2024-09-11T01:14:54.267162+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:14:54.267162+0000","last_undegraded":"2024-09-11T01:14:54.267162+0000","last_fullsized":"2024-09-11T01:14:54.267162+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":7541,"log_dups_size":0,"ondisk_log_size":7541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30859264,"num_objects":7534,"num_object_clones":0,"num_object_copies":15068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4546,"num_write_kb":18184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7528","reported_seq":7693,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:54.341780+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:14:54.341780+0000","last_peered":"2024-09-11T01:14:54.341780+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:14:54.341780+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":7528,"log_dups_size":0,"ondisk_log_size":7528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30814208,"num_objects":7523,"num_object_clones":0,"num_object_copies":15046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3869,"num_write_kb":15476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3572},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7432","reported_seq":7557,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:54.282927+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:14:54.282927+0000","last_peered":"2024-09-11T01:14:54.282927+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:14:54.282927+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":7432,"log_dups_size":0,"ondisk_log_size":7432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30416896,"num_objects":7426,"num_object_clones":0,"num_object_copies":14852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4289,"num_write_kb":17156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3440},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7556","reported_seq":7681,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:54.336451+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:14:54.336451+0000","last_peered":"2024-09-11T01:14:54.336451+0000","last_clean":"2024-09-11T01:14:54.336451+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:14:54.336451+0000","last_undegraded":"2024-09-11T01:14:54.336451+0000","last_fullsized":"2024-09-11T01:14:54.336451+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":7556,"log_dups_size":0,"ondisk_log_size":7556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30941184,"num_objects":7554,"num_object_clones":0,"num_object_copies":15108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7554,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4432,"num_write_kb":17728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7470","reported_seq":7593,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:53.582370+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:14:53.582370+0000","last_peered":"2024-09-11T01:14:53.582370+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:14:53.582370+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":7470,"log_dups_size":0,"ondisk_log_size":7470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30568448,"num_objects":7463,"num_object_clones":0,"num_object_copies":14926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5441,"num_write_kb":21764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":3530}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741922+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741922+0000","last_peered":"2024-09-11T01:13:09.741922+0000","last_clean":"2024-09-11T01:13:09.741922+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741922+0000","last_undegraded":"2024-09-11T01:13:09.741922+0000","last_fullsized":"2024-09-11T01:13:09.741922+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7438","reported_seq":7561,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:54.318729+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:14:54.318729+0000","last_peered":"2024-09-11T01:14:54.318729+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:14:54.318729+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":7438,"log_dups_size":0,"ondisk_log_size":7438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30433280,"num_objects":7430,"num_object_clones":0,"num_object_copies":14860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4025,"num_write_kb":16100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3446},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.910012+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.910012+0000","last_peered":"2024-09-11T01:13:09.910012+0000","last_clean":"2024-09-11T01:13:09.910012+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.910012+0000","last_undegraded":"2024-09-11T01:13:09.910012+0000","last_fullsized":"2024-09-11T01:13:09.910012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7486","reported_seq":7600,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:53.575596+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:14:53.575596+0000","last_peered":"2024-09-11T01:14:53.575596+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:14:53.575596+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":7486,"log_dups_size":0,"ondisk_log_size":7486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30642176,"num_objects":7481,"num_object_clones":0,"num_object_copies":14962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7481,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5303,"num_write_kb":21212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":3457}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7421","reported_seq":12936,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:14:53.618926+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:14:53.618926+0000","last_peered":"2024-09-11T01:14:53.618926+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:14:53.618926+0000","last_undegraded":"2024-09-11T01:14:53.618926+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":7421,"log_dups_size":0,"ondisk_log_size":7421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30371840,"num_objects":7415,"num_object_clones":0,"num_object_copies":14830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1227,"num_objects_unfound":0,"num_objects_dirty":7415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5206,"num_write_kb":20824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2700,"num_bytes_recovered":11051008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1227},{"shards":"2","objects":6188}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7582","reported_seq":7709,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:54.338013+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:14:54.338013+0000","last_peered":"2024-09-11T01:14:54.338013+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:14:54.338013+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":7582,"log_dups_size":0,"ondisk_log_size":7582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31047680,"num_objects":7580,"num_object_clones":0,"num_object_copies":15160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4319,"num_write_kb":17276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3630},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7425","reported_seq":7552,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:54.311379+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:14:54.311379+0000","last_peered":"2024-09-11T01:14:54.311379+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:14:54.311379+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":7425,"log_dups_size":0,"ondisk_log_size":7425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30380032,"num_objects":7417,"num_object_clones":0,"num_object_copies":14834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7417,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4080,"num_write_kb":16320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3478},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658174+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658174+0000","last_peered":"2024-09-11T01:13:09.658174+0000","last_clean":"2024-09-11T01:13:09.658174+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658174+0000","last_undegraded":"2024-09-11T01:13:09.658174+0000","last_fullsized":"2024-09-11T01:13:09.658174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7574","reported_seq":7700,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:54.267009+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:14:54.267009+0000","last_peered":"2024-09-11T01:14:54.267009+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:14:54.267009+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":7574,"log_dups_size":0,"ondisk_log_size":7574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31006720,"num_objects":7570,"num_object_clones":0,"num_object_copies":15140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4183,"num_write_kb":16732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3531},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7504","reported_seq":7633,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:54.364129+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:14:54.364129+0000","last_peered":"2024-09-11T01:14:54.364129+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:14:54.364129+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":7504,"log_dups_size":0,"ondisk_log_size":7504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30687849,"num_objects":7493,"num_object_clones":0,"num_object_copies":14986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4574,"num_write_kb":18296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3576},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":491332201,"num_objects":119955,"num_object_clones":0,"num_object_copies":239910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":1227,"num_objects_unfound":0,"num_objects_dirty":119955,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80886,"num_write_kb":323544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2701,"num_bytes_recovered":11055104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":991375360,"data_stored":991295343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120036,"ondisk_log_size":120036,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182441,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":693748,"kb_used_data":423332,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93678092,"statfs":{"total":96636764160,"available":95926366208,"internally_reserved":0,"allocated":433491968,"data_stored":433243149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":276885163},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986213,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":546600,"kb_used_data":366424,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93825240,"statfs":{"total":96636764160,"available":96077045760,"internally_reserved":0,"allocated":375218176,"data_stored":374973434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184479402},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659158,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":342428,"kb_used_data":176140,"kb_used_omap":4,"kb_used_meta":166267,"kb_avail":94029412,"statfs":{"total":96636764160,"available":96286117888,"internally_reserved":0,"allocated":180367360,"data_stored":180048961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":170258095},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,4,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":434159616,"data_stored":434156137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":181530624,"data_stored":181457565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":375685120,"data_stored":375681641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:14:55.361 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:14:55.397 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 151 20 120559 120539 3.11709 1.77734 0.0287453 0.0250538 2024-09-11T01:14:56.108 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:14:56.109 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:14:56.173 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":152,"stamp":"2024-09-11T01:14:54.587261+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":491332220,"num_objects":119957,"num_object_clones":0,"num_object_copies":239914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":1227,"num_objects_unfound":0,"num_objects_dirty":119957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80888,"num_write_kb":323546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2702,"num_bytes_recovered":11055123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120039,"ondisk_log_size":120039,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1582776,"kb_used_data":965896,"kb_used_omap":12,"kb_used_meta":616819,"kb_avail":281532744,"statfs":{"total":289910292480,"available":288289529856,"internally_reserved":0,"allocated":989077504,"data_stored":988265544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":631622660},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,6,7,9,14,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30359552,"num_objects":7412,"num_object_clones":0,"num_object_copies":14824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-488,"num_objects_unfound":0,"num_objects_dirty":7412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7412,"num_write_kb":29648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":488,"num_bytes_recovered":1998848,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003090"},"pg_stats":[{"pgid":"2.f","version":"43'7600","reported_seq":7673,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:54.371039+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:14:54.371039+0000","last_peered":"2024-09-11T01:14:54.371039+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:14:54.371039+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":7600,"log_dups_size":0,"ondisk_log_size":7600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31129600,"num_objects":7600,"num_object_clones":0,"num_object_copies":15200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7600,"num_write_kb":30400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3528},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7579","reported_seq":7676,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:53.612756+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:14:53.612756+0000","last_peered":"2024-09-11T01:14:53.612756+0000","last_clean":"2024-09-11T01:14:53.612756+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:14:53.612756+0000","last_undegraded":"2024-09-11T01:14:53.612756+0000","last_fullsized":"2024-09-11T01:14:53.612756+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":7579,"log_dups_size":0,"ondisk_log_size":7579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31019008,"num_objects":7573,"num_object_clones":0,"num_object_copies":15146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5818,"num_write_kb":23272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7441","reported_seq":7564,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:53.618115+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:14:53.618115+0000","last_peered":"2024-09-11T01:14:53.618115+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:14:53.618115+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":7441,"log_dups_size":0,"ondisk_log_size":7441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30461952,"num_objects":7437,"num_object_clones":0,"num_object_copies":14874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5742,"num_write_kb":22968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":3541}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7459","reported_seq":7524,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:53.567397+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:14:53.567397+0000","last_peered":"2024-09-11T01:14:53.567397+0000","last_clean":"2024-09-11T01:14:53.567397+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:14:53.567397+0000","last_undegraded":"2024-09-11T01:14:53.567397+0000","last_fullsized":"2024-09-11T01:14:53.567397+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":7459,"log_dups_size":0,"ondisk_log_size":7459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30552064,"num_objects":7459,"num_object_clones":0,"num_object_copies":14918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7459,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7459,"num_write_kb":29836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7541","reported_seq":7664,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:54.267162+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:14:54.267162+0000","last_peered":"2024-09-11T01:14:54.267162+0000","last_clean":"2024-09-11T01:14:54.267162+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:14:54.267162+0000","last_undegraded":"2024-09-11T01:14:54.267162+0000","last_fullsized":"2024-09-11T01:14:54.267162+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":7541,"log_dups_size":0,"ondisk_log_size":7541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30859264,"num_objects":7534,"num_object_clones":0,"num_object_copies":15068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4546,"num_write_kb":18184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7528","reported_seq":7693,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:54.341780+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:14:54.341780+0000","last_peered":"2024-09-11T01:14:54.341780+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:14:54.341780+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":7528,"log_dups_size":0,"ondisk_log_size":7528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30814208,"num_objects":7523,"num_object_clones":0,"num_object_copies":15046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3869,"num_write_kb":15476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3572},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7432","reported_seq":7557,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:54.282927+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:14:54.282927+0000","last_peered":"2024-09-11T01:14:54.282927+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:14:54.282927+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":7432,"log_dups_size":0,"ondisk_log_size":7432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30416896,"num_objects":7426,"num_object_clones":0,"num_object_copies":14852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4289,"num_write_kb":17156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3440},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7556","reported_seq":7681,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:54.336451+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:14:54.336451+0000","last_peered":"2024-09-11T01:14:54.336451+0000","last_clean":"2024-09-11T01:14:54.336451+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:14:54.336451+0000","last_undegraded":"2024-09-11T01:14:54.336451+0000","last_fullsized":"2024-09-11T01:14:54.336451+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":7556,"log_dups_size":0,"ondisk_log_size":7556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30941184,"num_objects":7554,"num_object_clones":0,"num_object_copies":15108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7554,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4432,"num_write_kb":17728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7470","reported_seq":7593,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:53.582370+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:14:53.582370+0000","last_peered":"2024-09-11T01:14:53.582370+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:14:53.582370+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":7470,"log_dups_size":0,"ondisk_log_size":7470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30568448,"num_objects":7463,"num_object_clones":0,"num_object_copies":14926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5441,"num_write_kb":21764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":3530}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741922+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741922+0000","last_peered":"2024-09-11T01:13:09.741922+0000","last_clean":"2024-09-11T01:13:09.741922+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741922+0000","last_undegraded":"2024-09-11T01:13:09.741922+0000","last_fullsized":"2024-09-11T01:13:09.741922+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7438","reported_seq":7561,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:54.318729+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:14:54.318729+0000","last_peered":"2024-09-11T01:14:54.318729+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:14:54.318729+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":7438,"log_dups_size":0,"ondisk_log_size":7438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30433280,"num_objects":7430,"num_object_clones":0,"num_object_copies":14860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4025,"num_write_kb":16100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3446},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.910012+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.910012+0000","last_peered":"2024-09-11T01:13:09.910012+0000","last_clean":"2024-09-11T01:13:09.910012+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.910012+0000","last_undegraded":"2024-09-11T01:13:09.910012+0000","last_fullsized":"2024-09-11T01:13:09.910012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7486","reported_seq":7600,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:53.575596+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:14:53.575596+0000","last_peered":"2024-09-11T01:14:53.575596+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:14:53.575596+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":7486,"log_dups_size":0,"ondisk_log_size":7486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30642176,"num_objects":7481,"num_object_clones":0,"num_object_copies":14962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7481,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5303,"num_write_kb":21212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":3457}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7421","reported_seq":12936,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:14:53.618926+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:14:53.618926+0000","last_peered":"2024-09-11T01:14:53.618926+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:14:53.618926+0000","last_undegraded":"2024-09-11T01:14:53.618926+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":7421,"log_dups_size":0,"ondisk_log_size":7421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30371840,"num_objects":7415,"num_object_clones":0,"num_object_copies":14830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1227,"num_objects_unfound":0,"num_objects_dirty":7415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5206,"num_write_kb":20824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2700,"num_bytes_recovered":11051008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1227},{"shards":"2","objects":6188}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7582","reported_seq":7709,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:54.338013+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:14:54.338013+0000","last_peered":"2024-09-11T01:14:54.338013+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:14:54.338013+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":7582,"log_dups_size":0,"ondisk_log_size":7582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31047680,"num_objects":7580,"num_object_clones":0,"num_object_copies":15160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4319,"num_write_kb":17276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3630},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7425","reported_seq":7552,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:54.311379+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:14:54.311379+0000","last_peered":"2024-09-11T01:14:54.311379+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:14:54.311379+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":7425,"log_dups_size":0,"ondisk_log_size":7425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30380032,"num_objects":7417,"num_object_clones":0,"num_object_copies":14834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7417,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4080,"num_write_kb":16320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3478},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658174+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658174+0000","last_peered":"2024-09-11T01:13:09.658174+0000","last_clean":"2024-09-11T01:13:09.658174+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658174+0000","last_undegraded":"2024-09-11T01:13:09.658174+0000","last_fullsized":"2024-09-11T01:13:09.658174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7574","reported_seq":7700,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:54.267009+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:14:54.267009+0000","last_peered":"2024-09-11T01:14:54.267009+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:14:54.267009+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":7574,"log_dups_size":0,"ondisk_log_size":7574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31006720,"num_objects":7570,"num_object_clones":0,"num_object_copies":15140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4183,"num_write_kb":16732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3531},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7504","reported_seq":7633,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:54.364129+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:14:54.364129+0000","last_peered":"2024-09-11T01:14:54.364129+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:14:54.364129+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":7504,"log_dups_size":0,"ondisk_log_size":7504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30687849,"num_objects":7493,"num_object_clones":0,"num_object_copies":14986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4574,"num_write_kb":18296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3576},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":491332201,"num_objects":119955,"num_object_clones":0,"num_object_copies":239910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":1227,"num_objects_unfound":0,"num_objects_dirty":119955,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80886,"num_write_kb":323544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2701,"num_bytes_recovered":11055104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":991375360,"data_stored":991295343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120036,"ondisk_log_size":120036,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182441,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":693748,"kb_used_data":423332,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93678092,"statfs":{"total":96636764160,"available":95926366208,"internally_reserved":0,"allocated":433491968,"data_stored":433243149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":276885163},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986213,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":546600,"kb_used_data":366424,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93825240,"statfs":{"total":96636764160,"available":96077045760,"internally_reserved":0,"allocated":375218176,"data_stored":374973434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184479402},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659158,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":342428,"kb_used_data":176140,"kb_used_omap":4,"kb_used_meta":166267,"kb_avail":94029412,"statfs":{"total":96636764160,"available":96286117888,"internally_reserved":0,"allocated":180367360,"data_stored":180048961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":170258095},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,4,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":434159616,"data_stored":434156137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":181530624,"data_stored":181457565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":375685120,"data_stored":375681641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:14:56.174 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:14:56.174 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'7600", 'reported_seq': 7673, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:54.371039+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:14:54.371039+0000', 'last_peered': '2024-09-11T01:14:54.371039+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:14:54.371039+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 7600, 'log_dups_size': 0, 'ondisk_log_size': 7600, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31129600, 'num_objects': 7600, 'num_object_clones': 0, 'num_object_copies': 15200, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7600, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7600, 'num_write_kb': 30400, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3528}, {'shards': '0,2', 'objects': 4072}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:56.175 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:14:56.175 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'7441", 'reported_seq': 7564, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:53.618115+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:14:53.618115+0000', 'last_peered': '2024-09-11T01:14:53.618115+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:14:53.618115+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 7441, 'log_dups_size': 0, 'ondisk_log_size': 7441, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30461952, 'num_objects': 7437, 'num_object_clones': 0, 'num_object_copies': 14874, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7437, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5742, 'num_write_kb': 22968, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3896}, {'shards': '2', 'objects': 3541}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:14:56.175 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:14:56.175 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'7528", 'reported_seq': 7693, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:54.341780+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:14:54.341780+0000', 'last_peered': '2024-09-11T01:14:54.341780+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:14:54.341780+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 7528, 'log_dups_size': 0, 'ondisk_log_size': 7528, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30814208, 'num_objects': 7523, 'num_object_clones': 0, 'num_object_copies': 15046, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7523, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 3869, 'num_write_kb': 15476, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3572}, {'shards': '0,2', 'objects': 3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:56.175 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:14:56.175 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'7432", 'reported_seq': 7557, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:54.282927+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:14:54.282927+0000', 'last_peered': '2024-09-11T01:14:54.282927+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:14:54.282927+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 7432, 'log_dups_size': 0, 'ondisk_log_size': 7432, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30416896, 'num_objects': 7426, 'num_object_clones': 0, 'num_object_copies': 14852, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7426, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4289, 'num_write_kb': 17156, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3440}, {'shards': '0,2', 'objects': 3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:56.175 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:14:56.175 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'7470", 'reported_seq': 7593, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:53.582370+0000', 'last_change': '2024-09-11T01:13:10.159069+0000', 'last_active': '2024-09-11T01:14:53.582370+0000', 'last_peered': '2024-09-11T01:14:53.582370+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:14:53.582370+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 7470, 'log_dups_size': 0, 'ondisk_log_size': 7470, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30568448, 'num_objects': 7463, 'num_object_clones': 0, 'num_object_copies': 14926, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7463, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5441, 'num_write_kb': 21764, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3932}, {'shards': '2', 'objects': 3530}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:14:56.175 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:14:56.175 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'7438", 'reported_seq': 7561, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:54.318729+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:14:54.318729+0000', 'last_peered': '2024-09-11T01:14:54.318729+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:14:54.318729+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 7438, 'log_dups_size': 0, 'ondisk_log_size': 7438, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30433280, 'num_objects': 7430, 'num_object_clones': 0, 'num_object_copies': 14860, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7430, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4025, 'num_write_kb': 16100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3446}, {'shards': '0,2', 'objects': 3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:56.176 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:14:56.176 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'7486", 'reported_seq': 7600, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:53.575596+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:14:53.575596+0000', 'last_peered': '2024-09-11T01:14:53.575596+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:14:53.575596+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 7486, 'log_dups_size': 0, 'ondisk_log_size': 7486, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30642176, 'num_objects': 7481, 'num_object_clones': 0, 'num_object_copies': 14962, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7481, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5303, 'num_write_kb': 21212, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4023}, {'shards': '2', 'objects': 3457}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:14:56.176 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:14:56.176 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'7421", 'reported_seq': 12936, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:14:53.618926+0000', 'last_change': '2024-09-11T01:13:10.217540+0000', 'last_active': '2024-09-11T01:14:53.618926+0000', 'last_peered': '2024-09-11T01:14:53.618926+0000', 'last_clean': '2024-09-11T01:13:07.638709+0000', 'last_became_active': '2024-09-11T01:13:10.125604+0000', 'last_became_peered': '2024-09-11T01:13:10.125604+0000', 'last_unstale': '2024-09-11T01:14:53.618926+0000', 'last_undegraded': '2024-09-11T01:14:53.618926+0000', 'last_fullsized': '2024-09-11T01:13:09.649604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2269", 'last_scrub_stamp': '2024-09-11T01:12:42.890920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:42.890920+0000', 'objects_scrubbed': 2209, 'log_size': 7421, 'log_dups_size': 0, 'ondisk_log_size': 7421, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 999, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30371840, 'num_objects': 7415, 'num_object_clones': 0, 'num_object_copies': 14830, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1227, 'num_objects_unfound': 0, 'num_objects_dirty': 7415, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5206, 'num_write_kb': 20824, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2700, 'num_bytes_recovered': 11051008, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1227}, {'shards': '2', 'objects': 6188}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:14:56.176 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:14:56.176 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'7582", 'reported_seq': 7709, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:54.338013+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:14:54.338013+0000', 'last_peered': '2024-09-11T01:14:54.338013+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:14:54.338013+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 7582, 'log_dups_size': 0, 'ondisk_log_size': 7582, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31047680, 'num_objects': 7580, 'num_object_clones': 0, 'num_object_copies': 15160, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7580, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4319, 'num_write_kb': 17276, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3630}, {'shards': '0,2', 'objects': 3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:56.176 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:14:56.176 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'7425", 'reported_seq': 7552, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:54.311379+0000', 'last_change': '2024-09-11T01:13:10.209668+0000', 'last_active': '2024-09-11T01:14:54.311379+0000', 'last_peered': '2024-09-11T01:14:54.311379+0000', 'last_clean': '2024-09-11T01:13:07.624565+0000', 'last_became_active': '2024-09-11T01:13:10.199795+0000', 'last_became_peered': '2024-09-11T01:13:10.199795+0000', 'last_unstale': '2024-09-11T01:14:54.311379+0000', 'last_undegraded': '2024-09-11T01:13:09.791198+0000', 'last_fullsized': '2024-09-11T01:13:09.652311+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3415", 'last_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'objects_scrubbed': 3337, 'log_size': 7425, 'log_dups_size': 0, 'ondisk_log_size': 7425, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1975, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30380032, 'num_objects': 7417, 'num_object_clones': 0, 'num_object_copies': 14834, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3942, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7417, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4080, 'num_write_kb': 16320, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3478}, {'shards': '0,2', 'objects': 3939}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:56.176 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:14:56.176 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'7574", 'reported_seq': 7700, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:54.267009+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:14:54.267009+0000', 'last_peered': '2024-09-11T01:14:54.267009+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:14:54.267009+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 7574, 'log_dups_size': 0, 'ondisk_log_size': 7574, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31006720, 'num_objects': 7570, 'num_object_clones': 0, 'num_object_copies': 15140, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7570, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4183, 'num_write_kb': 16732, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3531}, {'shards': '0,2', 'objects': 4039}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:56.176 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:14:56.177 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'7504", 'reported_seq': 7633, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:54.364129+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:14:54.364129+0000', 'last_peered': '2024-09-11T01:14:54.364129+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:14:54.364129+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 7504, 'log_dups_size': 0, 'ondisk_log_size': 7504, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30687849, 'num_objects': 7493, 'num_object_clones': 0, 'num_object_copies': 14986, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7493, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4574, 'num_write_kb': 18296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3576}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:14:56.177 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:14:56.398 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 152 20 121031 121011 3.10871 1.84375 0.138313 0.0251196 2024-09-11T01:14:57.311 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:14:57.311 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43711/239914 objects degraded (18.219%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":233,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":119957,"data_bytes":491332220,"bytes_used":1620762624,"bytes_avail":288289529856,"bytes_total":289910292480,"degraded_objects":43711,"degraded_total":239914,"degraded_ratio":0.18219445301232942,"misplaced_objects":1227,"misplaced_total":239914,"misplaced_ratio":0.0051143326358611832,"recovering_objects_per_sec":27,"recovering_bytes_per_sec":111588,"recovering_keys_per_sec":0,"num_objects_recovered":327,"num_bytes_recovered":1339392,"num_keys_recovered":0,"write_bytes_sec":1691566,"read_op_per_sec":0,"write_op_per_sec":412},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:14:57.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:14:57.347 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:14:57.402 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 153 20 121502 121482 3.10034 1.83984 0.060847 0.0251912 2024-09-11T01:14:58.205 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:14:58.206 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:14:58.265 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":153,"stamp":"2024-09-11T01:14:56.587725+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":491332220,"num_objects":119957,"num_object_clones":0,"num_object_copies":239914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":1227,"num_objects_unfound":0,"num_objects_dirty":119957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80888,"num_write_kb":323546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2702,"num_bytes_recovered":11055123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120039,"ondisk_log_size":120039,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1582776,"kb_used_data":965896,"kb_used_omap":12,"kb_used_meta":616819,"kb_avail":281532744,"statfs":{"total":289910292480,"available":288289529856,"internally_reserved":0,"allocated":989077504,"data_stored":988265544,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":631622660},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,6,7,9,14,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20303872,"num_objects":4957,"num_object_clones":0,"num_object_copies":9914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-327,"num_objects_unfound":0,"num_objects_dirty":4957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4957,"num_write_kb":19828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":327,"num_bytes_recovered":1339392,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002997"},"pg_stats":[{"pgid":"2.f","version":"43'7600","reported_seq":7673,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:54.371039+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:14:54.371039+0000","last_peered":"2024-09-11T01:14:54.371039+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:14:54.371039+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":7600,"log_dups_size":0,"ondisk_log_size":7600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31129600,"num_objects":7600,"num_object_clones":0,"num_object_copies":15200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7600,"num_write_kb":30400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3528},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7579","reported_seq":7676,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:53.612756+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:14:53.612756+0000","last_peered":"2024-09-11T01:14:53.612756+0000","last_clean":"2024-09-11T01:14:53.612756+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:14:53.612756+0000","last_undegraded":"2024-09-11T01:14:53.612756+0000","last_fullsized":"2024-09-11T01:14:53.612756+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":7579,"log_dups_size":0,"ondisk_log_size":7579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31019008,"num_objects":7573,"num_object_clones":0,"num_object_copies":15146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5818,"num_write_kb":23272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7441","reported_seq":7564,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:53.618115+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:14:53.618115+0000","last_peered":"2024-09-11T01:14:53.618115+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:14:53.618115+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":7441,"log_dups_size":0,"ondisk_log_size":7441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30461952,"num_objects":7437,"num_object_clones":0,"num_object_copies":14874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5742,"num_write_kb":22968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":3541}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7459","reported_seq":7524,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:53.567397+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:14:53.567397+0000","last_peered":"2024-09-11T01:14:53.567397+0000","last_clean":"2024-09-11T01:14:53.567397+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:14:53.567397+0000","last_undegraded":"2024-09-11T01:14:53.567397+0000","last_fullsized":"2024-09-11T01:14:53.567397+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":7459,"log_dups_size":0,"ondisk_log_size":7459,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30552064,"num_objects":7459,"num_object_clones":0,"num_object_copies":14918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7459,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7459,"num_write_kb":29836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7541","reported_seq":7664,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:54.267162+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:14:54.267162+0000","last_peered":"2024-09-11T01:14:54.267162+0000","last_clean":"2024-09-11T01:14:54.267162+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:14:54.267162+0000","last_undegraded":"2024-09-11T01:14:54.267162+0000","last_fullsized":"2024-09-11T01:14:54.267162+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":7541,"log_dups_size":0,"ondisk_log_size":7541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30859264,"num_objects":7534,"num_object_clones":0,"num_object_copies":15068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4546,"num_write_kb":18184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7528","reported_seq":7693,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:54.341780+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:14:54.341780+0000","last_peered":"2024-09-11T01:14:54.341780+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:14:54.341780+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":7528,"log_dups_size":0,"ondisk_log_size":7528,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30814208,"num_objects":7523,"num_object_clones":0,"num_object_copies":15046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3869,"num_write_kb":15476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3572},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7432","reported_seq":7557,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:54.282927+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:14:54.282927+0000","last_peered":"2024-09-11T01:14:54.282927+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:14:54.282927+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":7432,"log_dups_size":0,"ondisk_log_size":7432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30416896,"num_objects":7426,"num_object_clones":0,"num_object_copies":14852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4289,"num_write_kb":17156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3440},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7556","reported_seq":7681,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:54.336451+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:14:54.336451+0000","last_peered":"2024-09-11T01:14:54.336451+0000","last_clean":"2024-09-11T01:14:54.336451+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:14:54.336451+0000","last_undegraded":"2024-09-11T01:14:54.336451+0000","last_fullsized":"2024-09-11T01:14:54.336451+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":7556,"log_dups_size":0,"ondisk_log_size":7556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30941184,"num_objects":7554,"num_object_clones":0,"num_object_copies":15108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7554,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4432,"num_write_kb":17728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7470","reported_seq":7593,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:53.582370+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:14:53.582370+0000","last_peered":"2024-09-11T01:14:53.582370+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:14:53.582370+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":7470,"log_dups_size":0,"ondisk_log_size":7470,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30568448,"num_objects":7463,"num_object_clones":0,"num_object_copies":14926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5441,"num_write_kb":21764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":3530}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741922+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741922+0000","last_peered":"2024-09-11T01:13:09.741922+0000","last_clean":"2024-09-11T01:13:09.741922+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741922+0000","last_undegraded":"2024-09-11T01:13:09.741922+0000","last_fullsized":"2024-09-11T01:13:09.741922+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7438","reported_seq":7561,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:54.318729+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:14:54.318729+0000","last_peered":"2024-09-11T01:14:54.318729+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:14:54.318729+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":7438,"log_dups_size":0,"ondisk_log_size":7438,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30433280,"num_objects":7430,"num_object_clones":0,"num_object_copies":14860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4025,"num_write_kb":16100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3446},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.910012+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.910012+0000","last_peered":"2024-09-11T01:13:09.910012+0000","last_clean":"2024-09-11T01:13:09.910012+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.910012+0000","last_undegraded":"2024-09-11T01:13:09.910012+0000","last_fullsized":"2024-09-11T01:13:09.910012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7486","reported_seq":7600,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:53.575596+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:14:53.575596+0000","last_peered":"2024-09-11T01:14:53.575596+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:14:53.575596+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":7486,"log_dups_size":0,"ondisk_log_size":7486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30642176,"num_objects":7481,"num_object_clones":0,"num_object_copies":14962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7481,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5303,"num_write_kb":21212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":3457}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7421","reported_seq":12936,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:14:53.618926+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:14:53.618926+0000","last_peered":"2024-09-11T01:14:53.618926+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:14:53.618926+0000","last_undegraded":"2024-09-11T01:14:53.618926+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":7421,"log_dups_size":0,"ondisk_log_size":7421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30371840,"num_objects":7415,"num_object_clones":0,"num_object_copies":14830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1227,"num_objects_unfound":0,"num_objects_dirty":7415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5206,"num_write_kb":20824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2700,"num_bytes_recovered":11051008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1227},{"shards":"2","objects":6188}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7582","reported_seq":7709,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:54.338013+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:14:54.338013+0000","last_peered":"2024-09-11T01:14:54.338013+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:14:54.338013+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":7582,"log_dups_size":0,"ondisk_log_size":7582,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31047680,"num_objects":7580,"num_object_clones":0,"num_object_copies":15160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4319,"num_write_kb":17276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3630},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7425","reported_seq":7552,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:54.311379+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:14:54.311379+0000","last_peered":"2024-09-11T01:14:54.311379+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:14:54.311379+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":7425,"log_dups_size":0,"ondisk_log_size":7425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30380032,"num_objects":7417,"num_object_clones":0,"num_object_copies":14834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7417,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4080,"num_write_kb":16320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3478},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658174+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658174+0000","last_peered":"2024-09-11T01:13:09.658174+0000","last_clean":"2024-09-11T01:13:09.658174+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658174+0000","last_undegraded":"2024-09-11T01:13:09.658174+0000","last_fullsized":"2024-09-11T01:13:09.658174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7574","reported_seq":7700,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:54.267009+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:14:54.267009+0000","last_peered":"2024-09-11T01:14:54.267009+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:14:54.267009+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":7574,"log_dups_size":0,"ondisk_log_size":7574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31006720,"num_objects":7570,"num_object_clones":0,"num_object_copies":15140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7570,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4183,"num_write_kb":16732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3531},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7504","reported_seq":7633,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:54.364129+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:14:54.364129+0000","last_peered":"2024-09-11T01:14:54.364129+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:14:54.364129+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":7504,"log_dups_size":0,"ondisk_log_size":7504,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30687849,"num_objects":7493,"num_object_clones":0,"num_object_copies":14986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4574,"num_write_kb":18296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3576},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":491332201,"num_objects":119955,"num_object_clones":0,"num_object_copies":239910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":1227,"num_objects_unfound":0,"num_objects_dirty":119955,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":80886,"num_write_kb":323544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2701,"num_bytes_recovered":11055104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":991375360,"data_stored":991295343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":120036,"ondisk_log_size":120036,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182441,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":693748,"kb_used_data":423332,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93678092,"statfs":{"total":96636764160,"available":95926366208,"internally_reserved":0,"allocated":433491968,"data_stored":433243149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":276885163},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,5,4,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986213,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":546600,"kb_used_data":366424,"kb_used_omap":4,"kb_used_meta":180155,"kb_avail":93825240,"statfs":{"total":96636764160,"available":96077045760,"internally_reserved":0,"allocated":375218176,"data_stored":374973434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":184479402},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659158,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":342428,"kb_used_data":176140,"kb_used_omap":4,"kb_used_meta":166267,"kb_avail":94029412,"statfs":{"total":96636764160,"available":96286117888,"internally_reserved":0,"allocated":180367360,"data_stored":180048961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":170258095},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,4,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":434159616,"data_stored":434156137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":181530624,"data_stored":181457565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":375685120,"data_stored":375681641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:14:58.402 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 154 20 121972 121952 3.09213 1.83594 0.0110147 0.0252568 2024-09-11T01:14:59.402 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 155 20 122487 122467 3.08516 2.01172 0.0316123 0.0253154 2024-09-11T01:15:00.402 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 156 20 123001 122981 3.07826 2.00781 0.0143081 0.0253704 2024-09-11T01:15:01.267 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:01.404 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 157 20 123508 123488 3.07123 1.98047 0.0431374 0.0254274 2024-09-11T01:15:02.186 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:15:02.188 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:15:02.237 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":155,"stamp":"2024-09-11T01:15:00.588668+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":501084796,"num_objects":122338,"num_object_clones":0,"num_object_copies":244676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":1030,"num_objects_unfound":0,"num_objects_dirty":122338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83269,"num_write_kb":333070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2899,"num_bytes_recovered":11862035,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":122420,"ondisk_log_size":122420,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1608320,"kb_used_data":987276,"kb_used_omap":12,"kb_used_meta":620979,"kb_avail":281507200,"statfs":{"total":289910292480,"available":288263372800,"internally_reserved":0,"allocated":1010970624,"data_stored":1010158664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":635882500},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,10,12,8,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30056448,"num_objects":7338,"num_object_clones":0,"num_object_copies":14676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-524,"num_objects_unfound":0,"num_objects_dirty":7338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7338,"num_write_kb":29352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":524,"num_bytes_recovered":2146304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002864"},"pg_stats":[{"pgid":"2.f","version":"43'7753","reported_seq":7826,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:59.319563+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:14:59.319563+0000","last_peered":"2024-09-11T01:14:59.319563+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:14:59.319563+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":7753,"log_dups_size":0,"ondisk_log_size":7753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31756288,"num_objects":7753,"num_object_clones":0,"num_object_copies":15506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7753,"num_write_kb":31012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3681},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7730","reported_seq":7827,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:58.637693+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:14:58.637693+0000","last_peered":"2024-09-11T01:14:58.637693+0000","last_clean":"2024-09-11T01:14:58.637693+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:14:58.637693+0000","last_undegraded":"2024-09-11T01:14:58.637693+0000","last_fullsized":"2024-09-11T01:14:58.637693+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":7730,"log_dups_size":0,"ondisk_log_size":7730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31637504,"num_objects":7724,"num_object_clones":0,"num_object_copies":15448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7724,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5969,"num_write_kb":23876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7566","reported_seq":7689,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:58.642158+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:14:58.642158+0000","last_peered":"2024-09-11T01:14:58.642158+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:14:58.642158+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":7566,"log_dups_size":0,"ondisk_log_size":7566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30973952,"num_objects":7562,"num_object_clones":0,"num_object_copies":15124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5867,"num_write_kb":23468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":3666}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7607","reported_seq":7672,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:58.574683+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:14:58.574683+0000","last_peered":"2024-09-11T01:14:58.574683+0000","last_clean":"2024-09-11T01:14:58.574683+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:14:58.574683+0000","last_undegraded":"2024-09-11T01:14:58.574683+0000","last_fullsized":"2024-09-11T01:14:58.574683+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":7607,"log_dups_size":0,"ondisk_log_size":7607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31158272,"num_objects":7607,"num_object_clones":0,"num_object_copies":15214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7607,"num_write_kb":30428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7676","reported_seq":7799,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:59.382119+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:14:59.382119+0000","last_peered":"2024-09-11T01:14:59.382119+0000","last_clean":"2024-09-11T01:14:59.382119+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:14:59.382119+0000","last_undegraded":"2024-09-11T01:14:59.382119+0000","last_fullsized":"2024-09-11T01:14:59.382119+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":7676,"log_dups_size":0,"ondisk_log_size":7676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31412224,"num_objects":7669,"num_object_clones":0,"num_object_copies":15338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4681,"num_write_kb":18724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7691","reported_seq":7856,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:59.380080+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:14:59.380080+0000","last_peered":"2024-09-11T01:14:59.380080+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:14:59.380080+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":7691,"log_dups_size":0,"ondisk_log_size":7691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31481856,"num_objects":7686,"num_object_clones":0,"num_object_copies":15372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4032,"num_write_kb":16128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3735},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7587","reported_seq":7712,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:59.347132+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:14:59.347132+0000","last_peered":"2024-09-11T01:14:59.347132+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:14:59.347132+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":7587,"log_dups_size":0,"ondisk_log_size":7587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31051776,"num_objects":7581,"num_object_clones":0,"num_object_copies":15162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4444,"num_write_kb":17776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3595},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7712","reported_seq":7836,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:59.364889+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:14:59.364889+0000","last_peered":"2024-09-11T01:14:59.364889+0000","last_clean":"2024-09-11T01:14:59.364889+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:14:59.364889+0000","last_undegraded":"2024-09-11T01:14:59.364889+0000","last_fullsized":"2024-09-11T01:14:59.364889+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":7712,"log_dups_size":0,"ondisk_log_size":7712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31580160,"num_objects":7710,"num_object_clones":0,"num_object_copies":15420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4588,"num_write_kb":18352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7587","reported_seq":7710,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:58.599426+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:14:58.599426+0000","last_peered":"2024-09-11T01:14:58.599426+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:14:58.599426+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":7587,"log_dups_size":0,"ondisk_log_size":7587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31047680,"num_objects":7580,"num_object_clones":0,"num_object_copies":15160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5558,"num_write_kb":22232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":3647}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741922+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741922+0000","last_peered":"2024-09-11T01:13:09.741922+0000","last_clean":"2024-09-11T01:13:09.741922+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741922+0000","last_undegraded":"2024-09-11T01:13:09.741922+0000","last_fullsized":"2024-09-11T01:13:09.741922+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7608","reported_seq":7731,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:59.330556+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:14:59.330556+0000","last_peered":"2024-09-11T01:14:59.330556+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:14:59.330556+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":7608,"log_dups_size":0,"ondisk_log_size":7608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31129600,"num_objects":7600,"num_object_clones":0,"num_object_copies":15200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4195,"num_write_kb":16780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3616},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.910012+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.910012+0000","last_peered":"2024-09-11T01:13:09.910012+0000","last_clean":"2024-09-11T01:13:09.910012+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.910012+0000","last_undegraded":"2024-09-11T01:13:09.910012+0000","last_fullsized":"2024-09-11T01:13:09.910012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7639","reported_seq":7752,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:58.618176+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:14:58.618176+0000","last_peered":"2024-09-11T01:14:58.618176+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:14:58.618176+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":7639,"log_dups_size":0,"ondisk_log_size":7639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31268864,"num_objects":7634,"num_object_clones":0,"num_object_copies":15268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5456,"num_write_kb":21824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":3610}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7566","reported_seq":13474,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:14:58.655078+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:14:58.655078+0000","last_peered":"2024-09-11T01:14:58.655078+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:14:58.655078+0000","last_undegraded":"2024-09-11T01:14:58.655078+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":7566,"log_dups_size":0,"ondisk_log_size":7566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30965760,"num_objects":7560,"num_object_clones":0,"num_object_copies":15120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1030,"num_objects_unfound":0,"num_objects_dirty":7560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5351,"num_write_kb":21404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2897,"num_bytes_recovered":11857920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1030},{"shards":"2","objects":6530}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7728","reported_seq":7854,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:59.406056+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:14:59.406056+0000","last_peered":"2024-09-11T01:14:59.406056+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:14:59.406056+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":7728,"log_dups_size":0,"ondisk_log_size":7728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31645696,"num_objects":7726,"num_object_clones":0,"num_object_copies":15452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4465,"num_write_kb":17860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3776},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7580","reported_seq":7712,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:59.340381+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:14:59.340381+0000","last_peered":"2024-09-11T01:14:59.340381+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:14:59.340381+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":7580,"log_dups_size":0,"ondisk_log_size":7580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31014912,"num_objects":7572,"num_object_clones":0,"num_object_copies":15144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4235,"num_write_kb":16940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3633},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658174+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658174+0000","last_peered":"2024-09-11T01:13:09.658174+0000","last_clean":"2024-09-11T01:13:09.658174+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658174+0000","last_undegraded":"2024-09-11T01:13:09.658174+0000","last_fullsized":"2024-09-11T01:13:09.658174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7719","reported_seq":7849,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:59.329686+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:14:59.329686+0000","last_peered":"2024-09-11T01:14:59.329686+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:14:59.329686+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":7719,"log_dups_size":0,"ondisk_log_size":7719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31600640,"num_objects":7715,"num_object_clones":0,"num_object_copies":15430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4328,"num_write_kb":17312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3676},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7668","reported_seq":7797,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:59.362230+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:14:59.362230+0000","last_peered":"2024-09-11T01:14:59.362230+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:14:59.362230+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":7668,"log_dups_size":0,"ondisk_log_size":7668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31359593,"num_objects":7657,"num_object_clones":0,"num_object_copies":15314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4738,"num_write_kb":18952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3740},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":501084777,"num_objects":122336,"num_object_clones":0,"num_object_copies":244672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":1030,"num_objects_unfound":0,"num_objects_dirty":122336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83267,"num_write_kb":333068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2898,"num_bytes_recovered":11862016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1011802112,"data_stored":1011722095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":122417,"ondisk_log_size":122417,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182442,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":709440,"kb_used_data":430704,"kb_used_omap":4,"kb_used_meta":278715,"kb_avail":93662400,"statfs":{"total":96636764160,"available":95910297600,"internally_reserved":0,"allocated":441040896,"data_stored":440792077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":285404843},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,1,9,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986214,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":555228,"kb_used_data":370892,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93816612,"statfs":{"total":96636764160,"available":96068210688,"internally_reserved":0,"allocated":379793408,"data_stored":379548666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188739242},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,1,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659159,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":343652,"kb_used_data":185680,"kb_used_omap":4,"kb_used_meta":157947,"kb_avail":94028188,"statfs":{"total":96636764160,"available":96284864512,"internally_reserved":0,"allocated":190136320,"data_stored":189817921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":161738415},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,2,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":441692160,"data_stored":441688681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":189861888,"data_stored":189788829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":380248064,"data_stored":380244585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:02.239 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:02.404 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 158 20 123968 123948 3.06317 1.79688 0.0688647 0.0254962 2024-09-11T01:15:03.064 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:15:03.064 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:15:03.139 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2024-09-11T01:15:02.591642+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":501084796,"num_objects":122338,"num_object_clones":0,"num_object_copies":244676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":1030,"num_objects_unfound":0,"num_objects_dirty":122338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83269,"num_write_kb":333070,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2899,"num_bytes_recovered":11862035,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":122420,"ondisk_log_size":122420,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1608320,"kb_used_data":987276,"kb_used_omap":12,"kb_used_meta":620979,"kb_avail":281507200,"statfs":{"total":289910292480,"available":288263372800,"internally_reserved":0,"allocated":1010970624,"data_stored":1010158664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":635882500},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,10,12,8,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19832832,"num_objects":4842,"num_object_clones":0,"num_object_copies":9684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-363,"num_objects_unfound":0,"num_objects_dirty":4842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4842,"num_write_kb":19368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":363,"num_bytes_recovered":1486848,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005352"},"pg_stats":[{"pgid":"2.f","version":"43'7753","reported_seq":7826,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:59.319563+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:14:59.319563+0000","last_peered":"2024-09-11T01:14:59.319563+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:14:59.319563+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":7753,"log_dups_size":0,"ondisk_log_size":7753,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31756288,"num_objects":7753,"num_object_clones":0,"num_object_copies":15506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7753,"num_write_kb":31012,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3681},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7730","reported_seq":7827,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:58.637693+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:14:58.637693+0000","last_peered":"2024-09-11T01:14:58.637693+0000","last_clean":"2024-09-11T01:14:58.637693+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:14:58.637693+0000","last_undegraded":"2024-09-11T01:14:58.637693+0000","last_fullsized":"2024-09-11T01:14:58.637693+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":7730,"log_dups_size":0,"ondisk_log_size":7730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31637504,"num_objects":7724,"num_object_clones":0,"num_object_copies":15448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7724,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5969,"num_write_kb":23876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7566","reported_seq":7689,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:58.642158+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:14:58.642158+0000","last_peered":"2024-09-11T01:14:58.642158+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:14:58.642158+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":7566,"log_dups_size":0,"ondisk_log_size":7566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30973952,"num_objects":7562,"num_object_clones":0,"num_object_copies":15124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5867,"num_write_kb":23468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":3666}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7607","reported_seq":7672,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:58.574683+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:14:58.574683+0000","last_peered":"2024-09-11T01:14:58.574683+0000","last_clean":"2024-09-11T01:14:58.574683+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:14:58.574683+0000","last_undegraded":"2024-09-11T01:14:58.574683+0000","last_fullsized":"2024-09-11T01:14:58.574683+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":7607,"log_dups_size":0,"ondisk_log_size":7607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31158272,"num_objects":7607,"num_object_clones":0,"num_object_copies":15214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7607,"num_write_kb":30428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7676","reported_seq":7799,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:59.382119+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:14:59.382119+0000","last_peered":"2024-09-11T01:14:59.382119+0000","last_clean":"2024-09-11T01:14:59.382119+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:14:59.382119+0000","last_undegraded":"2024-09-11T01:14:59.382119+0000","last_fullsized":"2024-09-11T01:14:59.382119+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":7676,"log_dups_size":0,"ondisk_log_size":7676,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31412224,"num_objects":7669,"num_object_clones":0,"num_object_copies":15338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4681,"num_write_kb":18724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7691","reported_seq":7856,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:59.380080+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:14:59.380080+0000","last_peered":"2024-09-11T01:14:59.380080+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:14:59.380080+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":7691,"log_dups_size":0,"ondisk_log_size":7691,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31481856,"num_objects":7686,"num_object_clones":0,"num_object_copies":15372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4032,"num_write_kb":16128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3735},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7587","reported_seq":7712,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:59.347132+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:14:59.347132+0000","last_peered":"2024-09-11T01:14:59.347132+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:14:59.347132+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":7587,"log_dups_size":0,"ondisk_log_size":7587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31051776,"num_objects":7581,"num_object_clones":0,"num_object_copies":15162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7581,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4444,"num_write_kb":17776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3595},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7712","reported_seq":7836,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:14:59.364889+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:14:59.364889+0000","last_peered":"2024-09-11T01:14:59.364889+0000","last_clean":"2024-09-11T01:14:59.364889+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:14:59.364889+0000","last_undegraded":"2024-09-11T01:14:59.364889+0000","last_fullsized":"2024-09-11T01:14:59.364889+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":7712,"log_dups_size":0,"ondisk_log_size":7712,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31580160,"num_objects":7710,"num_object_clones":0,"num_object_copies":15420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4588,"num_write_kb":18352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7587","reported_seq":7710,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:58.599426+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:14:58.599426+0000","last_peered":"2024-09-11T01:14:58.599426+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:14:58.599426+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":7587,"log_dups_size":0,"ondisk_log_size":7587,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31047680,"num_objects":7580,"num_object_clones":0,"num_object_copies":15160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5558,"num_write_kb":22232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":3647}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741922+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741922+0000","last_peered":"2024-09-11T01:13:09.741922+0000","last_clean":"2024-09-11T01:13:09.741922+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741922+0000","last_undegraded":"2024-09-11T01:13:09.741922+0000","last_fullsized":"2024-09-11T01:13:09.741922+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7608","reported_seq":7731,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:59.330556+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:14:59.330556+0000","last_peered":"2024-09-11T01:14:59.330556+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:14:59.330556+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":7608,"log_dups_size":0,"ondisk_log_size":7608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31129600,"num_objects":7600,"num_object_clones":0,"num_object_copies":15200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7600,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4195,"num_write_kb":16780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3616},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.910012+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.910012+0000","last_peered":"2024-09-11T01:13:09.910012+0000","last_clean":"2024-09-11T01:13:09.910012+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.910012+0000","last_undegraded":"2024-09-11T01:13:09.910012+0000","last_fullsized":"2024-09-11T01:13:09.910012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7639","reported_seq":7752,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:58.618176+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:14:58.618176+0000","last_peered":"2024-09-11T01:14:58.618176+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:14:58.618176+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":7639,"log_dups_size":0,"ondisk_log_size":7639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31268864,"num_objects":7634,"num_object_clones":0,"num_object_copies":15268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5456,"num_write_kb":21824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":3610}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7566","reported_seq":13474,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:14:58.655078+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:14:58.655078+0000","last_peered":"2024-09-11T01:14:58.655078+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:14:58.655078+0000","last_undegraded":"2024-09-11T01:14:58.655078+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":7566,"log_dups_size":0,"ondisk_log_size":7566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30965760,"num_objects":7560,"num_object_clones":0,"num_object_copies":15120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1030,"num_objects_unfound":0,"num_objects_dirty":7560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5351,"num_write_kb":21404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2897,"num_bytes_recovered":11857920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1030},{"shards":"2","objects":6530}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7728","reported_seq":7854,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:59.406056+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:14:59.406056+0000","last_peered":"2024-09-11T01:14:59.406056+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:14:59.406056+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":7728,"log_dups_size":0,"ondisk_log_size":7728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31645696,"num_objects":7726,"num_object_clones":0,"num_object_copies":15452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4465,"num_write_kb":17860,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3776},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7580","reported_seq":7712,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:59.340381+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:14:59.340381+0000","last_peered":"2024-09-11T01:14:59.340381+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:14:59.340381+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":7580,"log_dups_size":0,"ondisk_log_size":7580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31014912,"num_objects":7572,"num_object_clones":0,"num_object_copies":15144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4235,"num_write_kb":16940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3633},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658174+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658174+0000","last_peered":"2024-09-11T01:13:09.658174+0000","last_clean":"2024-09-11T01:13:09.658174+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658174+0000","last_undegraded":"2024-09-11T01:13:09.658174+0000","last_fullsized":"2024-09-11T01:13:09.658174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7719","reported_seq":7849,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:59.329686+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:14:59.329686+0000","last_peered":"2024-09-11T01:14:59.329686+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:14:59.329686+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":7719,"log_dups_size":0,"ondisk_log_size":7719,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31600640,"num_objects":7715,"num_object_clones":0,"num_object_copies":15430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4328,"num_write_kb":17312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3676},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7668","reported_seq":7797,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:14:59.362230+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:14:59.362230+0000","last_peered":"2024-09-11T01:14:59.362230+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:14:59.362230+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":7668,"log_dups_size":0,"ondisk_log_size":7668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31359593,"num_objects":7657,"num_object_clones":0,"num_object_copies":15314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4738,"num_write_kb":18952,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3740},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":501084777,"num_objects":122336,"num_object_clones":0,"num_object_copies":244672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":1030,"num_objects_unfound":0,"num_objects_dirty":122336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":83267,"num_write_kb":333068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2898,"num_bytes_recovered":11862016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1011802112,"data_stored":1011722095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":122417,"ondisk_log_size":122417,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182442,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":709440,"kb_used_data":430704,"kb_used_omap":4,"kb_used_meta":278715,"kb_avail":93662400,"statfs":{"total":96636764160,"available":95910297600,"internally_reserved":0,"allocated":441040896,"data_stored":440792077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":285404843},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,1,9,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986214,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":555228,"kb_used_data":370892,"kb_used_omap":4,"kb_used_meta":184315,"kb_avail":93816612,"statfs":{"total":96636764160,"available":96068210688,"internally_reserved":0,"allocated":379793408,"data_stored":379548666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":188739242},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,1,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659159,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":343652,"kb_used_data":185680,"kb_used_omap":4,"kb_used_meta":157947,"kb_avail":94028188,"statfs":{"total":96636764160,"available":96284864512,"internally_reserved":0,"allocated":190136320,"data_stored":189817921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":161738415},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,4,2,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":441692160,"data_stored":441688681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":189861888,"data_stored":189788829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":380248064,"data_stored":380244585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:03.140 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:15:03.140 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'7753", 'reported_seq': 7826, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:59.319563+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:14:59.319563+0000', 'last_peered': '2024-09-11T01:14:59.319563+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:14:59.319563+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 7753, 'log_dups_size': 0, 'ondisk_log_size': 7753, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31756288, 'num_objects': 7753, 'num_object_clones': 0, 'num_object_copies': 15506, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7753, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7753, 'num_write_kb': 31012, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3681}, {'shards': '0,2', 'objects': 4072}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:03.141 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:15:03.141 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'7566", 'reported_seq': 7689, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:58.642158+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:14:58.642158+0000', 'last_peered': '2024-09-11T01:14:58.642158+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:14:58.642158+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 7566, 'log_dups_size': 0, 'ondisk_log_size': 7566, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30973952, 'num_objects': 7562, 'num_object_clones': 0, 'num_object_copies': 15124, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7562, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5867, 'num_write_kb': 23468, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3896}, {'shards': '2', 'objects': 3666}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:03.141 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:15:03.141 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'7691", 'reported_seq': 7856, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:59.380080+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:14:59.380080+0000', 'last_peered': '2024-09-11T01:14:59.380080+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:14:59.380080+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 7691, 'log_dups_size': 0, 'ondisk_log_size': 7691, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31481856, 'num_objects': 7686, 'num_object_clones': 0, 'num_object_copies': 15372, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7686, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4032, 'num_write_kb': 16128, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3735}, {'shards': '0,2', 'objects': 3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:03.141 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:15:03.141 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'7587", 'reported_seq': 7712, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:59.347132+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:14:59.347132+0000', 'last_peered': '2024-09-11T01:14:59.347132+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:14:59.347132+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 7587, 'log_dups_size': 0, 'ondisk_log_size': 7587, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31051776, 'num_objects': 7581, 'num_object_clones': 0, 'num_object_copies': 15162, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7581, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4444, 'num_write_kb': 17776, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3595}, {'shards': '0,2', 'objects': 3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:03.141 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:15:03.142 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'7587", 'reported_seq': 7710, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:58.599426+0000', 'last_change': '2024-09-11T01:13:10.159069+0000', 'last_active': '2024-09-11T01:14:58.599426+0000', 'last_peered': '2024-09-11T01:14:58.599426+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:14:58.599426+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 7587, 'log_dups_size': 0, 'ondisk_log_size': 7587, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31047680, 'num_objects': 7580, 'num_object_clones': 0, 'num_object_copies': 15160, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7580, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5558, 'num_write_kb': 22232, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3932}, {'shards': '2', 'objects': 3647}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:03.142 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:15:03.142 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'7608", 'reported_seq': 7731, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:59.330556+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:14:59.330556+0000', 'last_peered': '2024-09-11T01:14:59.330556+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:14:59.330556+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 7608, 'log_dups_size': 0, 'ondisk_log_size': 7608, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31129600, 'num_objects': 7600, 'num_object_clones': 0, 'num_object_copies': 15200, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7600, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4195, 'num_write_kb': 16780, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3616}, {'shards': '0,2', 'objects': 3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:03.142 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:15:03.142 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'7639", 'reported_seq': 7752, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:58.618176+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:14:58.618176+0000', 'last_peered': '2024-09-11T01:14:58.618176+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:14:58.618176+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 7639, 'log_dups_size': 0, 'ondisk_log_size': 7639, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31268864, 'num_objects': 7634, 'num_object_clones': 0, 'num_object_copies': 15268, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7634, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5456, 'num_write_kb': 21824, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4023}, {'shards': '2', 'objects': 3610}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:03.142 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:15:03.142 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'7566", 'reported_seq': 13474, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:14:58.655078+0000', 'last_change': '2024-09-11T01:13:10.217540+0000', 'last_active': '2024-09-11T01:14:58.655078+0000', 'last_peered': '2024-09-11T01:14:58.655078+0000', 'last_clean': '2024-09-11T01:13:07.638709+0000', 'last_became_active': '2024-09-11T01:13:10.125604+0000', 'last_became_peered': '2024-09-11T01:13:10.125604+0000', 'last_unstale': '2024-09-11T01:14:58.655078+0000', 'last_undegraded': '2024-09-11T01:14:58.655078+0000', 'last_fullsized': '2024-09-11T01:13:09.649604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2269", 'last_scrub_stamp': '2024-09-11T01:12:42.890920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:42.890920+0000', 'objects_scrubbed': 2209, 'log_size': 7566, 'log_dups_size': 0, 'ondisk_log_size': 7566, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 999, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30965760, 'num_objects': 7560, 'num_object_clones': 0, 'num_object_copies': 15120, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1030, 'num_objects_unfound': 0, 'num_objects_dirty': 7560, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5351, 'num_write_kb': 21404, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2897, 'num_bytes_recovered': 11857920, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1030}, {'shards': '2', 'objects': 6530}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:03.142 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:15:03.143 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'7728", 'reported_seq': 7854, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:59.406056+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:14:59.406056+0000', 'last_peered': '2024-09-11T01:14:59.406056+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:14:59.406056+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 7728, 'log_dups_size': 0, 'ondisk_log_size': 7728, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31645696, 'num_objects': 7726, 'num_object_clones': 0, 'num_object_copies': 15452, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7726, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4465, 'num_write_kb': 17860, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3776}, {'shards': '0,2', 'objects': 3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:03.143 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:15:03.143 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'7580", 'reported_seq': 7712, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:59.340381+0000', 'last_change': '2024-09-11T01:13:10.209668+0000', 'last_active': '2024-09-11T01:14:59.340381+0000', 'last_peered': '2024-09-11T01:14:59.340381+0000', 'last_clean': '2024-09-11T01:13:07.624565+0000', 'last_became_active': '2024-09-11T01:13:10.199795+0000', 'last_became_peered': '2024-09-11T01:13:10.199795+0000', 'last_unstale': '2024-09-11T01:14:59.340381+0000', 'last_undegraded': '2024-09-11T01:13:09.791198+0000', 'last_fullsized': '2024-09-11T01:13:09.652311+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3415", 'last_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'objects_scrubbed': 3337, 'log_size': 7580, 'log_dups_size': 0, 'ondisk_log_size': 7580, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1975, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31014912, 'num_objects': 7572, 'num_object_clones': 0, 'num_object_copies': 15144, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3942, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7572, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4235, 'num_write_kb': 16940, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3633}, {'shards': '0,2', 'objects': 3939}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:03.143 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:15:03.143 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'7719", 'reported_seq': 7849, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:59.329686+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:14:59.329686+0000', 'last_peered': '2024-09-11T01:14:59.329686+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:14:59.329686+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 7719, 'log_dups_size': 0, 'ondisk_log_size': 7719, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31600640, 'num_objects': 7715, 'num_object_clones': 0, 'num_object_copies': 15430, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7715, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4328, 'num_write_kb': 17312, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3676}, {'shards': '0,2', 'objects': 4039}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:03.143 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:15:03.143 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'7668", 'reported_seq': 7797, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:14:59.362230+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:14:59.362230+0000', 'last_peered': '2024-09-11T01:14:59.362230+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:14:59.362230+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 7668, 'log_dups_size': 0, 'ondisk_log_size': 7668, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31359593, 'num_objects': 7657, 'num_object_clones': 0, 'num_object_copies': 15314, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7657, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4738, 'num_write_kb': 18952, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3740}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:03.143 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:15:03.405 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 159 20 124426 124406 3.05513 1.78906 0.0531944 0.025564 2024-09-11T01:15:04.192 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:15:04.192 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43711/244676 objects degraded (17.865%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":240,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":122338,"data_bytes":501084796,"bytes_used":1646919680,"bytes_avail":288263372800,"bytes_total":289910292480,"degraded_objects":43711,"degraded_total":244676,"degraded_ratio":0.17864849842240355,"misplaced_objects":1030,"misplaced_total":244676,"misplaced_ratio":0.0042096486782520559,"recovering_objects_per_sec":30,"recovering_bytes_per_sec":123848,"recovering_keys_per_sec":0,"num_objects_recovered":363,"num_bytes_recovered":1486848,"num_keys_recovered":0,"write_bytes_sec":1651999,"read_op_per_sec":0,"write_op_per_sec":403},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:15:04.242 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:15:04.242 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:04.408 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout:2024-09-11T01:15:04.405897+0000 min lat: 0.00219431 max lat: 2.69211 avg lat: 0.0256257 2024-09-11T01:15:04.409 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T01:15:04.409 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 160 20 124892 124872 3.04741 1.82031 0.155023 0.0256257 2024-09-11T01:15:05.167 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:15:05.378 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:15:05.379 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2024-09-11T01:15:04.592073+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":511115900,"num_objects":124787,"num_object_clones":0,"num_object_copies":249574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":824,"num_objects_unfound":0,"num_objects_dirty":124787,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85718,"num_write_kb":342866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3105,"num_bytes_recovered":12705811,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124869,"ondisk_log_size":124869,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1628756,"kb_used_data":1006308,"kb_used_omap":12,"kb_used_meta":622387,"kb_avail":281486764,"statfs":{"total":289910292480,"available":288242446336,"internally_reserved":0,"allocated":1030459392,"data_stored":1029647432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":637324292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,4,10,7,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29863936,"num_objects":7291,"num_object_clones":0,"num_object_copies":14582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-569,"num_objects_unfound":0,"num_objects_dirty":7291,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7291,"num_write_kb":29164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":569,"num_bytes_recovered":2330624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005255"},"pg_stats":[{"pgid":"2.f","version":"43'7903","reported_seq":7976,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:04.398884+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:15:04.398884+0000","last_peered":"2024-09-11T01:15:04.398884+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:15:04.398884+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":7903,"log_dups_size":0,"ondisk_log_size":7903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32370688,"num_objects":7903,"num_object_clones":0,"num_object_copies":15806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7903,"num_write_kb":31612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3831},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7869","reported_seq":7966,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:15:03.609126+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:15:03.609126+0000","last_peered":"2024-09-11T01:15:03.609126+0000","last_clean":"2024-09-11T01:15:03.609126+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:15:03.609126+0000","last_undegraded":"2024-09-11T01:15:03.609126+0000","last_fullsized":"2024-09-11T01:15:03.609126+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":7869,"log_dups_size":0,"ondisk_log_size":7869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32206848,"num_objects":7863,"num_object_clones":0,"num_object_copies":15726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6108,"num_write_kb":24432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7726","reported_seq":7849,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:03.696389+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:15:03.696389+0000","last_peered":"2024-09-11T01:15:03.696389+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:15:03.696389+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":7726,"log_dups_size":0,"ondisk_log_size":7726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31629312,"num_objects":7722,"num_object_clones":0,"num_object_copies":15444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6027,"num_write_kb":24108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":3826}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7770","reported_seq":7835,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:15:03.670096+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:15:03.670096+0000","last_peered":"2024-09-11T01:15:03.670096+0000","last_clean":"2024-09-11T01:15:03.670096+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:15:03.670096+0000","last_undegraded":"2024-09-11T01:15:03.670096+0000","last_fullsized":"2024-09-11T01:15:03.670096+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":7770,"log_dups_size":0,"ondisk_log_size":7770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31825920,"num_objects":7770,"num_object_clones":0,"num_object_copies":15540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7770,"num_write_kb":31080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7809","reported_seq":7932,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:15:04.414109+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:15:04.414109+0000","last_peered":"2024-09-11T01:15:04.414109+0000","last_clean":"2024-09-11T01:15:04.414109+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:15:04.414109+0000","last_undegraded":"2024-09-11T01:15:04.414109+0000","last_fullsized":"2024-09-11T01:15:04.414109+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":7809,"log_dups_size":0,"ondisk_log_size":7809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31956992,"num_objects":7802,"num_object_clones":0,"num_object_copies":15604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4814,"num_write_kb":19256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7859","reported_seq":8024,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:04.460099+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:15:04.460099+0000","last_peered":"2024-09-11T01:15:04.460099+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:15:04.460099+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":7859,"log_dups_size":0,"ondisk_log_size":7859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32169984,"num_objects":7854,"num_object_clones":0,"num_object_copies":15708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4200,"num_write_kb":16800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3903},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7742","reported_seq":7867,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:04.426291+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:15:04.426291+0000","last_peered":"2024-09-11T01:15:04.426291+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:15:04.426291+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":7742,"log_dups_size":0,"ondisk_log_size":7742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31686656,"num_objects":7736,"num_object_clones":0,"num_object_copies":15472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4599,"num_write_kb":18396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3750},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7890","reported_seq":8015,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:15:04.330427+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:15:04.330427+0000","last_peered":"2024-09-11T01:15:04.330427+0000","last_clean":"2024-09-11T01:15:04.330427+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:15:04.330427+0000","last_undegraded":"2024-09-11T01:15:04.330427+0000","last_fullsized":"2024-09-11T01:15:04.330427+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":7890,"log_dups_size":0,"ondisk_log_size":7890,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32309248,"num_objects":7888,"num_object_clones":0,"num_object_copies":15776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4766,"num_write_kb":19064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.654802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.654802+0000","last_peered":"2024-09-11T01:13:09.654802+0000","last_clean":"2024-09-11T01:13:09.654802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.654802+0000","last_undegraded":"2024-09-11T01:13:09.654802+0000","last_fullsized":"2024-09-11T01:13:09.654802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7738","reported_seq":7861,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:03.681726+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:15:03.681726+0000","last_peered":"2024-09-11T01:15:03.681726+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:15:03.681726+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":7738,"log_dups_size":0,"ondisk_log_size":7738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31666176,"num_objects":7731,"num_object_clones":0,"num_object_copies":15462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5709,"num_write_kb":22836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":3798}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741922+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741922+0000","last_peered":"2024-09-11T01:13:09.741922+0000","last_clean":"2024-09-11T01:13:09.741922+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741922+0000","last_undegraded":"2024-09-11T01:13:09.741922+0000","last_fullsized":"2024-09-11T01:13:09.741922+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7750","reported_seq":7874,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:04.458501+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:15:04.458501+0000","last_peered":"2024-09-11T01:15:04.458501+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:15:04.458501+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":7750,"log_dups_size":0,"ondisk_log_size":7750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31711232,"num_objects":7742,"num_object_clones":0,"num_object_copies":15484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4337,"num_write_kb":17348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3758},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.910012+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.910012+0000","last_peered":"2024-09-11T01:13:09.910012+0000","last_clean":"2024-09-11T01:13:09.910012+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.910012+0000","last_undegraded":"2024-09-11T01:13:09.910012+0000","last_fullsized":"2024-09-11T01:13:09.910012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7790","reported_seq":7904,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:03.647567+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:15:03.647567+0000","last_peered":"2024-09-11T01:15:03.647567+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:15:03.647567+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":7790,"log_dups_size":0,"ondisk_log_size":7790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31887360,"num_objects":7785,"num_object_clones":0,"num_object_copies":15570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5607,"num_write_kb":22428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":3761}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7711","reported_seq":14032,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:03.666155+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:15:03.666155+0000","last_peered":"2024-09-11T01:15:03.666155+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:15:03.666155+0000","last_undegraded":"2024-09-11T01:15:03.666155+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":7711,"log_dups_size":0,"ondisk_log_size":7711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31559680,"num_objects":7705,"num_object_clones":0,"num_object_copies":15410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":824,"num_objects_unfound":0,"num_objects_dirty":7705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5496,"num_write_kb":21984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3103,"num_bytes_recovered":12701696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":824},{"shards":"2","objects":6881}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7896","reported_seq":8022,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:04.348511+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:15:04.348511+0000","last_peered":"2024-09-11T01:15:04.348511+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:15:04.348511+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":7896,"log_dups_size":0,"ondisk_log_size":7896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32333824,"num_objects":7894,"num_object_clones":0,"num_object_copies":15788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4633,"num_write_kb":18532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3944},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7726","reported_seq":7858,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:04.461051+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:15:04.461051+0000","last_peered":"2024-09-11T01:15:04.461051+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:15:04.461051+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":7726,"log_dups_size":0,"ondisk_log_size":7726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31612928,"num_objects":7718,"num_object_clones":0,"num_object_copies":15436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4381,"num_write_kb":17524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3779},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658174+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658174+0000","last_peered":"2024-09-11T01:13:09.658174+0000","last_clean":"2024-09-11T01:13:09.658174+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658174+0000","last_undegraded":"2024-09-11T01:13:09.658174+0000","last_fullsized":"2024-09-11T01:13:09.658174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7869","reported_seq":7999,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:04.374224+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:15:04.374224+0000","last_peered":"2024-09-11T01:15:04.374224+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:15:04.374224+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":7869,"log_dups_size":0,"ondisk_log_size":7869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32215040,"num_objects":7865,"num_object_clones":0,"num_object_copies":15730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4478,"num_write_kb":17912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3826},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7818","reported_seq":7947,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:04.448662+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:15:04.448662+0000","last_peered":"2024-09-11T01:15:04.448662+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:15:04.448662+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":7818,"log_dups_size":0,"ondisk_log_size":7818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31973993,"num_objects":7807,"num_object_clones":0,"num_object_copies":15614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4888,"num_write_kb":19552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3890},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":511115881,"num_objects":124785,"num_object_clones":0,"num_object_copies":249570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":824,"num_objects_unfound":0,"num_objects_dirty":124785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85716,"num_write_kb":342864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3104,"num_bytes_recovered":12705792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1032892416,"data_stored":1032812399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124866,"ondisk_log_size":124866,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182443,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":704308,"kb_used_data":438052,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93667532,"statfs":{"total":96636764160,"available":95915552768,"internally_reserved":0,"allocated":448565248,"data_stored":448316429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":272625323},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,6,3,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986215,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":569884,"kb_used_data":375820,"kb_used_omap":4,"kb_used_meta":194043,"kb_avail":93801956,"statfs":{"total":96636764160,"available":96053202944,"internally_reserved":0,"allocated":384839680,"data_stored":384594938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":198700714},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659160,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":354564,"kb_used_data":192436,"kb_used_omap":4,"kb_used_meta":162107,"kb_avail":94017276,"statfs":{"total":96636764160,"available":96273690624,"internally_reserved":0,"allocated":197054464,"data_stored":196736065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":165998255},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,4,4,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":449265664,"data_stored":449262185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":198324224,"data_stored":198251165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":385302528,"data_stored":385299049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:05.406 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 161 20 125362 125342 3.03989 1.83594 0.0762467 0.0256914 2024-09-11T01:15:06.406 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 162 20 125855 125835 3.03301 1.92578 0.031643 0.0257505 2024-09-11T01:15:07.406 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 163 20 126335 126315 3.02591 1.875 0.0216555 0.0258112 2024-09-11T01:15:08.381 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:08.406 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 164 20 126830 126810 3.01925 1.93359 0.0160233 0.0258671 2024-09-11T01:15:09.206 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:15:09.207 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:15:09.241 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":159,"stamp":"2024-09-11T01:15:08.593048+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":511115900,"num_objects":124787,"num_object_clones":0,"num_object_copies":249574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":824,"num_objects_unfound":0,"num_objects_dirty":124787,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85718,"num_write_kb":342866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3105,"num_bytes_recovered":12705811,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124869,"ondisk_log_size":124869,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1648116,"kb_used_data":1015940,"kb_used_omap":12,"kb_used_meta":632115,"kb_avail":281467404,"statfs":{"total":289910292480,"available":288222621696,"internally_reserved":0,"allocated":1040322560,"data_stored":1039510600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":647285764},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,5,8,4,7,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19783680,"num_objects":4830,"num_object_clones":0,"num_object_copies":9660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-403,"num_objects_unfound":0,"num_objects_dirty":4830,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4830,"num_write_kb":19320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":403,"num_bytes_recovered":1650688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005322"},"pg_stats":[{"pgid":"2.f","version":"43'7903","reported_seq":7976,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:04.398884+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:15:04.398884+0000","last_peered":"2024-09-11T01:15:04.398884+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:15:04.398884+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":7903,"log_dups_size":0,"ondisk_log_size":7903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32370688,"num_objects":7903,"num_object_clones":0,"num_object_copies":15806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7903,"num_write_kb":31612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3831},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7869","reported_seq":7966,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:15:03.609126+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:15:03.609126+0000","last_peered":"2024-09-11T01:15:03.609126+0000","last_clean":"2024-09-11T01:15:03.609126+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:15:03.609126+0000","last_undegraded":"2024-09-11T01:15:03.609126+0000","last_fullsized":"2024-09-11T01:15:03.609126+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":7869,"log_dups_size":0,"ondisk_log_size":7869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32206848,"num_objects":7863,"num_object_clones":0,"num_object_copies":15726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6108,"num_write_kb":24432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7726","reported_seq":7849,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:03.696389+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:15:03.696389+0000","last_peered":"2024-09-11T01:15:03.696389+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:15:03.696389+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":7726,"log_dups_size":0,"ondisk_log_size":7726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31629312,"num_objects":7722,"num_object_clones":0,"num_object_copies":15444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6027,"num_write_kb":24108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":3826}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7770","reported_seq":7835,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:15:03.670096+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:15:03.670096+0000","last_peered":"2024-09-11T01:15:03.670096+0000","last_clean":"2024-09-11T01:15:03.670096+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:15:03.670096+0000","last_undegraded":"2024-09-11T01:15:03.670096+0000","last_fullsized":"2024-09-11T01:15:03.670096+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":7770,"log_dups_size":0,"ondisk_log_size":7770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31825920,"num_objects":7770,"num_object_clones":0,"num_object_copies":15540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7770,"num_write_kb":31080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7809","reported_seq":7932,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:15:04.414109+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:15:04.414109+0000","last_peered":"2024-09-11T01:15:04.414109+0000","last_clean":"2024-09-11T01:15:04.414109+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:15:04.414109+0000","last_undegraded":"2024-09-11T01:15:04.414109+0000","last_fullsized":"2024-09-11T01:15:04.414109+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":7809,"log_dups_size":0,"ondisk_log_size":7809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31956992,"num_objects":7802,"num_object_clones":0,"num_object_copies":15604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4814,"num_write_kb":19256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7859","reported_seq":8024,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:04.460099+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:15:04.460099+0000","last_peered":"2024-09-11T01:15:04.460099+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:15:04.460099+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":7859,"log_dups_size":0,"ondisk_log_size":7859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32169984,"num_objects":7854,"num_object_clones":0,"num_object_copies":15708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4200,"num_write_kb":16800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3903},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7742","reported_seq":7867,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:04.426291+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:15:04.426291+0000","last_peered":"2024-09-11T01:15:04.426291+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:15:04.426291+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":7742,"log_dups_size":0,"ondisk_log_size":7742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31686656,"num_objects":7736,"num_object_clones":0,"num_object_copies":15472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4599,"num_write_kb":18396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3750},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7890","reported_seq":8015,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:15:04.330427+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:15:04.330427+0000","last_peered":"2024-09-11T01:15:04.330427+0000","last_clean":"2024-09-11T01:15:04.330427+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:15:04.330427+0000","last_undegraded":"2024-09-11T01:15:04.330427+0000","last_fullsized":"2024-09-11T01:15:04.330427+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":7890,"log_dups_size":0,"ondisk_log_size":7890,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32309248,"num_objects":7888,"num_object_clones":0,"num_object_copies":15776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4766,"num_write_kb":19064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.657802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.657802+0000","last_peered":"2024-09-11T01:13:09.657802+0000","last_clean":"2024-09-11T01:13:09.657802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.657802+0000","last_undegraded":"2024-09-11T01:13:09.657802+0000","last_fullsized":"2024-09-11T01:13:09.657802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7738","reported_seq":7861,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:03.681726+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:15:03.681726+0000","last_peered":"2024-09-11T01:15:03.681726+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:15:03.681726+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":7738,"log_dups_size":0,"ondisk_log_size":7738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31666176,"num_objects":7731,"num_object_clones":0,"num_object_copies":15462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5709,"num_write_kb":22836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":3798}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741922+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741922+0000","last_peered":"2024-09-11T01:13:09.741922+0000","last_clean":"2024-09-11T01:13:09.741922+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741922+0000","last_undegraded":"2024-09-11T01:13:09.741922+0000","last_fullsized":"2024-09-11T01:13:09.741922+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7750","reported_seq":7874,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:04.458501+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:15:04.458501+0000","last_peered":"2024-09-11T01:15:04.458501+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:15:04.458501+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":7750,"log_dups_size":0,"ondisk_log_size":7750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31711232,"num_objects":7742,"num_object_clones":0,"num_object_copies":15484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4337,"num_write_kb":17348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3758},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.910012+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.910012+0000","last_peered":"2024-09-11T01:13:09.910012+0000","last_clean":"2024-09-11T01:13:09.910012+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.910012+0000","last_undegraded":"2024-09-11T01:13:09.910012+0000","last_fullsized":"2024-09-11T01:13:09.910012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7790","reported_seq":7904,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:03.647567+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:15:03.647567+0000","last_peered":"2024-09-11T01:15:03.647567+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:15:03.647567+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":7790,"log_dups_size":0,"ondisk_log_size":7790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31887360,"num_objects":7785,"num_object_clones":0,"num_object_copies":15570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5607,"num_write_kb":22428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":3761}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7711","reported_seq":14032,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:03.666155+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:15:03.666155+0000","last_peered":"2024-09-11T01:15:03.666155+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:15:03.666155+0000","last_undegraded":"2024-09-11T01:15:03.666155+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":7711,"log_dups_size":0,"ondisk_log_size":7711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31559680,"num_objects":7705,"num_object_clones":0,"num_object_copies":15410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":824,"num_objects_unfound":0,"num_objects_dirty":7705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5496,"num_write_kb":21984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3103,"num_bytes_recovered":12701696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":824},{"shards":"2","objects":6881}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7896","reported_seq":8022,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:04.348511+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:15:04.348511+0000","last_peered":"2024-09-11T01:15:04.348511+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:15:04.348511+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":7896,"log_dups_size":0,"ondisk_log_size":7896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32333824,"num_objects":7894,"num_object_clones":0,"num_object_copies":15788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4633,"num_write_kb":18532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3944},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7726","reported_seq":7858,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:04.461051+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:15:04.461051+0000","last_peered":"2024-09-11T01:15:04.461051+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:15:04.461051+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":7726,"log_dups_size":0,"ondisk_log_size":7726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31612928,"num_objects":7718,"num_object_clones":0,"num_object_copies":15436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4381,"num_write_kb":17524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3779},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658174+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658174+0000","last_peered":"2024-09-11T01:13:09.658174+0000","last_clean":"2024-09-11T01:13:09.658174+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658174+0000","last_undegraded":"2024-09-11T01:13:09.658174+0000","last_fullsized":"2024-09-11T01:13:09.658174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7869","reported_seq":7999,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:04.374224+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:15:04.374224+0000","last_peered":"2024-09-11T01:15:04.374224+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:15:04.374224+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":7869,"log_dups_size":0,"ondisk_log_size":7869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32215040,"num_objects":7865,"num_object_clones":0,"num_object_copies":15730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4478,"num_write_kb":17912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3826},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7818","reported_seq":7947,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:04.448662+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:15:04.448662+0000","last_peered":"2024-09-11T01:15:04.448662+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:15:04.448662+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":7818,"log_dups_size":0,"ondisk_log_size":7818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31973993,"num_objects":7807,"num_object_clones":0,"num_object_copies":15614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4888,"num_write_kb":19552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3890},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":511115881,"num_objects":124785,"num_object_clones":0,"num_object_copies":249570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":824,"num_objects_unfound":0,"num_objects_dirty":124785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85716,"num_write_kb":342864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3104,"num_bytes_recovered":12705792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1041203200,"data_stored":1041123183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124866,"ondisk_log_size":124866,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182443,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":704308,"kb_used_data":438052,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93667532,"statfs":{"total":96636764160,"available":95915552768,"internally_reserved":0,"allocated":448565248,"data_stored":448316429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":272625323},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,6,3,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986215,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":569884,"kb_used_data":375820,"kb_used_omap":4,"kb_used_meta":194043,"kb_avail":93801956,"statfs":{"total":96636764160,"available":96053202944,"internally_reserved":0,"allocated":384839680,"data_stored":384594938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":198700714},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659161,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":373924,"kb_used_data":202068,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93997916,"statfs":{"total":96636764160,"available":96253865984,"internally_reserved":0,"allocated":206917632,"data_stored":206599233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":175959727},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,1,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":449265664,"data_stored":449262185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":206635008,"data_stored":206561949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":385302528,"data_stored":385299049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:09.242 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:09.406 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 165 20 127272 127252 3.01141 1.72656 0.023733 0.025934 2024-09-11T01:15:10.095 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:15:10.098 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:15:10.157 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":159,"stamp":"2024-09-11T01:15:08.593048+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":511115900,"num_objects":124787,"num_object_clones":0,"num_object_copies":249574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":824,"num_objects_unfound":0,"num_objects_dirty":124787,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85718,"num_write_kb":342866,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3105,"num_bytes_recovered":12705811,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124869,"ondisk_log_size":124869,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1648116,"kb_used_data":1015940,"kb_used_omap":12,"kb_used_meta":632115,"kb_avail":281467404,"statfs":{"total":289910292480,"available":288222621696,"internally_reserved":0,"allocated":1040322560,"data_stored":1039510600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":647285764},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,5,8,4,7,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19783680,"num_objects":4830,"num_object_clones":0,"num_object_copies":9660,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-403,"num_objects_unfound":0,"num_objects_dirty":4830,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4830,"num_write_kb":19320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":403,"num_bytes_recovered":1650688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005322"},"pg_stats":[{"pgid":"2.f","version":"43'7903","reported_seq":7976,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:04.398884+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:15:04.398884+0000","last_peered":"2024-09-11T01:15:04.398884+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:15:04.398884+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":7903,"log_dups_size":0,"ondisk_log_size":7903,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32370688,"num_objects":7903,"num_object_clones":0,"num_object_copies":15806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7903,"num_write_kb":31612,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3831},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'7869","reported_seq":7966,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:15:03.609126+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:15:03.609126+0000","last_peered":"2024-09-11T01:15:03.609126+0000","last_clean":"2024-09-11T01:15:03.609126+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:15:03.609126+0000","last_undegraded":"2024-09-11T01:15:03.609126+0000","last_fullsized":"2024-09-11T01:15:03.609126+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":7869,"log_dups_size":0,"ondisk_log_size":7869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32206848,"num_objects":7863,"num_object_clones":0,"num_object_copies":15726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6108,"num_write_kb":24432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7726","reported_seq":7849,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:03.696389+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:15:03.696389+0000","last_peered":"2024-09-11T01:15:03.696389+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:15:03.696389+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":7726,"log_dups_size":0,"ondisk_log_size":7726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31629312,"num_objects":7722,"num_object_clones":0,"num_object_copies":15444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7722,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6027,"num_write_kb":24108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":3826}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7770","reported_seq":7835,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:15:03.670096+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:15:03.670096+0000","last_peered":"2024-09-11T01:15:03.670096+0000","last_clean":"2024-09-11T01:15:03.670096+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:15:03.670096+0000","last_undegraded":"2024-09-11T01:15:03.670096+0000","last_fullsized":"2024-09-11T01:15:03.670096+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":7770,"log_dups_size":0,"ondisk_log_size":7770,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31825920,"num_objects":7770,"num_object_clones":0,"num_object_copies":15540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7770,"num_write_kb":31080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7809","reported_seq":7932,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:15:04.414109+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:15:04.414109+0000","last_peered":"2024-09-11T01:15:04.414109+0000","last_clean":"2024-09-11T01:15:04.414109+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:15:04.414109+0000","last_undegraded":"2024-09-11T01:15:04.414109+0000","last_fullsized":"2024-09-11T01:15:04.414109+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":7809,"log_dups_size":0,"ondisk_log_size":7809,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31956992,"num_objects":7802,"num_object_clones":0,"num_object_copies":15604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4814,"num_write_kb":19256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'7859","reported_seq":8024,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:04.460099+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:15:04.460099+0000","last_peered":"2024-09-11T01:15:04.460099+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:15:04.460099+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":7859,"log_dups_size":0,"ondisk_log_size":7859,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32169984,"num_objects":7854,"num_object_clones":0,"num_object_copies":15708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4200,"num_write_kb":16800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3903},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7742","reported_seq":7867,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:04.426291+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:15:04.426291+0000","last_peered":"2024-09-11T01:15:04.426291+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:15:04.426291+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":7742,"log_dups_size":0,"ondisk_log_size":7742,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31686656,"num_objects":7736,"num_object_clones":0,"num_object_copies":15472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4599,"num_write_kb":18396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3750},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'7890","reported_seq":8015,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:15:04.330427+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:15:04.330427+0000","last_peered":"2024-09-11T01:15:04.330427+0000","last_clean":"2024-09-11T01:15:04.330427+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:15:04.330427+0000","last_undegraded":"2024-09-11T01:15:04.330427+0000","last_fullsized":"2024-09-11T01:15:04.330427+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":7890,"log_dups_size":0,"ondisk_log_size":7890,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32309248,"num_objects":7888,"num_object_clones":0,"num_object_copies":15776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4766,"num_write_kb":19064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.657802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.657802+0000","last_peered":"2024-09-11T01:13:09.657802+0000","last_clean":"2024-09-11T01:13:09.657802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.657802+0000","last_undegraded":"2024-09-11T01:13:09.657802+0000","last_fullsized":"2024-09-11T01:13:09.657802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7738","reported_seq":7861,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:03.681726+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:15:03.681726+0000","last_peered":"2024-09-11T01:15:03.681726+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:15:03.681726+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":7738,"log_dups_size":0,"ondisk_log_size":7738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31666176,"num_objects":7731,"num_object_clones":0,"num_object_copies":15462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5709,"num_write_kb":22836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":3798}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741922+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741922+0000","last_peered":"2024-09-11T01:13:09.741922+0000","last_clean":"2024-09-11T01:13:09.741922+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741922+0000","last_undegraded":"2024-09-11T01:13:09.741922+0000","last_fullsized":"2024-09-11T01:13:09.741922+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7750","reported_seq":7874,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:04.458501+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:15:04.458501+0000","last_peered":"2024-09-11T01:15:04.458501+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:15:04.458501+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":7750,"log_dups_size":0,"ondisk_log_size":7750,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31711232,"num_objects":7742,"num_object_clones":0,"num_object_copies":15484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4337,"num_write_kb":17348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3758},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.910012+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.910012+0000","last_peered":"2024-09-11T01:13:09.910012+0000","last_clean":"2024-09-11T01:13:09.910012+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.910012+0000","last_undegraded":"2024-09-11T01:13:09.910012+0000","last_fullsized":"2024-09-11T01:13:09.910012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7790","reported_seq":7904,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:03.647567+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:15:03.647567+0000","last_peered":"2024-09-11T01:15:03.647567+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:15:03.647567+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":7790,"log_dups_size":0,"ondisk_log_size":7790,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31887360,"num_objects":7785,"num_object_clones":0,"num_object_copies":15570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5607,"num_write_kb":22428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":3761}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7711","reported_seq":14032,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:03.666155+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:15:03.666155+0000","last_peered":"2024-09-11T01:15:03.666155+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:15:03.666155+0000","last_undegraded":"2024-09-11T01:15:03.666155+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":7711,"log_dups_size":0,"ondisk_log_size":7711,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31559680,"num_objects":7705,"num_object_clones":0,"num_object_copies":15410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":824,"num_objects_unfound":0,"num_objects_dirty":7705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5496,"num_write_kb":21984,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3103,"num_bytes_recovered":12701696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":824},{"shards":"2","objects":6881}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'7896","reported_seq":8022,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:04.348511+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:15:04.348511+0000","last_peered":"2024-09-11T01:15:04.348511+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:15:04.348511+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":7896,"log_dups_size":0,"ondisk_log_size":7896,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32333824,"num_objects":7894,"num_object_clones":0,"num_object_copies":15788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7894,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4633,"num_write_kb":18532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3944},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7726","reported_seq":7858,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:04.461051+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:15:04.461051+0000","last_peered":"2024-09-11T01:15:04.461051+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:15:04.461051+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":7726,"log_dups_size":0,"ondisk_log_size":7726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31612928,"num_objects":7718,"num_object_clones":0,"num_object_copies":15436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4381,"num_write_kb":17524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3779},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658174+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658174+0000","last_peered":"2024-09-11T01:13:09.658174+0000","last_clean":"2024-09-11T01:13:09.658174+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658174+0000","last_undegraded":"2024-09-11T01:13:09.658174+0000","last_fullsized":"2024-09-11T01:13:09.658174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'7869","reported_seq":7999,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:04.374224+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:15:04.374224+0000","last_peered":"2024-09-11T01:15:04.374224+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:15:04.374224+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":7869,"log_dups_size":0,"ondisk_log_size":7869,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32215040,"num_objects":7865,"num_object_clones":0,"num_object_copies":15730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4478,"num_write_kb":17912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3826},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7818","reported_seq":7947,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:04.448662+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:15:04.448662+0000","last_peered":"2024-09-11T01:15:04.448662+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:15:04.448662+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":7818,"log_dups_size":0,"ondisk_log_size":7818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31973993,"num_objects":7807,"num_object_clones":0,"num_object_copies":15614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4888,"num_write_kb":19552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3890},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":511115881,"num_objects":124785,"num_object_clones":0,"num_object_copies":249570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":824,"num_objects_unfound":0,"num_objects_dirty":124785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":85716,"num_write_kb":342864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3104,"num_bytes_recovered":12705792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1041203200,"data_stored":1041123183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124866,"ondisk_log_size":124866,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182443,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":704308,"kb_used_data":438052,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93667532,"statfs":{"total":96636764160,"available":95915552768,"internally_reserved":0,"allocated":448565248,"data_stored":448316429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":272625323},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,6,3,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986215,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":569884,"kb_used_data":375820,"kb_used_omap":4,"kb_used_meta":194043,"kb_avail":93801956,"statfs":{"total":96636764160,"available":96053202944,"internally_reserved":0,"allocated":384839680,"data_stored":384594938,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":198700714},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659161,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":373924,"kb_used_data":202068,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93997916,"statfs":{"total":96636764160,"available":96253865984,"internally_reserved":0,"allocated":206917632,"data_stored":206599233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":175959727},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,1,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":449265664,"data_stored":449262185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":206635008,"data_stored":206561949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":385302528,"data_stored":385299049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:10.159 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:15:10.159 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'7903", 'reported_seq': 7976, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:04.398884+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:15:04.398884+0000', 'last_peered': '2024-09-11T01:15:04.398884+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:15:04.398884+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 7903, 'log_dups_size': 0, 'ondisk_log_size': 7903, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32370688, 'num_objects': 7903, 'num_object_clones': 0, 'num_object_copies': 15806, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7903, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7903, 'num_write_kb': 31612, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3831}, {'shards': '0,2', 'objects': 4072}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:10.159 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:15:10.159 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'7726", 'reported_seq': 7849, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:03.696389+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:15:03.696389+0000', 'last_peered': '2024-09-11T01:15:03.696389+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:15:03.696389+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 7726, 'log_dups_size': 0, 'ondisk_log_size': 7726, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31629312, 'num_objects': 7722, 'num_object_clones': 0, 'num_object_copies': 15444, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7722, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6027, 'num_write_kb': 24108, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3896}, {'shards': '2', 'objects': 3826}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:10.159 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:15:10.159 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'7859", 'reported_seq': 8024, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:04.460099+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:15:04.460099+0000', 'last_peered': '2024-09-11T01:15:04.460099+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:15:04.460099+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 7859, 'log_dups_size': 0, 'ondisk_log_size': 7859, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32169984, 'num_objects': 7854, 'num_object_clones': 0, 'num_object_copies': 15708, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7854, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4200, 'num_write_kb': 16800, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3903}, {'shards': '0,2', 'objects': 3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:10.159 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:15:10.159 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'7742", 'reported_seq': 7867, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:04.426291+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:15:04.426291+0000', 'last_peered': '2024-09-11T01:15:04.426291+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:15:04.426291+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 7742, 'log_dups_size': 0, 'ondisk_log_size': 7742, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31686656, 'num_objects': 7736, 'num_object_clones': 0, 'num_object_copies': 15472, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7736, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4599, 'num_write_kb': 18396, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3750}, {'shards': '0,2', 'objects': 3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:10.160 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:15:10.160 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'7738", 'reported_seq': 7861, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:03.681726+0000', 'last_change': '2024-09-11T01:13:10.159069+0000', 'last_active': '2024-09-11T01:15:03.681726+0000', 'last_peered': '2024-09-11T01:15:03.681726+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:15:03.681726+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 7738, 'log_dups_size': 0, 'ondisk_log_size': 7738, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31666176, 'num_objects': 7731, 'num_object_clones': 0, 'num_object_copies': 15462, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7731, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5709, 'num_write_kb': 22836, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3932}, {'shards': '2', 'objects': 3798}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:10.160 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:15:10.160 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'7750", 'reported_seq': 7874, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:04.458501+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:15:04.458501+0000', 'last_peered': '2024-09-11T01:15:04.458501+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:15:04.458501+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 7750, 'log_dups_size': 0, 'ondisk_log_size': 7750, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31711232, 'num_objects': 7742, 'num_object_clones': 0, 'num_object_copies': 15484, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7742, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4337, 'num_write_kb': 17348, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3758}, {'shards': '0,2', 'objects': 3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:10.160 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:15:10.160 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'7790", 'reported_seq': 7904, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:03.647567+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:15:03.647567+0000', 'last_peered': '2024-09-11T01:15:03.647567+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:15:03.647567+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 7790, 'log_dups_size': 0, 'ondisk_log_size': 7790, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31887360, 'num_objects': 7785, 'num_object_clones': 0, 'num_object_copies': 15570, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7785, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5607, 'num_write_kb': 22428, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4023}, {'shards': '2', 'objects': 3761}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:10.160 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:15:10.160 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'7711", 'reported_seq': 14032, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:15:03.666155+0000', 'last_change': '2024-09-11T01:13:10.217540+0000', 'last_active': '2024-09-11T01:15:03.666155+0000', 'last_peered': '2024-09-11T01:15:03.666155+0000', 'last_clean': '2024-09-11T01:13:07.638709+0000', 'last_became_active': '2024-09-11T01:13:10.125604+0000', 'last_became_peered': '2024-09-11T01:13:10.125604+0000', 'last_unstale': '2024-09-11T01:15:03.666155+0000', 'last_undegraded': '2024-09-11T01:15:03.666155+0000', 'last_fullsized': '2024-09-11T01:13:09.649604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2269", 'last_scrub_stamp': '2024-09-11T01:12:42.890920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:42.890920+0000', 'objects_scrubbed': 2209, 'log_size': 7711, 'log_dups_size': 0, 'ondisk_log_size': 7711, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 999, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31559680, 'num_objects': 7705, 'num_object_clones': 0, 'num_object_copies': 15410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 824, 'num_objects_unfound': 0, 'num_objects_dirty': 7705, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5496, 'num_write_kb': 21984, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3103, 'num_bytes_recovered': 12701696, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 824}, {'shards': '2', 'objects': 6881}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:10.160 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:15:10.160 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'7896", 'reported_seq': 8022, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:04.348511+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:15:04.348511+0000', 'last_peered': '2024-09-11T01:15:04.348511+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:15:04.348511+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 7896, 'log_dups_size': 0, 'ondisk_log_size': 7896, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32333824, 'num_objects': 7894, 'num_object_clones': 0, 'num_object_copies': 15788, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7894, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4633, 'num_write_kb': 18532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3944}, {'shards': '0,2', 'objects': 3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:10.161 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:15:10.161 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'7726", 'reported_seq': 7858, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:04.461051+0000', 'last_change': '2024-09-11T01:13:10.209668+0000', 'last_active': '2024-09-11T01:15:04.461051+0000', 'last_peered': '2024-09-11T01:15:04.461051+0000', 'last_clean': '2024-09-11T01:13:07.624565+0000', 'last_became_active': '2024-09-11T01:13:10.199795+0000', 'last_became_peered': '2024-09-11T01:13:10.199795+0000', 'last_unstale': '2024-09-11T01:15:04.461051+0000', 'last_undegraded': '2024-09-11T01:13:09.791198+0000', 'last_fullsized': '2024-09-11T01:13:09.652311+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3415", 'last_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'objects_scrubbed': 3337, 'log_size': 7726, 'log_dups_size': 0, 'ondisk_log_size': 7726, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1975, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31612928, 'num_objects': 7718, 'num_object_clones': 0, 'num_object_copies': 15436, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3942, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7718, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4381, 'num_write_kb': 17524, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3779}, {'shards': '0,2', 'objects': 3939}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:10.161 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:15:10.161 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'7869", 'reported_seq': 7999, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:04.374224+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:15:04.374224+0000', 'last_peered': '2024-09-11T01:15:04.374224+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:15:04.374224+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 7869, 'log_dups_size': 0, 'ondisk_log_size': 7869, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32215040, 'num_objects': 7865, 'num_object_clones': 0, 'num_object_copies': 15730, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7865, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4478, 'num_write_kb': 17912, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3826}, {'shards': '0,2', 'objects': 4039}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:10.161 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:15:10.161 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'7818", 'reported_seq': 7947, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:04.448662+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:15:04.448662+0000', 'last_peered': '2024-09-11T01:15:04.448662+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:15:04.448662+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 7818, 'log_dups_size': 0, 'ondisk_log_size': 7818, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31973993, 'num_objects': 7807, 'num_object_clones': 0, 'num_object_copies': 15614, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7807, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4888, 'num_write_kb': 19552, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3890}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:10.161 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:15:10.408 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 166 20 127731 127711 3.00404 1.79297 0.084452 0.0259986 2024-09-11T01:15:11.180 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:15:11.180 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43711/254392 objects degraded (17.183%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":247,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":127196,"data_bytes":520983164,"bytes_used":1731346432,"bytes_avail":288178946048,"bytes_total":289910292480,"degraded_objects":43711,"degraded_total":254392,"degraded_ratio":0.17182537186703983,"misplaced_objects":601,"misplaced_total":254392,"misplaced_ratio":0.0023624956759646528,"recovering_objects_per_sec":52,"recovering_bytes_per_sec":213506,"recovering_keys_per_sec":0,"num_objects_recovered":626,"num_bytes_recovered":2564096,"num_keys_recovered":0,"write_bytes_sec":2468963,"read_op_per_sec":0,"write_op_per_sec":602},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:15:11.226 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:15:11.226 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:11.408 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 167 20 128197 128177 2.99695 1.82031 0.0680176 0.0260598 2024-09-11T01:15:12.074 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:15:12.074 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:15:12.137 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-09-11T01:15:10.597617+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":520983164,"num_objects":127196,"num_object_clones":0,"num_object_copies":254392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":601,"num_objects_unfound":0,"num_objects_dirty":127196,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88127,"num_write_kb":352502,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3328,"num_bytes_recovered":13619219,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":127278,"ondisk_log_size":127278,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1690768,"kb_used_data":1028064,"kb_used_omap":12,"kb_used_meta":662643,"kb_avail":281424752,"statfs":{"total":289910292480,"available":288178946048,"internally_reserved":0,"allocated":1052737536,"data_stored":1051925576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":678546436},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,3,5,2,16,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29650944,"num_objects":7239,"num_object_clones":0,"num_object_copies":14478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-626,"num_objects_unfound":0,"num_objects_dirty":7239,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7239,"num_write_kb":28956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":626,"num_bytes_recovered":2564096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009469"},"pg_stats":[{"pgid":"2.f","version":"43'8069","reported_seq":8141,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:09.522569+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:15:09.522569+0000","last_peered":"2024-09-11T01:15:09.522569+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:15:09.522569+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":8069,"log_dups_size":0,"ondisk_log_size":8069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33050624,"num_objects":8069,"num_object_clones":0,"num_object_copies":16138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8069,"num_write_kb":32276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3997},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8027","reported_seq":8124,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:15:08.705076+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:15:08.705076+0000","last_peered":"2024-09-11T01:15:08.705076+0000","last_clean":"2024-09-11T01:15:08.705076+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:15:08.705076+0000","last_undegraded":"2024-09-11T01:15:08.705076+0000","last_fullsized":"2024-09-11T01:15:08.705076+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":8027,"log_dups_size":0,"ondisk_log_size":8027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32854016,"num_objects":8021,"num_object_clones":0,"num_object_copies":16042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8021,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6266,"num_write_kb":25064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'7866","reported_seq":7989,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:08.625183+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:15:08.625183+0000","last_peered":"2024-09-11T01:15:08.625183+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:15:08.625183+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":7866,"log_dups_size":0,"ondisk_log_size":7866,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32202752,"num_objects":7862,"num_object_clones":0,"num_object_copies":15724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6167,"num_write_kb":24668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":3966}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'7943","reported_seq":8008,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:15:08.599464+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:15:08.599464+0000","last_peered":"2024-09-11T01:15:08.599464+0000","last_clean":"2024-09-11T01:15:08.599464+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:15:08.599464+0000","last_undegraded":"2024-09-11T01:15:08.599464+0000","last_fullsized":"2024-09-11T01:15:08.599464+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":7943,"log_dups_size":0,"ondisk_log_size":7943,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32534528,"num_objects":7943,"num_object_clones":0,"num_object_copies":15886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7943,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7943,"num_write_kb":31772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'7952","reported_seq":8075,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:15:09.535375+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:15:09.535375+0000","last_peered":"2024-09-11T01:15:09.535375+0000","last_clean":"2024-09-11T01:15:09.535375+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:15:09.535375+0000","last_undegraded":"2024-09-11T01:15:09.535375+0000","last_fullsized":"2024-09-11T01:15:09.535375+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":7952,"log_dups_size":0,"ondisk_log_size":7952,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32542720,"num_objects":7945,"num_object_clones":0,"num_object_copies":15890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4957,"num_write_kb":19828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8006","reported_seq":8171,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:09.456725+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:15:09.456725+0000","last_peered":"2024-09-11T01:15:09.456725+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:15:09.456725+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":8006,"log_dups_size":0,"ondisk_log_size":8006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32772096,"num_objects":8001,"num_object_clones":0,"num_object_copies":16002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4347,"num_write_kb":17388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4050},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'7886","reported_seq":8012,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:09.517993+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:15:09.517993+0000","last_peered":"2024-09-11T01:15:09.517993+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:15:09.517993+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":7886,"log_dups_size":0,"ondisk_log_size":7886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32276480,"num_objects":7880,"num_object_clones":0,"num_object_copies":15760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7880,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4743,"num_write_kb":18972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3894},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8036","reported_seq":8161,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:15:09.529744+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:15:09.529744+0000","last_peered":"2024-09-11T01:15:09.529744+0000","last_clean":"2024-09-11T01:15:09.529744+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:15:09.529744+0000","last_undegraded":"2024-09-11T01:15:09.529744+0000","last_fullsized":"2024-09-11T01:15:09.529744+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":8036,"log_dups_size":0,"ondisk_log_size":8036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32907264,"num_objects":8034,"num_object_clones":0,"num_object_copies":16068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4912,"num_write_kb":19648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.657802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.657802+0000","last_peered":"2024-09-11T01:13:09.657802+0000","last_clean":"2024-09-11T01:13:09.657802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.657802+0000","last_undegraded":"2024-09-11T01:13:09.657802+0000","last_fullsized":"2024-09-11T01:13:09.657802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'7885","reported_seq":8008,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:08.556353+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:15:08.556353+0000","last_peered":"2024-09-11T01:15:08.556353+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:15:08.556353+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":7885,"log_dups_size":0,"ondisk_log_size":7885,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32268288,"num_objects":7878,"num_object_clones":0,"num_object_copies":15756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7878,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5856,"num_write_kb":23424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":3945}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741922+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741922+0000","last_peered":"2024-09-11T01:13:09.741922+0000","last_clean":"2024-09-11T01:13:09.741922+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741922+0000","last_undegraded":"2024-09-11T01:13:09.741922+0000","last_fullsized":"2024-09-11T01:13:09.741922+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'7893","reported_seq":8017,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:09.504532+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:15:09.504532+0000","last_peered":"2024-09-11T01:15:09.504532+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:15:09.504532+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":7893,"log_dups_size":0,"ondisk_log_size":7893,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32296960,"num_objects":7885,"num_object_clones":0,"num_object_copies":15770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4480,"num_write_kb":17920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3901},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.910012+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.910012+0000","last_peered":"2024-09-11T01:13:09.910012+0000","last_clean":"2024-09-11T01:13:09.910012+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.910012+0000","last_undegraded":"2024-09-11T01:13:09.910012+0000","last_fullsized":"2024-09-11T01:13:09.910012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'7956","reported_seq":8070,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:08.665273+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:15:08.665273+0000","last_peered":"2024-09-11T01:15:08.665273+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:15:08.665273+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":7956,"log_dups_size":0,"ondisk_log_size":7956,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32567296,"num_objects":7951,"num_object_clones":0,"num_object_copies":15902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5773,"num_write_kb":23092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":3927}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":74,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.768757+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.768757+0000","last_peered":"2024-09-11T01:13:09.768757+0000","last_clean":"2024-09-11T01:13:09.768757+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.768757+0000","last_undegraded":"2024-09-11T01:13:09.768757+0000","last_fullsized":"2024-09-11T01:13:09.768757+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7850","reported_seq":14617,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:08.715021+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:15:08.715021+0000","last_peered":"2024-09-11T01:15:08.715021+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:15:08.715021+0000","last_undegraded":"2024-09-11T01:15:08.715021+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":7850,"log_dups_size":0,"ondisk_log_size":7850,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32129024,"num_objects":7844,"num_object_clones":0,"num_object_copies":15688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":601,"num_objects_unfound":0,"num_objects_dirty":7844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5635,"num_write_kb":22540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3326,"num_bytes_recovered":13615104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":601},{"shards":"2","objects":7243}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8051","reported_seq":8178,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:09.499416+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:15:09.499416+0000","last_peered":"2024-09-11T01:15:09.499416+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:15:09.499416+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":8051,"log_dups_size":0,"ondisk_log_size":8051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32968704,"num_objects":8049,"num_object_clones":0,"num_object_copies":16098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4788,"num_write_kb":19152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4099},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.902951+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:09.902951+0000","last_peered":"2024-09-11T01:13:09.902951+0000","last_clean":"2024-09-11T01:13:09.902951+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:09.902951+0000","last_undegraded":"2024-09-11T01:13:09.902951+0000","last_fullsized":"2024-09-11T01:13:09.902951+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'7871","reported_seq":8002,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:09.467994+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:15:09.467994+0000","last_peered":"2024-09-11T01:15:09.467994+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:15:09.467994+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":7871,"log_dups_size":0,"ondisk_log_size":7871,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32206848,"num_objects":7863,"num_object_clones":0,"num_object_copies":15726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4526,"num_write_kb":18104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3924},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658174+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658174+0000","last_peered":"2024-09-11T01:13:09.658174+0000","last_clean":"2024-09-11T01:13:09.658174+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658174+0000","last_undegraded":"2024-09-11T01:13:09.658174+0000","last_fullsized":"2024-09-11T01:13:09.658174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8016","reported_seq":8146,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:09.540589+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:15:09.540589+0000","last_peered":"2024-09-11T01:15:09.540589+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:15:09.540589+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":8016,"log_dups_size":0,"ondisk_log_size":8016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32817152,"num_objects":8012,"num_object_clones":0,"num_object_copies":16024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4625,"num_write_kb":18500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3973},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.905475+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.905475+0000","last_peered":"2024-09-11T01:13:09.905475+0000","last_clean":"2024-09-11T01:13:09.905475+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.905475+0000","last_undegraded":"2024-09-11T01:13:09.905475+0000","last_fullsized":"2024-09-11T01:13:09.905475+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'7968","reported_seq":8097,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:09.454287+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:15:09.454287+0000","last_peered":"2024-09-11T01:15:09.454287+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:15:09.454287+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":7968,"log_dups_size":0,"ondisk_log_size":7968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32588393,"num_objects":7957,"num_object_clones":0,"num_object_copies":15914,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7957,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5038,"num_write_kb":20152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4040},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":520983145,"num_objects":127194,"num_object_clones":0,"num_object_copies":254388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":601,"num_objects_unfound":0,"num_objects_dirty":127194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":88125,"num_write_kb":352500,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3327,"num_bytes_recovered":13619200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1053552640,"data_stored":1053472623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":127275,"ondisk_log_size":127275,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182444,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":738028,"kb_used_data":445404,"kb_used_omap":4,"kb_used_meta":292603,"kb_avail":93633812,"statfs":{"total":96636764160,"available":95881023488,"internally_reserved":0,"allocated":456093696,"data_stored":455844877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":299626155},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,1,6,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986216,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":578816,"kb_used_data":380592,"kb_used_omap":4,"kb_used_meta":198203,"kb_avail":93793024,"statfs":{"total":96636764160,"available":96044056576,"internally_reserved":0,"allocated":389726208,"data_stored":389481466,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":202960554},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,0,6,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659161,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":373924,"kb_used_data":202068,"kb_used_omap":4,"kb_used_meta":171835,"kb_avail":93997916,"statfs":{"total":96636764160,"available":96253865984,"internally_reserved":0,"allocated":206917632,"data_stored":206599233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":175959727},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,1,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":456679424,"data_stored":456675945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":206635008,"data_stored":206561949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":390238208,"data_stored":390234729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:12.411 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 168 20 128653 128633 2.98972 1.78125 0.0351322 0.0261233 2024-09-11T01:15:13.408 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 169 20 129135 129115 2.98317 1.88281 0.0601387 0.0261801 2024-09-11T01:15:14.408 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 170 20 129610 129590 2.97653 1.85547 0.00508657 0.0262376 2024-09-11T01:15:15.139 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:15.410 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 171 20 130098 130078 2.97025 1.90625 0.017552 0.0262934 2024-09-11T01:15:15.902 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:15:15.902 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:15:15.980 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":162,"stamp":"2024-09-11T01:15:14.604653+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":530575996,"num_objects":129538,"num_object_clones":0,"num_object_copies":259076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":342,"num_objects_unfound":0,"num_objects_dirty":129538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90469,"num_write_kb":361870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":14680083,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129620,"ondisk_log_size":129620,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1704448,"kb_used_data":1045904,"kb_used_omap":12,"kb_used_meta":658483,"kb_avail":281411072,"statfs":{"total":289910292480,"available":288164937728,"internally_reserved":0,"allocated":1071005696,"data_stored":1070193736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":674286596},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,7,11,16,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":29491200,"num_objects":7200,"num_object_clones":0,"num_object_copies":14400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-688,"num_objects_unfound":0,"num_objects_dirty":7200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7200,"num_write_kb":28800,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":688,"num_bytes_recovered":2818048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.013010"},"pg_stats":[{"pgid":"2.f","version":"43'8223","reported_seq":8294,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:14.468360+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:15:14.468360+0000","last_peered":"2024-09-11T01:15:14.468360+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:15:14.468360+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":8223,"log_dups_size":0,"ondisk_log_size":8223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33681408,"num_objects":8223,"num_object_clones":0,"num_object_copies":16446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8223,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8223,"num_write_kb":32892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4151},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8172","reported_seq":8269,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:15:13.688323+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:15:13.688323+0000","last_peered":"2024-09-11T01:15:13.688323+0000","last_clean":"2024-09-11T01:15:13.688323+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:15:13.688323+0000","last_undegraded":"2024-09-11T01:15:13.688323+0000","last_fullsized":"2024-09-11T01:15:13.688323+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":8172,"log_dups_size":0,"ondisk_log_size":8172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33447936,"num_objects":8166,"num_object_clones":0,"num_object_copies":16332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6411,"num_write_kb":25644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8018","reported_seq":8140,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:13.770168+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:15:13.770168+0000","last_peered":"2024-09-11T01:15:13.770168+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:15:13.770168+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":8018,"log_dups_size":0,"ondisk_log_size":8018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32825344,"num_objects":8014,"num_object_clones":0,"num_object_copies":16028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6319,"num_write_kb":25276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":4118}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8088","reported_seq":8152,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:15:13.738590+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:15:13.738590+0000","last_peered":"2024-09-11T01:15:13.738590+0000","last_clean":"2024-09-11T01:15:13.738590+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:15:13.738590+0000","last_undegraded":"2024-09-11T01:15:13.738590+0000","last_fullsized":"2024-09-11T01:15:13.738590+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":8088,"log_dups_size":0,"ondisk_log_size":8088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33128448,"num_objects":8088,"num_object_clones":0,"num_object_copies":16176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8088,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8088,"num_write_kb":32352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8102","reported_seq":8225,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:15:14.526659+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:15:14.526659+0000","last_peered":"2024-09-11T01:15:14.526659+0000","last_clean":"2024-09-11T01:15:14.526659+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:15:14.526659+0000","last_undegraded":"2024-09-11T01:15:14.526659+0000","last_fullsized":"2024-09-11T01:15:14.526659+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":8102,"log_dups_size":0,"ondisk_log_size":8102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33157120,"num_objects":8095,"num_object_clones":0,"num_object_copies":16190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8095,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5107,"num_write_kb":20428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8151","reported_seq":8314,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:14.467498+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:15:14.467498+0000","last_peered":"2024-09-11T01:15:14.467498+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:15:14.467498+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":8151,"log_dups_size":0,"ondisk_log_size":8151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33366016,"num_objects":8146,"num_object_clones":0,"num_object_copies":16292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4492,"num_write_kb":17968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4195},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8031","reported_seq":8157,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:14.548150+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:15:14.548150+0000","last_peered":"2024-09-11T01:15:14.548150+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:15:14.548150+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":8031,"log_dups_size":0,"ondisk_log_size":8031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32870400,"num_objects":8025,"num_object_clones":0,"num_object_copies":16050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8025,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4888,"num_write_kb":19552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4039},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8171","reported_seq":8296,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:15:14.539557+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:15:14.539557+0000","last_peered":"2024-09-11T01:15:14.539557+0000","last_clean":"2024-09-11T01:15:14.539557+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:15:14.539557+0000","last_undegraded":"2024-09-11T01:15:14.539557+0000","last_fullsized":"2024-09-11T01:15:14.539557+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":8171,"log_dups_size":0,"ondisk_log_size":8171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33460224,"num_objects":8169,"num_object_clones":0,"num_object_copies":16338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5047,"num_write_kb":20188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.657802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.657802+0000","last_peered":"2024-09-11T01:13:09.657802+0000","last_clean":"2024-09-11T01:13:09.657802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.657802+0000","last_undegraded":"2024-09-11T01:13:09.657802+0000","last_fullsized":"2024-09-11T01:13:09.657802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8047","reported_seq":8170,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:13.738892+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:15:13.738892+0000","last_peered":"2024-09-11T01:15:13.738892+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:15:13.738892+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":8047,"log_dups_size":0,"ondisk_log_size":8047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32931840,"num_objects":8040,"num_object_clones":0,"num_object_copies":16080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6018,"num_write_kb":24072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":4107}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741922+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741922+0000","last_peered":"2024-09-11T01:13:09.741922+0000","last_clean":"2024-09-11T01:13:09.741922+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741922+0000","last_undegraded":"2024-09-11T01:13:09.741922+0000","last_fullsized":"2024-09-11T01:13:09.741922+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8022","reported_seq":8143,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:14.504240+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:15:14.504240+0000","last_peered":"2024-09-11T01:15:14.504240+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:15:14.504240+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":8022,"log_dups_size":0,"ondisk_log_size":8022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32825344,"num_objects":8014,"num_object_clones":0,"num_object_copies":16028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4609,"num_write_kb":18436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4030},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.910012+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.910012+0000","last_peered":"2024-09-11T01:13:09.910012+0000","last_clean":"2024-09-11T01:13:09.910012+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.910012+0000","last_undegraded":"2024-09-11T01:13:09.910012+0000","last_fullsized":"2024-09-11T01:13:09.910012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8115","reported_seq":8229,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:13.720223+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:15:13.720223+0000","last_peered":"2024-09-11T01:15:13.720223+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:15:13.720223+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":8115,"log_dups_size":0,"ondisk_log_size":8115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33218560,"num_objects":8110,"num_object_clones":0,"num_object_copies":16220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5932,"num_write_kb":23728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":4086}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.769458+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.769458+0000","last_peered":"2024-09-11T01:13:09.769458+0000","last_clean":"2024-09-11T01:13:09.769458+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.769458+0000","last_undegraded":"2024-09-11T01:13:09.769458+0000","last_fullsized":"2024-09-11T01:13:09.769458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7981","reported_seq":15265,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:13.745381+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:15:13.745381+0000","last_peered":"2024-09-11T01:15:13.745381+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:15:13.745381+0000","last_undegraded":"2024-09-11T01:15:13.745381+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":7981,"log_dups_size":0,"ondisk_log_size":7981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32665600,"num_objects":7975,"num_object_clones":0,"num_object_copies":15950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":342,"num_objects_unfound":0,"num_objects_dirty":7975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5766,"num_write_kb":23064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3585,"num_bytes_recovered":14675968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":342},{"shards":"2","objects":7633}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8207","reported_seq":8333,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:14.514414+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:15:14.514414+0000","last_peered":"2024-09-11T01:15:14.514414+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:15:14.514414+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33607680,"num_objects":8205,"num_object_clones":0,"num_object_copies":16410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4944,"num_write_kb":19776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4255},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:10.221233+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:10.221233+0000","last_peered":"2024-09-11T01:13:10.221233+0000","last_clean":"2024-09-11T01:13:10.221233+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:10.221233+0000","last_undegraded":"2024-09-11T01:13:10.221233+0000","last_fullsized":"2024-09-11T01:13:10.221233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8021","reported_seq":8153,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:14.528198+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:15:14.528198+0000","last_peered":"2024-09-11T01:15:14.528198+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:15:14.528198+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":8021,"log_dups_size":0,"ondisk_log_size":8021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32821248,"num_objects":8013,"num_object_clones":0,"num_object_copies":16026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4676,"num_write_kb":18704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4074},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658174+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658174+0000","last_peered":"2024-09-11T01:13:09.658174+0000","last_clean":"2024-09-11T01:13:09.658174+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658174+0000","last_undegraded":"2024-09-11T01:13:09.658174+0000","last_fullsized":"2024-09-11T01:13:09.658174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8162","reported_seq":8292,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:14.486020+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:15:14.486020+0000","last_peered":"2024-09-11T01:15:14.486020+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:15:14.486020+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":8162,"log_dups_size":0,"ondisk_log_size":8162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33415168,"num_objects":8158,"num_object_clones":0,"num_object_copies":16316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4771,"num_write_kb":19084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4119},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.916946+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.916946+0000","last_peered":"2024-09-11T01:13:09.916946+0000","last_clean":"2024-09-11T01:13:09.916946+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.916946+0000","last_undegraded":"2024-09-11T01:13:09.916946+0000","last_fullsized":"2024-09-11T01:13:09.916946+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8106","reported_seq":8235,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:14.500397+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:15:14.500397+0000","last_peered":"2024-09-11T01:15:14.500397+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:15:14.500397+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":8106,"log_dups_size":0,"ondisk_log_size":8106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33153641,"num_objects":8095,"num_object_clones":0,"num_object_copies":16190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8095,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5176,"num_write_kb":20704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4178},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":530575977,"num_objects":129536,"num_object_clones":0,"num_object_copies":259072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":342,"num_objects_unfound":0,"num_objects_dirty":129536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90467,"num_write_kb":361868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3586,"num_bytes_recovered":14680064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1073565696,"data_stored":1073485679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129617,"ondisk_log_size":129617,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182445,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":732348,"kb_used_data":452204,"kb_used_omap":4,"kb_used_meta":280123,"kb_avail":93639492,"statfs":{"total":96636764160,"available":95886839808,"internally_reserved":0,"allocated":463056896,"data_stored":462808077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":286846635},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,5,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986217,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":587724,"kb_used_data":385340,"kb_used_omap":4,"kb_used_meta":202363,"kb_avail":93784116,"statfs":{"total":96636764160,"available":96034934784,"internally_reserved":0,"allocated":394588160,"data_stored":394343418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":207220394},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,1,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659162,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":384376,"kb_used_data":208360,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93987464,"statfs":{"total":96636764160,"available":96243163136,"internally_reserved":0,"allocated":213360640,"data_stored":213042241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":180219567},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":463749120,"data_stored":463745641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":214753280,"data_stored":214680221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":395063296,"data_stored":395059817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:15.981 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:16.410 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 172 20 130554 130534 2.96334 1.78125 0.065562 0.0263569 2024-09-11T01:15:16.755 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:15:16.758 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:15:16.818 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":163,"stamp":"2024-09-11T01:15:16.609258+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":530575996,"num_objects":129538,"num_object_clones":0,"num_object_copies":259076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":342,"num_objects_unfound":0,"num_objects_dirty":129538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90469,"num_write_kb":361870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":14680083,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129620,"ondisk_log_size":129620,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1704448,"kb_used_data":1045904,"kb_used_omap":12,"kb_used_meta":658483,"kb_avail":281411072,"statfs":{"total":289910292480,"available":288164937728,"internally_reserved":0,"allocated":1071005696,"data_stored":1070193736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":674286596},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,7,11,16,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19460096,"num_objects":4751,"num_object_clones":0,"num_object_copies":9502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-482,"num_objects_unfound":0,"num_objects_dirty":4751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4751,"num_write_kb":19004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":482,"num_bytes_recovered":1974272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.017184"},"pg_stats":[{"pgid":"2.f","version":"43'8223","reported_seq":8294,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:14.468360+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:15:14.468360+0000","last_peered":"2024-09-11T01:15:14.468360+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:15:14.468360+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":8223,"log_dups_size":0,"ondisk_log_size":8223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33681408,"num_objects":8223,"num_object_clones":0,"num_object_copies":16446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8223,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8223,"num_write_kb":32892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4151},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8172","reported_seq":8269,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:15:13.688323+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:15:13.688323+0000","last_peered":"2024-09-11T01:15:13.688323+0000","last_clean":"2024-09-11T01:15:13.688323+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:15:13.688323+0000","last_undegraded":"2024-09-11T01:15:13.688323+0000","last_fullsized":"2024-09-11T01:15:13.688323+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":8172,"log_dups_size":0,"ondisk_log_size":8172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33447936,"num_objects":8166,"num_object_clones":0,"num_object_copies":16332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6411,"num_write_kb":25644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8018","reported_seq":8140,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:13.770168+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:15:13.770168+0000","last_peered":"2024-09-11T01:15:13.770168+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:15:13.770168+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":8018,"log_dups_size":0,"ondisk_log_size":8018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32825344,"num_objects":8014,"num_object_clones":0,"num_object_copies":16028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6319,"num_write_kb":25276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":4118}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8088","reported_seq":8152,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:15:13.738590+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:15:13.738590+0000","last_peered":"2024-09-11T01:15:13.738590+0000","last_clean":"2024-09-11T01:15:13.738590+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:15:13.738590+0000","last_undegraded":"2024-09-11T01:15:13.738590+0000","last_fullsized":"2024-09-11T01:15:13.738590+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":8088,"log_dups_size":0,"ondisk_log_size":8088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33128448,"num_objects":8088,"num_object_clones":0,"num_object_copies":16176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8088,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8088,"num_write_kb":32352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8102","reported_seq":8225,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:15:14.526659+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:15:14.526659+0000","last_peered":"2024-09-11T01:15:14.526659+0000","last_clean":"2024-09-11T01:15:14.526659+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:15:14.526659+0000","last_undegraded":"2024-09-11T01:15:14.526659+0000","last_fullsized":"2024-09-11T01:15:14.526659+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":8102,"log_dups_size":0,"ondisk_log_size":8102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33157120,"num_objects":8095,"num_object_clones":0,"num_object_copies":16190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8095,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5107,"num_write_kb":20428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8151","reported_seq":8314,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:14.467498+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:15:14.467498+0000","last_peered":"2024-09-11T01:15:14.467498+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:15:14.467498+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":8151,"log_dups_size":0,"ondisk_log_size":8151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33366016,"num_objects":8146,"num_object_clones":0,"num_object_copies":16292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4492,"num_write_kb":17968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4195},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8031","reported_seq":8157,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:14.548150+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:15:14.548150+0000","last_peered":"2024-09-11T01:15:14.548150+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:15:14.548150+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":8031,"log_dups_size":0,"ondisk_log_size":8031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32870400,"num_objects":8025,"num_object_clones":0,"num_object_copies":16050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8025,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4888,"num_write_kb":19552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4039},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8171","reported_seq":8296,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:15:14.539557+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:15:14.539557+0000","last_peered":"2024-09-11T01:15:14.539557+0000","last_clean":"2024-09-11T01:15:14.539557+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:15:14.539557+0000","last_undegraded":"2024-09-11T01:15:14.539557+0000","last_fullsized":"2024-09-11T01:15:14.539557+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":8171,"log_dups_size":0,"ondisk_log_size":8171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33460224,"num_objects":8169,"num_object_clones":0,"num_object_copies":16338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5047,"num_write_kb":20188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.657802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.657802+0000","last_peered":"2024-09-11T01:13:09.657802+0000","last_clean":"2024-09-11T01:13:09.657802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.657802+0000","last_undegraded":"2024-09-11T01:13:09.657802+0000","last_fullsized":"2024-09-11T01:13:09.657802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8047","reported_seq":8170,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:13.738892+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:15:13.738892+0000","last_peered":"2024-09-11T01:15:13.738892+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:15:13.738892+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":8047,"log_dups_size":0,"ondisk_log_size":8047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32931840,"num_objects":8040,"num_object_clones":0,"num_object_copies":16080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6018,"num_write_kb":24072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":4107}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741922+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741922+0000","last_peered":"2024-09-11T01:13:09.741922+0000","last_clean":"2024-09-11T01:13:09.741922+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741922+0000","last_undegraded":"2024-09-11T01:13:09.741922+0000","last_fullsized":"2024-09-11T01:13:09.741922+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8022","reported_seq":8143,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:14.504240+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:15:14.504240+0000","last_peered":"2024-09-11T01:15:14.504240+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:15:14.504240+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":8022,"log_dups_size":0,"ondisk_log_size":8022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32825344,"num_objects":8014,"num_object_clones":0,"num_object_copies":16028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4609,"num_write_kb":18436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4030},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.910012+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.910012+0000","last_peered":"2024-09-11T01:13:09.910012+0000","last_clean":"2024-09-11T01:13:09.910012+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.910012+0000","last_undegraded":"2024-09-11T01:13:09.910012+0000","last_fullsized":"2024-09-11T01:13:09.910012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8115","reported_seq":8229,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:13.720223+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:15:13.720223+0000","last_peered":"2024-09-11T01:15:13.720223+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:15:13.720223+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":8115,"log_dups_size":0,"ondisk_log_size":8115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33218560,"num_objects":8110,"num_object_clones":0,"num_object_copies":16220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5932,"num_write_kb":23728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":4086}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.769458+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.769458+0000","last_peered":"2024-09-11T01:13:09.769458+0000","last_clean":"2024-09-11T01:13:09.769458+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.769458+0000","last_undegraded":"2024-09-11T01:13:09.769458+0000","last_fullsized":"2024-09-11T01:13:09.769458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7981","reported_seq":15265,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:13.745381+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:15:13.745381+0000","last_peered":"2024-09-11T01:15:13.745381+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:15:13.745381+0000","last_undegraded":"2024-09-11T01:15:13.745381+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":7981,"log_dups_size":0,"ondisk_log_size":7981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32665600,"num_objects":7975,"num_object_clones":0,"num_object_copies":15950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":342,"num_objects_unfound":0,"num_objects_dirty":7975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5766,"num_write_kb":23064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3585,"num_bytes_recovered":14675968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":342},{"shards":"2","objects":7633}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8207","reported_seq":8333,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:14.514414+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:15:14.514414+0000","last_peered":"2024-09-11T01:15:14.514414+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:15:14.514414+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33607680,"num_objects":8205,"num_object_clones":0,"num_object_copies":16410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4944,"num_write_kb":19776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4255},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:10.221233+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:10.221233+0000","last_peered":"2024-09-11T01:13:10.221233+0000","last_clean":"2024-09-11T01:13:10.221233+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:10.221233+0000","last_undegraded":"2024-09-11T01:13:10.221233+0000","last_fullsized":"2024-09-11T01:13:10.221233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8021","reported_seq":8153,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:14.528198+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:15:14.528198+0000","last_peered":"2024-09-11T01:15:14.528198+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:15:14.528198+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":8021,"log_dups_size":0,"ondisk_log_size":8021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32821248,"num_objects":8013,"num_object_clones":0,"num_object_copies":16026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4676,"num_write_kb":18704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4074},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658174+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658174+0000","last_peered":"2024-09-11T01:13:09.658174+0000","last_clean":"2024-09-11T01:13:09.658174+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658174+0000","last_undegraded":"2024-09-11T01:13:09.658174+0000","last_fullsized":"2024-09-11T01:13:09.658174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8162","reported_seq":8292,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:14.486020+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:15:14.486020+0000","last_peered":"2024-09-11T01:15:14.486020+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:15:14.486020+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":8162,"log_dups_size":0,"ondisk_log_size":8162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33415168,"num_objects":8158,"num_object_clones":0,"num_object_copies":16316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4771,"num_write_kb":19084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4119},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.916946+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.916946+0000","last_peered":"2024-09-11T01:13:09.916946+0000","last_clean":"2024-09-11T01:13:09.916946+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.916946+0000","last_undegraded":"2024-09-11T01:13:09.916946+0000","last_fullsized":"2024-09-11T01:13:09.916946+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.661252+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.661252+0000","last_peered":"2024-09-11T01:13:09.661252+0000","last_clean":"2024-09-11T01:13:09.661252+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.661252+0000","last_undegraded":"2024-09-11T01:13:09.661252+0000","last_fullsized":"2024-09-11T01:13:09.661252+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8106","reported_seq":8235,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:14.500397+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:15:14.500397+0000","last_peered":"2024-09-11T01:15:14.500397+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:15:14.500397+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":8106,"log_dups_size":0,"ondisk_log_size":8106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33153641,"num_objects":8095,"num_object_clones":0,"num_object_copies":16190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8095,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5176,"num_write_kb":20704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4178},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":530575977,"num_objects":129536,"num_object_clones":0,"num_object_copies":259072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":342,"num_objects_unfound":0,"num_objects_dirty":129536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90467,"num_write_kb":361868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3586,"num_bytes_recovered":14680064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1073565696,"data_stored":1073485679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129617,"ondisk_log_size":129617,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182445,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":732348,"kb_used_data":452204,"kb_used_omap":4,"kb_used_meta":280123,"kb_avail":93639492,"statfs":{"total":96636764160,"available":95886839808,"internally_reserved":0,"allocated":463056896,"data_stored":462808077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":286846635},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,5,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986217,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":587724,"kb_used_data":385340,"kb_used_omap":4,"kb_used_meta":202363,"kb_avail":93784116,"statfs":{"total":96636764160,"available":96034934784,"internally_reserved":0,"allocated":394588160,"data_stored":394343418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":207220394},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,1,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659162,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":384376,"kb_used_data":208360,"kb_used_omap":4,"kb_used_meta":175995,"kb_avail":93987464,"statfs":{"total":96636764160,"available":96243163136,"internally_reserved":0,"allocated":213360640,"data_stored":213042241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":180219567},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":463749120,"data_stored":463745641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":214753280,"data_stored":214680221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":395063296,"data_stored":395059817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:16.819 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:15:16.819 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'8223", 'reported_seq': 8294, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:14.468360+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:15:14.468360+0000', 'last_peered': '2024-09-11T01:15:14.468360+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:15:14.468360+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 8223, 'log_dups_size': 0, 'ondisk_log_size': 8223, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33681408, 'num_objects': 8223, 'num_object_clones': 0, 'num_object_copies': 16446, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8223, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8223, 'num_write_kb': 32892, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4151}, {'shards': '0,2', 'objects': 4072}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:16.819 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:15:16.819 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'8018", 'reported_seq': 8140, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:13.770168+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:15:13.770168+0000', 'last_peered': '2024-09-11T01:15:13.770168+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:15:13.770168+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 8018, 'log_dups_size': 0, 'ondisk_log_size': 8018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32825344, 'num_objects': 8014, 'num_object_clones': 0, 'num_object_copies': 16028, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8014, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6319, 'num_write_kb': 25276, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3896}, {'shards': '2', 'objects': 4118}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:16.819 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:15:16.820 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'8151", 'reported_seq': 8314, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:14.467498+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:15:14.467498+0000', 'last_peered': '2024-09-11T01:15:14.467498+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:15:14.467498+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 8151, 'log_dups_size': 0, 'ondisk_log_size': 8151, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33366016, 'num_objects': 8146, 'num_object_clones': 0, 'num_object_copies': 16292, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8146, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4492, 'num_write_kb': 17968, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4195}, {'shards': '0,2', 'objects': 3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:16.820 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:15:16.820 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'8031", 'reported_seq': 8157, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:14.548150+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:15:14.548150+0000', 'last_peered': '2024-09-11T01:15:14.548150+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:15:14.548150+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 8031, 'log_dups_size': 0, 'ondisk_log_size': 8031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32870400, 'num_objects': 8025, 'num_object_clones': 0, 'num_object_copies': 16050, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8025, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4888, 'num_write_kb': 19552, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4039}, {'shards': '0,2', 'objects': 3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:16.820 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:15:16.820 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'8047", 'reported_seq': 8170, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:13.738892+0000', 'last_change': '2024-09-11T01:13:10.159069+0000', 'last_active': '2024-09-11T01:15:13.738892+0000', 'last_peered': '2024-09-11T01:15:13.738892+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:15:13.738892+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 8047, 'log_dups_size': 0, 'ondisk_log_size': 8047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32931840, 'num_objects': 8040, 'num_object_clones': 0, 'num_object_copies': 16080, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8040, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6018, 'num_write_kb': 24072, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3932}, {'shards': '2', 'objects': 4107}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:16.820 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:15:16.820 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'8022", 'reported_seq': 8143, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:14.504240+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:15:14.504240+0000', 'last_peered': '2024-09-11T01:15:14.504240+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:15:14.504240+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 8022, 'log_dups_size': 0, 'ondisk_log_size': 8022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32825344, 'num_objects': 8014, 'num_object_clones': 0, 'num_object_copies': 16028, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8014, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4609, 'num_write_kb': 18436, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4030}, {'shards': '0,2', 'objects': 3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:16.820 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:15:16.820 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'8115", 'reported_seq': 8229, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:13.720223+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:15:13.720223+0000', 'last_peered': '2024-09-11T01:15:13.720223+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:15:13.720223+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 8115, 'log_dups_size': 0, 'ondisk_log_size': 8115, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33218560, 'num_objects': 8110, 'num_object_clones': 0, 'num_object_copies': 16220, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8110, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5932, 'num_write_kb': 23728, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4023}, {'shards': '2', 'objects': 4086}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:16.820 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:15:16.821 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'7981", 'reported_seq': 15265, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:15:13.745381+0000', 'last_change': '2024-09-11T01:13:10.217540+0000', 'last_active': '2024-09-11T01:15:13.745381+0000', 'last_peered': '2024-09-11T01:15:13.745381+0000', 'last_clean': '2024-09-11T01:13:07.638709+0000', 'last_became_active': '2024-09-11T01:13:10.125604+0000', 'last_became_peered': '2024-09-11T01:13:10.125604+0000', 'last_unstale': '2024-09-11T01:15:13.745381+0000', 'last_undegraded': '2024-09-11T01:15:13.745381+0000', 'last_fullsized': '2024-09-11T01:13:09.649604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2269", 'last_scrub_stamp': '2024-09-11T01:12:42.890920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:42.890920+0000', 'objects_scrubbed': 2209, 'log_size': 7981, 'log_dups_size': 0, 'ondisk_log_size': 7981, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 999, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32665600, 'num_objects': 7975, 'num_object_clones': 0, 'num_object_copies': 15950, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 342, 'num_objects_unfound': 0, 'num_objects_dirty': 7975, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5766, 'num_write_kb': 23064, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3585, 'num_bytes_recovered': 14675968, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 342}, {'shards': '2', 'objects': 7633}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:16.821 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:15:16.821 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'8207", 'reported_seq': 8333, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:14.514414+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:15:14.514414+0000', 'last_peered': '2024-09-11T01:15:14.514414+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:15:14.514414+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 8207, 'log_dups_size': 0, 'ondisk_log_size': 8207, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33607680, 'num_objects': 8205, 'num_object_clones': 0, 'num_object_copies': 16410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8205, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4944, 'num_write_kb': 19776, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4255}, {'shards': '0,2', 'objects': 3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:16.821 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:15:16.821 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'8021", 'reported_seq': 8153, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:14.528198+0000', 'last_change': '2024-09-11T01:13:10.209668+0000', 'last_active': '2024-09-11T01:15:14.528198+0000', 'last_peered': '2024-09-11T01:15:14.528198+0000', 'last_clean': '2024-09-11T01:13:07.624565+0000', 'last_became_active': '2024-09-11T01:13:10.199795+0000', 'last_became_peered': '2024-09-11T01:13:10.199795+0000', 'last_unstale': '2024-09-11T01:15:14.528198+0000', 'last_undegraded': '2024-09-11T01:13:09.791198+0000', 'last_fullsized': '2024-09-11T01:13:09.652311+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3415", 'last_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'objects_scrubbed': 3337, 'log_size': 8021, 'log_dups_size': 0, 'ondisk_log_size': 8021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1975, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32821248, 'num_objects': 8013, 'num_object_clones': 0, 'num_object_copies': 16026, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3942, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8013, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4676, 'num_write_kb': 18704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4074}, {'shards': '0,2', 'objects': 3939}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:16.821 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:15:16.821 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'8162", 'reported_seq': 8292, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:14.486020+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:15:14.486020+0000', 'last_peered': '2024-09-11T01:15:14.486020+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:15:14.486020+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 8162, 'log_dups_size': 0, 'ondisk_log_size': 8162, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33415168, 'num_objects': 8158, 'num_object_clones': 0, 'num_object_copies': 16316, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8158, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4771, 'num_write_kb': 19084, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4119}, {'shards': '0,2', 'objects': 4039}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:16.821 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:15:16.821 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'8106", 'reported_seq': 8235, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:14.500397+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:15:14.500397+0000', 'last_peered': '2024-09-11T01:15:14.500397+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:15:14.500397+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 8106, 'log_dups_size': 0, 'ondisk_log_size': 8106, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33153641, 'num_objects': 8095, 'num_object_clones': 0, 'num_object_copies': 16190, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8095, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5176, 'num_write_kb': 20704, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4178}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:16.822 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:15:17.410 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 173 20 130992 130972 2.9561 1.71094 0.0656147 0.0264208 2024-09-11T01:15:17.606 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:15:17.606 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43711/259076 objects degraded (16.872%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":254,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":43,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":12},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":129538,"data_bytes":530575996,"bytes_used":1745354752,"bytes_avail":288164937728,"bytes_total":289910292480,"degraded_objects":43711,"degraded_total":259076,"degraded_ratio":0.16871883153977982,"misplaced_objects":342,"misplaced_total":259076,"misplaced_ratio":0.0013200759622658989,"recovering_objects_per_sec":40,"recovering_bytes_per_sec":164287,"recovering_keys_per_sec":0,"num_objects_recovered":482,"num_bytes_recovered":1974272,"num_keys_recovered":0,"write_bytes_sec":1619355,"read_op_per_sec":0,"write_op_per_sec":395},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:15:17.701 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:15:17.701 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:18.410 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 174 20 131452 131432 2.94944 1.79688 0.0610684 0.0264801 2024-09-11T01:15:18.687 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:15:18.688 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:15:18.768 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":164,"stamp":"2024-09-11T01:15:18.613302+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":530575996,"num_objects":129538,"num_object_clones":0,"num_object_copies":259076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":342,"num_objects_unfound":0,"num_objects_dirty":129538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90469,"num_write_kb":361870,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3587,"num_bytes_recovered":14680083,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129620,"ondisk_log_size":129620,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1739240,"kb_used_data":1054328,"kb_used_omap":12,"kb_used_meta":684851,"kb_avail":281376280,"statfs":{"total":289910292480,"available":288129310720,"internally_reserved":0,"allocated":1079631872,"data_stored":1078819912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":701287428},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,5,8,22,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19460096,"num_objects":4751,"num_object_clones":0,"num_object_copies":9502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-482,"num_objects_unfound":0,"num_objects_dirty":4751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4751,"num_write_kb":19004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":482,"num_bytes_recovered":1974272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.020702"},"pg_stats":[{"pgid":"2.f","version":"43'8223","reported_seq":8294,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:14.468360+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:15:14.468360+0000","last_peered":"2024-09-11T01:15:14.468360+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:15:14.468360+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":8223,"log_dups_size":0,"ondisk_log_size":8223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33681408,"num_objects":8223,"num_object_clones":0,"num_object_copies":16446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8223,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8223,"num_write_kb":32892,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4151},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8172","reported_seq":8269,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:15:13.688323+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:15:13.688323+0000","last_peered":"2024-09-11T01:15:13.688323+0000","last_clean":"2024-09-11T01:15:13.688323+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:15:13.688323+0000","last_undegraded":"2024-09-11T01:15:13.688323+0000","last_fullsized":"2024-09-11T01:15:13.688323+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":8172,"log_dups_size":0,"ondisk_log_size":8172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33447936,"num_objects":8166,"num_object_clones":0,"num_object_copies":16332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8166,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6411,"num_write_kb":25644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8018","reported_seq":8140,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:13.770168+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:15:13.770168+0000","last_peered":"2024-09-11T01:15:13.770168+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:15:13.770168+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":8018,"log_dups_size":0,"ondisk_log_size":8018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32825344,"num_objects":8014,"num_object_clones":0,"num_object_copies":16028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6319,"num_write_kb":25276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":4118}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8088","reported_seq":8152,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:15:13.738590+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:15:13.738590+0000","last_peered":"2024-09-11T01:15:13.738590+0000","last_clean":"2024-09-11T01:15:13.738590+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:15:13.738590+0000","last_undegraded":"2024-09-11T01:15:13.738590+0000","last_fullsized":"2024-09-11T01:15:13.738590+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":8088,"log_dups_size":0,"ondisk_log_size":8088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33128448,"num_objects":8088,"num_object_clones":0,"num_object_copies":16176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8088,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8088,"num_write_kb":32352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8102","reported_seq":8225,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:15:14.526659+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:15:14.526659+0000","last_peered":"2024-09-11T01:15:14.526659+0000","last_clean":"2024-09-11T01:15:14.526659+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:15:14.526659+0000","last_undegraded":"2024-09-11T01:15:14.526659+0000","last_fullsized":"2024-09-11T01:15:14.526659+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":8102,"log_dups_size":0,"ondisk_log_size":8102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33157120,"num_objects":8095,"num_object_clones":0,"num_object_copies":16190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8095,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5107,"num_write_kb":20428,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8151","reported_seq":8314,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:14.467498+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:15:14.467498+0000","last_peered":"2024-09-11T01:15:14.467498+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:15:14.467498+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":8151,"log_dups_size":0,"ondisk_log_size":8151,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33366016,"num_objects":8146,"num_object_clones":0,"num_object_copies":16292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8146,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4492,"num_write_kb":17968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4195},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8031","reported_seq":8157,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:14.548150+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:15:14.548150+0000","last_peered":"2024-09-11T01:15:14.548150+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:15:14.548150+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":8031,"log_dups_size":0,"ondisk_log_size":8031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32870400,"num_objects":8025,"num_object_clones":0,"num_object_copies":16050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8025,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4888,"num_write_kb":19552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4039},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8171","reported_seq":8296,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:15:14.539557+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:15:14.539557+0000","last_peered":"2024-09-11T01:15:14.539557+0000","last_clean":"2024-09-11T01:15:14.539557+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:15:14.539557+0000","last_undegraded":"2024-09-11T01:15:14.539557+0000","last_fullsized":"2024-09-11T01:15:14.539557+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":8171,"log_dups_size":0,"ondisk_log_size":8171,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33460224,"num_objects":8169,"num_object_clones":0,"num_object_copies":16338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5047,"num_write_kb":20188,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.657802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.657802+0000","last_peered":"2024-09-11T01:13:09.657802+0000","last_clean":"2024-09-11T01:13:09.657802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.657802+0000","last_undegraded":"2024-09-11T01:13:09.657802+0000","last_fullsized":"2024-09-11T01:13:09.657802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8047","reported_seq":8170,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:13.738892+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:15:13.738892+0000","last_peered":"2024-09-11T01:15:13.738892+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:15:13.738892+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":8047,"log_dups_size":0,"ondisk_log_size":8047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32931840,"num_objects":8040,"num_object_clones":0,"num_object_copies":16080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6018,"num_write_kb":24072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":4107}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741922+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741922+0000","last_peered":"2024-09-11T01:13:09.741922+0000","last_clean":"2024-09-11T01:13:09.741922+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741922+0000","last_undegraded":"2024-09-11T01:13:09.741922+0000","last_fullsized":"2024-09-11T01:13:09.741922+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8022","reported_seq":8143,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:14.504240+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:15:14.504240+0000","last_peered":"2024-09-11T01:15:14.504240+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:15:14.504240+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":8022,"log_dups_size":0,"ondisk_log_size":8022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32825344,"num_objects":8014,"num_object_clones":0,"num_object_copies":16028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4609,"num_write_kb":18436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4030},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.910012+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.910012+0000","last_peered":"2024-09-11T01:13:09.910012+0000","last_clean":"2024-09-11T01:13:09.910012+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.910012+0000","last_undegraded":"2024-09-11T01:13:09.910012+0000","last_fullsized":"2024-09-11T01:13:09.910012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8115","reported_seq":8229,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:13.720223+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:15:13.720223+0000","last_peered":"2024-09-11T01:15:13.720223+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:15:13.720223+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":8115,"log_dups_size":0,"ondisk_log_size":8115,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33218560,"num_objects":8110,"num_object_clones":0,"num_object_copies":16220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5932,"num_write_kb":23728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":4086}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.769458+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.769458+0000","last_peered":"2024-09-11T01:13:09.769458+0000","last_clean":"2024-09-11T01:13:09.769458+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.769458+0000","last_undegraded":"2024-09-11T01:13:09.769458+0000","last_fullsized":"2024-09-11T01:13:09.769458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'7981","reported_seq":15265,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:13.745381+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:15:13.745381+0000","last_peered":"2024-09-11T01:15:13.745381+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:15:13.745381+0000","last_undegraded":"2024-09-11T01:15:13.745381+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":7981,"log_dups_size":0,"ondisk_log_size":7981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32665600,"num_objects":7975,"num_object_clones":0,"num_object_copies":15950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":342,"num_objects_unfound":0,"num_objects_dirty":7975,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5766,"num_write_kb":23064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3585,"num_bytes_recovered":14675968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":342},{"shards":"2","objects":7633}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8207","reported_seq":8333,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:14.514414+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:15:14.514414+0000","last_peered":"2024-09-11T01:15:14.514414+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:15:14.514414+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":8207,"log_dups_size":0,"ondisk_log_size":8207,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33607680,"num_objects":8205,"num_object_clones":0,"num_object_copies":16410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4944,"num_write_kb":19776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4255},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:10.221233+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:10.221233+0000","last_peered":"2024-09-11T01:13:10.221233+0000","last_clean":"2024-09-11T01:13:10.221233+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:10.221233+0000","last_undegraded":"2024-09-11T01:13:10.221233+0000","last_fullsized":"2024-09-11T01:13:10.221233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8021","reported_seq":8153,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:14.528198+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:15:14.528198+0000","last_peered":"2024-09-11T01:15:14.528198+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:15:14.528198+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":8021,"log_dups_size":0,"ondisk_log_size":8021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32821248,"num_objects":8013,"num_object_clones":0,"num_object_copies":16026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8013,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4676,"num_write_kb":18704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4074},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658174+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658174+0000","last_peered":"2024-09-11T01:13:09.658174+0000","last_clean":"2024-09-11T01:13:09.658174+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658174+0000","last_undegraded":"2024-09-11T01:13:09.658174+0000","last_fullsized":"2024-09-11T01:13:09.658174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8162","reported_seq":8292,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:14.486020+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:15:14.486020+0000","last_peered":"2024-09-11T01:15:14.486020+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:15:14.486020+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":8162,"log_dups_size":0,"ondisk_log_size":8162,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33415168,"num_objects":8158,"num_object_clones":0,"num_object_copies":16316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4771,"num_write_kb":19084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4119},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.916946+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.916946+0000","last_peered":"2024-09-11T01:13:09.916946+0000","last_clean":"2024-09-11T01:13:09.916946+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.916946+0000","last_undegraded":"2024-09-11T01:13:09.916946+0000","last_fullsized":"2024-09-11T01:13:09.916946+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.664190+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.664190+0000","last_peered":"2024-09-11T01:13:09.664190+0000","last_clean":"2024-09-11T01:13:09.664190+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.664190+0000","last_undegraded":"2024-09-11T01:13:09.664190+0000","last_fullsized":"2024-09-11T01:13:09.664190+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8106","reported_seq":8235,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:14.500397+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:15:14.500397+0000","last_peered":"2024-09-11T01:15:14.500397+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:15:14.500397+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":8106,"log_dups_size":0,"ondisk_log_size":8106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33153641,"num_objects":8095,"num_object_clones":0,"num_object_copies":16190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8095,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5176,"num_write_kb":20704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4178},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":530575977,"num_objects":129536,"num_object_clones":0,"num_object_copies":259072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":342,"num_objects_unfound":0,"num_objects_dirty":129536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":90467,"num_write_kb":361868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3586,"num_bytes_recovered":14680064,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1082085376,"data_stored":1082005359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":129617,"ondisk_log_size":129617,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182445,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":732348,"kb_used_data":452204,"kb_used_omap":4,"kb_used_meta":280123,"kb_avail":93639492,"statfs":{"total":96636764160,"available":95886839808,"internally_reserved":0,"allocated":463056896,"data_stored":462808077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":286846635},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,5,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986217,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":587724,"kb_used_data":385340,"kb_used_omap":4,"kb_used_meta":202363,"kb_avail":93784116,"statfs":{"total":96636764160,"available":96034934784,"internally_reserved":0,"allocated":394588160,"data_stored":394343418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":207220394},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,1,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659163,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":419168,"kb_used_data":216784,"kb_used_omap":4,"kb_used_meta":202363,"kb_avail":93952672,"statfs":{"total":96636764160,"available":96207536128,"internally_reserved":0,"allocated":221986816,"data_stored":221668417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":207220399},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,2,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":463749120,"data_stored":463745641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":223272960,"data_stored":223199901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":395063296,"data_stored":395059817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:19.410 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 175 20 131926 131906 2.94316 1.85156 0.015439 0.0265349 2024-09-11T01:15:20.411 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 176 20 132410 132390 2.93718 1.89062 0.0146233 0.0265898 2024-09-11T01:15:21.411 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 177 20 132858 132838 2.93048 1.75 0.0082238 0.026625 2024-09-11T01:15:21.770 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:22.400 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:15:22.400 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:15:22.410 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 178 20 133153 133133 2.92049 1.15234 0.0432717 0.0267425 2024-09-11T01:15:22.441 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":167,"stamp":"2024-09-11T01:15:21.682415+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":540078716,"num_objects":131858,"num_object_clones":0,"num_object_copies":263716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":64,"num_objects_unfound":0,"num_objects_dirty":131858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":92789,"num_write_kb":371150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":15818771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131940,"ondisk_log_size":131940,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1797356,"kb_used_data":1066620,"kb_used_omap":12,"kb_used_meta":730675,"kb_avail":281318164,"statfs":{"total":289910292480,"available":288069799936,"internally_reserved":0,"allocated":1092218880,"data_stored":1091406920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":748211204},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,4,6,5,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19095552,"num_objects":4662,"num_object_clones":0,"num_object_copies":9324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-537,"num_objects_unfound":0,"num_objects_dirty":4662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4662,"num_write_kb":18648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":537,"num_bytes_recovered":2199552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.082007"},"pg_stats":[{"pgid":"2.f","version":"43'8371","reported_seq":8442,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:19.554126+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:15:19.554126+0000","last_peered":"2024-09-11T01:15:19.554126+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:15:19.554126+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":8371,"log_dups_size":0,"ondisk_log_size":8371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34287616,"num_objects":8371,"num_object_clones":0,"num_object_copies":16742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8371,"num_write_kb":33484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4299},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8320","reported_seq":8417,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:15:18.684414+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:15:18.684414+0000","last_peered":"2024-09-11T01:15:18.684414+0000","last_clean":"2024-09-11T01:15:18.684414+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:15:18.684414+0000","last_undegraded":"2024-09-11T01:15:18.684414+0000","last_fullsized":"2024-09-11T01:15:18.684414+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":8320,"log_dups_size":0,"ondisk_log_size":8320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34054144,"num_objects":8314,"num_object_clones":0,"num_object_copies":16628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6559,"num_write_kb":26236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8166","reported_seq":8289,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:18.745601+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:15:18.745601+0000","last_peered":"2024-09-11T01:15:18.745601+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:15:18.745601+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":8166,"log_dups_size":0,"ondisk_log_size":8166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33431552,"num_objects":8162,"num_object_clones":0,"num_object_copies":16324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6467,"num_write_kb":25868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":4266}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8227","reported_seq":8292,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:15:18.668703+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:15:18.668703+0000","last_peered":"2024-09-11T01:15:18.668703+0000","last_clean":"2024-09-11T01:15:18.668703+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:15:18.668703+0000","last_undegraded":"2024-09-11T01:15:18.668703+0000","last_fullsized":"2024-09-11T01:15:18.668703+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":8227,"log_dups_size":0,"ondisk_log_size":8227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33697792,"num_objects":8227,"num_object_clones":0,"num_object_copies":16454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8227,"num_write_kb":32908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8244","reported_seq":8367,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.577437+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:15:19.577437+0000","last_peered":"2024-09-11T01:15:19.577437+0000","last_clean":"2024-09-11T01:15:19.577437+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:15:19.577437+0000","last_undegraded":"2024-09-11T01:15:19.577437+0000","last_fullsized":"2024-09-11T01:15:19.577437+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":8244,"log_dups_size":0,"ondisk_log_size":8244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33738752,"num_objects":8237,"num_object_clones":0,"num_object_copies":16474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5249,"num_write_kb":20996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8307","reported_seq":8473,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:19.419836+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:15:19.419836+0000","last_peered":"2024-09-11T01:15:19.419836+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:15:19.419836+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":8307,"log_dups_size":0,"ondisk_log_size":8307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34004992,"num_objects":8302,"num_object_clones":0,"num_object_copies":16604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4648,"num_write_kb":18592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4351},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8164","reported_seq":8290,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:19.571759+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:15:19.571759+0000","last_peered":"2024-09-11T01:15:19.571759+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:15:19.571759+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":8164,"log_dups_size":0,"ondisk_log_size":8164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33415168,"num_objects":8158,"num_object_clones":0,"num_object_copies":16316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5021,"num_write_kb":20084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4172},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8302","reported_seq":8427,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.491332+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:15:19.491332+0000","last_peered":"2024-09-11T01:15:19.491332+0000","last_clean":"2024-09-11T01:15:19.491332+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:15:19.491332+0000","last_undegraded":"2024-09-11T01:15:19.491332+0000","last_fullsized":"2024-09-11T01:15:19.491332+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":8302,"log_dups_size":0,"ondisk_log_size":8302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33996800,"num_objects":8300,"num_object_clones":0,"num_object_copies":16600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5178,"num_write_kb":20712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.657802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.657802+0000","last_peered":"2024-09-11T01:13:09.657802+0000","last_clean":"2024-09-11T01:13:09.657802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.657802+0000","last_undegraded":"2024-09-11T01:13:09.657802+0000","last_fullsized":"2024-09-11T01:13:09.657802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8202","reported_seq":8323,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:18.726495+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:15:18.726495+0000","last_peered":"2024-09-11T01:15:18.726495+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:15:18.726495+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":8202,"log_dups_size":0,"ondisk_log_size":8202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33566720,"num_objects":8195,"num_object_clones":0,"num_object_copies":16390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6173,"num_write_kb":24692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":4262}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741922+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741922+0000","last_peered":"2024-09-11T01:13:09.741922+0000","last_clean":"2024-09-11T01:13:09.741922+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741922+0000","last_undegraded":"2024-09-11T01:13:09.741922+0000","last_fullsized":"2024-09-11T01:13:09.741922+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8177","reported_seq":8300,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:19.549267+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:15:19.549267+0000","last_peered":"2024-09-11T01:15:19.549267+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:15:19.549267+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":8177,"log_dups_size":0,"ondisk_log_size":8177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33460224,"num_objects":8169,"num_object_clones":0,"num_object_copies":16338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4764,"num_write_kb":19056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4185},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.910012+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.910012+0000","last_peered":"2024-09-11T01:13:09.910012+0000","last_clean":"2024-09-11T01:13:09.910012+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.910012+0000","last_undegraded":"2024-09-11T01:13:09.910012+0000","last_fullsized":"2024-09-11T01:13:09.910012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8278","reported_seq":8392,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:18.651930+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:15:18.651930+0000","last_peered":"2024-09-11T01:15:18.651930+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:15:18.651930+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":8278,"log_dups_size":0,"ondisk_log_size":8278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33886208,"num_objects":8273,"num_object_clones":0,"num_object_copies":16546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6095,"num_write_kb":24380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":4249}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.769458+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.769458+0000","last_peered":"2024-09-11T01:13:09.769458+0000","last_clean":"2024-09-11T01:13:09.769458+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.769458+0000","last_undegraded":"2024-09-11T01:13:09.769458+0000","last_fullsized":"2024-09-11T01:13:09.769458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8130","reported_seq":15971,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:18.792456+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:15:18.792456+0000","last_peered":"2024-09-11T01:15:18.792456+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:15:18.792456+0000","last_undegraded":"2024-09-11T01:15:18.792456+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":8130,"log_dups_size":0,"ondisk_log_size":8130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33275904,"num_objects":8124,"num_object_clones":0,"num_object_copies":16248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":64,"num_objects_unfound":0,"num_objects_dirty":8124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5915,"num_write_kb":23660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3863,"num_bytes_recovered":15814656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":64},{"shards":"2","objects":8060}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8349","reported_seq":8476,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:19.565539+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:15:19.565539+0000","last_peered":"2024-09-11T01:15:19.565539+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:15:19.565539+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":8349,"log_dups_size":0,"ondisk_log_size":8349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34189312,"num_objects":8347,"num_object_clones":0,"num_object_copies":16694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5086,"num_write_kb":20344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4397},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:10.221233+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:10.221233+0000","last_peered":"2024-09-11T01:13:10.221233+0000","last_clean":"2024-09-11T01:13:10.221233+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:10.221233+0000","last_undegraded":"2024-09-11T01:13:10.221233+0000","last_fullsized":"2024-09-11T01:13:10.221233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8155","reported_seq":8287,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:19.430546+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:15:19.430546+0000","last_peered":"2024-09-11T01:15:19.430546+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:15:19.430546+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":8155,"log_dups_size":0,"ondisk_log_size":8155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33370112,"num_objects":8147,"num_object_clones":0,"num_object_copies":16294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4810,"num_write_kb":19240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4208},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658174+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658174+0000","last_peered":"2024-09-11T01:13:09.658174+0000","last_clean":"2024-09-11T01:13:09.658174+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658174+0000","last_undegraded":"2024-09-11T01:13:09.658174+0000","last_fullsized":"2024-09-11T01:13:09.658174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8304","reported_seq":8432,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:19.566206+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:15:19.566206+0000","last_peered":"2024-09-11T01:15:19.566206+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:15:19.566206+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":8304,"log_dups_size":0,"ondisk_log_size":8304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33996800,"num_objects":8300,"num_object_clones":0,"num_object_copies":16600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4913,"num_write_kb":19652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4261},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.916946+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.916946+0000","last_peered":"2024-09-11T01:13:09.916946+0000","last_clean":"2024-09-11T01:13:09.916946+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.916946+0000","last_undegraded":"2024-09-11T01:13:09.916946+0000","last_fullsized":"2024-09-11T01:13:09.916946+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.664190+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.664190+0000","last_peered":"2024-09-11T01:13:09.664190+0000","last_clean":"2024-09-11T01:13:09.664190+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.664190+0000","last_undegraded":"2024-09-11T01:13:09.664190+0000","last_fullsized":"2024-09-11T01:13:09.664190+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8241","reported_seq":8370,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:19.568550+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:15:19.568550+0000","last_peered":"2024-09-11T01:15:19.568550+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:15:19.568550+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":8241,"log_dups_size":0,"ondisk_log_size":8241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33706601,"num_objects":8230,"num_object_clones":0,"num_object_copies":16460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5311,"num_write_kb":21244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4313},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":540078697,"num_objects":131856,"num_object_clones":0,"num_object_copies":263712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":64,"num_objects_unfound":0,"num_objects_dirty":131856,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":92787,"num_write_kb":371148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3864,"num_bytes_recovered":15818752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1093922816,"data_stored":1093842799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131937,"ondisk_log_size":131937,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182446,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":743332,"kb_used_data":459028,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93628508,"statfs":{"total":96636764160,"available":95875592192,"internally_reserved":0,"allocated":470044672,"data_stored":469795853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":291106475},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,5,3,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986218,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":634856,"kb_used_data":390808,"kb_used_omap":4,"kb_used_meta":244027,"kb_avail":93736984,"statfs":{"total":96636764160,"available":95986671616,"internally_reserved":0,"allocated":400187392,"data_stored":399942650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":249884330},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659163,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":419168,"kb_used_data":216784,"kb_used_omap":4,"kb_used_meta":202363,"kb_avail":93952672,"statfs":{"total":96636764160,"available":96207536128,"internally_reserved":0,"allocated":221986816,"data_stored":221668417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":207220399},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,2,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":470757376,"data_stored":470753897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":223272960,"data_stored":223199901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":399892480,"data_stored":399889001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:22.442 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:23.181 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:15:23.183 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:15:23.239 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":168,"stamp":"2024-09-11T01:15:22.614289+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":540078716,"num_objects":131858,"num_object_clones":0,"num_object_copies":263716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":64,"num_objects_unfound":0,"num_objects_dirty":131858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":92789,"num_write_kb":371150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3865,"num_bytes_recovered":15818771,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131940,"ondisk_log_size":131940,"up":48,"acting":36,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1797356,"kb_used_data":1066620,"kb_used_omap":12,"kb_used_meta":730675,"kb_avail":281318164,"statfs":{"total":289910292480,"available":288069799936,"internally_reserved":0,"allocated":1092218880,"data_stored":1091406920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":748211204},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,4,6,5,11,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":9502720,"num_objects":2320,"num_object_clones":0,"num_object_copies":4640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-278,"num_objects_unfound":0,"num_objects_dirty":2320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2320,"num_write_kb":9280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":1138688,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.009636"},"pg_stats":[{"pgid":"2.f","version":"43'8371","reported_seq":8442,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:19.554126+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:15:19.554126+0000","last_peered":"2024-09-11T01:15:19.554126+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:15:19.554126+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":8371,"log_dups_size":0,"ondisk_log_size":8371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34287616,"num_objects":8371,"num_object_clones":0,"num_object_copies":16742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8371,"num_write_kb":33484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4299},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"43'8320","reported_seq":8417,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:15:18.684414+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:15:18.684414+0000","last_peered":"2024-09-11T01:15:18.684414+0000","last_clean":"2024-09-11T01:15:18.684414+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:15:18.684414+0000","last_undegraded":"2024-09-11T01:15:18.684414+0000","last_fullsized":"2024-09-11T01:15:18.684414+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":8320,"log_dups_size":0,"ondisk_log_size":8320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34054144,"num_objects":8314,"num_object_clones":0,"num_object_copies":16628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6559,"num_write_kb":26236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"43'8166","reported_seq":8289,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:18.745601+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:15:18.745601+0000","last_peered":"2024-09-11T01:15:18.745601+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:15:18.745601+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":8166,"log_dups_size":0,"ondisk_log_size":8166,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33431552,"num_objects":8162,"num_object_clones":0,"num_object_copies":16324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8162,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6467,"num_write_kb":25868,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":4266}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"43'8227","reported_seq":8292,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:15:18.668703+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:15:18.668703+0000","last_peered":"2024-09-11T01:15:18.668703+0000","last_clean":"2024-09-11T01:15:18.668703+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:15:18.668703+0000","last_undegraded":"2024-09-11T01:15:18.668703+0000","last_fullsized":"2024-09-11T01:15:18.668703+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":8227,"log_dups_size":0,"ondisk_log_size":8227,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33697792,"num_objects":8227,"num_object_clones":0,"num_object_copies":16454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8227,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8227,"num_write_kb":32908,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8244","reported_seq":8367,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.577437+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:15:19.577437+0000","last_peered":"2024-09-11T01:15:19.577437+0000","last_clean":"2024-09-11T01:15:19.577437+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:15:19.577437+0000","last_undegraded":"2024-09-11T01:15:19.577437+0000","last_fullsized":"2024-09-11T01:15:19.577437+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":8244,"log_dups_size":0,"ondisk_log_size":8244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33738752,"num_objects":8237,"num_object_clones":0,"num_object_copies":16474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5249,"num_write_kb":20996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8307","reported_seq":8473,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:19.419836+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:15:19.419836+0000","last_peered":"2024-09-11T01:15:19.419836+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:15:19.419836+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":8307,"log_dups_size":0,"ondisk_log_size":8307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34004992,"num_objects":8302,"num_object_clones":0,"num_object_copies":16604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4648,"num_write_kb":18592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4351},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8164","reported_seq":8290,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:19.571759+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:15:19.571759+0000","last_peered":"2024-09-11T01:15:19.571759+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:15:19.571759+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":8164,"log_dups_size":0,"ondisk_log_size":8164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33415168,"num_objects":8158,"num_object_clones":0,"num_object_copies":16316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5021,"num_write_kb":20084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4172},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8302","reported_seq":8427,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.491332+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:15:19.491332+0000","last_peered":"2024-09-11T01:15:19.491332+0000","last_clean":"2024-09-11T01:15:19.491332+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:15:19.491332+0000","last_undegraded":"2024-09-11T01:15:19.491332+0000","last_fullsized":"2024-09-11T01:15:19.491332+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":8302,"log_dups_size":0,"ondisk_log_size":8302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33996800,"num_objects":8300,"num_object_clones":0,"num_object_copies":16600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5178,"num_write_kb":20712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.657802+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:13:09.657802+0000","last_peered":"2024-09-11T01:13:09.657802+0000","last_clean":"2024-09-11T01:13:09.657802+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:13:09.657802+0000","last_undegraded":"2024-09-11T01:13:09.657802+0000","last_fullsized":"2024-09-11T01:13:09.657802+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"43'8202","reported_seq":8323,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:18.726495+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:15:18.726495+0000","last_peered":"2024-09-11T01:15:18.726495+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:15:18.726495+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":8202,"log_dups_size":0,"ondisk_log_size":8202,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33566720,"num_objects":8195,"num_object_clones":0,"num_object_copies":16390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6173,"num_write_kb":24692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":4262}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.741922+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:13:09.741922+0000","last_peered":"2024-09-11T01:13:09.741922+0000","last_clean":"2024-09-11T01:13:09.741922+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:13:09.741922+0000","last_undegraded":"2024-09-11T01:13:09.741922+0000","last_fullsized":"2024-09-11T01:13:09.741922+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8177","reported_seq":8300,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:19.549267+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:15:19.549267+0000","last_peered":"2024-09-11T01:15:19.549267+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:15:19.549267+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":8177,"log_dups_size":0,"ondisk_log_size":8177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33460224,"num_objects":8169,"num_object_clones":0,"num_object_copies":16338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4764,"num_write_kb":19056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4185},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.910012+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.910012+0000","last_peered":"2024-09-11T01:13:09.910012+0000","last_clean":"2024-09-11T01:13:09.910012+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.910012+0000","last_undegraded":"2024-09-11T01:13:09.910012+0000","last_fullsized":"2024-09-11T01:13:09.910012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"43'8278","reported_seq":8392,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:18.651930+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:15:18.651930+0000","last_peered":"2024-09-11T01:15:18.651930+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:15:18.651930+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":8278,"log_dups_size":0,"ondisk_log_size":8278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33886208,"num_objects":8273,"num_object_clones":0,"num_object_copies":16546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6095,"num_write_kb":24380,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":4249}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":75,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.769458+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:13:09.769458+0000","last_peered":"2024-09-11T01:13:09.769458+0000","last_clean":"2024-09-11T01:13:09.769458+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:13:09.769458+0000","last_undegraded":"2024-09-11T01:13:09.769458+0000","last_fullsized":"2024-09-11T01:13:09.769458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"43'8130","reported_seq":15971,"reported_epoch":43,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:18.792456+0000","last_change":"2024-09-11T01:13:10.217540+0000","last_active":"2024-09-11T01:15:18.792456+0000","last_peered":"2024-09-11T01:15:18.792456+0000","last_clean":"2024-09-11T01:13:07.638709+0000","last_became_active":"2024-09-11T01:13:10.125604+0000","last_became_peered":"2024-09-11T01:13:10.125604+0000","last_unstale":"2024-09-11T01:15:18.792456+0000","last_undegraded":"2024-09-11T01:15:18.792456+0000","last_fullsized":"2024-09-11T01:13:09.649604+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":8130,"log_dups_size":0,"ondisk_log_size":8130,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33275904,"num_objects":8124,"num_object_clones":0,"num_object_copies":16248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":64,"num_objects_unfound":0,"num_objects_dirty":8124,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5915,"num_write_kb":23660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3863,"num_bytes_recovered":15814656,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":64},{"shards":"2","objects":8060}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8349","reported_seq":8476,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:19.565539+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:15:19.565539+0000","last_peered":"2024-09-11T01:15:19.565539+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:15:19.565539+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":8349,"log_dups_size":0,"ondisk_log_size":8349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34189312,"num_objects":8347,"num_object_clones":0,"num_object_copies":16694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5086,"num_write_kb":20344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4397},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:10.221233+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:13:10.221233+0000","last_peered":"2024-09-11T01:13:10.221233+0000","last_clean":"2024-09-11T01:13:10.221233+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:13:10.221233+0000","last_undegraded":"2024-09-11T01:13:10.221233+0000","last_fullsized":"2024-09-11T01:13:10.221233+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8155","reported_seq":8287,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:19.430546+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:15:19.430546+0000","last_peered":"2024-09-11T01:15:19.430546+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:15:19.430546+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":8155,"log_dups_size":0,"ondisk_log_size":8155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33370112,"num_objects":8147,"num_object_clones":0,"num_object_copies":16294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4810,"num_write_kb":19240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4208},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.658174+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:13:09.658174+0000","last_peered":"2024-09-11T01:13:09.658174+0000","last_clean":"2024-09-11T01:13:09.658174+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:13:09.658174+0000","last_undegraded":"2024-09-11T01:13:09.658174+0000","last_fullsized":"2024-09-11T01:13:09.658174+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8304","reported_seq":8432,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:19.566206+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:15:19.566206+0000","last_peered":"2024-09-11T01:15:19.566206+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:15:19.566206+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":8304,"log_dups_size":0,"ondisk_log_size":8304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33996800,"num_objects":8300,"num_object_clones":0,"num_object_copies":16600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4913,"num_write_kb":19652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4261},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":65,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.916946+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:13:09.916946+0000","last_peered":"2024-09-11T01:13:09.916946+0000","last_clean":"2024-09-11T01:13:09.916946+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:13:09.916946+0000","last_undegraded":"2024-09-11T01:13:09.916946+0000","last_fullsized":"2024-09-11T01:13:09.916946+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":70,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.664190+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:13:09.664190+0000","last_peered":"2024-09-11T01:13:09.664190+0000","last_clean":"2024-09-11T01:13:09.664190+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:13:09.664190+0000","last_undegraded":"2024-09-11T01:13:09.664190+0000","last_fullsized":"2024-09-11T01:13:09.664190+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8241","reported_seq":8370,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:19.568550+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:15:19.568550+0000","last_peered":"2024-09-11T01:15:19.568550+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:15:19.568550+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":8241,"log_dups_size":0,"ondisk_log_size":8241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33706601,"num_objects":8230,"num_object_clones":0,"num_object_copies":16460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5311,"num_write_kb":21244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4313},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":540078697,"num_objects":131856,"num_object_clones":0,"num_object_copies":263712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":64,"num_objects_unfound":0,"num_objects_dirty":131856,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":92787,"num_write_kb":371148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3864,"num_bytes_recovered":15818752,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1093922816,"data_stored":1093842799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":131937,"ondisk_log_size":131937,"up":32,"acting":20,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182446,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":743332,"kb_used_data":459028,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93628508,"statfs":{"total":96636764160,"available":95875592192,"internally_reserved":0,"allocated":470044672,"data_stored":469795853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":291106475},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,5,3,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986218,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":634856,"kb_used_data":390808,"kb_used_omap":4,"kb_used_meta":244027,"kb_avail":93736984,"statfs":{"total":96636764160,"available":95986671616,"internally_reserved":0,"allocated":400187392,"data_stored":399942650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":249884330},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,2],"upper_bound":16},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659163,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":419168,"kb_used_data":216784,"kb_used_omap":4,"kb_used_meta":202363,"kb_avail":93952672,"statfs":{"total":96636764160,"available":96207536128,"internally_reserved":0,"allocated":221986816,"data_stored":221668417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":207220399},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,2,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":470757376,"data_stored":470753897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":223272960,"data_stored":223199901,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":399892480,"data_stored":399889001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:23.241 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:15:23.241 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "43'8371", 'reported_seq': 8442, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:19.554126+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:15:19.554126+0000', 'last_peered': '2024-09-11T01:15:19.554126+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:15:19.554126+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 8371, 'log_dups_size': 0, 'ondisk_log_size': 8371, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34287616, 'num_objects': 8371, 'num_object_clones': 0, 'num_object_copies': 16742, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8371, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8371, 'num_write_kb': 33484, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4299}, {'shards': '0,2', 'objects': 4072}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:23.241 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:15:23.241 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "43'8166", 'reported_seq': 8289, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:18.745601+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:15:18.745601+0000', 'last_peered': '2024-09-11T01:15:18.745601+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:15:18.745601+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 8166, 'log_dups_size': 0, 'ondisk_log_size': 8166, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33431552, 'num_objects': 8162, 'num_object_clones': 0, 'num_object_copies': 16324, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8162, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6467, 'num_write_kb': 25868, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3896}, {'shards': '2', 'objects': 4266}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:23.241 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:15:23.241 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "43'8307", 'reported_seq': 8473, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:19.419836+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:15:19.419836+0000', 'last_peered': '2024-09-11T01:15:19.419836+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:15:19.419836+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 8307, 'log_dups_size': 0, 'ondisk_log_size': 8307, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34004992, 'num_objects': 8302, 'num_object_clones': 0, 'num_object_copies': 16604, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8302, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4648, 'num_write_kb': 18592, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4351}, {'shards': '0,2', 'objects': 3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:23.241 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:15:23.242 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "43'8164", 'reported_seq': 8290, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:19.571759+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:15:19.571759+0000', 'last_peered': '2024-09-11T01:15:19.571759+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:15:19.571759+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 8164, 'log_dups_size': 0, 'ondisk_log_size': 8164, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33415168, 'num_objects': 8158, 'num_object_clones': 0, 'num_object_copies': 16316, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8158, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5021, 'num_write_kb': 20084, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4172}, {'shards': '0,2', 'objects': 3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:23.242 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:15:23.242 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "43'8202", 'reported_seq': 8323, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:18.726495+0000', 'last_change': '2024-09-11T01:13:10.159069+0000', 'last_active': '2024-09-11T01:15:18.726495+0000', 'last_peered': '2024-09-11T01:15:18.726495+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:15:18.726495+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 8202, 'log_dups_size': 0, 'ondisk_log_size': 8202, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33566720, 'num_objects': 8195, 'num_object_clones': 0, 'num_object_copies': 16390, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8195, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6173, 'num_write_kb': 24692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3932}, {'shards': '2', 'objects': 4262}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:23.242 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:15:23.242 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "43'8177", 'reported_seq': 8300, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:19.549267+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:15:19.549267+0000', 'last_peered': '2024-09-11T01:15:19.549267+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:15:19.549267+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 8177, 'log_dups_size': 0, 'ondisk_log_size': 8177, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33460224, 'num_objects': 8169, 'num_object_clones': 0, 'num_object_copies': 16338, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8169, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4764, 'num_write_kb': 19056, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4185}, {'shards': '0,2', 'objects': 3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:23.242 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:15:23.242 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "43'8278", 'reported_seq': 8392, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:18.651930+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:15:18.651930+0000', 'last_peered': '2024-09-11T01:15:18.651930+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:15:18.651930+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 8278, 'log_dups_size': 0, 'ondisk_log_size': 8278, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33886208, 'num_objects': 8273, 'num_object_clones': 0, 'num_object_copies': 16546, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8273, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6095, 'num_write_kb': 24380, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4023}, {'shards': '2', 'objects': 4249}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:23.242 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.0 is not active+clean 2024-09-11T01:15:23.242 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.0', 'version': "43'8130", 'reported_seq': 15971, 'reported_epoch': 43, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:15:18.792456+0000', 'last_change': '2024-09-11T01:13:10.217540+0000', 'last_active': '2024-09-11T01:15:18.792456+0000', 'last_peered': '2024-09-11T01:15:18.792456+0000', 'last_clean': '2024-09-11T01:13:07.638709+0000', 'last_became_active': '2024-09-11T01:13:10.125604+0000', 'last_became_peered': '2024-09-11T01:13:10.125604+0000', 'last_unstale': '2024-09-11T01:15:18.792456+0000', 'last_undegraded': '2024-09-11T01:15:18.792456+0000', 'last_fullsized': '2024-09-11T01:13:09.649604+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2269", 'last_scrub_stamp': '2024-09-11T01:12:42.890920+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:42.890920+0000', 'objects_scrubbed': 2209, 'log_size': 8130, 'log_dups_size': 0, 'ondisk_log_size': 8130, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 999, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33275904, 'num_objects': 8124, 'num_object_clones': 0, 'num_object_copies': 16248, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 64, 'num_objects_unfound': 0, 'num_objects_dirty': 8124, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5915, 'num_write_kb': 23660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3863, 'num_bytes_recovered': 15814656, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 64}, {'shards': '2', 'objects': 8060}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:23.242 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:15:23.242 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "43'8349", 'reported_seq': 8476, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:19.565539+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:15:19.565539+0000', 'last_peered': '2024-09-11T01:15:19.565539+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:15:19.565539+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 8349, 'log_dups_size': 0, 'ondisk_log_size': 8349, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34189312, 'num_objects': 8347, 'num_object_clones': 0, 'num_object_copies': 16694, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8347, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5086, 'num_write_kb': 20344, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4397}, {'shards': '0,2', 'objects': 3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:23.243 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:15:23.243 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "43'8155", 'reported_seq': 8287, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:19.430546+0000', 'last_change': '2024-09-11T01:13:10.209668+0000', 'last_active': '2024-09-11T01:15:19.430546+0000', 'last_peered': '2024-09-11T01:15:19.430546+0000', 'last_clean': '2024-09-11T01:13:07.624565+0000', 'last_became_active': '2024-09-11T01:13:10.199795+0000', 'last_became_peered': '2024-09-11T01:13:10.199795+0000', 'last_unstale': '2024-09-11T01:15:19.430546+0000', 'last_undegraded': '2024-09-11T01:13:09.791198+0000', 'last_fullsized': '2024-09-11T01:13:09.652311+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3415", 'last_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'objects_scrubbed': 3337, 'log_size': 8155, 'log_dups_size': 0, 'ondisk_log_size': 8155, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1975, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33370112, 'num_objects': 8147, 'num_object_clones': 0, 'num_object_copies': 16294, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3942, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8147, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4810, 'num_write_kb': 19240, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4208}, {'shards': '0,2', 'objects': 3939}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:23.243 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:15:23.243 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "43'8304", 'reported_seq': 8432, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:19.566206+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:15:19.566206+0000', 'last_peered': '2024-09-11T01:15:19.566206+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:15:19.566206+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 8304, 'log_dups_size': 0, 'ondisk_log_size': 8304, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33996800, 'num_objects': 8300, 'num_object_clones': 0, 'num_object_copies': 16600, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8300, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4913, 'num_write_kb': 19652, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4261}, {'shards': '0,2', 'objects': 4039}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:23.243 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:15:23.243 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "43'8241", 'reported_seq': 8370, 'reported_epoch': 43, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:19.568550+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:15:19.568550+0000', 'last_peered': '2024-09-11T01:15:19.568550+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:15:19.568550+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 8241, 'log_dups_size': 0, 'ondisk_log_size': 8241, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33706601, 'num_objects': 8230, 'num_object_clones': 0, 'num_object_copies': 16460, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8230, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5311, 'num_write_kb': 21244, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4313}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:23.243 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:15:23.411 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 179 20 133615 133595 2.91426 1.80469 0.105268 0.0268004 2024-09-11T01:15:24.383 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:15:24.383 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 43711/263716 objects degraded (16.575%), 11 pgs degraded, 12 pgs undersized","count":23},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":260,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":12},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":11},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":131858,"data_bytes":540078716,"bytes_used":1840492544,"bytes_avail":288069799936,"bytes_total":289910292480,"degraded_objects":43711,"degraded_total":263716,"degraded_ratio":0.16575027681293514,"misplaced_objects":64,"misplaced_total":263716,"misplaced_ratio":0.00024268531298821455,"recovering_objects_per_sec":34,"recovering_bytes_per_sec":142164,"recovering_keys_per_sec":0,"num_objects_recovered":278,"num_bytes_recovered":1138688,"num_keys_recovered":0,"write_bytes_sec":1186410,"read_op_per_sec":0,"write_op_per_sec":289},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:15:24.411 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout:2024-09-11T01:15:24.411435+0000 min lat: 0.00219431 max lat: 2.69211 avg lat: 0.0268548 2024-09-11T01:15:24.411 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T01:15:24.411 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 180 20 134079 134059 2.90814 1.8125 0.0569645 0.0268548 2024-09-11T01:15:24.449 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:15:24.450 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:25.277 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:15:25.278 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:15:25.329 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2024-09-11T01:15:24.614764+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":543613564,"num_objects":132721,"num_object_clones":0,"num_object_copies":265442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":132721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":93652,"num_write_kb":374602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3929,"num_bytes_recovered":16080915,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132803,"ondisk_log_size":132803,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1806664,"kb_used_data":1080084,"kb_used_omap":12,"kb_used_meta":726515,"kb_avail":281308856,"statfs":{"total":289910292480,"available":288060268544,"internally_reserved":0,"allocated":1106006016,"data_stored":1105173266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":743951364},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,6,5,4,4,4,6,9],"upper_bound":2048},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":13037568,"num_objects":3183,"num_object_clones":0,"num_object_copies":6366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-342,"num_objects_unfound":0,"num_objects_dirty":3183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3183,"num_write_kb":12732,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":342,"num_bytes_recovered":1400832,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.005505"},"pg_stats":[{"pgid":"2.f","version":"43'8371","reported_seq":8442,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:19.554126+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:15:19.554126+0000","last_peered":"2024-09-11T01:15:19.554126+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:15:19.554126+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":8371,"log_dups_size":0,"ondisk_log_size":8371,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34287616,"num_objects":8371,"num_object_clones":0,"num_object_copies":16742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8371,"num_write_kb":33484,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4299},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'8454","reported_seq":8555,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:23.792485+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:15:23.792485+0000","last_peered":"2024-09-11T01:15:23.792485+0000","last_clean":"2024-09-11T01:15:23.792485+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:15:23.792485+0000","last_undegraded":"2024-09-11T01:15:23.792485+0000","last_fullsized":"2024-09-11T01:15:23.792485+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":8454,"log_dups_size":0,"ondisk_log_size":8454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34603008,"num_objects":8448,"num_object_clones":0,"num_object_copies":16896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6693,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'8321","reported_seq":8448,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:23.712138+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:15:23.712138+0000","last_peered":"2024-09-11T01:15:23.712138+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:15:23.712138+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":8321,"log_dups_size":0,"ondisk_log_size":8321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34066432,"num_objects":8317,"num_object_clones":0,"num_object_copies":16634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8317,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6622,"num_write_kb":26488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":4421}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'8380","reported_seq":8445,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:23.800663+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:15:23.800663+0000","last_peered":"2024-09-11T01:15:23.800663+0000","last_clean":"2024-09-11T01:15:23.800663+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:15:23.800663+0000","last_undegraded":"2024-09-11T01:15:23.800663+0000","last_fullsized":"2024-09-11T01:15:23.800663+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":8380,"log_dups_size":0,"ondisk_log_size":8380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34324480,"num_objects":8380,"num_object_clones":0,"num_object_copies":16760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8380,"num_write_kb":33520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"43'8244","reported_seq":8367,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.577437+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:15:19.577437+0000","last_peered":"2024-09-11T01:15:19.577437+0000","last_clean":"2024-09-11T01:15:19.577437+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:15:19.577437+0000","last_undegraded":"2024-09-11T01:15:19.577437+0000","last_fullsized":"2024-09-11T01:15:19.577437+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":8244,"log_dups_size":0,"ondisk_log_size":8244,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33738752,"num_objects":8237,"num_object_clones":0,"num_object_copies":16474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5249,"num_write_kb":20996,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"43'8307","reported_seq":8473,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:19.419836+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:15:19.419836+0000","last_peered":"2024-09-11T01:15:19.419836+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:15:19.419836+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":8307,"log_dups_size":0,"ondisk_log_size":8307,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34004992,"num_objects":8302,"num_object_clones":0,"num_object_copies":16604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4648,"num_write_kb":18592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4351},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"43'8164","reported_seq":8290,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:19.571759+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:15:19.571759+0000","last_peered":"2024-09-11T01:15:19.571759+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:15:19.571759+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":8164,"log_dups_size":0,"ondisk_log_size":8164,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33415168,"num_objects":8158,"num_object_clones":0,"num_object_copies":16316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5021,"num_write_kb":20084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4172},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"43'8302","reported_seq":8427,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:15:19.491332+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:15:19.491332+0000","last_peered":"2024-09-11T01:15:19.491332+0000","last_clean":"2024-09-11T01:15:19.491332+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:15:19.491332+0000","last_undegraded":"2024-09-11T01:15:19.491332+0000","last_fullsized":"2024-09-11T01:15:19.491332+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":8302,"log_dups_size":0,"ondisk_log_size":8302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33996800,"num_objects":8300,"num_object_clones":0,"num_object_copies":16600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5178,"num_write_kb":20712,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8348","reported_seq":8475,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:23.702107+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:15:23.702107+0000","last_peered":"2024-09-11T01:15:23.702107+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:15:23.702107+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":8348,"log_dups_size":0,"ondisk_log_size":8348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34164736,"num_objects":8341,"num_object_clones":0,"num_object_copies":16682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6319,"num_write_kb":25276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":4408}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692287+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692287+0000","last_peered":"2024-09-11T01:15:21.692287+0000","last_clean":"2024-09-11T01:15:21.692287+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692287+0000","last_undegraded":"2024-09-11T01:15:21.692287+0000","last_fullsized":"2024-09-11T01:15:21.692287+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"43'8177","reported_seq":8300,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:19.549267+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:15:19.549267+0000","last_peered":"2024-09-11T01:15:19.549267+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:15:19.549267+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":8177,"log_dups_size":0,"ondisk_log_size":8177,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33460224,"num_objects":8169,"num_object_clones":0,"num_object_copies":16338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4764,"num_write_kb":19056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4185},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":96,"reported_epoch":43,"state":"active+clean","last_fresh":"2024-09-11T01:13:09.910012+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:13:09.910012+0000","last_peered":"2024-09-11T01:13:09.910012+0000","last_clean":"2024-09-11T01:13:09.910012+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:13:09.910012+0000","last_undegraded":"2024-09-11T01:13:09.910012+0000","last_fullsized":"2024-09-11T01:13:09.910012+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'8412","reported_seq":8530,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:23.738773+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:15:23.738773+0000","last_peered":"2024-09-11T01:15:23.738773+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:15:23.738773+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":8412,"log_dups_size":0,"ondisk_log_size":8412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34435072,"num_objects":8407,"num_object_clones":0,"num_object_copies":16814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6229,"num_write_kb":24916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":4383}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8271","reported_seq":16251,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:23.752701+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:15:23.752701+0000","last_peered":"2024-09-11T01:15:23.752701+0000","last_clean":"2024-09-11T01:15:23.752701+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:15:23.752701+0000","last_undegraded":"2024-09-11T01:15:23.752701+0000","last_fullsized":"2024-09-11T01:15:23.752701+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":8271,"log_dups_size":0,"ondisk_log_size":8271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33853440,"num_objects":8265,"num_object_clones":0,"num_object_copies":16530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8265,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6056,"num_write_kb":24224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"43'8349","reported_seq":8476,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:19.565539+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:15:19.565539+0000","last_peered":"2024-09-11T01:15:19.565539+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:15:19.565539+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":8349,"log_dups_size":0,"ondisk_log_size":8349,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34189312,"num_objects":8347,"num_object_clones":0,"num_object_copies":16694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5086,"num_write_kb":20344,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4397},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"43'8155","reported_seq":8287,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:19.430546+0000","last_change":"2024-09-11T01:13:10.209668+0000","last_active":"2024-09-11T01:15:19.430546+0000","last_peered":"2024-09-11T01:15:19.430546+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:15:19.430546+0000","last_undegraded":"2024-09-11T01:13:09.791198+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":8155,"log_dups_size":0,"ondisk_log_size":8155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33370112,"num_objects":8147,"num_object_clones":0,"num_object_copies":16294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3942,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8147,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4810,"num_write_kb":19240,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4208},{"shards":"0,2","objects":3939}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694344+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694344+0000","last_peered":"2024-09-11T01:15:21.694344+0000","last_clean":"2024-09-11T01:15:21.694344+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694344+0000","last_undegraded":"2024-09-11T01:15:21.694344+0000","last_fullsized":"2024-09-11T01:15:21.694344+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"43'8304","reported_seq":8432,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:19.566206+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:15:19.566206+0000","last_peered":"2024-09-11T01:15:19.566206+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:15:19.566206+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":8304,"log_dups_size":0,"ondisk_log_size":8304,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33996800,"num_objects":8300,"num_object_clones":0,"num_object_copies":16600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4913,"num_write_kb":19652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4261},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"43'8241","reported_seq":8370,"reported_epoch":43,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:19.568550+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:15:19.568550+0000","last_peered":"2024-09-11T01:15:19.568550+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:15:19.568550+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":8241,"log_dups_size":0,"ondisk_log_size":8241,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33706601,"num_objects":8230,"num_object_clones":0,"num_object_copies":16460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5311,"num_write_kb":21244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4313},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":543613545,"num_objects":132719,"num_object_clones":0,"num_object_copies":265438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":43711,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":132719,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":93650,"num_write_kb":374600,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3928,"num_bytes_recovered":16080896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1106059264,"data_stored":1105979247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":132800,"ondisk_log_size":132800,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182446,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":743332,"kb_used_data":459028,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93628508,"statfs":{"total":96636764160,"available":95875592192,"internally_reserved":0,"allocated":470044672,"data_stored":469795853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":291106475},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,5,3,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986219,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":643520,"kb_used_data":395312,"kb_used_omap":4,"kb_used_meta":248187,"kb_avail":93728320,"statfs":{"total":96636764160,"available":95977799680,"internally_reserved":0,"allocated":404799488,"data_stored":404544351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":254144170},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1,4,6,9],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659164,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":419812,"kb_used_data":225744,"kb_used_omap":4,"kb_used_meta":194043,"kb_avail":93952028,"statfs":{"total":96636764160,"available":96206876672,"internally_reserved":0,"allocated":231161856,"data_stored":230833062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":198700719},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":470757376,"data_stored":470753897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":230809600,"data_stored":230736541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":404492288,"data_stored":404488809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:25.411 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 181 20 134540 134520 2.90202 1.80078 0.0153044 0.026913 2024-09-11T01:15:26.411 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 182 20 135011 134991 2.89619 1.83984 0.0138913 0.0269672 2024-09-11T01:15:27.413 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 183 20 135499 135479 2.89074 1.90625 0.00821926 0.0270174 2024-09-11T01:15:28.331 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:28.416 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 184 20 135980 135960 2.88521 1.87891 0.0143993 0.027071 2024-09-11T01:15:29.133 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:15:29.133 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:15:29.215 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":171,"stamp":"2024-09-11T01:15:28.615746+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":548926076,"num_objects":134018,"num_object_clones":0,"num_object_copies":268036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":3834,"num_objects_unfound":0,"num_objects_dirty":134018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94949,"num_write_kb":379790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4034,"num_bytes_recovered":16510995,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134100,"ondisk_log_size":134100,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1850944,"kb_used_data":1093840,"kb_used_omap":12,"kb_used_meta":757043,"kb_avail":281264576,"statfs":{"total":289910292480,"available":288014925824,"internally_reserved":0,"allocated":1120092160,"data_stored":1119249015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":775212036},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,3,8,6,5,2,4,6,9],"upper_bound":2048},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8847360,"num_objects":2160,"num_object_clones":0,"num_object_copies":4320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3942,"num_objects_misplaced":3770,"num_objects_unfound":0,"num_objects_dirty":2160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2160,"num_write_kb":8640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":692224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001942"},"pg_stats":[{"pgid":"2.f","version":"45'8506","reported_seq":8583,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:24.600739+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:15:24.600739+0000","last_peered":"2024-09-11T01:15:24.600739+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:15:24.600739+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":8506,"log_dups_size":0,"ondisk_log_size":8506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34840576,"num_objects":8506,"num_object_clones":0,"num_object_copies":17012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8506,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8506,"num_write_kb":34024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4434},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'8454","reported_seq":8555,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:23.792485+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:15:23.792485+0000","last_peered":"2024-09-11T01:15:23.792485+0000","last_clean":"2024-09-11T01:15:23.792485+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:15:23.792485+0000","last_undegraded":"2024-09-11T01:15:23.792485+0000","last_fullsized":"2024-09-11T01:15:23.792485+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":8454,"log_dups_size":0,"ondisk_log_size":8454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34603008,"num_objects":8448,"num_object_clones":0,"num_object_copies":16896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6693,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'8321","reported_seq":8448,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:23.712138+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:15:23.712138+0000","last_peered":"2024-09-11T01:15:23.712138+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:15:23.712138+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":8321,"log_dups_size":0,"ondisk_log_size":8321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34066432,"num_objects":8317,"num_object_clones":0,"num_object_copies":16634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8317,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6622,"num_write_kb":26488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":4421}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'8380","reported_seq":8445,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:23.800663+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:15:23.800663+0000","last_peered":"2024-09-11T01:15:23.800663+0000","last_clean":"2024-09-11T01:15:23.800663+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:15:23.800663+0000","last_undegraded":"2024-09-11T01:15:23.800663+0000","last_fullsized":"2024-09-11T01:15:23.800663+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":8380,"log_dups_size":0,"ondisk_log_size":8380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34324480,"num_objects":8380,"num_object_clones":0,"num_object_copies":16760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8380,"num_write_kb":33520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'8372","reported_seq":8498,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:24.647392+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:15:24.647392+0000","last_peered":"2024-09-11T01:15:24.647392+0000","last_clean":"2024-09-11T01:15:24.647392+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:15:24.647392+0000","last_undegraded":"2024-09-11T01:15:24.647392+0000","last_fullsized":"2024-09-11T01:15:24.647392+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":8372,"log_dups_size":0,"ondisk_log_size":8372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34263040,"num_objects":8365,"num_object_clones":0,"num_object_copies":16730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5377,"num_write_kb":21508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'8421","reported_seq":8591,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:24.460173+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:15:24.460173+0000","last_peered":"2024-09-11T01:15:24.460173+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:15:24.460173+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":8421,"log_dups_size":0,"ondisk_log_size":8421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34471936,"num_objects":8416,"num_object_clones":0,"num_object_copies":16832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4762,"num_write_kb":19048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4465},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'8285","reported_seq":8415,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:24.648330+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:15:24.648330+0000","last_peered":"2024-09-11T01:15:24.648330+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:15:24.648330+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":8285,"log_dups_size":0,"ondisk_log_size":8285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33910784,"num_objects":8279,"num_object_clones":0,"num_object_copies":16558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5142,"num_write_kb":20568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4293},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'8442","reported_seq":8571,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:24.607305+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:15:24.607305+0000","last_peered":"2024-09-11T01:15:24.607305+0000","last_clean":"2024-09-11T01:15:24.607305+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:15:24.607305+0000","last_undegraded":"2024-09-11T01:15:24.607305+0000","last_fullsized":"2024-09-11T01:15:24.607305+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":8442,"log_dups_size":0,"ondisk_log_size":8442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34570240,"num_objects":8440,"num_object_clones":0,"num_object_copies":16880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5318,"num_write_kb":21272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8348","reported_seq":8475,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:23.702107+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:15:23.702107+0000","last_peered":"2024-09-11T01:15:23.702107+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:15:23.702107+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":8348,"log_dups_size":0,"ondisk_log_size":8348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34164736,"num_objects":8341,"num_object_clones":0,"num_object_copies":16682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6319,"num_write_kb":25276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":4408}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692287+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692287+0000","last_peered":"2024-09-11T01:15:21.692287+0000","last_clean":"2024-09-11T01:15:21.692287+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692287+0000","last_undegraded":"2024-09-11T01:15:21.692287+0000","last_fullsized":"2024-09-11T01:15:21.692287+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'8308","reported_seq":8436,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:24.608615+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:15:24.608615+0000","last_peered":"2024-09-11T01:15:24.608615+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:15:24.608615+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":8308,"log_dups_size":0,"ondisk_log_size":8308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33996800,"num_objects":8300,"num_object_clones":0,"num_object_copies":16600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4895,"num_write_kb":19580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4316},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.685123+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.685123+0000","last_peered":"2024-09-11T01:15:21.685123+0000","last_clean":"2024-09-11T01:15:21.685123+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.685123+0000","last_undegraded":"2024-09-11T01:15:21.685123+0000","last_fullsized":"2024-09-11T01:15:21.685123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'8412","reported_seq":8530,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:23.738773+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:15:23.738773+0000","last_peered":"2024-09-11T01:15:23.738773+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:15:23.738773+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":8412,"log_dups_size":0,"ondisk_log_size":8412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34435072,"num_objects":8407,"num_object_clones":0,"num_object_copies":16814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6229,"num_write_kb":24916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":4383}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8271","reported_seq":16251,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:23.752701+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:15:23.752701+0000","last_peered":"2024-09-11T01:15:23.752701+0000","last_clean":"2024-09-11T01:15:23.752701+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:15:23.752701+0000","last_undegraded":"2024-09-11T01:15:23.752701+0000","last_fullsized":"2024-09-11T01:15:23.752701+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":8271,"log_dups_size":0,"ondisk_log_size":8271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33853440,"num_objects":8265,"num_object_clones":0,"num_object_copies":16530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8265,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6056,"num_write_kb":24224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'8474","reported_seq":8605,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:24.602515+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:15:24.602515+0000","last_peered":"2024-09-11T01:15:24.602515+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:15:24.602515+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":8474,"log_dups_size":0,"ondisk_log_size":8474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34701312,"num_objects":8472,"num_object_clones":0,"num_object_copies":16944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5211,"num_write_kb":20844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4522},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'8286","reported_seq":8632,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:24.652759+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:15:24.652759+0000","last_peered":"2024-09-11T01:15:24.652759+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:15:24.652759+0000","last_undegraded":"2024-09-11T01:15:24.652759+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":8286,"log_dups_size":0,"ondisk_log_size":8286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33906688,"num_objects":8278,"num_object_clones":0,"num_object_copies":16556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3834,"num_objects_unfound":0,"num_objects_dirty":8278,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4941,"num_write_kb":19764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":430080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4444},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694344+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694344+0000","last_peered":"2024-09-11T01:15:21.694344+0000","last_clean":"2024-09-11T01:15:21.694344+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694344+0000","last_undegraded":"2024-09-11T01:15:21.694344+0000","last_fullsized":"2024-09-11T01:15:21.694344+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'8425","reported_seq":8558,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:24.630903+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:15:24.630903+0000","last_peered":"2024-09-11T01:15:24.630903+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:15:24.630903+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":8425,"log_dups_size":0,"ondisk_log_size":8425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34492416,"num_objects":8421,"num_object_clones":0,"num_object_copies":16842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5034,"num_write_kb":20136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4382},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'8392","reported_seq":8525,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:24.597266+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:15:24.597266+0000","last_peered":"2024-09-11T01:15:24.597266+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:15:24.597266+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":8392,"log_dups_size":0,"ondisk_log_size":8392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34325097,"num_objects":8381,"num_object_clones":0,"num_object_copies":16762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5462,"num_write_kb":21848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4464},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":548926057,"num_objects":134016,"num_object_clones":0,"num_object_copies":268032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":3834,"num_objects_unfound":0,"num_objects_dirty":134016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94947,"num_write_kb":379788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4033,"num_bytes_recovered":16510976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1120243712,"data_stored":1120163695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134097,"ondisk_log_size":134097,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182447,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":776012,"kb_used_data":465340,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93595828,"statfs":{"total":96636764160,"available":95842127872,"internally_reserved":0,"allocated":476508160,"data_stored":476248946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":318107307},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,1,7,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986219,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":643520,"kb_used_data":395312,"kb_used_omap":4,"kb_used_meta":248187,"kb_avail":93728320,"statfs":{"total":96636764160,"available":95977799680,"internally_reserved":0,"allocated":404799488,"data_stored":404544351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":254144170},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1,4,6,9],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659165,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":431412,"kb_used_data":233188,"kb_used_omap":4,"kb_used_meta":198203,"kb_avail":93940428,"statfs":{"total":96636764160,"available":96194998272,"internally_reserved":0,"allocated":238784512,"data_stored":238455718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":202960559},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":477249536,"data_stored":477246057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":238501888,"data_stored":238428829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":404492288,"data_stored":404488809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:29.216 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:29.416 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 185 20 136425 136405 2.87901 1.73828 0.0802051 0.027127 2024-09-11T01:15:30.106 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:15:30.107 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:15:30.188 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":171,"stamp":"2024-09-11T01:15:28.615746+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":548926076,"num_objects":134018,"num_object_clones":0,"num_object_copies":268036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":3834,"num_objects_unfound":0,"num_objects_dirty":134018,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94949,"num_write_kb":379790,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4034,"num_bytes_recovered":16510995,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134100,"ondisk_log_size":134100,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1850944,"kb_used_data":1093840,"kb_used_omap":12,"kb_used_meta":757043,"kb_avail":281264576,"statfs":{"total":289910292480,"available":288014925824,"internally_reserved":0,"allocated":1120092160,"data_stored":1119249015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":775212036},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,3,8,6,5,2,4,6,9],"upper_bound":2048},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8847360,"num_objects":2160,"num_object_clones":0,"num_object_copies":4320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3942,"num_objects_misplaced":3770,"num_objects_unfound":0,"num_objects_dirty":2160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2160,"num_write_kb":8640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":169,"num_bytes_recovered":692224,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001942"},"pg_stats":[{"pgid":"2.f","version":"45'8506","reported_seq":8583,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:24.600739+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:15:24.600739+0000","last_peered":"2024-09-11T01:15:24.600739+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:15:24.600739+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":8506,"log_dups_size":0,"ondisk_log_size":8506,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34840576,"num_objects":8506,"num_object_clones":0,"num_object_copies":17012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8506,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8506,"num_write_kb":34024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4434},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'8454","reported_seq":8555,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:23.792485+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:15:23.792485+0000","last_peered":"2024-09-11T01:15:23.792485+0000","last_clean":"2024-09-11T01:15:23.792485+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:15:23.792485+0000","last_undegraded":"2024-09-11T01:15:23.792485+0000","last_fullsized":"2024-09-11T01:15:23.792485+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":8454,"log_dups_size":0,"ondisk_log_size":8454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34603008,"num_objects":8448,"num_object_clones":0,"num_object_copies":16896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8448,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6693,"num_write_kb":26772,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'8321","reported_seq":8448,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:23.712138+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:15:23.712138+0000","last_peered":"2024-09-11T01:15:23.712138+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:15:23.712138+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":8321,"log_dups_size":0,"ondisk_log_size":8321,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34066432,"num_objects":8317,"num_object_clones":0,"num_object_copies":16634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8317,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6622,"num_write_kb":26488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":4421}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'8380","reported_seq":8445,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:23.800663+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:15:23.800663+0000","last_peered":"2024-09-11T01:15:23.800663+0000","last_clean":"2024-09-11T01:15:23.800663+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:15:23.800663+0000","last_undegraded":"2024-09-11T01:15:23.800663+0000","last_fullsized":"2024-09-11T01:15:23.800663+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":8380,"log_dups_size":0,"ondisk_log_size":8380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34324480,"num_objects":8380,"num_object_clones":0,"num_object_copies":16760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8380,"num_write_kb":33520,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'8372","reported_seq":8498,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:24.647392+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:15:24.647392+0000","last_peered":"2024-09-11T01:15:24.647392+0000","last_clean":"2024-09-11T01:15:24.647392+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:15:24.647392+0000","last_undegraded":"2024-09-11T01:15:24.647392+0000","last_fullsized":"2024-09-11T01:15:24.647392+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":8372,"log_dups_size":0,"ondisk_log_size":8372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34263040,"num_objects":8365,"num_object_clones":0,"num_object_copies":16730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5377,"num_write_kb":21508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'8421","reported_seq":8591,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:24.460173+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:15:24.460173+0000","last_peered":"2024-09-11T01:15:24.460173+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:15:24.460173+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":8421,"log_dups_size":0,"ondisk_log_size":8421,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34471936,"num_objects":8416,"num_object_clones":0,"num_object_copies":16832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4762,"num_write_kb":19048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4465},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'8285","reported_seq":8415,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:24.648330+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:15:24.648330+0000","last_peered":"2024-09-11T01:15:24.648330+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:15:24.648330+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":8285,"log_dups_size":0,"ondisk_log_size":8285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33910784,"num_objects":8279,"num_object_clones":0,"num_object_copies":16558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8279,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5142,"num_write_kb":20568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4293},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'8442","reported_seq":8571,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:24.607305+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:15:24.607305+0000","last_peered":"2024-09-11T01:15:24.607305+0000","last_clean":"2024-09-11T01:15:24.607305+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:15:24.607305+0000","last_undegraded":"2024-09-11T01:15:24.607305+0000","last_fullsized":"2024-09-11T01:15:24.607305+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":8442,"log_dups_size":0,"ondisk_log_size":8442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34570240,"num_objects":8440,"num_object_clones":0,"num_object_copies":16880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5318,"num_write_kb":21272,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8348","reported_seq":8475,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:23.702107+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:15:23.702107+0000","last_peered":"2024-09-11T01:15:23.702107+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:15:23.702107+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":8348,"log_dups_size":0,"ondisk_log_size":8348,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34164736,"num_objects":8341,"num_object_clones":0,"num_object_copies":16682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6319,"num_write_kb":25276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":4408}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692287+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692287+0000","last_peered":"2024-09-11T01:15:21.692287+0000","last_clean":"2024-09-11T01:15:21.692287+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692287+0000","last_undegraded":"2024-09-11T01:15:21.692287+0000","last_fullsized":"2024-09-11T01:15:21.692287+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'8308","reported_seq":8436,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:24.608615+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:15:24.608615+0000","last_peered":"2024-09-11T01:15:24.608615+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:15:24.608615+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":8308,"log_dups_size":0,"ondisk_log_size":8308,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33996800,"num_objects":8300,"num_object_clones":0,"num_object_copies":16600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4895,"num_write_kb":19580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4316},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.685123+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.685123+0000","last_peered":"2024-09-11T01:15:21.685123+0000","last_clean":"2024-09-11T01:15:21.685123+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.685123+0000","last_undegraded":"2024-09-11T01:15:21.685123+0000","last_fullsized":"2024-09-11T01:15:21.685123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'8412","reported_seq":8530,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:23.738773+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:15:23.738773+0000","last_peered":"2024-09-11T01:15:23.738773+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:15:23.738773+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":8412,"log_dups_size":0,"ondisk_log_size":8412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34435072,"num_objects":8407,"num_object_clones":0,"num_object_copies":16814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6229,"num_write_kb":24916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":4383}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8271","reported_seq":16251,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:23.752701+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:15:23.752701+0000","last_peered":"2024-09-11T01:15:23.752701+0000","last_clean":"2024-09-11T01:15:23.752701+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:15:23.752701+0000","last_undegraded":"2024-09-11T01:15:23.752701+0000","last_fullsized":"2024-09-11T01:15:23.752701+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":8271,"log_dups_size":0,"ondisk_log_size":8271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33853440,"num_objects":8265,"num_object_clones":0,"num_object_copies":16530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8265,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6056,"num_write_kb":24224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'8474","reported_seq":8605,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:24.602515+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:15:24.602515+0000","last_peered":"2024-09-11T01:15:24.602515+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:15:24.602515+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":8474,"log_dups_size":0,"ondisk_log_size":8474,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34701312,"num_objects":8472,"num_object_clones":0,"num_object_copies":16944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5211,"num_write_kb":20844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4522},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'8286","reported_seq":8632,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:24.652759+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:15:24.652759+0000","last_peered":"2024-09-11T01:15:24.652759+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:15:24.652759+0000","last_undegraded":"2024-09-11T01:15:24.652759+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":8286,"log_dups_size":0,"ondisk_log_size":8286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33906688,"num_objects":8278,"num_object_clones":0,"num_object_copies":16556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3834,"num_objects_unfound":0,"num_objects_dirty":8278,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4941,"num_write_kb":19764,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":105,"num_bytes_recovered":430080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4444},{"shards":"0,2","objects":3834}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694344+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694344+0000","last_peered":"2024-09-11T01:15:21.694344+0000","last_clean":"2024-09-11T01:15:21.694344+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694344+0000","last_undegraded":"2024-09-11T01:15:21.694344+0000","last_fullsized":"2024-09-11T01:15:21.694344+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'8425","reported_seq":8558,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:24.630903+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:15:24.630903+0000","last_peered":"2024-09-11T01:15:24.630903+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:15:24.630903+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":8425,"log_dups_size":0,"ondisk_log_size":8425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34492416,"num_objects":8421,"num_object_clones":0,"num_object_copies":16842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5034,"num_write_kb":20136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4382},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'8392","reported_seq":8525,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:24.597266+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:15:24.597266+0000","last_peered":"2024-09-11T01:15:24.597266+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:15:24.597266+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":8392,"log_dups_size":0,"ondisk_log_size":8392,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34325097,"num_objects":8381,"num_object_clones":0,"num_object_copies":16762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5462,"num_write_kb":21848,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4464},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":548926057,"num_objects":134016,"num_object_clones":0,"num_object_copies":268032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":3834,"num_objects_unfound":0,"num_objects_dirty":134016,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":94947,"num_write_kb":379788,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4033,"num_bytes_recovered":16510976,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1120243712,"data_stored":1120163695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134097,"ondisk_log_size":134097,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182447,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":776012,"kb_used_data":465340,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93595828,"statfs":{"total":96636764160,"available":95842127872,"internally_reserved":0,"allocated":476508160,"data_stored":476248946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":318107307},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,1,1,7,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986219,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":643520,"kb_used_data":395312,"kb_used_omap":4,"kb_used_meta":248187,"kb_avail":93728320,"statfs":{"total":96636764160,"available":95977799680,"internally_reserved":0,"allocated":404799488,"data_stored":404544351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":254144170},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1,4,6,9],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659165,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":431412,"kb_used_data":233188,"kb_used_omap":4,"kb_used_meta":198203,"kb_avail":93940428,"statfs":{"total":96636764160,"available":96194998272,"internally_reserved":0,"allocated":238784512,"data_stored":238455718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":202960559},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":477249536,"data_stored":477246057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":238501888,"data_stored":238428829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":404492288,"data_stored":404488809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:30.189 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:15:30.190 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'8506", 'reported_seq': 8583, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:24.600739+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:15:24.600739+0000', 'last_peered': '2024-09-11T01:15:24.600739+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:15:24.600739+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 8506, 'log_dups_size': 0, 'ondisk_log_size': 8506, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34840576, 'num_objects': 8506, 'num_object_clones': 0, 'num_object_copies': 17012, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8506, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8506, 'num_write_kb': 34024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4434}, {'shards': '0,2', 'objects': 4072}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:30.190 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:15:30.190 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'8321", 'reported_seq': 8448, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:23.712138+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:15:23.712138+0000', 'last_peered': '2024-09-11T01:15:23.712138+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:15:23.712138+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 8321, 'log_dups_size': 0, 'ondisk_log_size': 8321, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34066432, 'num_objects': 8317, 'num_object_clones': 0, 'num_object_copies': 16634, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8317, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6622, 'num_write_kb': 26488, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3896}, {'shards': '2', 'objects': 4421}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:30.190 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:15:30.190 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'8421", 'reported_seq': 8591, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:24.460173+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:15:24.460173+0000', 'last_peered': '2024-09-11T01:15:24.460173+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:15:24.460173+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 8421, 'log_dups_size': 0, 'ondisk_log_size': 8421, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34471936, 'num_objects': 8416, 'num_object_clones': 0, 'num_object_copies': 16832, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8416, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4762, 'num_write_kb': 19048, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4465}, {'shards': '0,2', 'objects': 3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:30.190 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:15:30.190 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'8285", 'reported_seq': 8415, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:24.648330+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:15:24.648330+0000', 'last_peered': '2024-09-11T01:15:24.648330+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:15:24.648330+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 8285, 'log_dups_size': 0, 'ondisk_log_size': 8285, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33910784, 'num_objects': 8279, 'num_object_clones': 0, 'num_object_copies': 16558, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8279, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5142, 'num_write_kb': 20568, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4293}, {'shards': '0,2', 'objects': 3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:30.191 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:15:30.191 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'8348", 'reported_seq': 8475, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:23.702107+0000', 'last_change': '2024-09-11T01:13:10.159069+0000', 'last_active': '2024-09-11T01:15:23.702107+0000', 'last_peered': '2024-09-11T01:15:23.702107+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:15:23.702107+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 8348, 'log_dups_size': 0, 'ondisk_log_size': 8348, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34164736, 'num_objects': 8341, 'num_object_clones': 0, 'num_object_copies': 16682, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8341, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6319, 'num_write_kb': 25276, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3932}, {'shards': '2', 'objects': 4408}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:30.191 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:15:30.191 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'8308", 'reported_seq': 8436, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:24.608615+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:15:24.608615+0000', 'last_peered': '2024-09-11T01:15:24.608615+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:15:24.608615+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 8308, 'log_dups_size': 0, 'ondisk_log_size': 8308, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33996800, 'num_objects': 8300, 'num_object_clones': 0, 'num_object_copies': 16600, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8300, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4895, 'num_write_kb': 19580, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4316}, {'shards': '0,2', 'objects': 3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:30.191 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:15:30.191 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'8412", 'reported_seq': 8530, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:23.738773+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:15:23.738773+0000', 'last_peered': '2024-09-11T01:15:23.738773+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:15:23.738773+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 8412, 'log_dups_size': 0, 'ondisk_log_size': 8412, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34435072, 'num_objects': 8407, 'num_object_clones': 0, 'num_object_copies': 16814, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8407, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6229, 'num_write_kb': 24916, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4023}, {'shards': '2', 'objects': 4383}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:30.191 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:15:30.191 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'8474", 'reported_seq': 8605, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:24.602515+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:15:24.602515+0000', 'last_peered': '2024-09-11T01:15:24.602515+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:15:24.602515+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 8474, 'log_dups_size': 0, 'ondisk_log_size': 8474, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34701312, 'num_objects': 8472, 'num_object_clones': 0, 'num_object_copies': 16944, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8472, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5211, 'num_write_kb': 20844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4522}, {'shards': '0,2', 'objects': 3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:30.192 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:15:30.192 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'8286", 'reported_seq': 8632, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:15:24.652759+0000', 'last_change': '2024-09-11T01:15:19.913288+0000', 'last_active': '2024-09-11T01:15:24.652759+0000', 'last_peered': '2024-09-11T01:15:24.652759+0000', 'last_clean': '2024-09-11T01:13:07.624565+0000', 'last_became_active': '2024-09-11T01:13:10.199795+0000', 'last_became_peered': '2024-09-11T01:13:10.199795+0000', 'last_unstale': '2024-09-11T01:15:24.652759+0000', 'last_undegraded': '2024-09-11T01:15:24.652759+0000', 'last_fullsized': '2024-09-11T01:13:09.652311+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3415", 'last_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'objects_scrubbed': 3337, 'log_size': 8286, 'log_dups_size': 0, 'ondisk_log_size': 8286, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1975, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33906688, 'num_objects': 8278, 'num_object_clones': 0, 'num_object_copies': 16556, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3834, 'num_objects_unfound': 0, 'num_objects_dirty': 8278, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 4941, 'num_write_kb': 19764, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 105, 'num_bytes_recovered': 430080, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4444}, {'shards': '0,2', 'objects': 3834}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:30.192 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:15:30.192 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'8425", 'reported_seq': 8558, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:24.630903+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:15:24.630903+0000', 'last_peered': '2024-09-11T01:15:24.630903+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:15:24.630903+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 8425, 'log_dups_size': 0, 'ondisk_log_size': 8425, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34492416, 'num_objects': 8421, 'num_object_clones': 0, 'num_object_copies': 16842, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8421, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5034, 'num_write_kb': 20136, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4382}, {'shards': '0,2', 'objects': 4039}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:30.192 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:15:30.192 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'8392", 'reported_seq': 8525, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:24.597266+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:15:24.597266+0000', 'last_peered': '2024-09-11T01:15:24.597266+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:15:24.597266+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 8392, 'log_dups_size': 0, 'ondisk_log_size': 8392, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34325097, 'num_objects': 8381, 'num_object_clones': 0, 'num_object_copies': 16762, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8381, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5462, 'num_write_kb': 21848, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4464}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:30.193 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:15:30.416 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 186 20 136890 136870 2.87329 1.81641 0.0647905 0.0271821 2024-09-11T01:15:31.298 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:15:31.298 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39769/272744 objects degraded (14.581%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":267,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":136372,"data_bytes":558568060,"bytes_used":1898725376,"bytes_avail":288011567104,"bytes_total":289910292480,"degraded_objects":39769,"degraded_total":272744,"degraded_ratio":0.14581072360895198,"misplaced_objects":3751,"misplaced_total":272744,"misplaced_ratio":0.013752823160179511,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":103942,"recovering_keys_per_sec":0,"num_objects_recovered":252,"num_bytes_recovered":1032192,"num_keys_recovered":0,"write_bytes_sec":1861895,"read_op_per_sec":0,"write_op_per_sec":454},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:15:31.358 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:15:31.359 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:31.416 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 187 20 137343 137323 2.86739 1.76953 0.0328412 0.0272378 2024-09-11T01:15:32.228 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:15:32.228 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:15:32.293 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":172,"stamp":"2024-09-11T01:15:30.616224+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":558568060,"num_objects":136372,"num_object_clones":0,"num_object_copies":272744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":3751,"num_objects_unfound":0,"num_objects_dirty":136372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97303,"num_write_kb":389206,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4117,"num_bytes_recovered":16850963,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136454,"ondisk_log_size":136454,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1854224,"kb_used_data":1105440,"kb_used_omap":12,"kb_used_meta":748723,"kb_avail":281261296,"statfs":{"total":289910292480,"available":288011567104,"internally_reserved":0,"allocated":1131970560,"data_stored":1131127415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":766692356},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,8,4,6,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18489344,"num_objects":4514,"num_object_clones":0,"num_object_copies":9028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3942,"num_objects_misplaced":3687,"num_objects_unfound":0,"num_objects_dirty":4514,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4514,"num_write_kb":18056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":1032192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.930387"},"pg_stats":[{"pgid":"2.f","version":"45'8649","reported_seq":8726,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.665652+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:15:29.665652+0000","last_peered":"2024-09-11T01:15:29.665652+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:15:29.665652+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":8649,"log_dups_size":0,"ondisk_log_size":8649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35426304,"num_objects":8649,"num_object_clones":0,"num_object_copies":17298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8649,"num_write_kb":34596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4577},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'8602","reported_seq":8703,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:28.739265+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:15:28.739265+0000","last_peered":"2024-09-11T01:15:28.739265+0000","last_clean":"2024-09-11T01:15:28.739265+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:15:28.739265+0000","last_undegraded":"2024-09-11T01:15:28.739265+0000","last_fullsized":"2024-09-11T01:15:28.739265+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":8602,"log_dups_size":0,"ondisk_log_size":8602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35209216,"num_objects":8596,"num_object_clones":0,"num_object_copies":17192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6841,"num_write_kb":27364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'8495","reported_seq":8620,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:28.752077+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:15:28.752077+0000","last_peered":"2024-09-11T01:15:28.752077+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:15:28.752077+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":8495,"log_dups_size":0,"ondisk_log_size":8495,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34779136,"num_objects":8491,"num_object_clones":0,"num_object_copies":16982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6796,"num_write_kb":27184,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":4595}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'8537","reported_seq":8606,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:28.765112+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:15:28.765112+0000","last_peered":"2024-09-11T01:15:28.765112+0000","last_clean":"2024-09-11T01:15:28.765112+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:15:28.765112+0000","last_undegraded":"2024-09-11T01:15:28.765112+0000","last_fullsized":"2024-09-11T01:15:28.765112+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":8537,"log_dups_size":0,"ondisk_log_size":8537,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34967552,"num_objects":8537,"num_object_clones":0,"num_object_copies":17074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8537,"num_write_kb":34148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'8496","reported_seq":8623,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:29.646260+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:15:29.646260+0000","last_peered":"2024-09-11T01:15:29.646260+0000","last_clean":"2024-09-11T01:15:29.646260+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:15:29.646260+0000","last_undegraded":"2024-09-11T01:15:29.646260+0000","last_fullsized":"2024-09-11T01:15:29.646260+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":8496,"log_dups_size":0,"ondisk_log_size":8496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34770944,"num_objects":8489,"num_object_clones":0,"num_object_copies":16978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8489,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5501,"num_write_kb":22004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'8578","reported_seq":8748,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.508947+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:15:29.508947+0000","last_peered":"2024-09-11T01:15:29.508947+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:15:29.508947+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":8578,"log_dups_size":0,"ondisk_log_size":8578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35115008,"num_objects":8573,"num_object_clones":0,"num_object_copies":17146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4919,"num_write_kb":19676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4622},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'8432","reported_seq":8561,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.604857+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:15:29.604857+0000","last_peered":"2024-09-11T01:15:29.604857+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:15:29.604857+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":8432,"log_dups_size":0,"ondisk_log_size":8432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34512896,"num_objects":8426,"num_object_clones":0,"num_object_copies":16852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5289,"num_write_kb":21156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4440},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'8575","reported_seq":8704,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:29.575829+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:15:29.575829+0000","last_peered":"2024-09-11T01:15:29.575829+0000","last_clean":"2024-09-11T01:15:29.575829+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:15:29.575829+0000","last_undegraded":"2024-09-11T01:15:29.575829+0000","last_fullsized":"2024-09-11T01:15:29.575829+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":8575,"log_dups_size":0,"ondisk_log_size":8575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35115008,"num_objects":8573,"num_object_clones":0,"num_object_copies":17146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5451,"num_write_kb":21804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8514","reported_seq":8641,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:28.750875+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:15:28.750875+0000","last_peered":"2024-09-11T01:15:28.750875+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:15:28.750875+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":8514,"log_dups_size":0,"ondisk_log_size":8514,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34844672,"num_objects":8507,"num_object_clones":0,"num_object_copies":17014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6485,"num_write_kb":25940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":4574}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692287+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692287+0000","last_peered":"2024-09-11T01:15:21.692287+0000","last_clean":"2024-09-11T01:15:21.692287+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692287+0000","last_undegraded":"2024-09-11T01:15:21.692287+0000","last_fullsized":"2024-09-11T01:15:21.692287+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'8443","reported_seq":8571,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.639531+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:15:29.639531+0000","last_peered":"2024-09-11T01:15:29.639531+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:15:29.639531+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":8443,"log_dups_size":0,"ondisk_log_size":8443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34549760,"num_objects":8435,"num_object_clones":0,"num_object_copies":16870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5030,"num_write_kb":20120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4451},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.685123+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.685123+0000","last_peered":"2024-09-11T01:15:21.685123+0000","last_clean":"2024-09-11T01:15:21.685123+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.685123+0000","last_undegraded":"2024-09-11T01:15:21.685123+0000","last_fullsized":"2024-09-11T01:15:21.685123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'8542","reported_seq":8660,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:28.799325+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:15:28.799325+0000","last_peered":"2024-09-11T01:15:28.799325+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:15:28.799325+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":8542,"log_dups_size":0,"ondisk_log_size":8542,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34967552,"num_objects":8537,"num_object_clones":0,"num_object_copies":17074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6359,"num_write_kb":25436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":4513}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8418","reported_seq":16398,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:28.724685+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:15:28.724685+0000","last_peered":"2024-09-11T01:15:28.724685+0000","last_clean":"2024-09-11T01:15:28.724685+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:15:28.724685+0000","last_undegraded":"2024-09-11T01:15:28.724685+0000","last_fullsized":"2024-09-11T01:15:28.724685+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":8418,"log_dups_size":0,"ondisk_log_size":8418,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34455552,"num_objects":8412,"num_object_clones":0,"num_object_copies":16824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6203,"num_write_kb":24812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'8619","reported_seq":8750,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.549914+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:15:29.549914+0000","last_peered":"2024-09-11T01:15:29.549914+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:15:29.549914+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":8619,"log_dups_size":0,"ondisk_log_size":8619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35295232,"num_objects":8617,"num_object_clones":0,"num_object_copies":17234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5356,"num_write_kb":21424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4667},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'8435","reported_seq":8948,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:29.697933+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:15:29.697933+0000","last_peered":"2024-09-11T01:15:29.697933+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:15:29.697933+0000","last_undegraded":"2024-09-11T01:15:29.697933+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":8435,"log_dups_size":0,"ondisk_log_size":8435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34516992,"num_objects":8427,"num_object_clones":0,"num_object_copies":16854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3751,"num_objects_unfound":0,"num_objects_dirty":8427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5090,"num_write_kb":20360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":770048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4676},{"shards":"0,2","objects":3751}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694344+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694344+0000","last_peered":"2024-09-11T01:15:21.694344+0000","last_clean":"2024-09-11T01:15:21.694344+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694344+0000","last_undegraded":"2024-09-11T01:15:21.694344+0000","last_fullsized":"2024-09-11T01:15:21.694344+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'8564","reported_seq":8698,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.456257+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:15:29.456257+0000","last_peered":"2024-09-11T01:15:29.456257+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:15:29.456257+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":8564,"log_dups_size":0,"ondisk_log_size":8564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35061760,"num_objects":8560,"num_object_clones":0,"num_object_copies":17120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5173,"num_write_kb":20692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4521},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'8552","reported_seq":8685,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.663312+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:15:29.663312+0000","last_peered":"2024-09-11T01:15:29.663312+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:15:29.663312+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":8552,"log_dups_size":0,"ondisk_log_size":8552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34980457,"num_objects":8541,"num_object_clones":0,"num_object_copies":17082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5622,"num_write_kb":22488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4624},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":558568041,"num_objects":136370,"num_object_clones":0,"num_object_copies":272740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":3751,"num_objects_unfound":0,"num_objects_dirty":136370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":97301,"num_write_kb":389204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4116,"num_bytes_recovered":16850944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1132306432,"data_stored":1132226415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":136451,"ondisk_log_size":136451,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182448,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":770392,"kb_used_data":472200,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93601448,"statfs":{"total":96636764160,"available":95847882752,"internally_reserved":0,"allocated":483532800,"data_stored":483273586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":305327787},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,2,3,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986220,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":652420,"kb_used_data":400052,"kb_used_omap":4,"kb_used_meta":252347,"kb_avail":93719420,"statfs":{"total":96636764160,"available":95968686080,"internally_reserved":0,"allocated":409653248,"data_stored":409398111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":258404010},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659165,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":431412,"kb_used_data":233188,"kb_used_omap":4,"kb_used_meta":198203,"kb_avail":93940428,"statfs":{"total":96636764160,"available":96194998272,"internally_reserved":0,"allocated":238784512,"data_stored":238455718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":202960559},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":484450304,"data_stored":484446825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":238501888,"data_stored":238428829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":409354240,"data_stored":409350761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:32.416 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 188 20 137788 137768 2.86139 1.73828 0.0609845 0.0272942 2024-09-11T01:15:33.416 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 189 20 138262 138242 2.85605 1.85156 0.0897278 0.0273459 2024-09-11T01:15:34.416 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 190 20 138733 138713 2.8507 1.83984 0.0271561 0.0273966 2024-09-11T01:15:35.297 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:35.416 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 191 20 139199 139179 2.8453 1.82031 0.0181854 0.0274478 2024-09-11T01:15:36.179 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:15:36.182 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:15:36.288 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":174,"stamp":"2024-09-11T01:15:34.617613+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":562254460,"num_objects":137272,"num_object_clones":0,"num_object_copies":274544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":3751,"num_objects_unfound":0,"num_objects_dirty":137272,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98203,"num_write_kb":392806,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4117,"num_bytes_recovered":16850963,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137354,"ondisk_log_size":137354,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1879720,"kb_used_data":1117048,"kb_used_omap":12,"kb_used_meta":762611,"kb_avail":281235800,"statfs":{"total":289910292480,"available":287985459200,"internally_reserved":0,"allocated":1143857152,"data_stored":1143014007,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":780913668},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,0,6,9,8,11,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22175744,"num_objects":5414,"num_object_clones":0,"num_object_copies":10828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3942,"num_objects_misplaced":3687,"num_objects_unfound":0,"num_objects_dirty":5414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5414,"num_write_kb":21656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":252,"num_bytes_recovered":1032192,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003323"},"pg_stats":[{"pgid":"2.f","version":"45'8649","reported_seq":8726,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.665652+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:15:29.665652+0000","last_peered":"2024-09-11T01:15:29.665652+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:15:29.665652+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":8649,"log_dups_size":0,"ondisk_log_size":8649,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35426304,"num_objects":8649,"num_object_clones":0,"num_object_copies":17298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8649,"num_write_kb":34596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4577},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'8766","reported_seq":8866,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:33.792328+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:15:33.792328+0000","last_peered":"2024-09-11T01:15:33.792328+0000","last_clean":"2024-09-11T01:15:33.792328+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:15:33.792328+0000","last_undegraded":"2024-09-11T01:15:33.792328+0000","last_fullsized":"2024-09-11T01:15:33.792328+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":8766,"log_dups_size":0,"ondisk_log_size":8766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35880960,"num_objects":8760,"num_object_clones":0,"num_object_copies":17520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7005,"num_write_kb":28020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'8637","reported_seq":8764,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:33.702327+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:15:33.702327+0000","last_peered":"2024-09-11T01:15:33.702327+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:15:33.702327+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":8637,"log_dups_size":0,"ondisk_log_size":8637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35360768,"num_objects":8633,"num_object_clones":0,"num_object_copies":17266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6938,"num_write_kb":27752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":4737}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'8685","reported_seq":8754,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:33.823773+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:15:33.823773+0000","last_peered":"2024-09-11T01:15:33.823773+0000","last_clean":"2024-09-11T01:15:33.823773+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:15:33.823773+0000","last_undegraded":"2024-09-11T01:15:33.823773+0000","last_fullsized":"2024-09-11T01:15:33.823773+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":8685,"log_dups_size":0,"ondisk_log_size":8685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35573760,"num_objects":8685,"num_object_clones":0,"num_object_copies":17370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8685,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8685,"num_write_kb":34740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'8496","reported_seq":8623,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:29.646260+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:15:29.646260+0000","last_peered":"2024-09-11T01:15:29.646260+0000","last_clean":"2024-09-11T01:15:29.646260+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:15:29.646260+0000","last_undegraded":"2024-09-11T01:15:29.646260+0000","last_fullsized":"2024-09-11T01:15:29.646260+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":8496,"log_dups_size":0,"ondisk_log_size":8496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34770944,"num_objects":8489,"num_object_clones":0,"num_object_copies":16978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8489,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5501,"num_write_kb":22004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'8578","reported_seq":8748,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.508947+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:15:29.508947+0000","last_peered":"2024-09-11T01:15:29.508947+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:15:29.508947+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":8578,"log_dups_size":0,"ondisk_log_size":8578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35115008,"num_objects":8573,"num_object_clones":0,"num_object_copies":17146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4919,"num_write_kb":19676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4622},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'8432","reported_seq":8561,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.604857+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:15:29.604857+0000","last_peered":"2024-09-11T01:15:29.604857+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:15:29.604857+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":8432,"log_dups_size":0,"ondisk_log_size":8432,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34512896,"num_objects":8426,"num_object_clones":0,"num_object_copies":16852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8426,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5289,"num_write_kb":21156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4440},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'8575","reported_seq":8704,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:29.575829+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:15:29.575829+0000","last_peered":"2024-09-11T01:15:29.575829+0000","last_clean":"2024-09-11T01:15:29.575829+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:15:29.575829+0000","last_undegraded":"2024-09-11T01:15:29.575829+0000","last_fullsized":"2024-09-11T01:15:29.575829+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":8575,"log_dups_size":0,"ondisk_log_size":8575,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35115008,"num_objects":8573,"num_object_clones":0,"num_object_copies":17146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5451,"num_write_kb":21804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8656","reported_seq":8783,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:33.807677+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:15:33.807677+0000","last_peered":"2024-09-11T01:15:33.807677+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:15:33.807677+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":8656,"log_dups_size":0,"ondisk_log_size":8656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35426304,"num_objects":8649,"num_object_clones":0,"num_object_copies":17298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6627,"num_write_kb":26508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":4716}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692287+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692287+0000","last_peered":"2024-09-11T01:15:21.692287+0000","last_clean":"2024-09-11T01:15:21.692287+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692287+0000","last_undegraded":"2024-09-11T01:15:21.692287+0000","last_fullsized":"2024-09-11T01:15:21.692287+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'8443","reported_seq":8571,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.639531+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:15:29.639531+0000","last_peered":"2024-09-11T01:15:29.639531+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:15:29.639531+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":8443,"log_dups_size":0,"ondisk_log_size":8443,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34549760,"num_objects":8435,"num_object_clones":0,"num_object_copies":16870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5030,"num_write_kb":20120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4451},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.685123+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.685123+0000","last_peered":"2024-09-11T01:15:21.685123+0000","last_clean":"2024-09-11T01:15:21.685123+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.685123+0000","last_undegraded":"2024-09-11T01:15:21.685123+0000","last_fullsized":"2024-09-11T01:15:21.685123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'8709","reported_seq":8827,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:33.802841+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:15:33.802841+0000","last_peered":"2024-09-11T01:15:33.802841+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:15:33.802841+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":8709,"log_dups_size":0,"ondisk_log_size":8709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35651584,"num_objects":8704,"num_object_clones":0,"num_object_copies":17408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6526,"num_write_kb":26104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":4680}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8555","reported_seq":16535,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:33.836486+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:15:33.836486+0000","last_peered":"2024-09-11T01:15:33.836486+0000","last_clean":"2024-09-11T01:15:33.836486+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:15:33.836486+0000","last_undegraded":"2024-09-11T01:15:33.836486+0000","last_fullsized":"2024-09-11T01:15:33.836486+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":8555,"log_dups_size":0,"ondisk_log_size":8555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35016704,"num_objects":8549,"num_object_clones":0,"num_object_copies":17098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6340,"num_write_kb":25360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'8619","reported_seq":8750,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.549914+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:15:29.549914+0000","last_peered":"2024-09-11T01:15:29.549914+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:15:29.549914+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":8619,"log_dups_size":0,"ondisk_log_size":8619,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35295232,"num_objects":8617,"num_object_clones":0,"num_object_copies":17234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5356,"num_write_kb":21424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4667},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'8435","reported_seq":8948,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:29.697933+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:15:29.697933+0000","last_peered":"2024-09-11T01:15:29.697933+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:15:29.697933+0000","last_undegraded":"2024-09-11T01:15:29.697933+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":8435,"log_dups_size":0,"ondisk_log_size":8435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34516992,"num_objects":8427,"num_object_clones":0,"num_object_copies":16854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3751,"num_objects_unfound":0,"num_objects_dirty":8427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5090,"num_write_kb":20360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":188,"num_bytes_recovered":770048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4676},{"shards":"0,2","objects":3751}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694344+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694344+0000","last_peered":"2024-09-11T01:15:21.694344+0000","last_clean":"2024-09-11T01:15:21.694344+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694344+0000","last_undegraded":"2024-09-11T01:15:21.694344+0000","last_fullsized":"2024-09-11T01:15:21.694344+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'8564","reported_seq":8698,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.456257+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:15:29.456257+0000","last_peered":"2024-09-11T01:15:29.456257+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:15:29.456257+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":8564,"log_dups_size":0,"ondisk_log_size":8564,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35061760,"num_objects":8560,"num_object_clones":0,"num_object_copies":17120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5173,"num_write_kb":20692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4521},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'8552","reported_seq":8685,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:29.663312+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:15:29.663312+0000","last_peered":"2024-09-11T01:15:29.663312+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:15:29.663312+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":8552,"log_dups_size":0,"ondisk_log_size":8552,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34980457,"num_objects":8541,"num_object_clones":0,"num_object_copies":17082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5622,"num_write_kb":22488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4624},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":562254441,"num_objects":137270,"num_object_clones":0,"num_object_copies":274540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":3751,"num_objects_unfound":0,"num_objects_dirty":137270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":98201,"num_write_kb":392804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4116,"num_bytes_recovered":16850944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1144569856,"data_stored":1144489839,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137351,"ondisk_log_size":137351,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182448,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":770392,"kb_used_data":472200,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93601448,"statfs":{"total":96636764160,"available":95847882752,"internally_reserved":0,"allocated":483532800,"data_stored":483273586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":305327787},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,3,2,3,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986221,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":679188,"kb_used_data":404612,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93692652,"statfs":{"total":96636764160,"available":95941275648,"internally_reserved":0,"allocated":414322688,"data_stored":414067551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,0,4,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659166,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":430140,"kb_used_data":240236,"kb_used_omap":4,"kb_used_meta":189883,"kb_avail":93941700,"statfs":{"total":96636764160,"available":96196300800,"internally_reserved":0,"allocated":246001664,"data_stored":245672870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":194440879},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,3,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":484450304,"data_stored":484446825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":245940224,"data_stored":245867165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":414179328,"data_stored":414175849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:36.290 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:36.421 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 192 20 139641 139621 2.83946 1.72656 0.0313865 0.0275051 2024-09-11T01:15:37.116 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:15:37.117 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:15:37.156 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":175,"stamp":"2024-09-11T01:15:36.620384+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":568230524,"num_objects":138731,"num_object_clones":0,"num_object_copies":277462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":3667,"num_objects_unfound":0,"num_objects_dirty":138731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99662,"num_write_kb":398642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4201,"num_bytes_recovered":17195027,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138813,"ondisk_log_size":138813,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1896232,"kb_used_data":1125240,"kb_used_omap":12,"kb_used_meta":770931,"kb_avail":281219288,"statfs":{"total":289910292480,"available":287968550912,"internally_reserved":0,"allocated":1152245760,"data_stored":1151402615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":789433348},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,4,4,9,12,9,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":24616960,"num_objects":6010,"num_object_clones":0,"num_object_copies":12020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-3942,"num_objects_misplaced":3667,"num_objects_unfound":0,"num_objects_dirty":6010,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6010,"num_write_kb":24040,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":1114112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005620"},"pg_stats":[{"pgid":"2.f","version":"45'8798","reported_seq":8875,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:34.663620+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:15:34.663620+0000","last_peered":"2024-09-11T01:15:34.663620+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:15:34.663620+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":8798,"log_dups_size":0,"ondisk_log_size":8798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36036608,"num_objects":8798,"num_object_clones":0,"num_object_copies":17596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8798,"num_write_kb":35192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4726},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'8766","reported_seq":8866,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:33.792328+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:15:33.792328+0000","last_peered":"2024-09-11T01:15:33.792328+0000","last_clean":"2024-09-11T01:15:33.792328+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:15:33.792328+0000","last_undegraded":"2024-09-11T01:15:33.792328+0000","last_fullsized":"2024-09-11T01:15:33.792328+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":8766,"log_dups_size":0,"ondisk_log_size":8766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35880960,"num_objects":8760,"num_object_clones":0,"num_object_copies":17520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7005,"num_write_kb":28020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'8637","reported_seq":8764,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:33.702327+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:15:33.702327+0000","last_peered":"2024-09-11T01:15:33.702327+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:15:33.702327+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":8637,"log_dups_size":0,"ondisk_log_size":8637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35360768,"num_objects":8633,"num_object_clones":0,"num_object_copies":17266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6938,"num_write_kb":27752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":4737}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'8685","reported_seq":8754,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:33.823773+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:15:33.823773+0000","last_peered":"2024-09-11T01:15:33.823773+0000","last_clean":"2024-09-11T01:15:33.823773+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:15:33.823773+0000","last_undegraded":"2024-09-11T01:15:33.823773+0000","last_fullsized":"2024-09-11T01:15:33.823773+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":8685,"log_dups_size":0,"ondisk_log_size":8685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35573760,"num_objects":8685,"num_object_clones":0,"num_object_copies":17370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8685,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8685,"num_write_kb":34740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'8641","reported_seq":8768,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:34.672242+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:15:34.672242+0000","last_peered":"2024-09-11T01:15:34.672242+0000","last_clean":"2024-09-11T01:15:34.672242+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:15:34.672242+0000","last_undegraded":"2024-09-11T01:15:34.672242+0000","last_fullsized":"2024-09-11T01:15:34.672242+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":8641,"log_dups_size":0,"ondisk_log_size":8641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35364864,"num_objects":8634,"num_object_clones":0,"num_object_copies":17268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5646,"num_write_kb":22584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'8733","reported_seq":8900,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:34.683274+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:15:34.683274+0000","last_peered":"2024-09-11T01:15:34.683274+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:15:34.683274+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":8733,"log_dups_size":0,"ondisk_log_size":8733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35749888,"num_objects":8728,"num_object_clones":0,"num_object_copies":17456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5074,"num_write_kb":20296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4777},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'8574","reported_seq":8704,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:34.669008+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:15:34.669008+0000","last_peered":"2024-09-11T01:15:34.669008+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:15:34.669008+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":8574,"log_dups_size":0,"ondisk_log_size":8574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35094528,"num_objects":8568,"num_object_clones":0,"num_object_copies":17136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5431,"num_write_kb":21724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4582},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'8715","reported_seq":8844,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:34.695931+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:15:34.695931+0000","last_peered":"2024-09-11T01:15:34.695931+0000","last_clean":"2024-09-11T01:15:34.695931+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:15:34.695931+0000","last_undegraded":"2024-09-11T01:15:34.695931+0000","last_fullsized":"2024-09-11T01:15:34.695931+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":8715,"log_dups_size":0,"ondisk_log_size":8715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35688448,"num_objects":8713,"num_object_clones":0,"num_object_copies":17426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5591,"num_write_kb":22364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8656","reported_seq":8783,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:33.807677+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:15:33.807677+0000","last_peered":"2024-09-11T01:15:33.807677+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:15:33.807677+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":8656,"log_dups_size":0,"ondisk_log_size":8656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35426304,"num_objects":8649,"num_object_clones":0,"num_object_copies":17298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6627,"num_write_kb":26508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":4716}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692287+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692287+0000","last_peered":"2024-09-11T01:15:21.692287+0000","last_clean":"2024-09-11T01:15:21.692287+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692287+0000","last_undegraded":"2024-09-11T01:15:21.692287+0000","last_fullsized":"2024-09-11T01:15:21.692287+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'8584","reported_seq":8711,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:34.730017+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:15:34.730017+0000","last_peered":"2024-09-11T01:15:34.730017+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:15:34.730017+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":8584,"log_dups_size":0,"ondisk_log_size":8584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35127296,"num_objects":8576,"num_object_clones":0,"num_object_copies":17152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5171,"num_write_kb":20684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4592},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.685123+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.685123+0000","last_peered":"2024-09-11T01:15:21.685123+0000","last_clean":"2024-09-11T01:15:21.685123+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.685123+0000","last_undegraded":"2024-09-11T01:15:21.685123+0000","last_fullsized":"2024-09-11T01:15:21.685123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'8709","reported_seq":8827,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:33.802841+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:15:33.802841+0000","last_peered":"2024-09-11T01:15:33.802841+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:15:33.802841+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":8709,"log_dups_size":0,"ondisk_log_size":8709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35651584,"num_objects":8704,"num_object_clones":0,"num_object_copies":17408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6526,"num_write_kb":26104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":4680}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8555","reported_seq":16535,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:33.836486+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:15:33.836486+0000","last_peered":"2024-09-11T01:15:33.836486+0000","last_clean":"2024-09-11T01:15:33.836486+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:15:33.836486+0000","last_undegraded":"2024-09-11T01:15:33.836486+0000","last_fullsized":"2024-09-11T01:15:33.836486+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":8555,"log_dups_size":0,"ondisk_log_size":8555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35016704,"num_objects":8549,"num_object_clones":0,"num_object_copies":17098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6340,"num_write_kb":25360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'8768","reported_seq":8898,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:34.735257+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:15:34.735257+0000","last_peered":"2024-09-11T01:15:34.735257+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:15:34.735257+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":8768,"log_dups_size":0,"ondisk_log_size":8768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35905536,"num_objects":8766,"num_object_clones":0,"num_object_copies":17532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5505,"num_write_kb":22020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4816},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'8593","reported_seq":9274,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:34.732446+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:15:34.732446+0000","last_peered":"2024-09-11T01:15:34.732446+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:15:34.732446+0000","last_undegraded":"2024-09-11T01:15:34.732446+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":8593,"log_dups_size":0,"ondisk_log_size":8593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35164160,"num_objects":8585,"num_object_clones":0,"num_object_copies":17170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3667,"num_objects_unfound":0,"num_objects_dirty":8585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5248,"num_write_kb":20992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":1114112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4918},{"shards":"0,2","objects":3667}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694344+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694344+0000","last_peered":"2024-09-11T01:15:21.694344+0000","last_clean":"2024-09-11T01:15:21.694344+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694344+0000","last_undegraded":"2024-09-11T01:15:21.694344+0000","last_fullsized":"2024-09-11T01:15:21.694344+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'8700","reported_seq":8834,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:34.697985+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:15:34.697985+0000","last_peered":"2024-09-11T01:15:34.697985+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:15:34.697985+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":8700,"log_dups_size":0,"ondisk_log_size":8700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35618816,"num_objects":8696,"num_object_clones":0,"num_object_copies":17392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8696,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5309,"num_write_kb":21236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4657},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'8696","reported_seq":8829,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:34.713012+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:15:34.713012+0000","last_peered":"2024-09-11T01:15:34.713012+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:15:34.713012+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":8696,"log_dups_size":0,"ondisk_log_size":8696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35570281,"num_objects":8685,"num_object_clones":0,"num_object_copies":17370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8685,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5766,"num_write_kb":23064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4768},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":568230505,"num_objects":138729,"num_object_clones":0,"num_object_copies":277458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":3667,"num_objects_unfound":0,"num_objects_dirty":138729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99660,"num_write_kb":398640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4200,"num_bytes_recovered":17195008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1151746048,"data_stored":1151666031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138810,"ondisk_log_size":138810,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182449,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":786904,"kb_used_data":480392,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93584936,"statfs":{"total":96636764160,"available":95830974464,"internally_reserved":0,"allocated":491921408,"data_stored":491662194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":313847467},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,4,1,2,7,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986221,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":679188,"kb_used_data":404612,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93692652,"statfs":{"total":96636764160,"available":95941275648,"internally_reserved":0,"allocated":414322688,"data_stored":414067551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,0,4,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659166,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":430140,"kb_used_data":240236,"kb_used_omap":4,"kb_used_meta":189883,"kb_avail":93941700,"statfs":{"total":96636764160,"available":96196300800,"internally_reserved":0,"allocated":246001664,"data_stored":245672870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":194440879},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,3,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":491626496,"data_stored":491623017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":245940224,"data_stored":245867165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":414179328,"data_stored":414175849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:37.158 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:15:37.158 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'8798", 'reported_seq': 8875, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:34.663620+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:15:34.663620+0000', 'last_peered': '2024-09-11T01:15:34.663620+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:15:34.663620+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 8798, 'log_dups_size': 0, 'ondisk_log_size': 8798, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36036608, 'num_objects': 8798, 'num_object_clones': 0, 'num_object_copies': 17596, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8798, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8798, 'num_write_kb': 35192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4726}, {'shards': '0,2', 'objects': 4072}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:37.158 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:15:37.158 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'8637", 'reported_seq': 8764, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:33.702327+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:15:33.702327+0000', 'last_peered': '2024-09-11T01:15:33.702327+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:15:33.702327+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 8637, 'log_dups_size': 0, 'ondisk_log_size': 8637, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35360768, 'num_objects': 8633, 'num_object_clones': 0, 'num_object_copies': 17266, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8633, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6938, 'num_write_kb': 27752, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3896}, {'shards': '2', 'objects': 4737}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:37.158 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:15:37.158 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'8733", 'reported_seq': 8900, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:34.683274+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:15:34.683274+0000', 'last_peered': '2024-09-11T01:15:34.683274+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:15:34.683274+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 8733, 'log_dups_size': 0, 'ondisk_log_size': 8733, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35749888, 'num_objects': 8728, 'num_object_clones': 0, 'num_object_copies': 17456, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8728, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5074, 'num_write_kb': 20296, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4777}, {'shards': '0,2', 'objects': 3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:37.158 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:15:37.158 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'8574", 'reported_seq': 8704, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:34.669008+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:15:34.669008+0000', 'last_peered': '2024-09-11T01:15:34.669008+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:15:34.669008+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 8574, 'log_dups_size': 0, 'ondisk_log_size': 8574, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35094528, 'num_objects': 8568, 'num_object_clones': 0, 'num_object_copies': 17136, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8568, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5431, 'num_write_kb': 21724, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4582}, {'shards': '0,2', 'objects': 3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:37.159 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:15:37.159 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'8656", 'reported_seq': 8783, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:33.807677+0000', 'last_change': '2024-09-11T01:13:10.159069+0000', 'last_active': '2024-09-11T01:15:33.807677+0000', 'last_peered': '2024-09-11T01:15:33.807677+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:15:33.807677+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 8656, 'log_dups_size': 0, 'ondisk_log_size': 8656, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35426304, 'num_objects': 8649, 'num_object_clones': 0, 'num_object_copies': 17298, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8649, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6627, 'num_write_kb': 26508, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3932}, {'shards': '2', 'objects': 4716}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:37.159 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:15:37.159 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'8584", 'reported_seq': 8711, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:34.730017+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:15:34.730017+0000', 'last_peered': '2024-09-11T01:15:34.730017+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:15:34.730017+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 8584, 'log_dups_size': 0, 'ondisk_log_size': 8584, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35127296, 'num_objects': 8576, 'num_object_clones': 0, 'num_object_copies': 17152, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8576, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5171, 'num_write_kb': 20684, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4592}, {'shards': '0,2', 'objects': 3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:37.159 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:15:37.159 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'8709", 'reported_seq': 8827, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:33.802841+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:15:33.802841+0000', 'last_peered': '2024-09-11T01:15:33.802841+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:15:33.802841+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 8709, 'log_dups_size': 0, 'ondisk_log_size': 8709, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35651584, 'num_objects': 8704, 'num_object_clones': 0, 'num_object_copies': 17408, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8704, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6526, 'num_write_kb': 26104, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4023}, {'shards': '2', 'objects': 4680}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:37.159 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:15:37.159 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'8768", 'reported_seq': 8898, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:34.735257+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:15:34.735257+0000', 'last_peered': '2024-09-11T01:15:34.735257+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:15:34.735257+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 8768, 'log_dups_size': 0, 'ondisk_log_size': 8768, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35905536, 'num_objects': 8766, 'num_object_clones': 0, 'num_object_copies': 17532, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8766, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5505, 'num_write_kb': 22020, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4816}, {'shards': '0,2', 'objects': 3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:37.159 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:15:37.159 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'8593", 'reported_seq': 9274, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:15:34.732446+0000', 'last_change': '2024-09-11T01:15:19.913288+0000', 'last_active': '2024-09-11T01:15:34.732446+0000', 'last_peered': '2024-09-11T01:15:34.732446+0000', 'last_clean': '2024-09-11T01:13:07.624565+0000', 'last_became_active': '2024-09-11T01:13:10.199795+0000', 'last_became_peered': '2024-09-11T01:13:10.199795+0000', 'last_unstale': '2024-09-11T01:15:34.732446+0000', 'last_undegraded': '2024-09-11T01:15:34.732446+0000', 'last_fullsized': '2024-09-11T01:13:09.652311+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3415", 'last_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'objects_scrubbed': 3337, 'log_size': 8593, 'log_dups_size': 0, 'ondisk_log_size': 8593, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1975, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35164160, 'num_objects': 8585, 'num_object_clones': 0, 'num_object_copies': 17170, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3667, 'num_objects_unfound': 0, 'num_objects_dirty': 8585, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5248, 'num_write_kb': 20992, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 272, 'num_bytes_recovered': 1114112, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4918}, {'shards': '0,2', 'objects': 3667}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:37.160 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:15:37.160 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'8700", 'reported_seq': 8834, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:34.697985+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:15:34.697985+0000', 'last_peered': '2024-09-11T01:15:34.697985+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:15:34.697985+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 8700, 'log_dups_size': 0, 'ondisk_log_size': 8700, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35618816, 'num_objects': 8696, 'num_object_clones': 0, 'num_object_copies': 17392, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8696, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5309, 'num_write_kb': 21236, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4657}, {'shards': '0,2', 'objects': 4039}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:37.160 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:15:37.160 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'8696", 'reported_seq': 8829, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:34.713012+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:15:34.713012+0000', 'last_peered': '2024-09-11T01:15:34.713012+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:15:34.713012+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 8696, 'log_dups_size': 0, 'ondisk_log_size': 8696, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35570281, 'num_objects': 8685, 'num_object_clones': 0, 'num_object_copies': 17370, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8685, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5766, 'num_write_kb': 23064, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4768}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:37.160 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:15:37.421 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 193 20 140075 140055 2.83348 1.69531 0.0603012 0.0275611 2024-09-11T01:15:38.096 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:15:38.096 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39769/277462 objects degraded (14.333%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":274,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":138731,"data_bytes":568230524,"bytes_used":1941741568,"bytes_avail":287968550912,"bytes_total":289910292480,"degraded_objects":39769,"degraded_total":277462,"degraded_ratio":0.14333133906625051,"misplaced_objects":3667,"misplaced_total":277462,"misplaced_ratio":0.013216224203674737,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":92799,"recovering_keys_per_sec":0,"num_objects_recovered":272,"num_bytes_recovered":1114112,"num_keys_recovered":0,"write_bytes_sec":2050452,"read_op_per_sec":0,"write_op_per_sec":500},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:15:38.133 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:15:38.133 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:38.422 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 194 20 140528 140508 2.82799 1.76953 0.023409 0.0276159 2024-09-11T01:15:38.882 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:15:38.882 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:15:38.945 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":176,"stamp":"2024-09-11T01:15:38.620873+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":568230524,"num_objects":138731,"num_object_clones":0,"num_object_copies":277462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":3667,"num_objects_unfound":0,"num_objects_dirty":138731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99662,"num_write_kb":398642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4201,"num_bytes_recovered":17195027,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138813,"ondisk_log_size":138813,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1907684,"kb_used_data":1132532,"kb_used_omap":12,"kb_used_meta":775091,"kb_avail":281207836,"statfs":{"total":289910292480,"available":287956824064,"internally_reserved":0,"allocated":1159712768,"data_stored":1158869623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":793693188},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,5,6,1,8,12,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19304448,"num_objects":4713,"num_object_clones":0,"num_object_copies":9426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-167,"num_objects_unfound":0,"num_objects_dirty":4713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4713,"num_write_kb":18852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":167,"num_bytes_recovered":684032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005633"},"pg_stats":[{"pgid":"2.f","version":"45'8798","reported_seq":8875,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:34.663620+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:15:34.663620+0000","last_peered":"2024-09-11T01:15:34.663620+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:15:34.663620+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":8798,"log_dups_size":0,"ondisk_log_size":8798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36036608,"num_objects":8798,"num_object_clones":0,"num_object_copies":17596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8798,"num_write_kb":35192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4726},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'8766","reported_seq":8866,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:33.792328+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:15:33.792328+0000","last_peered":"2024-09-11T01:15:33.792328+0000","last_clean":"2024-09-11T01:15:33.792328+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:15:33.792328+0000","last_undegraded":"2024-09-11T01:15:33.792328+0000","last_fullsized":"2024-09-11T01:15:33.792328+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":8766,"log_dups_size":0,"ondisk_log_size":8766,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35880960,"num_objects":8760,"num_object_clones":0,"num_object_copies":17520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7005,"num_write_kb":28020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'8637","reported_seq":8764,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:33.702327+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:15:33.702327+0000","last_peered":"2024-09-11T01:15:33.702327+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:15:33.702327+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":8637,"log_dups_size":0,"ondisk_log_size":8637,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35360768,"num_objects":8633,"num_object_clones":0,"num_object_copies":17266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6938,"num_write_kb":27752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":4737}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'8685","reported_seq":8754,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:33.823773+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:15:33.823773+0000","last_peered":"2024-09-11T01:15:33.823773+0000","last_clean":"2024-09-11T01:15:33.823773+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:15:33.823773+0000","last_undegraded":"2024-09-11T01:15:33.823773+0000","last_fullsized":"2024-09-11T01:15:33.823773+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":8685,"log_dups_size":0,"ondisk_log_size":8685,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35573760,"num_objects":8685,"num_object_clones":0,"num_object_copies":17370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8685,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8685,"num_write_kb":34740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'8641","reported_seq":8768,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:34.672242+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:15:34.672242+0000","last_peered":"2024-09-11T01:15:34.672242+0000","last_clean":"2024-09-11T01:15:34.672242+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:15:34.672242+0000","last_undegraded":"2024-09-11T01:15:34.672242+0000","last_fullsized":"2024-09-11T01:15:34.672242+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":8641,"log_dups_size":0,"ondisk_log_size":8641,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35364864,"num_objects":8634,"num_object_clones":0,"num_object_copies":17268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5646,"num_write_kb":22584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'8733","reported_seq":8900,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:34.683274+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:15:34.683274+0000","last_peered":"2024-09-11T01:15:34.683274+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:15:34.683274+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":8733,"log_dups_size":0,"ondisk_log_size":8733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35749888,"num_objects":8728,"num_object_clones":0,"num_object_copies":17456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5074,"num_write_kb":20296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4777},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'8574","reported_seq":8704,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:34.669008+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:15:34.669008+0000","last_peered":"2024-09-11T01:15:34.669008+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:15:34.669008+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":8574,"log_dups_size":0,"ondisk_log_size":8574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35094528,"num_objects":8568,"num_object_clones":0,"num_object_copies":17136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8568,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5431,"num_write_kb":21724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4582},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'8715","reported_seq":8844,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:34.695931+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:15:34.695931+0000","last_peered":"2024-09-11T01:15:34.695931+0000","last_clean":"2024-09-11T01:15:34.695931+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:15:34.695931+0000","last_undegraded":"2024-09-11T01:15:34.695931+0000","last_fullsized":"2024-09-11T01:15:34.695931+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":8715,"log_dups_size":0,"ondisk_log_size":8715,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35688448,"num_objects":8713,"num_object_clones":0,"num_object_copies":17426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5591,"num_write_kb":22364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8656","reported_seq":8783,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:33.807677+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:15:33.807677+0000","last_peered":"2024-09-11T01:15:33.807677+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:15:33.807677+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":8656,"log_dups_size":0,"ondisk_log_size":8656,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35426304,"num_objects":8649,"num_object_clones":0,"num_object_copies":17298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6627,"num_write_kb":26508,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":4716}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692287+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692287+0000","last_peered":"2024-09-11T01:15:21.692287+0000","last_clean":"2024-09-11T01:15:21.692287+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692287+0000","last_undegraded":"2024-09-11T01:15:21.692287+0000","last_fullsized":"2024-09-11T01:15:21.692287+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'8584","reported_seq":8711,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:34.730017+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:15:34.730017+0000","last_peered":"2024-09-11T01:15:34.730017+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:15:34.730017+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":8584,"log_dups_size":0,"ondisk_log_size":8584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35127296,"num_objects":8576,"num_object_clones":0,"num_object_copies":17152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5171,"num_write_kb":20684,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4592},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.685123+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.685123+0000","last_peered":"2024-09-11T01:15:21.685123+0000","last_clean":"2024-09-11T01:15:21.685123+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.685123+0000","last_undegraded":"2024-09-11T01:15:21.685123+0000","last_fullsized":"2024-09-11T01:15:21.685123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'8709","reported_seq":8827,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:33.802841+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:15:33.802841+0000","last_peered":"2024-09-11T01:15:33.802841+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:15:33.802841+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":8709,"log_dups_size":0,"ondisk_log_size":8709,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35651584,"num_objects":8704,"num_object_clones":0,"num_object_copies":17408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8704,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6526,"num_write_kb":26104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":4680}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8555","reported_seq":16535,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:33.836486+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:15:33.836486+0000","last_peered":"2024-09-11T01:15:33.836486+0000","last_clean":"2024-09-11T01:15:33.836486+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:15:33.836486+0000","last_undegraded":"2024-09-11T01:15:33.836486+0000","last_fullsized":"2024-09-11T01:15:33.836486+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":8555,"log_dups_size":0,"ondisk_log_size":8555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35016704,"num_objects":8549,"num_object_clones":0,"num_object_copies":17098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8549,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6340,"num_write_kb":25360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'8768","reported_seq":8898,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:34.735257+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:15:34.735257+0000","last_peered":"2024-09-11T01:15:34.735257+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:15:34.735257+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":8768,"log_dups_size":0,"ondisk_log_size":8768,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35905536,"num_objects":8766,"num_object_clones":0,"num_object_copies":17532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5505,"num_write_kb":22020,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4816},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'8593","reported_seq":9274,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:34.732446+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:15:34.732446+0000","last_peered":"2024-09-11T01:15:34.732446+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:15:34.732446+0000","last_undegraded":"2024-09-11T01:15:34.732446+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":8593,"log_dups_size":0,"ondisk_log_size":8593,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35164160,"num_objects":8585,"num_object_clones":0,"num_object_copies":17170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3667,"num_objects_unfound":0,"num_objects_dirty":8585,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5248,"num_write_kb":20992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":1114112,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4918},{"shards":"0,2","objects":3667}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694344+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694344+0000","last_peered":"2024-09-11T01:15:21.694344+0000","last_clean":"2024-09-11T01:15:21.694344+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694344+0000","last_undegraded":"2024-09-11T01:15:21.694344+0000","last_fullsized":"2024-09-11T01:15:21.694344+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'8700","reported_seq":8834,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:34.697985+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:15:34.697985+0000","last_peered":"2024-09-11T01:15:34.697985+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:15:34.697985+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":8700,"log_dups_size":0,"ondisk_log_size":8700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35618816,"num_objects":8696,"num_object_clones":0,"num_object_copies":17392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8696,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5309,"num_write_kb":21236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4657},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'8696","reported_seq":8829,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:34.713012+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:15:34.713012+0000","last_peered":"2024-09-11T01:15:34.713012+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:15:34.713012+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":8696,"log_dups_size":0,"ondisk_log_size":8696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35570281,"num_objects":8685,"num_object_clones":0,"num_object_copies":17370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8685,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5766,"num_write_kb":23064,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4768},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":568230505,"num_objects":138729,"num_object_clones":0,"num_object_copies":277458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":3667,"num_objects_unfound":0,"num_objects_dirty":138729,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":99660,"num_write_kb":398640,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4200,"num_bytes_recovered":17195008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1159086080,"data_stored":1159006063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":138810,"ondisk_log_size":138810,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182449,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":786904,"kb_used_data":480392,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93584936,"statfs":{"total":96636764160,"available":95830974464,"internally_reserved":0,"allocated":491921408,"data_stored":491662194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":313847467},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,4,1,2,7,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986221,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":679188,"kb_used_data":404612,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93692652,"statfs":{"total":96636764160,"available":95941275648,"internally_reserved":0,"allocated":414322688,"data_stored":414067551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,0,4,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659167,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":441592,"kb_used_data":247528,"kb_used_omap":4,"kb_used_meta":194043,"kb_avail":93930248,"statfs":{"total":96636764160,"available":96184573952,"internally_reserved":0,"allocated":253468672,"data_stored":253139878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":198700719},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,0,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":491626496,"data_stored":491623017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":253280256,"data_stored":253207197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":414179328,"data_stored":414175849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:39.422 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 195 20 140989 140969 2.82272 1.80078 0.0300194 0.0276702 2024-09-11T01:15:40.422 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 196 20 141447 141427 2.81745 1.78906 0.0270179 0.0277206 2024-09-11T01:15:41.423 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 197 20 141915 141895 2.81241 1.82812 0.0432005 0.0277698 2024-09-11T01:15:41.947 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:42.423 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 198 20 142374 142354 2.80726 1.79297 0.0181946 0.0278217 2024-09-11T01:15:42.884 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:15:42.885 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:15:42.980 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":178,"stamp":"2024-09-11T01:15:42.622345+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":577479292,"num_objects":140989,"num_object_clones":0,"num_object_copies":281978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":3574,"num_objects_unfound":0,"num_objects_dirty":140989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101920,"num_write_kb":407674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4294,"num_bytes_recovered":17575955,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141071,"ondisk_log_size":141071,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1910668,"kb_used_data":1143832,"kb_used_omap":12,"kb_used_meta":766771,"kb_avail":281204852,"statfs":{"total":289910292480,"available":287953768448,"internally_reserved":0,"allocated":1171283968,"data_stored":1170440823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":785173508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,4,3,8,14,6,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18911232,"num_objects":4617,"num_object_clones":0,"num_object_copies":9234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-177,"num_objects_unfound":0,"num_objects_dirty":4617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4617,"num_write_kb":18468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":724992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006121"},"pg_stats":[{"pgid":"2.f","version":"45'8919","reported_seq":8996,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.639749+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:15:39.639749+0000","last_peered":"2024-09-11T01:15:39.639749+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:15:39.639749+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":8919,"log_dups_size":0,"ondisk_log_size":8919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36532224,"num_objects":8919,"num_object_clones":0,"num_object_copies":17838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8919,"num_write_kb":35676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4847},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'8907","reported_seq":9008,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:38.811849+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:15:38.811849+0000","last_peered":"2024-09-11T01:15:38.811849+0000","last_clean":"2024-09-11T01:15:38.811849+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:15:38.811849+0000","last_undegraded":"2024-09-11T01:15:38.811849+0000","last_fullsized":"2024-09-11T01:15:38.811849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":8907,"log_dups_size":0,"ondisk_log_size":8907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36458496,"num_objects":8901,"num_object_clones":0,"num_object_copies":17802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7146,"num_write_kb":28584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'8779","reported_seq":8905,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:38.811787+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:15:38.811787+0000","last_peered":"2024-09-11T01:15:38.811787+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:15:38.811787+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":8779,"log_dups_size":0,"ondisk_log_size":8779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35942400,"num_objects":8775,"num_object_clones":0,"num_object_copies":17550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7080,"num_write_kb":28320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":4879}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'8835","reported_seq":8904,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:38.724668+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:15:38.724668+0000","last_peered":"2024-09-11T01:15:38.724668+0000","last_clean":"2024-09-11T01:15:38.724668+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:15:38.724668+0000","last_undegraded":"2024-09-11T01:15:38.724668+0000","last_fullsized":"2024-09-11T01:15:38.724668+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":8835,"log_dups_size":0,"ondisk_log_size":8835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36188160,"num_objects":8835,"num_object_clones":0,"num_object_copies":17670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8835,"num_write_kb":35340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'8788","reported_seq":8915,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:39.773513+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:15:39.773513+0000","last_peered":"2024-09-11T01:15:39.773513+0000","last_clean":"2024-09-11T01:15:39.773513+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:15:39.773513+0000","last_undegraded":"2024-09-11T01:15:39.773513+0000","last_fullsized":"2024-09-11T01:15:39.773513+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":8788,"log_dups_size":0,"ondisk_log_size":8788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35966976,"num_objects":8781,"num_object_clones":0,"num_object_copies":17562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5793,"num_write_kb":23172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'8875","reported_seq":9045,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.765707+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:15:39.765707+0000","last_peered":"2024-09-11T01:15:39.765707+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:15:39.765707+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":8875,"log_dups_size":0,"ondisk_log_size":8875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36331520,"num_objects":8870,"num_object_clones":0,"num_object_copies":17740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5216,"num_write_kb":20864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4919},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'8702","reported_seq":8832,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.730415+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:15:39.730415+0000","last_peered":"2024-09-11T01:15:39.730415+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:15:39.730415+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":8702,"log_dups_size":0,"ondisk_log_size":8702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35618816,"num_objects":8696,"num_object_clones":0,"num_object_copies":17392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8696,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5559,"num_write_kb":22236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4710},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'8856","reported_seq":8985,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:39.793343+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:15:39.793343+0000","last_peered":"2024-09-11T01:15:39.793343+0000","last_clean":"2024-09-11T01:15:39.793343+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:15:39.793343+0000","last_undegraded":"2024-09-11T01:15:39.793343+0000","last_fullsized":"2024-09-11T01:15:39.793343+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":8856,"log_dups_size":0,"ondisk_log_size":8856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36265984,"num_objects":8854,"num_object_clones":0,"num_object_copies":17708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5732,"num_write_kb":22928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8783","reported_seq":8910,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:38.742001+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:15:38.742001+0000","last_peered":"2024-09-11T01:15:38.742001+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:15:38.742001+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":8783,"log_dups_size":0,"ondisk_log_size":8783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35946496,"num_objects":8776,"num_object_clones":0,"num_object_copies":17552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6754,"num_write_kb":27016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":4843}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692287+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692287+0000","last_peered":"2024-09-11T01:15:21.692287+0000","last_clean":"2024-09-11T01:15:21.692287+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692287+0000","last_undegraded":"2024-09-11T01:15:21.692287+0000","last_fullsized":"2024-09-11T01:15:21.692287+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'8724","reported_seq":8851,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.781879+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:15:39.781879+0000","last_peered":"2024-09-11T01:15:39.781879+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:15:39.781879+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":8724,"log_dups_size":0,"ondisk_log_size":8724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35700736,"num_objects":8716,"num_object_clones":0,"num_object_copies":17432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5311,"num_write_kb":21244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4732},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.685123+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.685123+0000","last_peered":"2024-09-11T01:15:21.685123+0000","last_clean":"2024-09-11T01:15:21.685123+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.685123+0000","last_undegraded":"2024-09-11T01:15:21.685123+0000","last_fullsized":"2024-09-11T01:15:21.685123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'8870","reported_seq":8987,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:38.837351+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:15:38.837351+0000","last_peered":"2024-09-11T01:15:38.837351+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:15:38.837351+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":8870,"log_dups_size":0,"ondisk_log_size":8870,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36311040,"num_objects":8865,"num_object_clones":0,"num_object_copies":17730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6687,"num_write_kb":26748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":4841}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8704","reported_seq":16684,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:38.770592+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:15:38.770592+0000","last_peered":"2024-09-11T01:15:38.770592+0000","last_clean":"2024-09-11T01:15:38.770592+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:15:38.770592+0000","last_undegraded":"2024-09-11T01:15:38.770592+0000","last_fullsized":"2024-09-11T01:15:38.770592+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":8704,"log_dups_size":0,"ondisk_log_size":8704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35627008,"num_objects":8698,"num_object_clones":0,"num_object_copies":17396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8698,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6489,"num_write_kb":25956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'8928","reported_seq":9059,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.704628+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:15:39.704628+0000","last_peered":"2024-09-11T01:15:39.704628+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:15:39.704628+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":8928,"log_dups_size":0,"ondisk_log_size":8928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36560896,"num_objects":8926,"num_object_clones":0,"num_object_copies":17852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5665,"num_write_kb":22660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4976},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'8728","reported_seq":9593,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:39.828179+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:15:39.828179+0000","last_peered":"2024-09-11T01:15:39.828179+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:15:39.828179+0000","last_undegraded":"2024-09-11T01:15:39.828179+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":8728,"log_dups_size":0,"ondisk_log_size":8728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35717120,"num_objects":8720,"num_object_clones":0,"num_object_copies":17440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3574,"num_objects_unfound":0,"num_objects_dirty":8720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5383,"num_write_kb":21532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":365,"num_bytes_recovered":1495040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5146},{"shards":"0,2","objects":3574}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694344+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694344+0000","last_peered":"2024-09-11T01:15:21.694344+0000","last_clean":"2024-09-11T01:15:21.694344+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694344+0000","last_undegraded":"2024-09-11T01:15:21.694344+0000","last_fullsized":"2024-09-11T01:15:21.694344+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'8843","reported_seq":8977,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.762690+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:15:39.762690+0000","last_peered":"2024-09-11T01:15:39.762690+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:15:39.762690+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":8843,"log_dups_size":0,"ondisk_log_size":8843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36204544,"num_objects":8839,"num_object_clones":0,"num_object_copies":17678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5452,"num_write_kb":21808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4800},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'8827","reported_seq":8959,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.735883+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:15:39.735883+0000","last_peered":"2024-09-11T01:15:39.735883+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:15:39.735883+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":8827,"log_dups_size":0,"ondisk_log_size":8827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36106857,"num_objects":8816,"num_object_clones":0,"num_object_copies":17632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8816,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5897,"num_write_kb":23588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4899},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":577479273,"num_objects":140987,"num_object_clones":0,"num_object_copies":281974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":3574,"num_objects_unfound":0,"num_objects_dirty":140987,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101918,"num_write_kb":407672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4293,"num_bytes_recovered":17575936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1170747392,"data_stored":1170667375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141068,"ondisk_log_size":141068,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182450,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":797704,"kb_used_data":487032,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93574136,"statfs":{"total":96636764160,"available":95819915264,"internally_reserved":0,"allocated":498720768,"data_stored":498461554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":318107307},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,3,0,9,1,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986222,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":671372,"kb_used_data":409272,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93700468,"statfs":{"total":96636764160,"available":95949279232,"internally_reserved":0,"allocated":419094528,"data_stored":418839391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268365482},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,6,2,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659167,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":441592,"kb_used_data":247528,"kb_used_omap":4,"kb_used_meta":194043,"kb_avail":93930248,"statfs":{"total":96636764160,"available":96184573952,"internally_reserved":0,"allocated":253468672,"data_stored":253139878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":198700719},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,0,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":498573312,"data_stored":498569833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":253280256,"data_stored":253207197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":418893824,"data_stored":418890345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:42.981 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:43.427 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 199 20 142812 142792 2.80175 1.71094 0.0833063 0.0278744 2024-09-11T01:15:43.803 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:15:43.804 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:15:43.837 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":178,"stamp":"2024-09-11T01:15:42.622345+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":577479292,"num_objects":140989,"num_object_clones":0,"num_object_copies":281978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":3574,"num_objects_unfound":0,"num_objects_dirty":140989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101920,"num_write_kb":407674,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4294,"num_bytes_recovered":17575955,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141071,"ondisk_log_size":141071,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1910668,"kb_used_data":1143832,"kb_used_omap":12,"kb_used_meta":766771,"kb_avail":281204852,"statfs":{"total":289910292480,"available":287953768448,"internally_reserved":0,"allocated":1171283968,"data_stored":1170440823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":785173508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,4,3,8,14,6,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18911232,"num_objects":4617,"num_object_clones":0,"num_object_copies":9234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-177,"num_objects_unfound":0,"num_objects_dirty":4617,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4617,"num_write_kb":18468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":724992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006121"},"pg_stats":[{"pgid":"2.f","version":"45'8919","reported_seq":8996,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.639749+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:15:39.639749+0000","last_peered":"2024-09-11T01:15:39.639749+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:15:39.639749+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":8919,"log_dups_size":0,"ondisk_log_size":8919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36532224,"num_objects":8919,"num_object_clones":0,"num_object_copies":17838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8919,"num_write_kb":35676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4847},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'8907","reported_seq":9008,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:38.811849+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:15:38.811849+0000","last_peered":"2024-09-11T01:15:38.811849+0000","last_clean":"2024-09-11T01:15:38.811849+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:15:38.811849+0000","last_undegraded":"2024-09-11T01:15:38.811849+0000","last_fullsized":"2024-09-11T01:15:38.811849+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":8907,"log_dups_size":0,"ondisk_log_size":8907,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36458496,"num_objects":8901,"num_object_clones":0,"num_object_copies":17802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7146,"num_write_kb":28584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'8779","reported_seq":8905,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:38.811787+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:15:38.811787+0000","last_peered":"2024-09-11T01:15:38.811787+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:15:38.811787+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":8779,"log_dups_size":0,"ondisk_log_size":8779,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35942400,"num_objects":8775,"num_object_clones":0,"num_object_copies":17550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7080,"num_write_kb":28320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":4879}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'8835","reported_seq":8904,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:38.724668+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:15:38.724668+0000","last_peered":"2024-09-11T01:15:38.724668+0000","last_clean":"2024-09-11T01:15:38.724668+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:15:38.724668+0000","last_undegraded":"2024-09-11T01:15:38.724668+0000","last_fullsized":"2024-09-11T01:15:38.724668+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":8835,"log_dups_size":0,"ondisk_log_size":8835,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36188160,"num_objects":8835,"num_object_clones":0,"num_object_copies":17670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8835,"num_write_kb":35340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'8788","reported_seq":8915,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:39.773513+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:15:39.773513+0000","last_peered":"2024-09-11T01:15:39.773513+0000","last_clean":"2024-09-11T01:15:39.773513+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:15:39.773513+0000","last_undegraded":"2024-09-11T01:15:39.773513+0000","last_fullsized":"2024-09-11T01:15:39.773513+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":8788,"log_dups_size":0,"ondisk_log_size":8788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35966976,"num_objects":8781,"num_object_clones":0,"num_object_copies":17562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5793,"num_write_kb":23172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'8875","reported_seq":9045,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.765707+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:15:39.765707+0000","last_peered":"2024-09-11T01:15:39.765707+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:15:39.765707+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":8875,"log_dups_size":0,"ondisk_log_size":8875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36331520,"num_objects":8870,"num_object_clones":0,"num_object_copies":17740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5216,"num_write_kb":20864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4919},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'8702","reported_seq":8832,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.730415+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:15:39.730415+0000","last_peered":"2024-09-11T01:15:39.730415+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:15:39.730415+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":8702,"log_dups_size":0,"ondisk_log_size":8702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35618816,"num_objects":8696,"num_object_clones":0,"num_object_copies":17392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8696,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5559,"num_write_kb":22236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4710},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'8856","reported_seq":8985,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:39.793343+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:15:39.793343+0000","last_peered":"2024-09-11T01:15:39.793343+0000","last_clean":"2024-09-11T01:15:39.793343+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:15:39.793343+0000","last_undegraded":"2024-09-11T01:15:39.793343+0000","last_fullsized":"2024-09-11T01:15:39.793343+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":8856,"log_dups_size":0,"ondisk_log_size":8856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36265984,"num_objects":8854,"num_object_clones":0,"num_object_copies":17708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5732,"num_write_kb":22928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8783","reported_seq":8910,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:38.742001+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:15:38.742001+0000","last_peered":"2024-09-11T01:15:38.742001+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:15:38.742001+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":8783,"log_dups_size":0,"ondisk_log_size":8783,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35946496,"num_objects":8776,"num_object_clones":0,"num_object_copies":17552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6754,"num_write_kb":27016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":4843}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692287+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692287+0000","last_peered":"2024-09-11T01:15:21.692287+0000","last_clean":"2024-09-11T01:15:21.692287+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692287+0000","last_undegraded":"2024-09-11T01:15:21.692287+0000","last_fullsized":"2024-09-11T01:15:21.692287+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'8724","reported_seq":8851,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.781879+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:15:39.781879+0000","last_peered":"2024-09-11T01:15:39.781879+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:15:39.781879+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":8724,"log_dups_size":0,"ondisk_log_size":8724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35700736,"num_objects":8716,"num_object_clones":0,"num_object_copies":17432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5311,"num_write_kb":21244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4732},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.685123+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.685123+0000","last_peered":"2024-09-11T01:15:21.685123+0000","last_clean":"2024-09-11T01:15:21.685123+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.685123+0000","last_undegraded":"2024-09-11T01:15:21.685123+0000","last_fullsized":"2024-09-11T01:15:21.685123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'8870","reported_seq":8987,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:38.837351+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:15:38.837351+0000","last_peered":"2024-09-11T01:15:38.837351+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:15:38.837351+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":8870,"log_dups_size":0,"ondisk_log_size":8870,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36311040,"num_objects":8865,"num_object_clones":0,"num_object_copies":17730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6687,"num_write_kb":26748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":4841}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8704","reported_seq":16684,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:38.770592+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:15:38.770592+0000","last_peered":"2024-09-11T01:15:38.770592+0000","last_clean":"2024-09-11T01:15:38.770592+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:15:38.770592+0000","last_undegraded":"2024-09-11T01:15:38.770592+0000","last_fullsized":"2024-09-11T01:15:38.770592+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":8704,"log_dups_size":0,"ondisk_log_size":8704,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35627008,"num_objects":8698,"num_object_clones":0,"num_object_copies":17396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8698,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6489,"num_write_kb":25956,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'8928","reported_seq":9059,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.704628+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:15:39.704628+0000","last_peered":"2024-09-11T01:15:39.704628+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:15:39.704628+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":8928,"log_dups_size":0,"ondisk_log_size":8928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36560896,"num_objects":8926,"num_object_clones":0,"num_object_copies":17852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5665,"num_write_kb":22660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4976},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'8728","reported_seq":9593,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:39.828179+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:15:39.828179+0000","last_peered":"2024-09-11T01:15:39.828179+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:15:39.828179+0000","last_undegraded":"2024-09-11T01:15:39.828179+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":8728,"log_dups_size":0,"ondisk_log_size":8728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35717120,"num_objects":8720,"num_object_clones":0,"num_object_copies":17440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3574,"num_objects_unfound":0,"num_objects_dirty":8720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5383,"num_write_kb":21532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":365,"num_bytes_recovered":1495040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5146},{"shards":"0,2","objects":3574}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694344+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694344+0000","last_peered":"2024-09-11T01:15:21.694344+0000","last_clean":"2024-09-11T01:15:21.694344+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694344+0000","last_undegraded":"2024-09-11T01:15:21.694344+0000","last_fullsized":"2024-09-11T01:15:21.694344+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'8843","reported_seq":8977,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.762690+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:15:39.762690+0000","last_peered":"2024-09-11T01:15:39.762690+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:15:39.762690+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":8843,"log_dups_size":0,"ondisk_log_size":8843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36204544,"num_objects":8839,"num_object_clones":0,"num_object_copies":17678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5452,"num_write_kb":21808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4800},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'8827","reported_seq":8959,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.735883+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:15:39.735883+0000","last_peered":"2024-09-11T01:15:39.735883+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:15:39.735883+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":8827,"log_dups_size":0,"ondisk_log_size":8827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36106857,"num_objects":8816,"num_object_clones":0,"num_object_copies":17632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8816,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5897,"num_write_kb":23588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4899},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":577479273,"num_objects":140987,"num_object_clones":0,"num_object_copies":281974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":3574,"num_objects_unfound":0,"num_objects_dirty":140987,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":101918,"num_write_kb":407672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4293,"num_bytes_recovered":17575936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1170747392,"data_stored":1170667375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141068,"ondisk_log_size":141068,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182450,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":797704,"kb_used_data":487032,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93574136,"statfs":{"total":96636764160,"available":95819915264,"internally_reserved":0,"allocated":498720768,"data_stored":498461554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":318107307},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,3,0,9,1,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986222,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":671372,"kb_used_data":409272,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93700468,"statfs":{"total":96636764160,"available":95949279232,"internally_reserved":0,"allocated":419094528,"data_stored":418839391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268365482},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,6,2,5,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659167,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":441592,"kb_used_data":247528,"kb_used_omap":4,"kb_used_meta":194043,"kb_avail":93930248,"statfs":{"total":96636764160,"available":96184573952,"internally_reserved":0,"allocated":253468672,"data_stored":253139878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":198700719},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,0,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":498573312,"data_stored":498569833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":253280256,"data_stored":253207197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":418893824,"data_stored":418890345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:43.838 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:15:43.838 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'8919", 'reported_seq': 8996, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:39.639749+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:15:39.639749+0000', 'last_peered': '2024-09-11T01:15:39.639749+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:15:39.639749+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 8919, 'log_dups_size': 0, 'ondisk_log_size': 8919, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36532224, 'num_objects': 8919, 'num_object_clones': 0, 'num_object_copies': 17838, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8919, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8919, 'num_write_kb': 35676, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4847}, {'shards': '0,2', 'objects': 4072}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:43.838 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:15:43.838 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'8779", 'reported_seq': 8905, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:38.811787+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:15:38.811787+0000', 'last_peered': '2024-09-11T01:15:38.811787+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:15:38.811787+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 8779, 'log_dups_size': 0, 'ondisk_log_size': 8779, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35942400, 'num_objects': 8775, 'num_object_clones': 0, 'num_object_copies': 17550, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8775, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7080, 'num_write_kb': 28320, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3896}, {'shards': '2', 'objects': 4879}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:43.838 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:15:43.838 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'8875", 'reported_seq': 9045, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:39.765707+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:15:39.765707+0000', 'last_peered': '2024-09-11T01:15:39.765707+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:15:39.765707+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 8875, 'log_dups_size': 0, 'ondisk_log_size': 8875, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36331520, 'num_objects': 8870, 'num_object_clones': 0, 'num_object_copies': 17740, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8870, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5216, 'num_write_kb': 20864, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4919}, {'shards': '0,2', 'objects': 3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:43.839 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:15:43.839 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'8702", 'reported_seq': 8832, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:39.730415+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:15:39.730415+0000', 'last_peered': '2024-09-11T01:15:39.730415+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:15:39.730415+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 8702, 'log_dups_size': 0, 'ondisk_log_size': 8702, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35618816, 'num_objects': 8696, 'num_object_clones': 0, 'num_object_copies': 17392, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8696, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5559, 'num_write_kb': 22236, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4710}, {'shards': '0,2', 'objects': 3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:43.839 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:15:43.839 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'8783", 'reported_seq': 8910, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:38.742001+0000', 'last_change': '2024-09-11T01:13:10.159069+0000', 'last_active': '2024-09-11T01:15:38.742001+0000', 'last_peered': '2024-09-11T01:15:38.742001+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:15:38.742001+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 8783, 'log_dups_size': 0, 'ondisk_log_size': 8783, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35946496, 'num_objects': 8776, 'num_object_clones': 0, 'num_object_copies': 17552, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8776, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6754, 'num_write_kb': 27016, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3932}, {'shards': '2', 'objects': 4843}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:43.839 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:15:43.839 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'8724", 'reported_seq': 8851, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:39.781879+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:15:39.781879+0000', 'last_peered': '2024-09-11T01:15:39.781879+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:15:39.781879+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 8724, 'log_dups_size': 0, 'ondisk_log_size': 8724, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35700736, 'num_objects': 8716, 'num_object_clones': 0, 'num_object_copies': 17432, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8716, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5311, 'num_write_kb': 21244, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4732}, {'shards': '0,2', 'objects': 3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:43.839 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:15:43.839 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'8870", 'reported_seq': 8987, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:38.837351+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:15:38.837351+0000', 'last_peered': '2024-09-11T01:15:38.837351+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:15:38.837351+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 8870, 'log_dups_size': 0, 'ondisk_log_size': 8870, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36311040, 'num_objects': 8865, 'num_object_clones': 0, 'num_object_copies': 17730, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8865, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6687, 'num_write_kb': 26748, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4023}, {'shards': '2', 'objects': 4841}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:43.839 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:15:43.839 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'8928", 'reported_seq': 9059, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:39.704628+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:15:39.704628+0000', 'last_peered': '2024-09-11T01:15:39.704628+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:15:39.704628+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 8928, 'log_dups_size': 0, 'ondisk_log_size': 8928, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36560896, 'num_objects': 8926, 'num_object_clones': 0, 'num_object_copies': 17852, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8926, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5665, 'num_write_kb': 22660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4976}, {'shards': '0,2', 'objects': 3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:43.840 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:15:43.840 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'8728", 'reported_seq': 9593, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:15:39.828179+0000', 'last_change': '2024-09-11T01:15:19.913288+0000', 'last_active': '2024-09-11T01:15:39.828179+0000', 'last_peered': '2024-09-11T01:15:39.828179+0000', 'last_clean': '2024-09-11T01:13:07.624565+0000', 'last_became_active': '2024-09-11T01:13:10.199795+0000', 'last_became_peered': '2024-09-11T01:13:10.199795+0000', 'last_unstale': '2024-09-11T01:15:39.828179+0000', 'last_undegraded': '2024-09-11T01:15:39.828179+0000', 'last_fullsized': '2024-09-11T01:13:09.652311+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3415", 'last_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'objects_scrubbed': 3337, 'log_size': 8728, 'log_dups_size': 0, 'ondisk_log_size': 8728, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1975, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35717120, 'num_objects': 8720, 'num_object_clones': 0, 'num_object_copies': 17440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3574, 'num_objects_unfound': 0, 'num_objects_dirty': 8720, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5383, 'num_write_kb': 21532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 365, 'num_bytes_recovered': 1495040, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5146}, {'shards': '0,2', 'objects': 3574}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:43.840 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:15:43.840 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'8843", 'reported_seq': 8977, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:39.762690+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:15:39.762690+0000', 'last_peered': '2024-09-11T01:15:39.762690+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:15:39.762690+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 8843, 'log_dups_size': 0, 'ondisk_log_size': 8843, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36204544, 'num_objects': 8839, 'num_object_clones': 0, 'num_object_copies': 17678, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8839, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5452, 'num_write_kb': 21808, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4800}, {'shards': '0,2', 'objects': 4039}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:43.840 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:15:43.840 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'8827", 'reported_seq': 8959, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:39.735883+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:15:39.735883+0000', 'last_peered': '2024-09-11T01:15:39.735883+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:15:39.735883+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 8827, 'log_dups_size': 0, 'ondisk_log_size': 8827, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36106857, 'num_objects': 8816, 'num_object_clones': 0, 'num_object_copies': 17632, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8816, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5897, 'num_write_kb': 23588, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4899}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:43.840 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:15:44.423 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout:2024-09-11T01:15:44.424544+0000 min lat: 0.00219431 max lat: 2.69211 avg lat: 0.02793 2024-09-11T01:15:44.424 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T01:15:44.424 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 200 20 143259 143239 2.79648 1.74609 0.0178481 0.02793 2024-09-11T01:15:44.787 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:15:44.787 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39769/281978 objects degraded (14.104%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":281,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":140989,"data_bytes":577479292,"bytes_used":1956524032,"bytes_avail":287953768448,"bytes_total":289910292480,"degraded_objects":39769,"degraded_total":281978,"degraded_ratio":0.14103582548993185,"misplaced_objects":3574,"misplaced_total":281978,"misplaced_ratio":0.01267474767535056,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":60385,"recovering_keys_per_sec":0,"num_objects_recovered":177,"num_bytes_recovered":724992,"num_keys_recovered":0,"write_bytes_sec":1575132,"read_op_per_sec":0,"write_op_per_sec":384},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:15:44.837 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:15:44.837 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:45.424 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 201 20 143683 143663 2.79081 1.65625 0.0932904 0.0279838 2024-09-11T01:15:45.781 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:15:45.781 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:15:45.839 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":179,"stamp":"2024-09-11T01:15:44.622847+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":580989564,"num_objects":141846,"num_object_clones":0,"num_object_copies":283692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":3574,"num_objects_unfound":0,"num_objects_dirty":141846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102777,"num_write_kb":411102,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4294,"num_bytes_recovered":17575955,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141928,"ondisk_log_size":141928,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1935152,"kb_used_data":1155840,"kb_used_omap":12,"kb_used_meta":779251,"kb_avail":281180368,"statfs":{"total":289910292480,"available":287928696832,"internally_reserved":0,"allocated":1183580160,"data_stored":1182737015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":797953028},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,3,4,3,14,7,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22421504,"num_objects":5474,"num_object_clones":0,"num_object_copies":10948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-177,"num_objects_unfound":0,"num_objects_dirty":5474,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5474,"num_write_kb":21896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":177,"num_bytes_recovered":724992,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006063"},"pg_stats":[{"pgid":"2.f","version":"45'8919","reported_seq":8996,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.639749+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:15:39.639749+0000","last_peered":"2024-09-11T01:15:39.639749+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:15:39.639749+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":8919,"log_dups_size":0,"ondisk_log_size":8919,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36532224,"num_objects":8919,"num_object_clones":0,"num_object_copies":17838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8919,"num_write_kb":35676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4847},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9035","reported_seq":9135,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:43.824355+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:15:43.824355+0000","last_peered":"2024-09-11T01:15:43.824355+0000","last_clean":"2024-09-11T01:15:43.824355+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:15:43.824355+0000","last_undegraded":"2024-09-11T01:15:43.824355+0000","last_fullsized":"2024-09-11T01:15:43.824355+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":9035,"log_dups_size":0,"ondisk_log_size":9035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36982784,"num_objects":9029,"num_object_clones":0,"num_object_copies":18058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7274,"num_write_kb":29096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'8928","reported_seq":9055,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:43.685183+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:15:43.685183+0000","last_peered":"2024-09-11T01:15:43.685183+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:15:43.685183+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":8928,"log_dups_size":0,"ondisk_log_size":8928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36552704,"num_objects":8924,"num_object_clones":0,"num_object_copies":17848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":28916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":5028}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'8978","reported_seq":9047,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:43.823289+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:15:43.823289+0000","last_peered":"2024-09-11T01:15:43.823289+0000","last_clean":"2024-09-11T01:15:43.823289+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:15:43.823289+0000","last_undegraded":"2024-09-11T01:15:43.823289+0000","last_fullsized":"2024-09-11T01:15:43.823289+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":8978,"log_dups_size":0,"ondisk_log_size":8978,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36773888,"num_objects":8978,"num_object_clones":0,"num_object_copies":17956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8978,"num_write_kb":35912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'8788","reported_seq":8915,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:39.773513+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:15:39.773513+0000","last_peered":"2024-09-11T01:15:39.773513+0000","last_clean":"2024-09-11T01:15:39.773513+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:15:39.773513+0000","last_undegraded":"2024-09-11T01:15:39.773513+0000","last_fullsized":"2024-09-11T01:15:39.773513+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":8788,"log_dups_size":0,"ondisk_log_size":8788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35966976,"num_objects":8781,"num_object_clones":0,"num_object_copies":17562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5793,"num_write_kb":23172,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'8875","reported_seq":9045,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.765707+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:15:39.765707+0000","last_peered":"2024-09-11T01:15:39.765707+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:15:39.765707+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":8875,"log_dups_size":0,"ondisk_log_size":8875,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36331520,"num_objects":8870,"num_object_clones":0,"num_object_copies":17740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5216,"num_write_kb":20864,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4919},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'8702","reported_seq":8832,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.730415+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:15:39.730415+0000","last_peered":"2024-09-11T01:15:39.730415+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:15:39.730415+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":8702,"log_dups_size":0,"ondisk_log_size":8702,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35618816,"num_objects":8696,"num_object_clones":0,"num_object_copies":17392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8696,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5559,"num_write_kb":22236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4710},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'8856","reported_seq":8985,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:39.793343+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:15:39.793343+0000","last_peered":"2024-09-11T01:15:39.793343+0000","last_clean":"2024-09-11T01:15:39.793343+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:15:39.793343+0000","last_undegraded":"2024-09-11T01:15:39.793343+0000","last_fullsized":"2024-09-11T01:15:39.793343+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":8856,"log_dups_size":0,"ondisk_log_size":8856,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36265984,"num_objects":8854,"num_object_clones":0,"num_object_copies":17708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8854,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5732,"num_write_kb":22928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8934","reported_seq":9061,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:43.653930+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:15:43.653930+0000","last_peered":"2024-09-11T01:15:43.653930+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:15:43.653930+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":8934,"log_dups_size":0,"ondisk_log_size":8934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36564992,"num_objects":8927,"num_object_clones":0,"num_object_copies":17854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6905,"num_write_kb":27620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":4994}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692287+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692287+0000","last_peered":"2024-09-11T01:15:21.692287+0000","last_clean":"2024-09-11T01:15:21.692287+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692287+0000","last_undegraded":"2024-09-11T01:15:21.692287+0000","last_fullsized":"2024-09-11T01:15:21.692287+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'8724","reported_seq":8851,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.781879+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:15:39.781879+0000","last_peered":"2024-09-11T01:15:39.781879+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:15:39.781879+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":8724,"log_dups_size":0,"ondisk_log_size":8724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35700736,"num_objects":8716,"num_object_clones":0,"num_object_copies":17432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5311,"num_write_kb":21244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4732},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.685123+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.685123+0000","last_peered":"2024-09-11T01:15:21.685123+0000","last_clean":"2024-09-11T01:15:21.685123+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.685123+0000","last_undegraded":"2024-09-11T01:15:21.685123+0000","last_fullsized":"2024-09-11T01:15:21.685123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9011","reported_seq":9129,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:43.709480+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:15:43.709480+0000","last_peered":"2024-09-11T01:15:43.709480+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:15:43.709480+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":9011,"log_dups_size":0,"ondisk_log_size":9011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36888576,"num_objects":9006,"num_object_clones":0,"num_object_copies":18012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6828,"num_write_kb":27312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":4982}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8849","reported_seq":16829,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:43.788552+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:15:43.788552+0000","last_peered":"2024-09-11T01:15:43.788552+0000","last_clean":"2024-09-11T01:15:43.788552+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:15:43.788552+0000","last_undegraded":"2024-09-11T01:15:43.788552+0000","last_fullsized":"2024-09-11T01:15:43.788552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":8849,"log_dups_size":0,"ondisk_log_size":8849,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36220928,"num_objects":8843,"num_object_clones":0,"num_object_copies":17686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6634,"num_write_kb":26536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'8928","reported_seq":9059,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.704628+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:15:39.704628+0000","last_peered":"2024-09-11T01:15:39.704628+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:15:39.704628+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":8928,"log_dups_size":0,"ondisk_log_size":8928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36560896,"num_objects":8926,"num_object_clones":0,"num_object_copies":17852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5665,"num_write_kb":22660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4976},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'8728","reported_seq":9593,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:39.828179+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:15:39.828179+0000","last_peered":"2024-09-11T01:15:39.828179+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:15:39.828179+0000","last_undegraded":"2024-09-11T01:15:39.828179+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":8728,"log_dups_size":0,"ondisk_log_size":8728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35717120,"num_objects":8720,"num_object_clones":0,"num_object_copies":17440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3574,"num_objects_unfound":0,"num_objects_dirty":8720,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5383,"num_write_kb":21532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":365,"num_bytes_recovered":1495040,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5146},{"shards":"0,2","objects":3574}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694344+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694344+0000","last_peered":"2024-09-11T01:15:21.694344+0000","last_clean":"2024-09-11T01:15:21.694344+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694344+0000","last_undegraded":"2024-09-11T01:15:21.694344+0000","last_fullsized":"2024-09-11T01:15:21.694344+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'8843","reported_seq":8977,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.762690+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:15:39.762690+0000","last_peered":"2024-09-11T01:15:39.762690+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:15:39.762690+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":8843,"log_dups_size":0,"ondisk_log_size":8843,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36204544,"num_objects":8839,"num_object_clones":0,"num_object_copies":17678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5452,"num_write_kb":21808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4800},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'8827","reported_seq":8959,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:39.735883+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:15:39.735883+0000","last_peered":"2024-09-11T01:15:39.735883+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:15:39.735883+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":8827,"log_dups_size":0,"ondisk_log_size":8827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36106857,"num_objects":8816,"num_object_clones":0,"num_object_copies":17632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8816,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5897,"num_write_kb":23588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4899},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":580989545,"num_objects":141844,"num_object_clones":0,"num_object_copies":283688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":3574,"num_objects_unfound":0,"num_objects_dirty":141844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":102775,"num_write_kb":411100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4293,"num_bytes_recovered":17575936,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1182941184,"data_stored":1182861167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":141925,"ondisk_log_size":141925,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182450,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":797704,"kb_used_data":487032,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93574136,"statfs":{"total":96636764160,"available":95819915264,"internally_reserved":0,"allocated":498720768,"data_stored":498461554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":318107307},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,3,0,9,1,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986223,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":680172,"kb_used_data":413916,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93691668,"statfs":{"total":96636764160,"available":95940268032,"internally_reserved":0,"allocated":423849984,"data_stored":423594847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,0,1,1,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659168,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":457276,"kb_used_data":254892,"kb_used_omap":4,"kb_used_meta":202363,"kb_avail":93914564,"statfs":{"total":96636764160,"available":96168513536,"internally_reserved":0,"allocated":261009408,"data_stored":260680614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":207220399},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":498573312,"data_stored":498569833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":260788224,"data_stored":260715165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":423579648,"data_stored":423576169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:46.426 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 202 20 144144 144124 2.78587 1.80078 0.0575274 0.0280357 2024-09-11T01:15:47.427 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 203 20 144578 144558 2.78049 1.69531 0.00591981 0.0280881 2024-09-11T01:15:48.427 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 204 20 145057 145037 2.77603 1.87109 0.0345615 0.0281349 2024-09-11T01:15:48.841 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:49.430 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 205 20 145501 145481 2.77091 1.73438 0.0815451 0.0281861 2024-09-11T01:15:49.845 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:15:49.851 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:15:49.917 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":181,"stamp":"2024-09-11T01:15:48.627355+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":586822268,"num_objects":143270,"num_object_clones":0,"num_object_copies":286540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":3482,"num_objects_unfound":0,"num_objects_dirty":143270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104201,"num_write_kb":416798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4386,"num_bytes_recovered":17952787,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143352,"ondisk_log_size":143352,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1927788,"kb_used_data":1169276,"kb_used_omap":12,"kb_used_meta":758451,"kb_avail":281187732,"statfs":{"total":289910292480,"available":287936237568,"internally_reserved":0,"allocated":1197338624,"data_stored":1196495479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":776653828},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,3,2,6,7,19,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18591744,"num_objects":4539,"num_object_clones":0,"num_object_copies":9078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-185,"num_objects_unfound":0,"num_objects_dirty":4539,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4539,"num_write_kb":18156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":757760,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006970"},"pg_stats":[{"pgid":"2.f","version":"45'9069","reported_seq":9143,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:44.844120+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:15:44.844120+0000","last_peered":"2024-09-11T01:15:44.844120+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:15:44.844120+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":9069,"log_dups_size":0,"ondisk_log_size":9069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37146624,"num_objects":9069,"num_object_clones":0,"num_object_copies":18138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9069,"num_write_kb":36276,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4997},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9035","reported_seq":9135,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:43.824355+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:15:43.824355+0000","last_peered":"2024-09-11T01:15:43.824355+0000","last_clean":"2024-09-11T01:15:43.824355+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:15:43.824355+0000","last_undegraded":"2024-09-11T01:15:43.824355+0000","last_fullsized":"2024-09-11T01:15:43.824355+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":9035,"log_dups_size":0,"ondisk_log_size":9035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36982784,"num_objects":9029,"num_object_clones":0,"num_object_copies":18058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7274,"num_write_kb":29096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'8928","reported_seq":9055,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:43.685183+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:15:43.685183+0000","last_peered":"2024-09-11T01:15:43.685183+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:15:43.685183+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":8928,"log_dups_size":0,"ondisk_log_size":8928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36552704,"num_objects":8924,"num_object_clones":0,"num_object_copies":17848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7229,"num_write_kb":28916,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":5028}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'8978","reported_seq":9047,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:43.823289+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:15:43.823289+0000","last_peered":"2024-09-11T01:15:43.823289+0000","last_clean":"2024-09-11T01:15:43.823289+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:15:43.823289+0000","last_undegraded":"2024-09-11T01:15:43.823289+0000","last_fullsized":"2024-09-11T01:15:43.823289+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":8978,"log_dups_size":0,"ondisk_log_size":8978,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36773888,"num_objects":8978,"num_object_clones":0,"num_object_copies":17956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8978,"num_write_kb":35912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'8929","reported_seq":9056,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:44.806242+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:15:44.806242+0000","last_peered":"2024-09-11T01:15:44.806242+0000","last_clean":"2024-09-11T01:15:44.806242+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:15:44.806242+0000","last_undegraded":"2024-09-11T01:15:44.806242+0000","last_fullsized":"2024-09-11T01:15:44.806242+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":8929,"log_dups_size":0,"ondisk_log_size":8929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36544512,"num_objects":8922,"num_object_clones":0,"num_object_copies":17844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8922,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5934,"num_write_kb":23736,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9017","reported_seq":9187,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:44.824210+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:15:44.824210+0000","last_peered":"2024-09-11T01:15:44.824210+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:15:44.824210+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":9017,"log_dups_size":0,"ondisk_log_size":9017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36913152,"num_objects":9012,"num_object_clones":0,"num_object_copies":18024,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9012,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5358,"num_write_kb":21432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5061},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'8839","reported_seq":8964,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:44.735558+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:15:44.735558+0000","last_peered":"2024-09-11T01:15:44.735558+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:15:44.735558+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":8839,"log_dups_size":0,"ondisk_log_size":8839,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36179968,"num_objects":8833,"num_object_clones":0,"num_object_copies":17666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8833,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5696,"num_write_kb":22784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4847},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9008","reported_seq":9136,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:44.812891+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:15:44.812891+0000","last_peered":"2024-09-11T01:15:44.812891+0000","last_clean":"2024-09-11T01:15:44.812891+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:15:44.812891+0000","last_undegraded":"2024-09-11T01:15:44.812891+0000","last_fullsized":"2024-09-11T01:15:44.812891+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":9008,"log_dups_size":0,"ondisk_log_size":9008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36888576,"num_objects":9006,"num_object_clones":0,"num_object_copies":18012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5884,"num_write_kb":23536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'8934","reported_seq":9061,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:43.653930+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:15:43.653930+0000","last_peered":"2024-09-11T01:15:43.653930+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:15:43.653930+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":8934,"log_dups_size":0,"ondisk_log_size":8934,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36564992,"num_objects":8927,"num_object_clones":0,"num_object_copies":17854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6905,"num_write_kb":27620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":4994}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692287+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692287+0000","last_peered":"2024-09-11T01:15:21.692287+0000","last_clean":"2024-09-11T01:15:21.692287+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692287+0000","last_undegraded":"2024-09-11T01:15:21.692287+0000","last_fullsized":"2024-09-11T01:15:21.692287+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'8865","reported_seq":8993,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:44.701125+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:15:44.701125+0000","last_peered":"2024-09-11T01:15:44.701125+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:15:44.701125+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":8865,"log_dups_size":0,"ondisk_log_size":8865,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36278272,"num_objects":8857,"num_object_clones":0,"num_object_copies":17714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5452,"num_write_kb":21808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4873},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.685123+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.685123+0000","last_peered":"2024-09-11T01:15:21.685123+0000","last_clean":"2024-09-11T01:15:21.685123+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.685123+0000","last_undegraded":"2024-09-11T01:15:21.685123+0000","last_fullsized":"2024-09-11T01:15:21.685123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9011","reported_seq":9129,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:43.709480+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:15:43.709480+0000","last_peered":"2024-09-11T01:15:43.709480+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:15:43.709480+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":9011,"log_dups_size":0,"ondisk_log_size":9011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36888576,"num_objects":9006,"num_object_clones":0,"num_object_copies":18012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6828,"num_write_kb":27312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":4982}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8849","reported_seq":16829,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:43.788552+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:15:43.788552+0000","last_peered":"2024-09-11T01:15:43.788552+0000","last_clean":"2024-09-11T01:15:43.788552+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:15:43.788552+0000","last_undegraded":"2024-09-11T01:15:43.788552+0000","last_fullsized":"2024-09-11T01:15:43.788552+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":8849,"log_dups_size":0,"ondisk_log_size":8849,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36220928,"num_objects":8843,"num_object_clones":0,"num_object_copies":17686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8843,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6634,"num_write_kb":26536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9071","reported_seq":9200,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:44.841745+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:15:44.841745+0000","last_peered":"2024-09-11T01:15:44.841745+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:15:44.841745+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":9071,"log_dups_size":0,"ondisk_log_size":9071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37146624,"num_objects":9069,"num_object_clones":0,"num_object_copies":18138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5808,"num_write_kb":23232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5119},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'8864","reported_seq":9915,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:44.826296+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:15:44.826296+0000","last_peered":"2024-09-11T01:15:44.826296+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:15:44.826296+0000","last_undegraded":"2024-09-11T01:15:44.826296+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":8864,"log_dups_size":0,"ondisk_log_size":8864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36274176,"num_objects":8856,"num_object_clones":0,"num_object_copies":17712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3482,"num_objects_unfound":0,"num_objects_dirty":8856,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5519,"num_write_kb":22076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":457,"num_bytes_recovered":1871872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5374},{"shards":"0,2","objects":3482}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694344+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694344+0000","last_peered":"2024-09-11T01:15:21.694344+0000","last_clean":"2024-09-11T01:15:21.694344+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694344+0000","last_undegraded":"2024-09-11T01:15:21.694344+0000","last_fullsized":"2024-09-11T01:15:21.694344+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'8985","reported_seq":9119,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:44.803890+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:15:44.803890+0000","last_peered":"2024-09-11T01:15:44.803890+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:15:44.803890+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":8985,"log_dups_size":0,"ondisk_log_size":8985,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36786176,"num_objects":8981,"num_object_clones":0,"num_object_copies":17962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8981,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5594,"num_write_kb":22376,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4942},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'8967","reported_seq":9100,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:44.741646+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:15:44.741646+0000","last_peered":"2024-09-11T01:15:44.741646+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:15:44.741646+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":8967,"log_dups_size":0,"ondisk_log_size":8967,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36680297,"num_objects":8956,"num_object_clones":0,"num_object_copies":17912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8956,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6037,"num_write_kb":24148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5039},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":586822249,"num_objects":143268,"num_object_clones":0,"num_object_copies":286536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":3482,"num_objects_unfound":0,"num_objects_dirty":143268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":104199,"num_write_kb":416796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4385,"num_bytes_recovered":17952768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1197019136,"data_stored":1196939119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":143349,"ondisk_log_size":143349,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182451,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":795960,"kb_used_data":493608,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93575880,"statfs":{"total":96636764160,"available":95821701120,"internally_reserved":0,"allocated":505454592,"data_stored":505195378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":309587627},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,1,5,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986223,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":680172,"kb_used_data":413916,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93691668,"statfs":{"total":96636764160,"available":95940268032,"internally_reserved":0,"allocated":423849984,"data_stored":423594847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,0,1,1,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659169,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451656,"kb_used_data":261752,"kb_used_omap":4,"kb_used_meta":189883,"kb_avail":93920184,"statfs":{"total":96636764160,"available":96174268416,"internally_reserved":0,"allocated":268034048,"data_stored":267705254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":194440879},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,4,1,10,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":505495552,"data_stored":505492073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":267943936,"data_stored":267870877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":423579648,"data_stored":423576169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:49.919 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:50.430 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 206 20 145912 145892 2.76526 1.60547 0.0392521 0.0282438 2024-09-11T01:15:50.903 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:15:50.907 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:15:50.988 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":182,"stamp":"2024-09-11T01:15:50.627852+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596095612,"num_objects":145534,"num_object_clones":0,"num_object_copies":291068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":3390,"num_objects_unfound":0,"num_objects_dirty":145534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106465,"num_write_kb":425854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4478,"num_bytes_recovered":18329619,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145616,"ondisk_log_size":145616,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1947672,"kb_used_data":1180840,"kb_used_omap":12,"kb_used_meta":766771,"kb_avail":281167848,"statfs":{"total":289910292480,"available":287915876352,"internally_reserved":0,"allocated":1209180160,"data_stored":1208337015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":785173508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,0,1,10,6,15,3,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27865088,"num_objects":6803,"num_object_clones":0,"num_object_copies":13606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-277,"num_objects_unfound":0,"num_objects_dirty":6803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6803,"num_write_kb":27212,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":277,"num_bytes_recovered":1134592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006978"},"pg_stats":[{"pgid":"2.f","version":"45'9195","reported_seq":9272,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:49.858991+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:15:49.858991+0000","last_peered":"2024-09-11T01:15:49.858991+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:15:49.858991+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":9195,"log_dups_size":0,"ondisk_log_size":9195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37662720,"num_objects":9195,"num_object_clones":0,"num_object_copies":18390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9195,"num_write_kb":36780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5123},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9193","reported_seq":9294,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:48.759081+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:15:48.759081+0000","last_peered":"2024-09-11T01:15:48.759081+0000","last_clean":"2024-09-11T01:15:48.759081+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:15:48.759081+0000","last_undegraded":"2024-09-11T01:15:48.759081+0000","last_fullsized":"2024-09-11T01:15:48.759081+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":9193,"log_dups_size":0,"ondisk_log_size":9193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629952,"num_objects":9187,"num_object_clones":0,"num_object_copies":18374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7432,"num_write_kb":29728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9088","reported_seq":9215,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:48.829477+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:15:48.829477+0000","last_peered":"2024-09-11T01:15:48.829477+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:15:48.829477+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":9088,"log_dups_size":0,"ondisk_log_size":9088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37208064,"num_objects":9084,"num_object_clones":0,"num_object_copies":18168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7389,"num_write_kb":29556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":5188}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9137","reported_seq":9206,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:48.827722+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:15:48.827722+0000","last_peered":"2024-09-11T01:15:48.827722+0000","last_clean":"2024-09-11T01:15:48.827722+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:15:48.827722+0000","last_undegraded":"2024-09-11T01:15:48.827722+0000","last_fullsized":"2024-09-11T01:15:48.827722+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":9137,"log_dups_size":0,"ondisk_log_size":9137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37425152,"num_objects":9137,"num_object_clones":0,"num_object_copies":18274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9137,"num_write_kb":36548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9075","reported_seq":9202,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:49.820455+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:15:49.820455+0000","last_peered":"2024-09-11T01:15:49.820455+0000","last_clean":"2024-09-11T01:15:49.820455+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:15:49.820455+0000","last_undegraded":"2024-09-11T01:15:49.820455+0000","last_fullsized":"2024-09-11T01:15:49.820455+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":9075,"log_dups_size":0,"ondisk_log_size":9075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37142528,"num_objects":9068,"num_object_clones":0,"num_object_copies":18136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6080,"num_write_kb":24320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9159","reported_seq":9329,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:49.837929+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:15:49.837929+0000","last_peered":"2024-09-11T01:15:49.837929+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:15:49.837929+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":9159,"log_dups_size":0,"ondisk_log_size":9159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37494784,"num_objects":9154,"num_object_clones":0,"num_object_copies":18308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5500,"num_write_kb":22000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5203},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9002","reported_seq":9132,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:49.796775+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:15:49.796775+0000","last_peered":"2024-09-11T01:15:49.796775+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:15:49.796775+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":9002,"log_dups_size":0,"ondisk_log_size":9002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36847616,"num_objects":8996,"num_object_clones":0,"num_object_copies":17992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5859,"num_write_kb":23436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5010},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9143","reported_seq":9272,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:49.809728+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:15:49.809728+0000","last_peered":"2024-09-11T01:15:49.809728+0000","last_clean":"2024-09-11T01:15:49.809728+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:15:49.809728+0000","last_undegraded":"2024-09-11T01:15:49.809728+0000","last_fullsized":"2024-09-11T01:15:49.809728+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":9143,"log_dups_size":0,"ondisk_log_size":9143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37441536,"num_objects":9141,"num_object_clones":0,"num_object_copies":18282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9141,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6019,"num_write_kb":24076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9071","reported_seq":9198,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:48.825905+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:15:48.825905+0000","last_peered":"2024-09-11T01:15:48.825905+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:15:48.825905+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":9071,"log_dups_size":0,"ondisk_log_size":9071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37126144,"num_objects":9064,"num_object_clones":0,"num_object_copies":18128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7042,"num_write_kb":28168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":5131}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692287+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692287+0000","last_peered":"2024-09-11T01:15:21.692287+0000","last_clean":"2024-09-11T01:15:21.692287+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692287+0000","last_undegraded":"2024-09-11T01:15:21.692287+0000","last_fullsized":"2024-09-11T01:15:21.692287+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9017","reported_seq":9145,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:49.789114+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:15:49.789114+0000","last_peered":"2024-09-11T01:15:49.789114+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:15:49.789114+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":9017,"log_dups_size":0,"ondisk_log_size":9017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36900864,"num_objects":9009,"num_object_clones":0,"num_object_copies":18018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5604,"num_write_kb":22416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5025},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.685123+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.685123+0000","last_peered":"2024-09-11T01:15:21.685123+0000","last_clean":"2024-09-11T01:15:21.685123+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.685123+0000","last_undegraded":"2024-09-11T01:15:21.685123+0000","last_fullsized":"2024-09-11T01:15:21.685123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9139","reported_seq":9257,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:48.693003+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:15:48.693003+0000","last_peered":"2024-09-11T01:15:48.693003+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:15:48.693003+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":9139,"log_dups_size":0,"ondisk_log_size":9139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37412864,"num_objects":9134,"num_object_clones":0,"num_object_copies":18268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6956,"num_write_kb":27824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":5110}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8983","reported_seq":16963,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:48.738041+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:15:48.738041+0000","last_peered":"2024-09-11T01:15:48.738041+0000","last_clean":"2024-09-11T01:15:48.738041+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:15:48.738041+0000","last_undegraded":"2024-09-11T01:15:48.738041+0000","last_fullsized":"2024-09-11T01:15:48.738041+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":8983,"log_dups_size":0,"ondisk_log_size":8983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36769792,"num_objects":8977,"num_object_clones":0,"num_object_copies":17954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6768,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9190","reported_seq":9321,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:49.808189+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:15:49.808189+0000","last_peered":"2024-09-11T01:15:49.808189+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:15:49.808189+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":9190,"log_dups_size":0,"ondisk_log_size":9190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37634048,"num_objects":9188,"num_object_clones":0,"num_object_copies":18376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5927,"num_write_kb":23708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5238},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'8997","reported_seq":10232,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:49.868487+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:15:49.868487+0000","last_peered":"2024-09-11T01:15:49.868487+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:15:49.868487+0000","last_undegraded":"2024-09-11T01:15:49.868487+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":8997,"log_dups_size":0,"ondisk_log_size":8997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36818944,"num_objects":8989,"num_object_clones":0,"num_object_copies":17978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3390,"num_objects_unfound":0,"num_objects_dirty":8989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5652,"num_write_kb":22608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":549,"num_bytes_recovered":2248704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5599},{"shards":"0,2","objects":3390}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694344+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694344+0000","last_peered":"2024-09-11T01:15:21.694344+0000","last_clean":"2024-09-11T01:15:21.694344+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694344+0000","last_undegraded":"2024-09-11T01:15:21.694344+0000","last_fullsized":"2024-09-11T01:15:21.694344+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9117","reported_seq":9251,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:49.836316+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:15:49.836316+0000","last_peered":"2024-09-11T01:15:49.836316+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:15:49.836316+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":9117,"log_dups_size":0,"ondisk_log_size":9117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37326848,"num_objects":9113,"num_object_clones":0,"num_object_copies":18226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5726,"num_write_kb":22904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5074},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9107","reported_seq":9237,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:49.867129+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:15:49.867129+0000","last_peered":"2024-09-11T01:15:49.867129+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:15:49.867129+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":9107,"log_dups_size":0,"ondisk_log_size":9107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37253737,"num_objects":9096,"num_object_clones":0,"num_object_copies":18192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6177,"num_write_kb":24708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5179},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":596095593,"num_objects":145532,"num_object_clones":0,"num_object_copies":291064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":3390,"num_objects_unfound":0,"num_objects_dirty":145532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106463,"num_write_kb":425852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4477,"num_bytes_recovered":18329600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1208737792,"data_stored":1208657775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145613,"ondisk_log_size":145613,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182452,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":807076,"kb_used_data":500564,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93564764,"statfs":{"total":96636764160,"available":95810318336,"internally_reserved":0,"allocated":512577536,"data_stored":512318322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":313847467},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,1,3,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986224,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":688940,"kb_used_data":418524,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93682900,"statfs":{"total":96636764160,"available":95931289600,"internally_reserved":0,"allocated":428568576,"data_stored":428313439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276885162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,1,1,1,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659169,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451656,"kb_used_data":261752,"kb_used_omap":4,"kb_used_meta":189883,"kb_avail":93920184,"statfs":{"total":96636764160,"available":96174268416,"internally_reserved":0,"allocated":268034048,"data_stored":267705254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":194440879},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,4,1,10,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":512430080,"data_stored":512426601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":267943936,"data_stored":267870877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":428363776,"data_stored":428360297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:50.989 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:15:50.989 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'9195", 'reported_seq': 9272, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:49.858991+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:15:49.858991+0000', 'last_peered': '2024-09-11T01:15:49.858991+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:15:49.858991+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 9195, 'log_dups_size': 0, 'ondisk_log_size': 9195, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37662720, 'num_objects': 9195, 'num_object_clones': 0, 'num_object_copies': 18390, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9195, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9195, 'num_write_kb': 36780, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5123}, {'shards': '0,2', 'objects': 4072}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:50.989 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:15:50.989 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'9088", 'reported_seq': 9215, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:48.829477+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:15:48.829477+0000', 'last_peered': '2024-09-11T01:15:48.829477+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:15:48.829477+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 9088, 'log_dups_size': 0, 'ondisk_log_size': 9088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37208064, 'num_objects': 9084, 'num_object_clones': 0, 'num_object_copies': 18168, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9084, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7389, 'num_write_kb': 29556, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3896}, {'shards': '2', 'objects': 5188}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:50.990 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:15:50.990 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'9159", 'reported_seq': 9329, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:49.837929+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:15:49.837929+0000', 'last_peered': '2024-09-11T01:15:49.837929+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:15:49.837929+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 9159, 'log_dups_size': 0, 'ondisk_log_size': 9159, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37494784, 'num_objects': 9154, 'num_object_clones': 0, 'num_object_copies': 18308, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9154, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5500, 'num_write_kb': 22000, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5203}, {'shards': '0,2', 'objects': 3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:50.990 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:15:50.990 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'9002", 'reported_seq': 9132, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:49.796775+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:15:49.796775+0000', 'last_peered': '2024-09-11T01:15:49.796775+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:15:49.796775+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 9002, 'log_dups_size': 0, 'ondisk_log_size': 9002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36847616, 'num_objects': 8996, 'num_object_clones': 0, 'num_object_copies': 17992, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8996, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5859, 'num_write_kb': 23436, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5010}, {'shards': '0,2', 'objects': 3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:50.990 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:15:50.990 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'9071", 'reported_seq': 9198, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:48.825905+0000', 'last_change': '2024-09-11T01:13:10.159069+0000', 'last_active': '2024-09-11T01:15:48.825905+0000', 'last_peered': '2024-09-11T01:15:48.825905+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:15:48.825905+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 9071, 'log_dups_size': 0, 'ondisk_log_size': 9071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37126144, 'num_objects': 9064, 'num_object_clones': 0, 'num_object_copies': 18128, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9064, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7042, 'num_write_kb': 28168, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3932}, {'shards': '2', 'objects': 5131}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:50.990 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:15:50.990 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'9017", 'reported_seq': 9145, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:49.789114+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:15:49.789114+0000', 'last_peered': '2024-09-11T01:15:49.789114+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:15:49.789114+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 9017, 'log_dups_size': 0, 'ondisk_log_size': 9017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36900864, 'num_objects': 9009, 'num_object_clones': 0, 'num_object_copies': 18018, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9009, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5604, 'num_write_kb': 22416, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5025}, {'shards': '0,2', 'objects': 3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:50.991 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:15:50.991 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'9139", 'reported_seq': 9257, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:48.693003+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:15:48.693003+0000', 'last_peered': '2024-09-11T01:15:48.693003+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:15:48.693003+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 9139, 'log_dups_size': 0, 'ondisk_log_size': 9139, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37412864, 'num_objects': 9134, 'num_object_clones': 0, 'num_object_copies': 18268, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9134, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6956, 'num_write_kb': 27824, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4023}, {'shards': '2', 'objects': 5110}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:50.991 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:15:50.991 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'9190", 'reported_seq': 9321, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:49.808189+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:15:49.808189+0000', 'last_peered': '2024-09-11T01:15:49.808189+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:15:49.808189+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 9190, 'log_dups_size': 0, 'ondisk_log_size': 9190, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37634048, 'num_objects': 9188, 'num_object_clones': 0, 'num_object_copies': 18376, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9188, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5927, 'num_write_kb': 23708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5238}, {'shards': '0,2', 'objects': 3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:50.991 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:15:50.991 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'8997", 'reported_seq': 10232, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:15:49.868487+0000', 'last_change': '2024-09-11T01:15:19.913288+0000', 'last_active': '2024-09-11T01:15:49.868487+0000', 'last_peered': '2024-09-11T01:15:49.868487+0000', 'last_clean': '2024-09-11T01:13:07.624565+0000', 'last_became_active': '2024-09-11T01:13:10.199795+0000', 'last_became_peered': '2024-09-11T01:13:10.199795+0000', 'last_unstale': '2024-09-11T01:15:49.868487+0000', 'last_undegraded': '2024-09-11T01:15:49.868487+0000', 'last_fullsized': '2024-09-11T01:13:09.652311+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3415", 'last_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'objects_scrubbed': 3337, 'log_size': 8997, 'log_dups_size': 0, 'ondisk_log_size': 8997, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1975, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36818944, 'num_objects': 8989, 'num_object_clones': 0, 'num_object_copies': 17978, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3390, 'num_objects_unfound': 0, 'num_objects_dirty': 8989, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5652, 'num_write_kb': 22608, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 549, 'num_bytes_recovered': 2248704, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5599}, {'shards': '0,2', 'objects': 3390}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:50.991 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:15:50.991 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'9117", 'reported_seq': 9251, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:49.836316+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:15:49.836316+0000', 'last_peered': '2024-09-11T01:15:49.836316+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:15:49.836316+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 9117, 'log_dups_size': 0, 'ondisk_log_size': 9117, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37326848, 'num_objects': 9113, 'num_object_clones': 0, 'num_object_copies': 18226, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9113, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5726, 'num_write_kb': 22904, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5074}, {'shards': '0,2', 'objects': 4039}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:50.991 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:15:50.991 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'9107", 'reported_seq': 9237, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:49.867129+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:15:49.867129+0000', 'last_peered': '2024-09-11T01:15:49.867129+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:15:49.867129+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 9107, 'log_dups_size': 0, 'ondisk_log_size': 9107, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37253737, 'num_objects': 9096, 'num_object_clones': 0, 'num_object_copies': 18192, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9096, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6177, 'num_write_kb': 24708, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5179}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:50.992 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:15:51.430 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 207 20 146346 146326 2.76009 1.69531 0.0954451 0.0282963 2024-09-11T01:15:52.049 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:15:52.049 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39769/291068 objects degraded (13.663%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":288,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":145534,"data_bytes":596095612,"bytes_used":1994416128,"bytes_avail":287915876352,"bytes_total":289910292480,"degraded_objects":39769,"degraded_total":291068,"degraded_ratio":0.13663130265092693,"misplaced_objects":3390,"misplaced_total":291068,"misplaced_ratio":0.01164676295573543,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":94494,"recovering_keys_per_sec":0,"num_objects_recovered":277,"num_bytes_recovered":1134592,"num_keys_recovered":0,"write_bytes_sec":2320741,"read_op_per_sec":0,"write_op_per_sec":566},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:15:52.102 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:15:52.102 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:52.432 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 208 20 146784 146764 2.75505 1.71094 0.0105302 0.0283493 2024-09-11T01:15:52.977 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:15:52.978 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:15:53.045 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":183,"stamp":"2024-09-11T01:15:52.628355+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596095612,"num_objects":145534,"num_object_clones":0,"num_object_copies":291068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":3390,"num_objects_unfound":0,"num_objects_dirty":145534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106465,"num_write_kb":425854,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4478,"num_bytes_recovered":18329619,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145616,"ondisk_log_size":145616,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1947672,"kb_used_data":1180840,"kb_used_omap":12,"kb_used_meta":766771,"kb_avail":281167848,"statfs":{"total":289910292480,"available":287915876352,"internally_reserved":0,"allocated":1209180160,"data_stored":1208337015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":785173508},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,0,1,10,6,15,3,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18616320,"num_objects":4545,"num_object_clones":0,"num_object_copies":9090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-184,"num_objects_unfound":0,"num_objects_dirty":4545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4545,"num_write_kb":18180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":753664,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006522"},"pg_stats":[{"pgid":"2.f","version":"45'9195","reported_seq":9272,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:49.858991+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:15:49.858991+0000","last_peered":"2024-09-11T01:15:49.858991+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:15:49.858991+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":9195,"log_dups_size":0,"ondisk_log_size":9195,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37662720,"num_objects":9195,"num_object_clones":0,"num_object_copies":18390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9195,"num_write_kb":36780,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5123},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9193","reported_seq":9294,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:48.759081+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:15:48.759081+0000","last_peered":"2024-09-11T01:15:48.759081+0000","last_clean":"2024-09-11T01:15:48.759081+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:15:48.759081+0000","last_undegraded":"2024-09-11T01:15:48.759081+0000","last_fullsized":"2024-09-11T01:15:48.759081+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":9193,"log_dups_size":0,"ondisk_log_size":9193,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37629952,"num_objects":9187,"num_object_clones":0,"num_object_copies":18374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7432,"num_write_kb":29728,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9088","reported_seq":9215,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:48.829477+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:15:48.829477+0000","last_peered":"2024-09-11T01:15:48.829477+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:15:48.829477+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":9088,"log_dups_size":0,"ondisk_log_size":9088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37208064,"num_objects":9084,"num_object_clones":0,"num_object_copies":18168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9084,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7389,"num_write_kb":29556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":5188}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9137","reported_seq":9206,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:48.827722+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:15:48.827722+0000","last_peered":"2024-09-11T01:15:48.827722+0000","last_clean":"2024-09-11T01:15:48.827722+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:15:48.827722+0000","last_undegraded":"2024-09-11T01:15:48.827722+0000","last_fullsized":"2024-09-11T01:15:48.827722+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":9137,"log_dups_size":0,"ondisk_log_size":9137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37425152,"num_objects":9137,"num_object_clones":0,"num_object_copies":18274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9137,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9137,"num_write_kb":36548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9075","reported_seq":9202,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:49.820455+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:15:49.820455+0000","last_peered":"2024-09-11T01:15:49.820455+0000","last_clean":"2024-09-11T01:15:49.820455+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:15:49.820455+0000","last_undegraded":"2024-09-11T01:15:49.820455+0000","last_fullsized":"2024-09-11T01:15:49.820455+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":9075,"log_dups_size":0,"ondisk_log_size":9075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37142528,"num_objects":9068,"num_object_clones":0,"num_object_copies":18136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6080,"num_write_kb":24320,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9159","reported_seq":9329,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:49.837929+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:15:49.837929+0000","last_peered":"2024-09-11T01:15:49.837929+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:15:49.837929+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":9159,"log_dups_size":0,"ondisk_log_size":9159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37494784,"num_objects":9154,"num_object_clones":0,"num_object_copies":18308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9154,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5500,"num_write_kb":22000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5203},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9002","reported_seq":9132,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:49.796775+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:15:49.796775+0000","last_peered":"2024-09-11T01:15:49.796775+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:15:49.796775+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":9002,"log_dups_size":0,"ondisk_log_size":9002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36847616,"num_objects":8996,"num_object_clones":0,"num_object_copies":17992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8996,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5859,"num_write_kb":23436,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5010},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9143","reported_seq":9272,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:49.809728+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:15:49.809728+0000","last_peered":"2024-09-11T01:15:49.809728+0000","last_clean":"2024-09-11T01:15:49.809728+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:15:49.809728+0000","last_undegraded":"2024-09-11T01:15:49.809728+0000","last_fullsized":"2024-09-11T01:15:49.809728+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":9143,"log_dups_size":0,"ondisk_log_size":9143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37441536,"num_objects":9141,"num_object_clones":0,"num_object_copies":18282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9141,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6019,"num_write_kb":24076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9071","reported_seq":9198,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:48.825905+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:15:48.825905+0000","last_peered":"2024-09-11T01:15:48.825905+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:15:48.825905+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":9071,"log_dups_size":0,"ondisk_log_size":9071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37126144,"num_objects":9064,"num_object_clones":0,"num_object_copies":18128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7042,"num_write_kb":28168,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":5131}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692287+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692287+0000","last_peered":"2024-09-11T01:15:21.692287+0000","last_clean":"2024-09-11T01:15:21.692287+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692287+0000","last_undegraded":"2024-09-11T01:15:21.692287+0000","last_fullsized":"2024-09-11T01:15:21.692287+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9017","reported_seq":9145,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:49.789114+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:15:49.789114+0000","last_peered":"2024-09-11T01:15:49.789114+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:15:49.789114+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":9017,"log_dups_size":0,"ondisk_log_size":9017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36900864,"num_objects":9009,"num_object_clones":0,"num_object_copies":18018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5604,"num_write_kb":22416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5025},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.685123+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.685123+0000","last_peered":"2024-09-11T01:15:21.685123+0000","last_clean":"2024-09-11T01:15:21.685123+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.685123+0000","last_undegraded":"2024-09-11T01:15:21.685123+0000","last_fullsized":"2024-09-11T01:15:21.685123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9139","reported_seq":9257,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:48.693003+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:15:48.693003+0000","last_peered":"2024-09-11T01:15:48.693003+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:15:48.693003+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":9139,"log_dups_size":0,"ondisk_log_size":9139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37412864,"num_objects":9134,"num_object_clones":0,"num_object_copies":18268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6956,"num_write_kb":27824,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":5110}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'8983","reported_seq":16963,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:48.738041+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:15:48.738041+0000","last_peered":"2024-09-11T01:15:48.738041+0000","last_clean":"2024-09-11T01:15:48.738041+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:15:48.738041+0000","last_undegraded":"2024-09-11T01:15:48.738041+0000","last_fullsized":"2024-09-11T01:15:48.738041+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":8983,"log_dups_size":0,"ondisk_log_size":8983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36769792,"num_objects":8977,"num_object_clones":0,"num_object_copies":17954,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8977,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6768,"num_write_kb":27072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9190","reported_seq":9321,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:49.808189+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:15:49.808189+0000","last_peered":"2024-09-11T01:15:49.808189+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:15:49.808189+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":9190,"log_dups_size":0,"ondisk_log_size":9190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37634048,"num_objects":9188,"num_object_clones":0,"num_object_copies":18376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5927,"num_write_kb":23708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5238},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'8997","reported_seq":10232,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:49.868487+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:15:49.868487+0000","last_peered":"2024-09-11T01:15:49.868487+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:15:49.868487+0000","last_undegraded":"2024-09-11T01:15:49.868487+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":8997,"log_dups_size":0,"ondisk_log_size":8997,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36818944,"num_objects":8989,"num_object_clones":0,"num_object_copies":17978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3390,"num_objects_unfound":0,"num_objects_dirty":8989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5652,"num_write_kb":22608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":549,"num_bytes_recovered":2248704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5599},{"shards":"0,2","objects":3390}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694344+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694344+0000","last_peered":"2024-09-11T01:15:21.694344+0000","last_clean":"2024-09-11T01:15:21.694344+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694344+0000","last_undegraded":"2024-09-11T01:15:21.694344+0000","last_fullsized":"2024-09-11T01:15:21.694344+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9117","reported_seq":9251,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:49.836316+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:15:49.836316+0000","last_peered":"2024-09-11T01:15:49.836316+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:15:49.836316+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":9117,"log_dups_size":0,"ondisk_log_size":9117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37326848,"num_objects":9113,"num_object_clones":0,"num_object_copies":18226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5726,"num_write_kb":22904,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5074},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9107","reported_seq":9237,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:49.867129+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:15:49.867129+0000","last_peered":"2024-09-11T01:15:49.867129+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:15:49.867129+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":9107,"log_dups_size":0,"ondisk_log_size":9107,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37253737,"num_objects":9096,"num_object_clones":0,"num_object_copies":18192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6177,"num_write_kb":24708,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5179},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":596095593,"num_objects":145532,"num_object_clones":0,"num_object_copies":291064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":3390,"num_objects_unfound":0,"num_objects_dirty":145532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":106463,"num_write_kb":425852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4477,"num_bytes_recovered":18329600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1208737792,"data_stored":1208657775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145613,"ondisk_log_size":145613,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182452,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":807076,"kb_used_data":500564,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93564764,"statfs":{"total":96636764160,"available":95810318336,"internally_reserved":0,"allocated":512577536,"data_stored":512318322,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":313847467},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,0,1,3,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986224,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":688940,"kb_used_data":418524,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93682900,"statfs":{"total":96636764160,"available":95931289600,"internally_reserved":0,"allocated":428568576,"data_stored":428313439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276885162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,1,1,1,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659169,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451656,"kb_used_data":261752,"kb_used_omap":4,"kb_used_meta":189883,"kb_avail":93920184,"statfs":{"total":96636764160,"available":96174268416,"internally_reserved":0,"allocated":268034048,"data_stored":267705254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":194440879},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,4,1,10,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":512430080,"data_stored":512426601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":267943936,"data_stored":267870877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":428363776,"data_stored":428360297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:53.430 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 209 20 147220 147200 2.75001 1.70312 0.106262 0.0284002 2024-09-11T01:15:54.431 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 210 20 147674 147654 2.74536 1.77344 0.0337236 0.0284491 2024-09-11T01:15:55.431 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 211 20 148136 148116 2.74091 1.80469 0.0478708 0.0284955 2024-09-11T01:15:56.047 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:56.430 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 212 20 148578 148558 2.73612 1.72656 0.0557968 0.0285445 2024-09-11T01:15:56.746 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:15:56.747 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:15:56.850 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":185,"stamp":"2024-09-11T01:15:56.629233+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":604918396,"num_objects":147688,"num_object_clones":0,"num_object_copies":295376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":3291,"num_objects_unfound":0,"num_objects_dirty":147688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108619,"num_write_kb":434470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4577,"num_bytes_recovered":18735123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147770,"ondisk_log_size":147770,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1999760,"kb_used_data":1198240,"kb_used_omap":12,"kb_used_meta":801459,"kb_avail":281115760,"statfs":{"total":289910292480,"available":287862538240,"internally_reserved":0,"allocated":1226997760,"data_stored":1226154615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":820694020},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,0,4,5,10,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23928832,"num_objects":5842,"num_object_clones":0,"num_object_copies":11684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-283,"num_objects_unfound":0,"num_objects_dirty":5842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5842,"num_write_kb":23368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":1159168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006386"},"pg_stats":[{"pgid":"2.f","version":"45'9338","reported_seq":9415,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.866399+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:15:54.866399+0000","last_peered":"2024-09-11T01:15:54.866399+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:15:54.866399+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":9338,"log_dups_size":0,"ondisk_log_size":9338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38248448,"num_objects":9338,"num_object_clones":0,"num_object_copies":18676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9338,"num_write_kb":37352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5266},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9312","reported_seq":9413,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:53.680760+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:15:53.680760+0000","last_peered":"2024-09-11T01:15:53.680760+0000","last_clean":"2024-09-11T01:15:53.680760+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:15:53.680760+0000","last_undegraded":"2024-09-11T01:15:53.680760+0000","last_fullsized":"2024-09-11T01:15:53.680760+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":9312,"log_dups_size":0,"ondisk_log_size":9312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38117376,"num_objects":9306,"num_object_clones":0,"num_object_copies":18612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7551,"num_write_kb":30204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9228","reported_seq":9354,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:53.809798+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:15:53.809798+0000","last_peered":"2024-09-11T01:15:53.809798+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:15:53.809798+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":9228,"log_dups_size":0,"ondisk_log_size":9228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37781504,"num_objects":9224,"num_object_clones":0,"num_object_copies":18448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7529,"num_write_kb":30116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":5328}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9256","reported_seq":9325,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:53.880526+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:15:53.880526+0000","last_peered":"2024-09-11T01:15:53.880526+0000","last_clean":"2024-09-11T01:15:53.880526+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:15:53.880526+0000","last_undegraded":"2024-09-11T01:15:53.880526+0000","last_fullsized":"2024-09-11T01:15:53.880526+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":9256,"log_dups_size":0,"ondisk_log_size":9256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37912576,"num_objects":9256,"num_object_clones":0,"num_object_copies":18512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9256,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9214","reported_seq":9341,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:54.829577+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:15:54.829577+0000","last_peered":"2024-09-11T01:15:54.829577+0000","last_clean":"2024-09-11T01:15:54.829577+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:15:54.829577+0000","last_undegraded":"2024-09-11T01:15:54.829577+0000","last_fullsized":"2024-09-11T01:15:54.829577+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":9214,"log_dups_size":0,"ondisk_log_size":9214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37711872,"num_objects":9207,"num_object_clones":0,"num_object_copies":18414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6219,"num_write_kb":24876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9297","reported_seq":9467,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.882546+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:15:54.882546+0000","last_peered":"2024-09-11T01:15:54.882546+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:15:54.882546+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":9297,"log_dups_size":0,"ondisk_log_size":9297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38060032,"num_objects":9292,"num_object_clones":0,"num_object_copies":18584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5638,"num_write_kb":22552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5341},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9128","reported_seq":9258,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.852428+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:15:54.852428+0000","last_peered":"2024-09-11T01:15:54.852428+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:15:54.852428+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":9128,"log_dups_size":0,"ondisk_log_size":9128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37363712,"num_objects":9122,"num_object_clones":0,"num_object_copies":18244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5985,"num_write_kb":23940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5136},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9279","reported_seq":9408,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:54.837642+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:15:54.837642+0000","last_peered":"2024-09-11T01:15:54.837642+0000","last_clean":"2024-09-11T01:15:54.837642+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:15:54.837642+0000","last_undegraded":"2024-09-11T01:15:54.837642+0000","last_fullsized":"2024-09-11T01:15:54.837642+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":9279,"log_dups_size":0,"ondisk_log_size":9279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37998592,"num_objects":9277,"num_object_clones":0,"num_object_copies":18554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6155,"num_write_kb":24620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9196","reported_seq":9323,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:53.754572+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:15:53.754572+0000","last_peered":"2024-09-11T01:15:53.754572+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:15:53.754572+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":9196,"log_dups_size":0,"ondisk_log_size":9196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37638144,"num_objects":9189,"num_object_clones":0,"num_object_copies":18378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7167,"num_write_kb":28668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":5256}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692287+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692287+0000","last_peered":"2024-09-11T01:15:21.692287+0000","last_clean":"2024-09-11T01:15:21.692287+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692287+0000","last_undegraded":"2024-09-11T01:15:21.692287+0000","last_fullsized":"2024-09-11T01:15:21.692287+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9143","reported_seq":9270,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.845881+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:15:54.845881+0000","last_peered":"2024-09-11T01:15:54.845881+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:15:54.845881+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":9143,"log_dups_size":0,"ondisk_log_size":9143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37416960,"num_objects":9135,"num_object_clones":0,"num_object_copies":18270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5730,"num_write_kb":22920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5151},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.685123+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.685123+0000","last_peered":"2024-09-11T01:15:21.685123+0000","last_clean":"2024-09-11T01:15:21.685123+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.685123+0000","last_undegraded":"2024-09-11T01:15:21.685123+0000","last_fullsized":"2024-09-11T01:15:21.685123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9287","reported_seq":9405,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:53.864505+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:15:53.864505+0000","last_peered":"2024-09-11T01:15:53.864505+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:15:53.864505+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":9287,"log_dups_size":0,"ondisk_log_size":9287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38019072,"num_objects":9282,"num_object_clones":0,"num_object_copies":18564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7104,"num_write_kb":28416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":5258}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9109","reported_seq":17089,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:53.779397+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:15:53.779397+0000","last_peered":"2024-09-11T01:15:53.779397+0000","last_clean":"2024-09-11T01:15:53.779397+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:15:53.779397+0000","last_undegraded":"2024-09-11T01:15:53.779397+0000","last_fullsized":"2024-09-11T01:15:53.779397+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":9109,"log_dups_size":0,"ondisk_log_size":9109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37285888,"num_objects":9103,"num_object_clones":0,"num_object_copies":18206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6894,"num_write_kb":27576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9334","reported_seq":9465,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.861913+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:15:54.861913+0000","last_peered":"2024-09-11T01:15:54.861913+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:15:54.861913+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":9334,"log_dups_size":0,"ondisk_log_size":9334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38223872,"num_objects":9332,"num_object_clones":0,"num_object_copies":18664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6071,"num_write_kb":24284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5382},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9128","reported_seq":10559,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:54.886548+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:15:54.886548+0000","last_peered":"2024-09-11T01:15:54.886548+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:15:54.886548+0000","last_undegraded":"2024-09-11T01:15:54.886548+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":9128,"log_dups_size":0,"ondisk_log_size":9128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37355520,"num_objects":9120,"num_object_clones":0,"num_object_copies":18240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3291,"num_objects_unfound":0,"num_objects_dirty":9120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5783,"num_write_kb":23132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":648,"num_bytes_recovered":2654208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5829},{"shards":"0,2","objects":3291}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694344+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694344+0000","last_peered":"2024-09-11T01:15:21.694344+0000","last_clean":"2024-09-11T01:15:21.694344+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694344+0000","last_undegraded":"2024-09-11T01:15:21.694344+0000","last_fullsized":"2024-09-11T01:15:21.694344+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9273","reported_seq":9405,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.821000+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:15:54.821000+0000","last_peered":"2024-09-11T01:15:54.821000+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:15:54.821000+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":9273,"log_dups_size":0,"ondisk_log_size":9273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37965824,"num_objects":9269,"num_object_clones":0,"num_object_copies":18538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5882,"num_write_kb":23528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5230},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9245","reported_seq":9378,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.783618+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:15:54.783618+0000","last_peered":"2024-09-11T01:15:54.783618+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:15:54.783618+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":9245,"log_dups_size":0,"ondisk_log_size":9245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37818985,"num_objects":9234,"num_object_clones":0,"num_object_copies":18468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6315,"num_write_kb":25260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5317},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":604918377,"num_objects":147686,"num_object_clones":0,"num_object_copies":295372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":3291,"num_objects_unfound":0,"num_objects_dirty":147686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108617,"num_write_kb":434468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4576,"num_bytes_recovered":18735104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1226911744,"data_stored":1226831727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147767,"ondisk_log_size":147767,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182453,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":817660,"kb_used_data":506988,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93554180,"statfs":{"total":96636764160,"available":95799480320,"internally_reserved":0,"allocated":519155712,"data_stored":518896498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":318107307},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,0,2,2,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986225,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":697276,"kb_used_data":422700,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93674564,"statfs":{"total":96636764160,"available":95922753536,"internally_reserved":0,"allocated":432844800,"data_stored":432589663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659170,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":484824,"kb_used_data":268552,"kb_used_omap":4,"kb_used_meta":216251,"kb_avail":93887016,"statfs":{"total":96636764160,"available":96140304384,"internally_reserved":0,"allocated":274997248,"data_stored":274668454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":221441711},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":519131136,"data_stored":519127657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":275083264,"data_stored":275010205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":432697344,"data_stored":432693865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:56.852 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:57.436 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 213 20 148998 148978 2.73093 1.64062 0.0985005 0.0285993 2024-09-11T01:15:57.655 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:15:57.665 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:15:57.744 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":185,"stamp":"2024-09-11T01:15:56.629233+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":604918396,"num_objects":147688,"num_object_clones":0,"num_object_copies":295376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":3291,"num_objects_unfound":0,"num_objects_dirty":147688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108619,"num_write_kb":434470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4577,"num_bytes_recovered":18735123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147770,"ondisk_log_size":147770,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1999760,"kb_used_data":1198240,"kb_used_omap":12,"kb_used_meta":801459,"kb_avail":281115760,"statfs":{"total":289910292480,"available":287862538240,"internally_reserved":0,"allocated":1226997760,"data_stored":1226154615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":820694020},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,0,4,5,10,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23928832,"num_objects":5842,"num_object_clones":0,"num_object_copies":11684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-283,"num_objects_unfound":0,"num_objects_dirty":5842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5842,"num_write_kb":23368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":1159168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006386"},"pg_stats":[{"pgid":"2.f","version":"45'9338","reported_seq":9415,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.866399+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:15:54.866399+0000","last_peered":"2024-09-11T01:15:54.866399+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:15:54.866399+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":9338,"log_dups_size":0,"ondisk_log_size":9338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38248448,"num_objects":9338,"num_object_clones":0,"num_object_copies":18676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9338,"num_write_kb":37352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5266},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9312","reported_seq":9413,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:53.680760+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:15:53.680760+0000","last_peered":"2024-09-11T01:15:53.680760+0000","last_clean":"2024-09-11T01:15:53.680760+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:15:53.680760+0000","last_undegraded":"2024-09-11T01:15:53.680760+0000","last_fullsized":"2024-09-11T01:15:53.680760+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":9312,"log_dups_size":0,"ondisk_log_size":9312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38117376,"num_objects":9306,"num_object_clones":0,"num_object_copies":18612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7551,"num_write_kb":30204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9228","reported_seq":9354,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:53.809798+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:15:53.809798+0000","last_peered":"2024-09-11T01:15:53.809798+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:15:53.809798+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":9228,"log_dups_size":0,"ondisk_log_size":9228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37781504,"num_objects":9224,"num_object_clones":0,"num_object_copies":18448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7529,"num_write_kb":30116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":5328}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9256","reported_seq":9325,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:53.880526+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:15:53.880526+0000","last_peered":"2024-09-11T01:15:53.880526+0000","last_clean":"2024-09-11T01:15:53.880526+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:15:53.880526+0000","last_undegraded":"2024-09-11T01:15:53.880526+0000","last_fullsized":"2024-09-11T01:15:53.880526+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":9256,"log_dups_size":0,"ondisk_log_size":9256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37912576,"num_objects":9256,"num_object_clones":0,"num_object_copies":18512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9256,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9214","reported_seq":9341,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:54.829577+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:15:54.829577+0000","last_peered":"2024-09-11T01:15:54.829577+0000","last_clean":"2024-09-11T01:15:54.829577+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:15:54.829577+0000","last_undegraded":"2024-09-11T01:15:54.829577+0000","last_fullsized":"2024-09-11T01:15:54.829577+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":9214,"log_dups_size":0,"ondisk_log_size":9214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37711872,"num_objects":9207,"num_object_clones":0,"num_object_copies":18414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6219,"num_write_kb":24876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9297","reported_seq":9467,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.882546+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:15:54.882546+0000","last_peered":"2024-09-11T01:15:54.882546+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:15:54.882546+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":9297,"log_dups_size":0,"ondisk_log_size":9297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38060032,"num_objects":9292,"num_object_clones":0,"num_object_copies":18584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5638,"num_write_kb":22552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5341},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9128","reported_seq":9258,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.852428+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:15:54.852428+0000","last_peered":"2024-09-11T01:15:54.852428+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:15:54.852428+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":9128,"log_dups_size":0,"ondisk_log_size":9128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37363712,"num_objects":9122,"num_object_clones":0,"num_object_copies":18244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5985,"num_write_kb":23940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5136},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9279","reported_seq":9408,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:54.837642+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:15:54.837642+0000","last_peered":"2024-09-11T01:15:54.837642+0000","last_clean":"2024-09-11T01:15:54.837642+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:15:54.837642+0000","last_undegraded":"2024-09-11T01:15:54.837642+0000","last_fullsized":"2024-09-11T01:15:54.837642+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":9279,"log_dups_size":0,"ondisk_log_size":9279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37998592,"num_objects":9277,"num_object_clones":0,"num_object_copies":18554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6155,"num_write_kb":24620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9196","reported_seq":9323,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:53.754572+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:15:53.754572+0000","last_peered":"2024-09-11T01:15:53.754572+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:15:53.754572+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":9196,"log_dups_size":0,"ondisk_log_size":9196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37638144,"num_objects":9189,"num_object_clones":0,"num_object_copies":18378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7167,"num_write_kb":28668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":5256}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692287+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692287+0000","last_peered":"2024-09-11T01:15:21.692287+0000","last_clean":"2024-09-11T01:15:21.692287+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692287+0000","last_undegraded":"2024-09-11T01:15:21.692287+0000","last_fullsized":"2024-09-11T01:15:21.692287+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9143","reported_seq":9270,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.845881+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:15:54.845881+0000","last_peered":"2024-09-11T01:15:54.845881+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:15:54.845881+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":9143,"log_dups_size":0,"ondisk_log_size":9143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37416960,"num_objects":9135,"num_object_clones":0,"num_object_copies":18270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5730,"num_write_kb":22920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5151},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.685123+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.685123+0000","last_peered":"2024-09-11T01:15:21.685123+0000","last_clean":"2024-09-11T01:15:21.685123+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.685123+0000","last_undegraded":"2024-09-11T01:15:21.685123+0000","last_fullsized":"2024-09-11T01:15:21.685123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9287","reported_seq":9405,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:53.864505+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:15:53.864505+0000","last_peered":"2024-09-11T01:15:53.864505+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:15:53.864505+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":9287,"log_dups_size":0,"ondisk_log_size":9287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38019072,"num_objects":9282,"num_object_clones":0,"num_object_copies":18564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7104,"num_write_kb":28416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":5258}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9109","reported_seq":17089,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:53.779397+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:15:53.779397+0000","last_peered":"2024-09-11T01:15:53.779397+0000","last_clean":"2024-09-11T01:15:53.779397+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:15:53.779397+0000","last_undegraded":"2024-09-11T01:15:53.779397+0000","last_fullsized":"2024-09-11T01:15:53.779397+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":9109,"log_dups_size":0,"ondisk_log_size":9109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37285888,"num_objects":9103,"num_object_clones":0,"num_object_copies":18206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6894,"num_write_kb":27576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9334","reported_seq":9465,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.861913+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:15:54.861913+0000","last_peered":"2024-09-11T01:15:54.861913+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:15:54.861913+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":9334,"log_dups_size":0,"ondisk_log_size":9334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38223872,"num_objects":9332,"num_object_clones":0,"num_object_copies":18664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6071,"num_write_kb":24284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5382},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9128","reported_seq":10559,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:54.886548+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:15:54.886548+0000","last_peered":"2024-09-11T01:15:54.886548+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:15:54.886548+0000","last_undegraded":"2024-09-11T01:15:54.886548+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":9128,"log_dups_size":0,"ondisk_log_size":9128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37355520,"num_objects":9120,"num_object_clones":0,"num_object_copies":18240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3291,"num_objects_unfound":0,"num_objects_dirty":9120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5783,"num_write_kb":23132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":648,"num_bytes_recovered":2654208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5829},{"shards":"0,2","objects":3291}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694344+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694344+0000","last_peered":"2024-09-11T01:15:21.694344+0000","last_clean":"2024-09-11T01:15:21.694344+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694344+0000","last_undegraded":"2024-09-11T01:15:21.694344+0000","last_fullsized":"2024-09-11T01:15:21.694344+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9273","reported_seq":9405,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.821000+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:15:54.821000+0000","last_peered":"2024-09-11T01:15:54.821000+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:15:54.821000+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":9273,"log_dups_size":0,"ondisk_log_size":9273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37965824,"num_objects":9269,"num_object_clones":0,"num_object_copies":18538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5882,"num_write_kb":23528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5230},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9245","reported_seq":9378,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.783618+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:15:54.783618+0000","last_peered":"2024-09-11T01:15:54.783618+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:15:54.783618+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":9245,"log_dups_size":0,"ondisk_log_size":9245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37818985,"num_objects":9234,"num_object_clones":0,"num_object_copies":18468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6315,"num_write_kb":25260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5317},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":604918377,"num_objects":147686,"num_object_clones":0,"num_object_copies":295372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":3291,"num_objects_unfound":0,"num_objects_dirty":147686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108617,"num_write_kb":434468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4576,"num_bytes_recovered":18735104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1226911744,"data_stored":1226831727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147767,"ondisk_log_size":147767,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182453,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":817660,"kb_used_data":506988,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93554180,"statfs":{"total":96636764160,"available":95799480320,"internally_reserved":0,"allocated":519155712,"data_stored":518896498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":318107307},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,0,2,2,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986225,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":697276,"kb_used_data":422700,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93674564,"statfs":{"total":96636764160,"available":95922753536,"internally_reserved":0,"allocated":432844800,"data_stored":432589663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659170,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":484824,"kb_used_data":268552,"kb_used_omap":4,"kb_used_meta":216251,"kb_avail":93887016,"statfs":{"total":96636764160,"available":96140304384,"internally_reserved":0,"allocated":274997248,"data_stored":274668454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":221441711},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,5,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":519131136,"data_stored":519127657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":275083264,"data_stored":275010205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":432697344,"data_stored":432693865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:15:57.746 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:15:57.746 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'9338", 'reported_seq': 9415, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:54.866399+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:15:54.866399+0000', 'last_peered': '2024-09-11T01:15:54.866399+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:15:54.866399+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 9338, 'log_dups_size': 0, 'ondisk_log_size': 9338, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38248448, 'num_objects': 9338, 'num_object_clones': 0, 'num_object_copies': 18676, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9338, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9338, 'num_write_kb': 37352, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5266}, {'shards': '0,2', 'objects': 4072}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:57.747 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:15:57.747 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'9228", 'reported_seq': 9354, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:53.809798+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:15:53.809798+0000', 'last_peered': '2024-09-11T01:15:53.809798+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:15:53.809798+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 9228, 'log_dups_size': 0, 'ondisk_log_size': 9228, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37781504, 'num_objects': 9224, 'num_object_clones': 0, 'num_object_copies': 18448, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9224, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7529, 'num_write_kb': 30116, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3896}, {'shards': '2', 'objects': 5328}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:57.747 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:15:57.747 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'9297", 'reported_seq': 9467, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:54.882546+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:15:54.882546+0000', 'last_peered': '2024-09-11T01:15:54.882546+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:15:54.882546+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 9297, 'log_dups_size': 0, 'ondisk_log_size': 9297, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38060032, 'num_objects': 9292, 'num_object_clones': 0, 'num_object_copies': 18584, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9292, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5638, 'num_write_kb': 22552, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5341}, {'shards': '0,2', 'objects': 3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:57.747 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:15:57.747 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'9128", 'reported_seq': 9258, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:54.852428+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:15:54.852428+0000', 'last_peered': '2024-09-11T01:15:54.852428+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:15:54.852428+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 9128, 'log_dups_size': 0, 'ondisk_log_size': 9128, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37363712, 'num_objects': 9122, 'num_object_clones': 0, 'num_object_copies': 18244, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9122, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5985, 'num_write_kb': 23940, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5136}, {'shards': '0,2', 'objects': 3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:57.747 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:15:57.747 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'9196", 'reported_seq': 9323, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:53.754572+0000', 'last_change': '2024-09-11T01:13:10.159069+0000', 'last_active': '2024-09-11T01:15:53.754572+0000', 'last_peered': '2024-09-11T01:15:53.754572+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:15:53.754572+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 9196, 'log_dups_size': 0, 'ondisk_log_size': 9196, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37638144, 'num_objects': 9189, 'num_object_clones': 0, 'num_object_copies': 18378, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9189, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7167, 'num_write_kb': 28668, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3932}, {'shards': '2', 'objects': 5256}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:57.748 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:15:57.748 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'9143", 'reported_seq': 9270, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:54.845881+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:15:54.845881+0000', 'last_peered': '2024-09-11T01:15:54.845881+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:15:54.845881+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 9143, 'log_dups_size': 0, 'ondisk_log_size': 9143, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37416960, 'num_objects': 9135, 'num_object_clones': 0, 'num_object_copies': 18270, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9135, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5730, 'num_write_kb': 22920, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5151}, {'shards': '0,2', 'objects': 3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:57.748 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:15:57.748 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'9287", 'reported_seq': 9405, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:53.864505+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:15:53.864505+0000', 'last_peered': '2024-09-11T01:15:53.864505+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:15:53.864505+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 9287, 'log_dups_size': 0, 'ondisk_log_size': 9287, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38019072, 'num_objects': 9282, 'num_object_clones': 0, 'num_object_copies': 18564, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9282, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7104, 'num_write_kb': 28416, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4023}, {'shards': '2', 'objects': 5258}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:15:57.748 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:15:57.748 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'9334", 'reported_seq': 9465, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:54.861913+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:15:54.861913+0000', 'last_peered': '2024-09-11T01:15:54.861913+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:15:54.861913+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 9334, 'log_dups_size': 0, 'ondisk_log_size': 9334, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38223872, 'num_objects': 9332, 'num_object_clones': 0, 'num_object_copies': 18664, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9332, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6071, 'num_write_kb': 24284, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5382}, {'shards': '0,2', 'objects': 3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:57.748 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:15:57.748 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'9128", 'reported_seq': 10559, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:15:54.886548+0000', 'last_change': '2024-09-11T01:15:19.913288+0000', 'last_active': '2024-09-11T01:15:54.886548+0000', 'last_peered': '2024-09-11T01:15:54.886548+0000', 'last_clean': '2024-09-11T01:13:07.624565+0000', 'last_became_active': '2024-09-11T01:13:10.199795+0000', 'last_became_peered': '2024-09-11T01:13:10.199795+0000', 'last_unstale': '2024-09-11T01:15:54.886548+0000', 'last_undegraded': '2024-09-11T01:15:54.886548+0000', 'last_fullsized': '2024-09-11T01:13:09.652311+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3415", 'last_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'objects_scrubbed': 3337, 'log_size': 9128, 'log_dups_size': 0, 'ondisk_log_size': 9128, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1975, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37355520, 'num_objects': 9120, 'num_object_clones': 0, 'num_object_copies': 18240, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3291, 'num_objects_unfound': 0, 'num_objects_dirty': 9120, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5783, 'num_write_kb': 23132, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 648, 'num_bytes_recovered': 2654208, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5829}, {'shards': '0,2', 'objects': 3291}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:57.749 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:15:57.749 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'9273", 'reported_seq': 9405, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:54.821000+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:15:54.821000+0000', 'last_peered': '2024-09-11T01:15:54.821000+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:15:54.821000+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 9273, 'log_dups_size': 0, 'ondisk_log_size': 9273, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37965824, 'num_objects': 9269, 'num_object_clones': 0, 'num_object_copies': 18538, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9269, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5882, 'num_write_kb': 23528, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5230}, {'shards': '0,2', 'objects': 4039}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:57.749 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:15:57.749 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'9245", 'reported_seq': 9378, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:54.783618+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:15:54.783618+0000', 'last_peered': '2024-09-11T01:15:54.783618+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:15:54.783618+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 9245, 'log_dups_size': 0, 'ondisk_log_size': 9245, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37818985, 'num_objects': 9234, 'num_object_clones': 0, 'num_object_copies': 18468, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9234, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6315, 'num_write_kb': 25260, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5317}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:15:57.749 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:15:58.435 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 214 20 149426 149406 2.72598 1.67188 0.0109008 0.0286498 2024-09-11T01:15:58.863 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:15:58.864 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39769/295376 objects degraded (13.464%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":295,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":147688,"data_bytes":604918396,"bytes_used":2058915840,"bytes_avail":287851376640,"bytes_total":289910292480,"degraded_objects":39769,"degraded_total":295376,"degraded_ratio":0.13463856237473593,"misplaced_objects":3291,"misplaced_total":295376,"misplaced_ratio":0.0111417312171605,"recovering_objects_per_sec":15,"recovering_bytes_per_sec":65179,"recovering_keys_per_sec":0,"num_objects_recovered":191,"num_bytes_recovered":782336,"num_keys_recovered":0,"write_bytes_sec":1507658,"read_op_per_sec":0,"write_op_per_sec":368},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:15:58.923 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:15:58.923 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:15:59.435 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 215 20 149857 149837 2.72113 1.68359 0.122697 0.0287032 2024-09-11T01:15:59.775 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:15:59.776 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:15:59.841 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":186,"stamp":"2024-09-11T01:15:58.629667+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":604918396,"num_objects":147688,"num_object_clones":0,"num_object_copies":295376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":3291,"num_objects_unfound":0,"num_objects_dirty":147688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108619,"num_write_kb":434470,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4577,"num_bytes_recovered":18735123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147770,"ondisk_log_size":147770,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2010660,"kb_used_data":1204980,"kb_used_omap":12,"kb_used_meta":805619,"kb_avail":281104860,"statfs":{"total":289910292480,"available":287851376640,"internally_reserved":0,"allocated":1233899520,"data_stored":1233056375,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":824953860},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,2,0,4,6,6,10,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18096128,"num_objects":4418,"num_object_clones":0,"num_object_copies":8836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-191,"num_objects_unfound":0,"num_objects_dirty":4418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4418,"num_write_kb":17672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":191,"num_bytes_recovered":782336,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002805"},"pg_stats":[{"pgid":"2.f","version":"45'9338","reported_seq":9415,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.866399+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:15:54.866399+0000","last_peered":"2024-09-11T01:15:54.866399+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:15:54.866399+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":9338,"log_dups_size":0,"ondisk_log_size":9338,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38248448,"num_objects":9338,"num_object_clones":0,"num_object_copies":18676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9338,"num_write_kb":37352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5266},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9312","reported_seq":9413,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:53.680760+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:15:53.680760+0000","last_peered":"2024-09-11T01:15:53.680760+0000","last_clean":"2024-09-11T01:15:53.680760+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:15:53.680760+0000","last_undegraded":"2024-09-11T01:15:53.680760+0000","last_fullsized":"2024-09-11T01:15:53.680760+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":9312,"log_dups_size":0,"ondisk_log_size":9312,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38117376,"num_objects":9306,"num_object_clones":0,"num_object_copies":18612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7551,"num_write_kb":30204,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9228","reported_seq":9354,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:53.809798+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:15:53.809798+0000","last_peered":"2024-09-11T01:15:53.809798+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:15:53.809798+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":9228,"log_dups_size":0,"ondisk_log_size":9228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37781504,"num_objects":9224,"num_object_clones":0,"num_object_copies":18448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7529,"num_write_kb":30116,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":5328}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9256","reported_seq":9325,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:53.880526+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:15:53.880526+0000","last_peered":"2024-09-11T01:15:53.880526+0000","last_clean":"2024-09-11T01:15:53.880526+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:15:53.880526+0000","last_undegraded":"2024-09-11T01:15:53.880526+0000","last_fullsized":"2024-09-11T01:15:53.880526+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":9256,"log_dups_size":0,"ondisk_log_size":9256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37912576,"num_objects":9256,"num_object_clones":0,"num_object_copies":18512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9256,"num_write_kb":37024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9214","reported_seq":9341,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:54.829577+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:15:54.829577+0000","last_peered":"2024-09-11T01:15:54.829577+0000","last_clean":"2024-09-11T01:15:54.829577+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:15:54.829577+0000","last_undegraded":"2024-09-11T01:15:54.829577+0000","last_fullsized":"2024-09-11T01:15:54.829577+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":9214,"log_dups_size":0,"ondisk_log_size":9214,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37711872,"num_objects":9207,"num_object_clones":0,"num_object_copies":18414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6219,"num_write_kb":24876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9297","reported_seq":9467,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.882546+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:15:54.882546+0000","last_peered":"2024-09-11T01:15:54.882546+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:15:54.882546+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":9297,"log_dups_size":0,"ondisk_log_size":9297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38060032,"num_objects":9292,"num_object_clones":0,"num_object_copies":18584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9292,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5638,"num_write_kb":22552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5341},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9128","reported_seq":9258,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.852428+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:15:54.852428+0000","last_peered":"2024-09-11T01:15:54.852428+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:15:54.852428+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":9128,"log_dups_size":0,"ondisk_log_size":9128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37363712,"num_objects":9122,"num_object_clones":0,"num_object_copies":18244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5985,"num_write_kb":23940,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5136},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9279","reported_seq":9408,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:54.837642+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:15:54.837642+0000","last_peered":"2024-09-11T01:15:54.837642+0000","last_clean":"2024-09-11T01:15:54.837642+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:15:54.837642+0000","last_undegraded":"2024-09-11T01:15:54.837642+0000","last_fullsized":"2024-09-11T01:15:54.837642+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":9279,"log_dups_size":0,"ondisk_log_size":9279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37998592,"num_objects":9277,"num_object_clones":0,"num_object_copies":18554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6155,"num_write_kb":24620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9196","reported_seq":9323,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:53.754572+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:15:53.754572+0000","last_peered":"2024-09-11T01:15:53.754572+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:15:53.754572+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":9196,"log_dups_size":0,"ondisk_log_size":9196,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37638144,"num_objects":9189,"num_object_clones":0,"num_object_copies":18378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7167,"num_write_kb":28668,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":5256}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692287+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692287+0000","last_peered":"2024-09-11T01:15:21.692287+0000","last_clean":"2024-09-11T01:15:21.692287+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692287+0000","last_undegraded":"2024-09-11T01:15:21.692287+0000","last_fullsized":"2024-09-11T01:15:21.692287+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9143","reported_seq":9270,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.845881+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:15:54.845881+0000","last_peered":"2024-09-11T01:15:54.845881+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:15:54.845881+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":9143,"log_dups_size":0,"ondisk_log_size":9143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37416960,"num_objects":9135,"num_object_clones":0,"num_object_copies":18270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5730,"num_write_kb":22920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5151},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.685123+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.685123+0000","last_peered":"2024-09-11T01:15:21.685123+0000","last_clean":"2024-09-11T01:15:21.685123+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.685123+0000","last_undegraded":"2024-09-11T01:15:21.685123+0000","last_fullsized":"2024-09-11T01:15:21.685123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9287","reported_seq":9405,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:53.864505+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:15:53.864505+0000","last_peered":"2024-09-11T01:15:53.864505+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:15:53.864505+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":9287,"log_dups_size":0,"ondisk_log_size":9287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38019072,"num_objects":9282,"num_object_clones":0,"num_object_copies":18564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7104,"num_write_kb":28416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":5258}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9109","reported_seq":17089,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:53.779397+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:15:53.779397+0000","last_peered":"2024-09-11T01:15:53.779397+0000","last_clean":"2024-09-11T01:15:53.779397+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:15:53.779397+0000","last_undegraded":"2024-09-11T01:15:53.779397+0000","last_fullsized":"2024-09-11T01:15:53.779397+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":9109,"log_dups_size":0,"ondisk_log_size":9109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37285888,"num_objects":9103,"num_object_clones":0,"num_object_copies":18206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6894,"num_write_kb":27576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9334","reported_seq":9465,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.861913+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:15:54.861913+0000","last_peered":"2024-09-11T01:15:54.861913+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:15:54.861913+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":9334,"log_dups_size":0,"ondisk_log_size":9334,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38223872,"num_objects":9332,"num_object_clones":0,"num_object_copies":18664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6071,"num_write_kb":24284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5382},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9128","reported_seq":10559,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:54.886548+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:15:54.886548+0000","last_peered":"2024-09-11T01:15:54.886548+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:15:54.886548+0000","last_undegraded":"2024-09-11T01:15:54.886548+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":9128,"log_dups_size":0,"ondisk_log_size":9128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37355520,"num_objects":9120,"num_object_clones":0,"num_object_copies":18240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3291,"num_objects_unfound":0,"num_objects_dirty":9120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5783,"num_write_kb":23132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":648,"num_bytes_recovered":2654208,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5829},{"shards":"0,2","objects":3291}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694344+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694344+0000","last_peered":"2024-09-11T01:15:21.694344+0000","last_clean":"2024-09-11T01:15:21.694344+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694344+0000","last_undegraded":"2024-09-11T01:15:21.694344+0000","last_fullsized":"2024-09-11T01:15:21.694344+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9273","reported_seq":9405,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.821000+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:15:54.821000+0000","last_peered":"2024-09-11T01:15:54.821000+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:15:54.821000+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":9273,"log_dups_size":0,"ondisk_log_size":9273,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37965824,"num_objects":9269,"num_object_clones":0,"num_object_copies":18538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9269,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5882,"num_write_kb":23528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5230},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9245","reported_seq":9378,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:54.783618+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:15:54.783618+0000","last_peered":"2024-09-11T01:15:54.783618+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:15:54.783618+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":9245,"log_dups_size":0,"ondisk_log_size":9245,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37818985,"num_objects":9234,"num_object_clones":0,"num_object_copies":18468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6315,"num_write_kb":25260,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5317},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":604918377,"num_objects":147686,"num_object_clones":0,"num_object_copies":295372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":3291,"num_objects_unfound":0,"num_objects_dirty":147686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":108617,"num_write_kb":434468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4576,"num_bytes_recovered":18735104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1234100224,"data_stored":1234020207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":147767,"ondisk_log_size":147767,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182453,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":817660,"kb_used_data":506988,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93554180,"statfs":{"total":96636764160,"available":95799480320,"internally_reserved":0,"allocated":519155712,"data_stored":518896498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":318107307},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,0,2,2,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986225,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":697276,"kb_used_data":422700,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93674564,"statfs":{"total":96636764160,"available":95922753536,"internally_reserved":0,"allocated":432844800,"data_stored":432589663,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659171,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":495724,"kb_used_data":275292,"kb_used_omap":4,"kb_used_meta":220411,"kb_avail":93876116,"statfs":{"total":96636764160,"available":96129142784,"internally_reserved":0,"allocated":281899008,"data_stored":281570214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":225701551},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,1,3,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":519131136,"data_stored":519127657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":282271744,"data_stored":282198685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":432697344,"data_stored":432693865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:00.435 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 216 20 150299 150279 2.71653 1.72656 0.0924261 0.0287525 2024-09-11T01:16:01.435 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 217 20 150749 150729 2.71211 1.75781 0.0272987 0.028797 2024-09-11T01:16:02.436 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 218 20 151215 151195 2.70802 1.82031 0.0289547 0.0288414 2024-09-11T01:16:02.843 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:03.435 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 219 20 151648 151628 2.70338 1.69141 0.0914713 0.0288898 2024-09-11T01:16:03.779 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:16:03.780 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:16:03.844 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":188,"stamp":"2024-09-11T01:16:02.630638+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":613966460,"num_objects":149897,"num_object_clones":0,"num_object_copies":299794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":3194,"num_objects_unfound":0,"num_objects_dirty":149897,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110828,"num_write_kb":443306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4674,"num_bytes_recovered":19132435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149979,"ondisk_log_size":149979,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2001140,"kb_used_data":1216260,"kb_used_omap":12,"kb_used_meta":784819,"kb_avail":281114380,"statfs":{"total":289910292480,"available":287861125120,"internally_reserved":0,"allocated":1245450240,"data_stored":1244607095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":803654660},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,2,4,9,6,10,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17870848,"num_objects":4363,"num_object_clones":0,"num_object_copies":8726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-196,"num_objects_unfound":0,"num_objects_dirty":4363,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4363,"num_write_kb":17452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":196,"num_bytes_recovered":802816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002786"},"pg_stats":[{"pgid":"2.f","version":"45'9461","reported_seq":9538,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:59.932846+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:15:59.932846+0000","last_peered":"2024-09-11T01:15:59.932846+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:15:59.932846+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":9461,"log_dups_size":0,"ondisk_log_size":9461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38752256,"num_objects":9461,"num_object_clones":0,"num_object_copies":18922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9461,"num_write_kb":37844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5389},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9481","reported_seq":9577,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:58.895838+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:15:58.895838+0000","last_peered":"2024-09-11T01:15:58.895838+0000","last_clean":"2024-09-11T01:15:58.895838+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:15:58.895838+0000","last_undegraded":"2024-09-11T01:15:58.895838+0000","last_fullsized":"2024-09-11T01:15:58.895838+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":9481,"log_dups_size":0,"ondisk_log_size":9481,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38809600,"num_objects":9475,"num_object_clones":0,"num_object_copies":18950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7720,"num_write_kb":30880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9368","reported_seq":9494,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:58.882719+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:15:58.882719+0000","last_peered":"2024-09-11T01:15:58.882719+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:15:58.882719+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":9368,"log_dups_size":0,"ondisk_log_size":9368,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38354944,"num_objects":9364,"num_object_clones":0,"num_object_copies":18728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7669,"num_write_kb":30676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":5468}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9394","reported_seq":9463,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:58.860329+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:15:58.860329+0000","last_peered":"2024-09-11T01:15:58.860329+0000","last_clean":"2024-09-11T01:15:58.860329+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:15:58.860329+0000","last_undegraded":"2024-09-11T01:15:58.860329+0000","last_fullsized":"2024-09-11T01:15:58.860329+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":9394,"log_dups_size":0,"ondisk_log_size":9394,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38477824,"num_objects":9394,"num_object_clones":0,"num_object_copies":18788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9394,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9394,"num_write_kb":37576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9364","reported_seq":9491,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:59.806608+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:15:59.806608+0000","last_peered":"2024-09-11T01:15:59.806608+0000","last_clean":"2024-09-11T01:15:59.806608+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:15:59.806608+0000","last_undegraded":"2024-09-11T01:15:59.806608+0000","last_fullsized":"2024-09-11T01:15:59.806608+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":9364,"log_dups_size":0,"ondisk_log_size":9364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38326272,"num_objects":9357,"num_object_clones":0,"num_object_copies":18714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6369,"num_write_kb":25476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9435","reported_seq":9605,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:59.901443+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:15:59.901443+0000","last_peered":"2024-09-11T01:15:59.901443+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:15:59.901443+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":9435,"log_dups_size":0,"ondisk_log_size":9435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38625280,"num_objects":9430,"num_object_clones":0,"num_object_copies":18860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5776,"num_write_kb":23104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5479},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9289","reported_seq":9418,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:59.839633+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:15:59.839633+0000","last_peered":"2024-09-11T01:15:59.839633+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:15:59.839633+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":9289,"log_dups_size":0,"ondisk_log_size":9289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38023168,"num_objects":9283,"num_object_clones":0,"num_object_copies":18566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6146,"num_write_kb":24584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5297},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9437","reported_seq":9566,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:59.884885+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:15:59.884885+0000","last_peered":"2024-09-11T01:15:59.884885+0000","last_clean":"2024-09-11T01:15:59.884885+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:15:59.884885+0000","last_undegraded":"2024-09-11T01:15:59.884885+0000","last_fullsized":"2024-09-11T01:15:59.884885+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":9437,"log_dups_size":0,"ondisk_log_size":9437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38645760,"num_objects":9435,"num_object_clones":0,"num_object_copies":18870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6313,"num_write_kb":25252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9315","reported_seq":9442,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:58.783957+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:15:58.783957+0000","last_peered":"2024-09-11T01:15:58.783957+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:15:58.783957+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":9315,"log_dups_size":0,"ondisk_log_size":9315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38125568,"num_objects":9308,"num_object_clones":0,"num_object_copies":18616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7286,"num_write_kb":29144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":5375}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692287+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692287+0000","last_peered":"2024-09-11T01:15:21.692287+0000","last_clean":"2024-09-11T01:15:21.692287+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692287+0000","last_undegraded":"2024-09-11T01:15:21.692287+0000","last_fullsized":"2024-09-11T01:15:21.692287+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9291","reported_seq":9417,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:59.901179+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:15:59.901179+0000","last_peered":"2024-09-11T01:15:59.901179+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:15:59.901179+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":9291,"log_dups_size":0,"ondisk_log_size":9291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38023168,"num_objects":9283,"num_object_clones":0,"num_object_copies":18566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5878,"num_write_kb":23512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5299},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.685123+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.685123+0000","last_peered":"2024-09-11T01:15:21.685123+0000","last_clean":"2024-09-11T01:15:21.685123+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.685123+0000","last_undegraded":"2024-09-11T01:15:21.685123+0000","last_fullsized":"2024-09-11T01:15:21.685123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9416","reported_seq":9533,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:58.879859+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:15:58.879859+0000","last_peered":"2024-09-11T01:15:58.879859+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:15:58.879859+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":9416,"log_dups_size":0,"ondisk_log_size":9416,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38547456,"num_objects":9411,"num_object_clones":0,"num_object_copies":18822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7233,"num_write_kb":28932,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":5387}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9234","reported_seq":17214,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:58.871223+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:15:58.871223+0000","last_peered":"2024-09-11T01:15:58.871223+0000","last_clean":"2024-09-11T01:15:58.871223+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:15:58.871223+0000","last_undegraded":"2024-09-11T01:15:58.871223+0000","last_fullsized":"2024-09-11T01:15:58.871223+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":9234,"log_dups_size":0,"ondisk_log_size":9234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37797888,"num_objects":9228,"num_object_clones":0,"num_object_copies":18456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7019,"num_write_kb":28076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9477","reported_seq":9607,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:59.828869+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:15:59.828869+0000","last_peered":"2024-09-11T01:15:59.828869+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:15:59.828869+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":9477,"log_dups_size":0,"ondisk_log_size":9477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38809600,"num_objects":9475,"num_object_clones":0,"num_object_copies":18950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6214,"num_write_kb":24856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5525},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9250","reported_seq":10877,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:59.948532+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:15:59.948532+0000","last_peered":"2024-09-11T01:15:59.948532+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:15:59.948532+0000","last_undegraded":"2024-09-11T01:15:59.948532+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":9250,"log_dups_size":0,"ondisk_log_size":9250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37855232,"num_objects":9242,"num_object_clones":0,"num_object_copies":18484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3194,"num_objects_unfound":0,"num_objects_dirty":9242,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5905,"num_write_kb":23620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":745,"num_bytes_recovered":3051520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6048},{"shards":"0,2","objects":3194}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694344+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694344+0000","last_peered":"2024-09-11T01:15:21.694344+0000","last_clean":"2024-09-11T01:15:21.694344+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694344+0000","last_undegraded":"2024-09-11T01:15:21.694344+0000","last_fullsized":"2024-09-11T01:15:21.694344+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9397","reported_seq":9529,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:59.805950+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:15:59.805950+0000","last_peered":"2024-09-11T01:15:59.805950+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:15:59.805950+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":9397,"log_dups_size":0,"ondisk_log_size":9397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38473728,"num_objects":9393,"num_object_clones":0,"num_object_copies":18786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9393,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6006,"num_write_kb":24024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5354},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9367","reported_seq":9500,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:59.898392+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:15:59.898392+0000","last_peered":"2024-09-11T01:15:59.898392+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:15:59.898392+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":9367,"log_dups_size":0,"ondisk_log_size":9367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38318697,"num_objects":9356,"num_object_clones":0,"num_object_copies":18712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6437,"num_write_kb":25748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5439},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":613966441,"num_objects":149895,"num_object_clones":0,"num_object_copies":299790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":3194,"num_objects_unfound":0,"num_objects_dirty":149895,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":110826,"num_write_kb":443304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4673,"num_bytes_recovered":19132416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1245519872,"data_stored":1245439855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":149976,"ondisk_log_size":149976,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182454,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":816224,"kb_used_data":513872,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93555616,"statfs":{"total":96636764160,"available":95800950784,"internally_reserved":0,"allocated":526204928,"data_stored":525945714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":309587627},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,5,4,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986226,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":689192,"kb_used_data":427096,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93682648,"statfs":{"total":96636764160,"available":95931031552,"internally_reserved":0,"allocated":437346304,"data_stored":437091167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268365482},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,1,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659171,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":495724,"kb_used_data":275292,"kb_used_omap":4,"kb_used_meta":220411,"kb_avail":93876116,"statfs":{"total":96636764160,"available":96129142784,"internally_reserved":0,"allocated":281899008,"data_stored":281570214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":225701551},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,1,3,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":526016512,"data_stored":526013033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":282271744,"data_stored":282198685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":437231616,"data_stored":437228137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:03.846 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:04.435 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout:2024-09-11T01:16:04.436498+0000 min lat: 0.00219431 max lat: 2.69211 avg lat: 0.0289405 2024-09-11T01:16:04.435 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2024-09-11T01:16:04.436 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 220 20 152084 152064 2.69884 1.70312 0.0297307 0.0289405 2024-09-11T01:16:04.747 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:16:04.748 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:16:04.803 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":189,"stamp":"2024-09-11T01:16:04.636388+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":617439868,"num_objects":150745,"num_object_clones":0,"num_object_copies":301490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":3194,"num_objects_unfound":0,"num_objects_dirty":150745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111676,"num_write_kb":446698,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4674,"num_bytes_recovered":19132435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150827,"ondisk_log_size":150827,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2026588,"kb_used_data":1227820,"kb_used_omap":12,"kb_used_meta":798707,"kb_avail":281088932,"statfs":{"total":289910292480,"available":287835066368,"internally_reserved":0,"allocated":1257287680,"data_stored":1256444535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":817875972},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,9,11,11,11,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21344256,"num_objects":5211,"num_object_clones":0,"num_object_copies":10422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-196,"num_objects_unfound":0,"num_objects_dirty":5211,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5211,"num_write_kb":20844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":196,"num_bytes_recovered":802816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008033"},"pg_stats":[{"pgid":"2.f","version":"45'9461","reported_seq":9538,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:59.932846+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:15:59.932846+0000","last_peered":"2024-09-11T01:15:59.932846+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:15:59.932846+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":9461,"log_dups_size":0,"ondisk_log_size":9461,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38752256,"num_objects":9461,"num_object_clones":0,"num_object_copies":18922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9461,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9461,"num_write_kb":37844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5389},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9625","reported_seq":9726,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:03.842380+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:16:03.842380+0000","last_peered":"2024-09-11T01:16:03.842380+0000","last_clean":"2024-09-11T01:16:03.842380+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:16:03.842380+0000","last_undegraded":"2024-09-11T01:16:03.842380+0000","last_fullsized":"2024-09-11T01:16:03.842380+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":9625,"log_dups_size":0,"ondisk_log_size":9625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39399424,"num_objects":9619,"num_object_clones":0,"num_object_copies":19238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7864,"num_write_kb":31456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9541","reported_seq":9668,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:03.844185+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:16:03.844185+0000","last_peered":"2024-09-11T01:16:03.844185+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:16:03.844185+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":9541,"log_dups_size":0,"ondisk_log_size":9541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39063552,"num_objects":9537,"num_object_clones":0,"num_object_copies":19074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7842,"num_write_kb":31368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":5641}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9518","reported_seq":9587,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:03.833608+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:16:03.833608+0000","last_peered":"2024-09-11T01:16:03.833608+0000","last_clean":"2024-09-11T01:16:03.833608+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:16:03.833608+0000","last_undegraded":"2024-09-11T01:16:03.833608+0000","last_fullsized":"2024-09-11T01:16:03.833608+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":9518,"log_dups_size":0,"ondisk_log_size":9518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38985728,"num_objects":9518,"num_object_clones":0,"num_object_copies":19036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9518,"num_write_kb":38072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9364","reported_seq":9491,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:59.806608+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:15:59.806608+0000","last_peered":"2024-09-11T01:15:59.806608+0000","last_clean":"2024-09-11T01:15:59.806608+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:15:59.806608+0000","last_undegraded":"2024-09-11T01:15:59.806608+0000","last_fullsized":"2024-09-11T01:15:59.806608+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":9364,"log_dups_size":0,"ondisk_log_size":9364,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38326272,"num_objects":9357,"num_object_clones":0,"num_object_copies":18714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9357,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6369,"num_write_kb":25476,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9435","reported_seq":9605,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:59.901443+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:15:59.901443+0000","last_peered":"2024-09-11T01:15:59.901443+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:15:59.901443+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":9435,"log_dups_size":0,"ondisk_log_size":9435,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38625280,"num_objects":9430,"num_object_clones":0,"num_object_copies":18860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5776,"num_write_kb":23104,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5479},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9289","reported_seq":9418,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:59.839633+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:15:59.839633+0000","last_peered":"2024-09-11T01:15:59.839633+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:15:59.839633+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":9289,"log_dups_size":0,"ondisk_log_size":9289,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38023168,"num_objects":9283,"num_object_clones":0,"num_object_copies":18566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6146,"num_write_kb":24584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5297},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9437","reported_seq":9566,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:59.884885+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:15:59.884885+0000","last_peered":"2024-09-11T01:15:59.884885+0000","last_clean":"2024-09-11T01:15:59.884885+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:15:59.884885+0000","last_undegraded":"2024-09-11T01:15:59.884885+0000","last_fullsized":"2024-09-11T01:15:59.884885+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":9437,"log_dups_size":0,"ondisk_log_size":9437,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38645760,"num_objects":9435,"num_object_clones":0,"num_object_copies":18870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6313,"num_write_kb":25252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9465","reported_seq":9592,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:03.840709+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:16:03.840709+0000","last_peered":"2024-09-11T01:16:03.840709+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:16:03.840709+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":9465,"log_dups_size":0,"ondisk_log_size":9465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38739968,"num_objects":9458,"num_object_clones":0,"num_object_copies":18916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7436,"num_write_kb":29744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":5525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692287+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692287+0000","last_peered":"2024-09-11T01:15:21.692287+0000","last_clean":"2024-09-11T01:15:21.692287+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692287+0000","last_undegraded":"2024-09-11T01:15:21.692287+0000","last_fullsized":"2024-09-11T01:15:21.692287+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9291","reported_seq":9417,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:59.901179+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:15:59.901179+0000","last_peered":"2024-09-11T01:15:59.901179+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:15:59.901179+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":9291,"log_dups_size":0,"ondisk_log_size":9291,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38023168,"num_objects":9283,"num_object_clones":0,"num_object_copies":18566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9283,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5878,"num_write_kb":23512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5299},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":100,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.685123+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.685123+0000","last_peered":"2024-09-11T01:15:21.685123+0000","last_clean":"2024-09-11T01:15:21.685123+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.685123+0000","last_undegraded":"2024-09-11T01:15:21.685123+0000","last_fullsized":"2024-09-11T01:15:21.685123+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9549","reported_seq":9665,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:03.855280+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:16:03.855280+0000","last_peered":"2024-09-11T01:16:03.855280+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:16:03.855280+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":9549,"log_dups_size":0,"ondisk_log_size":9549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39092224,"num_objects":9544,"num_object_clones":0,"num_object_copies":19088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7366,"num_write_kb":29464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":5520}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9358","reported_seq":17338,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:03.893044+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:16:03.893044+0000","last_peered":"2024-09-11T01:16:03.893044+0000","last_clean":"2024-09-11T01:16:03.893044+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:16:03.893044+0000","last_undegraded":"2024-09-11T01:16:03.893044+0000","last_fullsized":"2024-09-11T01:16:03.893044+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":9358,"log_dups_size":0,"ondisk_log_size":9358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38305792,"num_objects":9352,"num_object_clones":0,"num_object_copies":18704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7143,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9477","reported_seq":9607,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:59.828869+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:15:59.828869+0000","last_peered":"2024-09-11T01:15:59.828869+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:15:59.828869+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":9477,"log_dups_size":0,"ondisk_log_size":9477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38809600,"num_objects":9475,"num_object_clones":0,"num_object_copies":18950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6214,"num_write_kb":24856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5525},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9250","reported_seq":10877,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:15:59.948532+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:15:59.948532+0000","last_peered":"2024-09-11T01:15:59.948532+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:15:59.948532+0000","last_undegraded":"2024-09-11T01:15:59.948532+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":9250,"log_dups_size":0,"ondisk_log_size":9250,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37855232,"num_objects":9242,"num_object_clones":0,"num_object_copies":18484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3194,"num_objects_unfound":0,"num_objects_dirty":9242,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5905,"num_write_kb":23620,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":745,"num_bytes_recovered":3051520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6048},{"shards":"0,2","objects":3194}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694344+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694344+0000","last_peered":"2024-09-11T01:15:21.694344+0000","last_clean":"2024-09-11T01:15:21.694344+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694344+0000","last_undegraded":"2024-09-11T01:15:21.694344+0000","last_fullsized":"2024-09-11T01:15:21.694344+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9397","reported_seq":9529,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:59.805950+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:15:59.805950+0000","last_peered":"2024-09-11T01:15:59.805950+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:15:59.805950+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":9397,"log_dups_size":0,"ondisk_log_size":9397,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38473728,"num_objects":9393,"num_object_clones":0,"num_object_copies":18786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9393,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6006,"num_write_kb":24024,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5354},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9367","reported_seq":9500,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:15:59.898392+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:15:59.898392+0000","last_peered":"2024-09-11T01:15:59.898392+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:15:59.898392+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":9367,"log_dups_size":0,"ondisk_log_size":9367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38318697,"num_objects":9356,"num_object_clones":0,"num_object_copies":18712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6437,"num_write_kb":25748,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5439},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617439849,"num_objects":150743,"num_object_clones":0,"num_object_copies":301486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":3194,"num_objects_unfound":0,"num_objects_dirty":150743,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":111674,"num_write_kb":446696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4673,"num_bytes_recovered":19132416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1257545728,"data_stored":1257465711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":150824,"ondisk_log_size":150824,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182454,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":816224,"kb_used_data":513872,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93555616,"statfs":{"total":96636764160,"available":95800950784,"internally_reserved":0,"allocated":526204928,"data_stored":525945714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":309587627},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,5,4,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986227,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":697828,"kb_used_data":431572,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93674012,"statfs":{"total":96636764160,"available":95922188288,"internally_reserved":0,"allocated":441929728,"data_stored":441674591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,6,3,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659172,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":512536,"kb_used_data":282376,"kb_used_omap":4,"kb_used_meta":230139,"kb_avail":93859304,"statfs":{"total":96636764160,"available":96111927296,"internally_reserved":0,"allocated":289153024,"data_stored":288824230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":235663023},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,4,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":526016512,"data_stored":526013033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":289636352,"data_stored":289563293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":441892864,"data_stored":441889385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:04.804 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:16:04.804 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'9461", 'reported_seq': 9538, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:59.932846+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:15:59.932846+0000', 'last_peered': '2024-09-11T01:15:59.932846+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:15:59.932846+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 9461, 'log_dups_size': 0, 'ondisk_log_size': 9461, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38752256, 'num_objects': 9461, 'num_object_clones': 0, 'num_object_copies': 18922, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9461, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9461, 'num_write_kb': 37844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5389}, {'shards': '0,2', 'objects': 4072}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:04.804 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:16:04.804 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'9541", 'reported_seq': 9668, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:03.844185+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:16:03.844185+0000', 'last_peered': '2024-09-11T01:16:03.844185+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:16:03.844185+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 9541, 'log_dups_size': 0, 'ondisk_log_size': 9541, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39063552, 'num_objects': 9537, 'num_object_clones': 0, 'num_object_copies': 19074, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9537, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7842, 'num_write_kb': 31368, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3896}, {'shards': '2', 'objects': 5641}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:04.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:16:04.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'9435", 'reported_seq': 9605, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:59.901443+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:15:59.901443+0000', 'last_peered': '2024-09-11T01:15:59.901443+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:15:59.901443+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 9435, 'log_dups_size': 0, 'ondisk_log_size': 9435, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38625280, 'num_objects': 9430, 'num_object_clones': 0, 'num_object_copies': 18860, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9430, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5776, 'num_write_kb': 23104, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5479}, {'shards': '0,2', 'objects': 3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:04.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:16:04.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'9289", 'reported_seq': 9418, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:59.839633+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:15:59.839633+0000', 'last_peered': '2024-09-11T01:15:59.839633+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:15:59.839633+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 9289, 'log_dups_size': 0, 'ondisk_log_size': 9289, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38023168, 'num_objects': 9283, 'num_object_clones': 0, 'num_object_copies': 18566, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9283, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6146, 'num_write_kb': 24584, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5297}, {'shards': '0,2', 'objects': 3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:04.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:16:04.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'9465", 'reported_seq': 9592, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:03.840709+0000', 'last_change': '2024-09-11T01:13:10.159069+0000', 'last_active': '2024-09-11T01:16:03.840709+0000', 'last_peered': '2024-09-11T01:16:03.840709+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:16:03.840709+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 9465, 'log_dups_size': 0, 'ondisk_log_size': 9465, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38739968, 'num_objects': 9458, 'num_object_clones': 0, 'num_object_copies': 18916, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9458, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7436, 'num_write_kb': 29744, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3932}, {'shards': '2', 'objects': 5525}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:04.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:16:04.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'9291", 'reported_seq': 9417, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:59.901179+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:15:59.901179+0000', 'last_peered': '2024-09-11T01:15:59.901179+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:15:59.901179+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 9291, 'log_dups_size': 0, 'ondisk_log_size': 9291, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38023168, 'num_objects': 9283, 'num_object_clones': 0, 'num_object_copies': 18566, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9283, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5878, 'num_write_kb': 23512, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5299}, {'shards': '0,2', 'objects': 3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:04.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:16:04.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'9549", 'reported_seq': 9665, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:03.855280+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:16:03.855280+0000', 'last_peered': '2024-09-11T01:16:03.855280+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:16:03.855280+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 9549, 'log_dups_size': 0, 'ondisk_log_size': 9549, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39092224, 'num_objects': 9544, 'num_object_clones': 0, 'num_object_copies': 19088, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9544, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7366, 'num_write_kb': 29464, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4023}, {'shards': '2', 'objects': 5520}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:04.806 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:16:04.806 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'9477", 'reported_seq': 9607, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:59.828869+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:15:59.828869+0000', 'last_peered': '2024-09-11T01:15:59.828869+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:15:59.828869+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 9477, 'log_dups_size': 0, 'ondisk_log_size': 9477, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38809600, 'num_objects': 9475, 'num_object_clones': 0, 'num_object_copies': 18950, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9475, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6214, 'num_write_kb': 24856, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5525}, {'shards': '0,2', 'objects': 3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:04.806 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:16:04.806 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'9250", 'reported_seq': 10877, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:15:59.948532+0000', 'last_change': '2024-09-11T01:15:19.913288+0000', 'last_active': '2024-09-11T01:15:59.948532+0000', 'last_peered': '2024-09-11T01:15:59.948532+0000', 'last_clean': '2024-09-11T01:13:07.624565+0000', 'last_became_active': '2024-09-11T01:13:10.199795+0000', 'last_became_peered': '2024-09-11T01:13:10.199795+0000', 'last_unstale': '2024-09-11T01:15:59.948532+0000', 'last_undegraded': '2024-09-11T01:15:59.948532+0000', 'last_fullsized': '2024-09-11T01:13:09.652311+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3415", 'last_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'objects_scrubbed': 3337, 'log_size': 9250, 'log_dups_size': 0, 'ondisk_log_size': 9250, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1975, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37855232, 'num_objects': 9242, 'num_object_clones': 0, 'num_object_copies': 18484, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 3194, 'num_objects_unfound': 0, 'num_objects_dirty': 9242, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 5905, 'num_write_kb': 23620, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 745, 'num_bytes_recovered': 3051520, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6048}, {'shards': '0,2', 'objects': 3194}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:04.806 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:16:04.806 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'9397", 'reported_seq': 9529, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:59.805950+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:15:59.805950+0000', 'last_peered': '2024-09-11T01:15:59.805950+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:15:59.805950+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 9397, 'log_dups_size': 0, 'ondisk_log_size': 9397, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38473728, 'num_objects': 9393, 'num_object_clones': 0, 'num_object_copies': 18786, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9393, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6006, 'num_write_kb': 24024, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5354}, {'shards': '0,2', 'objects': 4039}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:04.806 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:16:04.806 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'9367", 'reported_seq': 9500, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:15:59.898392+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:15:59.898392+0000', 'last_peered': '2024-09-11T01:15:59.898392+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:15:59.898392+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 9367, 'log_dups_size': 0, 'ondisk_log_size': 9367, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38318697, 'num_objects': 9356, 'num_object_clones': 0, 'num_object_copies': 18712, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9356, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6437, 'num_write_kb': 25748, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5439}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:04.807 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:16:05.435 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 221 20 152516 152496 2.69426 1.6875 0.123864 0.0289885 2024-09-11T01:16:05.875 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:16:05.875 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39769/301490 objects degraded (13.191%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":302,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":150745,"data_bytes":617439868,"bytes_used":2075226112,"bytes_avail":287835066368,"bytes_total":289910292480,"degraded_objects":39769,"degraded_total":301490,"degraded_ratio":0.13190818932634582,"misplaced_objects":3194,"misplaced_total":301490,"misplaced_ratio":0.010594049553882384,"recovering_objects_per_sec":16,"recovering_bytes_per_sec":66856,"recovering_keys_per_sec":0,"num_objects_recovered":196,"num_bytes_recovered":802816,"num_keys_recovered":0,"write_bytes_sec":1777497,"read_op_per_sec":0,"write_op_per_sec":433},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:16:05.926 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:16:05.926 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:06.439 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 222 20 152937 152917 2.68953 1.64453 0.050046 0.0290372 2024-09-11T01:16:06.829 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:16:06.829 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:16:06.885 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":190,"stamp":"2024-09-11T01:16:06.636871+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":623166076,"num_objects":152143,"num_object_clones":0,"num_object_copies":304286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":3094,"num_objects_unfound":0,"num_objects_dirty":152143,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":113074,"num_write_kb":452290,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4774,"num_bytes_recovered":19542035,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152225,"ondisk_log_size":152225,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2055364,"kb_used_data":1234388,"kb_used_omap":12,"kb_used_meta":820915,"kb_avail":281060156,"statfs":{"total":289910292480,"available":287805599744,"internally_reserved":0,"allocated":1264013312,"data_stored":1263170167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":840616964},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,8,11,13,14,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23887872,"num_objects":5832,"num_object_clones":0,"num_object_copies":11664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-296,"num_objects_unfound":0,"num_objects_dirty":5832,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5832,"num_write_kb":23328,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":1212416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008098"},"pg_stats":[{"pgid":"2.f","version":"45'9603","reported_seq":9680,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:04.938586+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:16:04.938586+0000","last_peered":"2024-09-11T01:16:04.938586+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:16:04.938586+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":9603,"log_dups_size":0,"ondisk_log_size":9603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39333888,"num_objects":9603,"num_object_clones":0,"num_object_copies":19206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9603,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9603,"num_write_kb":38412,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5531},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9625","reported_seq":9726,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:03.842380+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:16:03.842380+0000","last_peered":"2024-09-11T01:16:03.842380+0000","last_clean":"2024-09-11T01:16:03.842380+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:16:03.842380+0000","last_undegraded":"2024-09-11T01:16:03.842380+0000","last_fullsized":"2024-09-11T01:16:03.842380+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":9625,"log_dups_size":0,"ondisk_log_size":9625,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39399424,"num_objects":9619,"num_object_clones":0,"num_object_copies":19238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7864,"num_write_kb":31456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9541","reported_seq":9668,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:03.844185+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:16:03.844185+0000","last_peered":"2024-09-11T01:16:03.844185+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:16:03.844185+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":9541,"log_dups_size":0,"ondisk_log_size":9541,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39063552,"num_objects":9537,"num_object_clones":0,"num_object_copies":19074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9537,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7842,"num_write_kb":31368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":5641}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9518","reported_seq":9587,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:03.833608+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:16:03.833608+0000","last_peered":"2024-09-11T01:16:03.833608+0000","last_clean":"2024-09-11T01:16:03.833608+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:16:03.833608+0000","last_undegraded":"2024-09-11T01:16:03.833608+0000","last_fullsized":"2024-09-11T01:16:03.833608+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":9518,"log_dups_size":0,"ondisk_log_size":9518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38985728,"num_objects":9518,"num_object_clones":0,"num_object_copies":19036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9518,"num_write_kb":38072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9499","reported_seq":9626,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:04.946675+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:16:04.946675+0000","last_peered":"2024-09-11T01:16:04.946675+0000","last_clean":"2024-09-11T01:16:04.946675+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:16:04.946675+0000","last_undegraded":"2024-09-11T01:16:04.946675+0000","last_fullsized":"2024-09-11T01:16:04.946675+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":9499,"log_dups_size":0,"ondisk_log_size":9499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38879232,"num_objects":9492,"num_object_clones":0,"num_object_copies":18984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6504,"num_write_kb":26016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9578","reported_seq":9748,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:04.909630+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:16:04.909630+0000","last_peered":"2024-09-11T01:16:04.909630+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:16:04.909630+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":9578,"log_dups_size":0,"ondisk_log_size":9578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39211008,"num_objects":9573,"num_object_clones":0,"num_object_copies":19146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9573,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5919,"num_write_kb":23676,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5622},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9423","reported_seq":9552,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:04.811964+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:16:04.811964+0000","last_peered":"2024-09-11T01:16:04.811964+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:16:04.811964+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":9423,"log_dups_size":0,"ondisk_log_size":9423,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38572032,"num_objects":9417,"num_object_clones":0,"num_object_copies":18834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9417,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6280,"num_write_kb":25120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5431},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9578","reported_seq":9707,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:04.829425+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:16:04.829425+0000","last_peered":"2024-09-11T01:16:04.829425+0000","last_clean":"2024-09-11T01:16:04.829425+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:16:04.829425+0000","last_undegraded":"2024-09-11T01:16:04.829425+0000","last_fullsized":"2024-09-11T01:16:04.829425+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":9578,"log_dups_size":0,"ondisk_log_size":9578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39223296,"num_objects":9576,"num_object_clones":0,"num_object_copies":19152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6454,"num_write_kb":25816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9465","reported_seq":9592,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:03.840709+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:16:03.840709+0000","last_peered":"2024-09-11T01:16:03.840709+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:16:03.840709+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":9465,"log_dups_size":0,"ondisk_log_size":9465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38739968,"num_objects":9458,"num_object_clones":0,"num_object_copies":18916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7436,"num_write_kb":29744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":5525}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692287+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692287+0000","last_peered":"2024-09-11T01:15:21.692287+0000","last_clean":"2024-09-11T01:15:21.692287+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692287+0000","last_undegraded":"2024-09-11T01:15:21.692287+0000","last_fullsized":"2024-09-11T01:15:21.692287+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9430","reported_seq":9558,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:04.952134+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:16:04.952134+0000","last_peered":"2024-09-11T01:16:04.952134+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:16:04.952134+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":9430,"log_dups_size":0,"ondisk_log_size":9430,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38592512,"num_objects":9422,"num_object_clones":0,"num_object_copies":18844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9422,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6017,"num_write_kb":24068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5438},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9549","reported_seq":9665,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:03.855280+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:16:03.855280+0000","last_peered":"2024-09-11T01:16:03.855280+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:16:03.855280+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":9549,"log_dups_size":0,"ondisk_log_size":9549,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39092224,"num_objects":9544,"num_object_clones":0,"num_object_copies":19088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7366,"num_write_kb":29464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":5520}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9358","reported_seq":17338,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:03.893044+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:16:03.893044+0000","last_peered":"2024-09-11T01:16:03.893044+0000","last_clean":"2024-09-11T01:16:03.893044+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:16:03.893044+0000","last_undegraded":"2024-09-11T01:16:03.893044+0000","last_fullsized":"2024-09-11T01:16:03.893044+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":9358,"log_dups_size":0,"ondisk_log_size":9358,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38305792,"num_objects":9352,"num_object_clones":0,"num_object_copies":18704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7143,"num_write_kb":28572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9600","reported_seq":9731,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:04.938563+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:16:04.938563+0000","last_peered":"2024-09-11T01:16:04.938563+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:16:04.938563+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":9600,"log_dups_size":0,"ondisk_log_size":9600,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39313408,"num_objects":9598,"num_object_clones":0,"num_object_copies":19196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9598,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6337,"num_write_kb":25348,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5648},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9407","reported_seq":11231,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:04.978765+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:16:04.978765+0000","last_peered":"2024-09-11T01:16:04.978765+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:16:04.978765+0000","last_undegraded":"2024-09-11T01:16:04.978765+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":9407,"log_dups_size":0,"ondisk_log_size":9407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38498304,"num_objects":9399,"num_object_clones":0,"num_object_copies":18798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":3094,"num_objects_unfound":0,"num_objects_dirty":9399,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6062,"num_write_kb":24248,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":845,"num_bytes_recovered":3461120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6305},{"shards":"0,2","objects":3094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694344+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694344+0000","last_peered":"2024-09-11T01:15:21.694344+0000","last_clean":"2024-09-11T01:15:21.694344+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694344+0000","last_undegraded":"2024-09-11T01:15:21.694344+0000","last_fullsized":"2024-09-11T01:15:21.694344+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9535","reported_seq":9668,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:04.928581+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:16:04.928581+0000","last_peered":"2024-09-11T01:16:04.928581+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:16:04.928581+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":9535,"log_dups_size":0,"ondisk_log_size":9535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39038976,"num_objects":9531,"num_object_clones":0,"num_object_copies":19062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9531,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6144,"num_write_kb":24576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5492},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9513","reported_seq":9646,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:04.934816+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:16:04.934816+0000","last_peered":"2024-09-11T01:16:04.934816+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:16:04.934816+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":9513,"log_dups_size":0,"ondisk_log_size":9513,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38916713,"num_objects":9502,"num_object_clones":0,"num_object_copies":19004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6583,"num_write_kb":26332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5585},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":623166057,"num_objects":152141,"num_object_clones":0,"num_object_copies":304282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":3094,"num_objects_unfound":0,"num_objects_dirty":152141,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":113072,"num_write_kb":452288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4773,"num_bytes_recovered":19542016,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1264357376,"data_stored":1264277359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":152222,"ondisk_log_size":152222,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182455,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":845000,"kb_used_data":520440,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93526840,"statfs":{"total":96636764160,"available":95771484160,"internally_reserved":0,"allocated":532930560,"data_stored":532671346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":332328619},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,5,6,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986227,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":697828,"kb_used_data":431572,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93674012,"statfs":{"total":96636764160,"available":95922188288,"internally_reserved":0,"allocated":441929728,"data_stored":441674591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,6,3,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659172,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":512536,"kb_used_data":282376,"kb_used_omap":4,"kb_used_meta":230139,"kb_avail":93859304,"statfs":{"total":96636764160,"available":96111927296,"internally_reserved":0,"allocated":289153024,"data_stored":288824230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":235663023},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,4,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532828160,"data_stored":532824681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":289636352,"data_stored":289563293,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":441892864,"data_stored":441889385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:07.436 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 223 20 153369 153349 2.68504 1.6875 0.0213956 0.029089 2024-09-11T01:16:08.439 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 224 20 153820 153800 2.68088 1.76172 0.0551078 0.0291329 2024-09-11T01:16:09.439 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 225 20 154275 154255 2.67686 1.77734 0.129544 0.0291771 2024-09-11T01:16:09.887 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:10.439 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 226 20 154711 154691 2.67255 1.70312 0.0363805 0.0292249 2024-09-11T01:16:10.759 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:16:10.762 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:16:10.858 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":192,"stamp":"2024-09-11T01:16:10.637774+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":632148604,"num_objects":154336,"num_object_clones":0,"num_object_copies":308672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2995,"num_objects_unfound":0,"num_objects_dirty":154336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115267,"num_write_kb":461062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4873,"num_bytes_recovered":19947539,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154418,"ondisk_log_size":154418,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2089964,"kb_used_data":1252348,"kb_used_omap":12,"kb_used_meta":837555,"kb_avail":281025556,"statfs":{"total":289910292480,"available":287770169344,"internally_reserved":0,"allocated":1282404352,"data_stored":1281561207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":857656324},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,2,0,4,11,8,14,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27230208,"num_objects":6648,"num_object_clones":0,"num_object_copies":13296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-296,"num_objects_unfound":0,"num_objects_dirty":6648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6648,"num_write_kb":26592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":1212416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008106"},"pg_stats":[{"pgid":"2.f","version":"45'9740","reported_seq":9814,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:10.046430+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:16:10.046430+0000","last_peered":"2024-09-11T01:16:10.046430+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:16:10.046430+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":9740,"log_dups_size":0,"ondisk_log_size":9740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39895040,"num_objects":9740,"num_object_clones":0,"num_object_copies":19480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9740,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9740,"num_write_kb":38960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5668},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9761","reported_seq":9862,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:08.746577+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:16:08.746577+0000","last_peered":"2024-09-11T01:16:08.746577+0000","last_clean":"2024-09-11T01:16:08.746577+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:16:08.746577+0000","last_undegraded":"2024-09-11T01:16:08.746577+0000","last_fullsized":"2024-09-11T01:16:08.746577+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":9761,"log_dups_size":0,"ondisk_log_size":9761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39956480,"num_objects":9755,"num_object_clones":0,"num_object_copies":19510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8000,"num_write_kb":32000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9663","reported_seq":9790,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:08.818118+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:16:08.818118+0000","last_peered":"2024-09-11T01:16:08.818118+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:16:08.818118+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":9663,"log_dups_size":0,"ondisk_log_size":9663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39563264,"num_objects":9659,"num_object_clones":0,"num_object_copies":19318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7964,"num_write_kb":31856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":5763}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9668","reported_seq":9737,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:08.796588+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:16:08.796588+0000","last_peered":"2024-09-11T01:16:08.796588+0000","last_clean":"2024-09-11T01:16:08.796588+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:16:08.796588+0000","last_undegraded":"2024-09-11T01:16:08.796588+0000","last_fullsized":"2024-09-11T01:16:08.796588+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":9668,"log_dups_size":0,"ondisk_log_size":9668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39600128,"num_objects":9668,"num_object_clones":0,"num_object_copies":19336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9668,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9668,"num_write_kb":38672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9638","reported_seq":9765,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:09.996104+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:16:09.996104+0000","last_peered":"2024-09-11T01:16:09.996104+0000","last_clean":"2024-09-11T01:16:09.996104+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:16:09.996104+0000","last_undegraded":"2024-09-11T01:16:09.996104+0000","last_fullsized":"2024-09-11T01:16:09.996104+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":9638,"log_dups_size":0,"ondisk_log_size":9638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39448576,"num_objects":9631,"num_object_clones":0,"num_object_copies":19262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6643,"num_write_kb":26572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9707","reported_seq":9877,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:09.918905+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:16:09.918905+0000","last_peered":"2024-09-11T01:16:09.918905+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:16:09.918905+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":9707,"log_dups_size":0,"ondisk_log_size":9707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39739392,"num_objects":9702,"num_object_clones":0,"num_object_copies":19404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6048,"num_write_kb":24192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5751},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9572","reported_seq":9702,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:10.021363+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:16:10.021363+0000","last_peered":"2024-09-11T01:16:10.021363+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:16:10.021363+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":9572,"log_dups_size":0,"ondisk_log_size":9572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39182336,"num_objects":9566,"num_object_clones":0,"num_object_copies":19132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6429,"num_write_kb":25716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5580},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9717","reported_seq":9846,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:09.887440+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:16:09.887440+0000","last_peered":"2024-09-11T01:16:09.887440+0000","last_clean":"2024-09-11T01:16:09.887440+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:16:09.887440+0000","last_undegraded":"2024-09-11T01:16:09.887440+0000","last_fullsized":"2024-09-11T01:16:09.887440+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":9717,"log_dups_size":0,"ondisk_log_size":9717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39792640,"num_objects":9715,"num_object_clones":0,"num_object_copies":19430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6593,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9599","reported_seq":9726,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:08.712129+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:16:08.712129+0000","last_peered":"2024-09-11T01:16:08.712129+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:16:08.712129+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":9599,"log_dups_size":0,"ondisk_log_size":9599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39288832,"num_objects":9592,"num_object_clones":0,"num_object_copies":19184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7570,"num_write_kb":30280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":5659}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692287+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692287+0000","last_peered":"2024-09-11T01:15:21.692287+0000","last_clean":"2024-09-11T01:15:21.692287+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692287+0000","last_undegraded":"2024-09-11T01:15:21.692287+0000","last_fullsized":"2024-09-11T01:15:21.692287+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9553","reported_seq":9681,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:09.999300+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:16:09.999300+0000","last_peered":"2024-09-11T01:16:09.999300+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:16:09.999300+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":9553,"log_dups_size":0,"ondisk_log_size":9553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39096320,"num_objects":9545,"num_object_clones":0,"num_object_copies":19090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6140,"num_write_kb":24560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5561},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9696","reported_seq":9814,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:08.670323+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:16:08.670323+0000","last_peered":"2024-09-11T01:16:08.670323+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:16:08.670323+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":9696,"log_dups_size":0,"ondisk_log_size":9696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39694336,"num_objects":9691,"num_object_clones":0,"num_object_copies":19382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7513,"num_write_kb":30052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":5667}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9519","reported_seq":17500,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:08.794718+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:16:08.794718+0000","last_peered":"2024-09-11T01:16:08.794718+0000","last_clean":"2024-09-11T01:16:08.794718+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:16:08.794718+0000","last_undegraded":"2024-09-11T01:16:08.794718+0000","last_fullsized":"2024-09-11T01:16:08.794718+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":9519,"log_dups_size":0,"ondisk_log_size":9519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38965248,"num_objects":9513,"num_object_clones":0,"num_object_copies":19026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9513,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7304,"num_write_kb":29216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9745","reported_seq":9876,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:09.904499+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:16:09.904499+0000","last_peered":"2024-09-11T01:16:09.904499+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:16:09.904499+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":9745,"log_dups_size":0,"ondisk_log_size":9745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39907328,"num_objects":9743,"num_object_clones":0,"num_object_copies":19486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9743,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6482,"num_write_kb":25928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5793},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9548","reported_seq":11573,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:10.050185+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:16:10.050185+0000","last_peered":"2024-09-11T01:16:10.050185+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:16:10.050185+0000","last_undegraded":"2024-09-11T01:16:10.050185+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":9548,"log_dups_size":0,"ondisk_log_size":9548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39075840,"num_objects":9540,"num_object_clones":0,"num_object_copies":19080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2995,"num_objects_unfound":0,"num_objects_dirty":9540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6203,"num_write_kb":24812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":944,"num_bytes_recovered":3866624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6545},{"shards":"0,2","objects":2995}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694344+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694344+0000","last_peered":"2024-09-11T01:15:21.694344+0000","last_clean":"2024-09-11T01:15:21.694344+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694344+0000","last_undegraded":"2024-09-11T01:15:21.694344+0000","last_fullsized":"2024-09-11T01:15:21.694344+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9679","reported_seq":9813,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:09.996934+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:16:09.996934+0000","last_peered":"2024-09-11T01:16:09.996934+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:16:09.996934+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":9679,"log_dups_size":0,"ondisk_log_size":9679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39628800,"num_objects":9675,"num_object_clones":0,"num_object_copies":19350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6288,"num_write_kb":25152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5636},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9610","reported_seq":9743,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:10.025440+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:16:10.025440+0000","last_peered":"2024-09-11T01:16:10.025440+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:16:10.025440+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":9610,"log_dups_size":0,"ondisk_log_size":9610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39314025,"num_objects":9599,"num_object_clones":0,"num_object_copies":19198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9599,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6680,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5682},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":632148585,"num_objects":154334,"num_object_clones":0,"num_object_copies":308668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2995,"num_objects_unfound":0,"num_objects_dirty":154334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115265,"num_write_kb":461060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4872,"num_bytes_recovered":19947520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1282732032,"data_stored":1282652015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154415,"ondisk_log_size":154415,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182456,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":859840,"kb_used_data":526960,"kb_used_omap":4,"kb_used_meta":332859,"kb_avail":93512000,"statfs":{"total":96636764160,"available":95756288000,"internally_reserved":0,"allocated":539607040,"data_stored":539347826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":340848299},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1,0,1,6,2,8,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986228,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":706588,"kb_used_data":436172,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93665252,"statfs":{"total":96636764160,"available":95913218048,"internally_reserved":0,"allocated":446640128,"data_stored":446384991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276885162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,0,1,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659173,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523536,"kb_used_data":289216,"kb_used_omap":4,"kb_used_meta":234299,"kb_avail":93848304,"statfs":{"total":96636764160,"available":96100663296,"internally_reserved":0,"allocated":296157184,"data_stored":295828390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":239922863},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":539570176,"data_stored":539566697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":296714240,"data_stored":296641181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":446447616,"data_stored":446444137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:10.859 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:11.439 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 227 20 155147 155127 2.66829 1.70312 0.0623902 0.0292715 2024-09-11T01:16:11.665 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:16:11.666 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:16:11.715 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":192,"stamp":"2024-09-11T01:16:10.637774+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":632148604,"num_objects":154336,"num_object_clones":0,"num_object_copies":308672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2995,"num_objects_unfound":0,"num_objects_dirty":154336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115267,"num_write_kb":461062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4873,"num_bytes_recovered":19947539,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154418,"ondisk_log_size":154418,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2089964,"kb_used_data":1252348,"kb_used_omap":12,"kb_used_meta":837555,"kb_avail":281025556,"statfs":{"total":289910292480,"available":287770169344,"internally_reserved":0,"allocated":1282404352,"data_stored":1281561207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":857656324},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,2,0,4,11,8,14,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27230208,"num_objects":6648,"num_object_clones":0,"num_object_copies":13296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-296,"num_objects_unfound":0,"num_objects_dirty":6648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6648,"num_write_kb":26592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":296,"num_bytes_recovered":1212416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008106"},"pg_stats":[{"pgid":"2.f","version":"45'9740","reported_seq":9814,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:10.046430+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:16:10.046430+0000","last_peered":"2024-09-11T01:16:10.046430+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:16:10.046430+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":9740,"log_dups_size":0,"ondisk_log_size":9740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39895040,"num_objects":9740,"num_object_clones":0,"num_object_copies":19480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9740,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9740,"num_write_kb":38960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5668},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9761","reported_seq":9862,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:08.746577+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:16:08.746577+0000","last_peered":"2024-09-11T01:16:08.746577+0000","last_clean":"2024-09-11T01:16:08.746577+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:16:08.746577+0000","last_undegraded":"2024-09-11T01:16:08.746577+0000","last_fullsized":"2024-09-11T01:16:08.746577+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":9761,"log_dups_size":0,"ondisk_log_size":9761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39956480,"num_objects":9755,"num_object_clones":0,"num_object_copies":19510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8000,"num_write_kb":32000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9663","reported_seq":9790,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:08.818118+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:16:08.818118+0000","last_peered":"2024-09-11T01:16:08.818118+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:16:08.818118+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":9663,"log_dups_size":0,"ondisk_log_size":9663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39563264,"num_objects":9659,"num_object_clones":0,"num_object_copies":19318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7964,"num_write_kb":31856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":5763}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9668","reported_seq":9737,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:08.796588+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:16:08.796588+0000","last_peered":"2024-09-11T01:16:08.796588+0000","last_clean":"2024-09-11T01:16:08.796588+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:16:08.796588+0000","last_undegraded":"2024-09-11T01:16:08.796588+0000","last_fullsized":"2024-09-11T01:16:08.796588+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":9668,"log_dups_size":0,"ondisk_log_size":9668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39600128,"num_objects":9668,"num_object_clones":0,"num_object_copies":19336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9668,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9668,"num_write_kb":38672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9638","reported_seq":9765,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:09.996104+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:16:09.996104+0000","last_peered":"2024-09-11T01:16:09.996104+0000","last_clean":"2024-09-11T01:16:09.996104+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:16:09.996104+0000","last_undegraded":"2024-09-11T01:16:09.996104+0000","last_fullsized":"2024-09-11T01:16:09.996104+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":9638,"log_dups_size":0,"ondisk_log_size":9638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39448576,"num_objects":9631,"num_object_clones":0,"num_object_copies":19262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6643,"num_write_kb":26572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9707","reported_seq":9877,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:09.918905+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:16:09.918905+0000","last_peered":"2024-09-11T01:16:09.918905+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:16:09.918905+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":9707,"log_dups_size":0,"ondisk_log_size":9707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39739392,"num_objects":9702,"num_object_clones":0,"num_object_copies":19404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6048,"num_write_kb":24192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5751},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9572","reported_seq":9702,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:10.021363+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:16:10.021363+0000","last_peered":"2024-09-11T01:16:10.021363+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:16:10.021363+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":9572,"log_dups_size":0,"ondisk_log_size":9572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39182336,"num_objects":9566,"num_object_clones":0,"num_object_copies":19132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6429,"num_write_kb":25716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5580},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9717","reported_seq":9846,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:09.887440+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:16:09.887440+0000","last_peered":"2024-09-11T01:16:09.887440+0000","last_clean":"2024-09-11T01:16:09.887440+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:16:09.887440+0000","last_undegraded":"2024-09-11T01:16:09.887440+0000","last_fullsized":"2024-09-11T01:16:09.887440+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":9717,"log_dups_size":0,"ondisk_log_size":9717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39792640,"num_objects":9715,"num_object_clones":0,"num_object_copies":19430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6593,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9599","reported_seq":9726,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:08.712129+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:16:08.712129+0000","last_peered":"2024-09-11T01:16:08.712129+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:16:08.712129+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":9599,"log_dups_size":0,"ondisk_log_size":9599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39288832,"num_objects":9592,"num_object_clones":0,"num_object_copies":19184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7570,"num_write_kb":30280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":5659}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692287+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692287+0000","last_peered":"2024-09-11T01:15:21.692287+0000","last_clean":"2024-09-11T01:15:21.692287+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692287+0000","last_undegraded":"2024-09-11T01:15:21.692287+0000","last_fullsized":"2024-09-11T01:15:21.692287+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9553","reported_seq":9681,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:09.999300+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:16:09.999300+0000","last_peered":"2024-09-11T01:16:09.999300+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:16:09.999300+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":9553,"log_dups_size":0,"ondisk_log_size":9553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39096320,"num_objects":9545,"num_object_clones":0,"num_object_copies":19090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6140,"num_write_kb":24560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5561},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9696","reported_seq":9814,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:08.670323+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:16:08.670323+0000","last_peered":"2024-09-11T01:16:08.670323+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:16:08.670323+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":9696,"log_dups_size":0,"ondisk_log_size":9696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39694336,"num_objects":9691,"num_object_clones":0,"num_object_copies":19382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7513,"num_write_kb":30052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":5667}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9519","reported_seq":17500,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:08.794718+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:16:08.794718+0000","last_peered":"2024-09-11T01:16:08.794718+0000","last_clean":"2024-09-11T01:16:08.794718+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:16:08.794718+0000","last_undegraded":"2024-09-11T01:16:08.794718+0000","last_fullsized":"2024-09-11T01:16:08.794718+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":9519,"log_dups_size":0,"ondisk_log_size":9519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38965248,"num_objects":9513,"num_object_clones":0,"num_object_copies":19026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9513,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7304,"num_write_kb":29216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9745","reported_seq":9876,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:09.904499+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:16:09.904499+0000","last_peered":"2024-09-11T01:16:09.904499+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:16:09.904499+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":9745,"log_dups_size":0,"ondisk_log_size":9745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39907328,"num_objects":9743,"num_object_clones":0,"num_object_copies":19486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9743,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6482,"num_write_kb":25928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5793},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9548","reported_seq":11573,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:10.050185+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:16:10.050185+0000","last_peered":"2024-09-11T01:16:10.050185+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:16:10.050185+0000","last_undegraded":"2024-09-11T01:16:10.050185+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":9548,"log_dups_size":0,"ondisk_log_size":9548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39075840,"num_objects":9540,"num_object_clones":0,"num_object_copies":19080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2995,"num_objects_unfound":0,"num_objects_dirty":9540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6203,"num_write_kb":24812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":944,"num_bytes_recovered":3866624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6545},{"shards":"0,2","objects":2995}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694344+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694344+0000","last_peered":"2024-09-11T01:15:21.694344+0000","last_clean":"2024-09-11T01:15:21.694344+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694344+0000","last_undegraded":"2024-09-11T01:15:21.694344+0000","last_fullsized":"2024-09-11T01:15:21.694344+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9679","reported_seq":9813,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:09.996934+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:16:09.996934+0000","last_peered":"2024-09-11T01:16:09.996934+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:16:09.996934+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":9679,"log_dups_size":0,"ondisk_log_size":9679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39628800,"num_objects":9675,"num_object_clones":0,"num_object_copies":19350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6288,"num_write_kb":25152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5636},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9610","reported_seq":9743,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:10.025440+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:16:10.025440+0000","last_peered":"2024-09-11T01:16:10.025440+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:16:10.025440+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":9610,"log_dups_size":0,"ondisk_log_size":9610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39314025,"num_objects":9599,"num_object_clones":0,"num_object_copies":19198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9599,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6680,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5682},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":632148585,"num_objects":154334,"num_object_clones":0,"num_object_copies":308668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2995,"num_objects_unfound":0,"num_objects_dirty":154334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115265,"num_write_kb":461060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4872,"num_bytes_recovered":19947520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1282732032,"data_stored":1282652015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154415,"ondisk_log_size":154415,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182456,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":859840,"kb_used_data":526960,"kb_used_omap":4,"kb_used_meta":332859,"kb_avail":93512000,"statfs":{"total":96636764160,"available":95756288000,"internally_reserved":0,"allocated":539607040,"data_stored":539347826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":340848299},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1,0,1,6,2,8,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986228,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":706588,"kb_used_data":436172,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93665252,"statfs":{"total":96636764160,"available":95913218048,"internally_reserved":0,"allocated":446640128,"data_stored":446384991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276885162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,0,1,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659173,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523536,"kb_used_data":289216,"kb_used_omap":4,"kb_used_meta":234299,"kb_avail":93848304,"statfs":{"total":96636764160,"available":96100663296,"internally_reserved":0,"allocated":296157184,"data_stored":295828390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":239922863},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":539570176,"data_stored":539566697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":296714240,"data_stored":296641181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":446447616,"data_stored":446444137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:11.717 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:16:11.717 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'9740", 'reported_seq': 9814, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:10.046430+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:16:10.046430+0000', 'last_peered': '2024-09-11T01:16:10.046430+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:16:10.046430+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 9740, 'log_dups_size': 0, 'ondisk_log_size': 9740, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39895040, 'num_objects': 9740, 'num_object_clones': 0, 'num_object_copies': 19480, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9740, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9740, 'num_write_kb': 38960, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5668}, {'shards': '0,2', 'objects': 4072}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:11.717 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:16:11.717 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'9663", 'reported_seq': 9790, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:08.818118+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:16:08.818118+0000', 'last_peered': '2024-09-11T01:16:08.818118+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:16:08.818118+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 9663, 'log_dups_size': 0, 'ondisk_log_size': 9663, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39563264, 'num_objects': 9659, 'num_object_clones': 0, 'num_object_copies': 19318, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9659, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7964, 'num_write_kb': 31856, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3896}, {'shards': '2', 'objects': 5763}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:11.717 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:16:11.717 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'9707", 'reported_seq': 9877, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:09.918905+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:16:09.918905+0000', 'last_peered': '2024-09-11T01:16:09.918905+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:16:09.918905+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 9707, 'log_dups_size': 0, 'ondisk_log_size': 9707, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39739392, 'num_objects': 9702, 'num_object_clones': 0, 'num_object_copies': 19404, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9702, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6048, 'num_write_kb': 24192, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5751}, {'shards': '0,2', 'objects': 3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:11.717 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:16:11.718 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'9572", 'reported_seq': 9702, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:10.021363+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:16:10.021363+0000', 'last_peered': '2024-09-11T01:16:10.021363+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:16:10.021363+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 9572, 'log_dups_size': 0, 'ondisk_log_size': 9572, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39182336, 'num_objects': 9566, 'num_object_clones': 0, 'num_object_copies': 19132, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9566, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6429, 'num_write_kb': 25716, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5580}, {'shards': '0,2', 'objects': 3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:11.718 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:16:11.718 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'9599", 'reported_seq': 9726, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:08.712129+0000', 'last_change': '2024-09-11T01:13:10.159069+0000', 'last_active': '2024-09-11T01:16:08.712129+0000', 'last_peered': '2024-09-11T01:16:08.712129+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:16:08.712129+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 9599, 'log_dups_size': 0, 'ondisk_log_size': 9599, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39288832, 'num_objects': 9592, 'num_object_clones': 0, 'num_object_copies': 19184, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9592, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7570, 'num_write_kb': 30280, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3932}, {'shards': '2', 'objects': 5659}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:11.718 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:16:11.718 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'9553", 'reported_seq': 9681, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:09.999300+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:16:09.999300+0000', 'last_peered': '2024-09-11T01:16:09.999300+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:16:09.999300+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 9553, 'log_dups_size': 0, 'ondisk_log_size': 9553, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39096320, 'num_objects': 9545, 'num_object_clones': 0, 'num_object_copies': 19090, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9545, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6140, 'num_write_kb': 24560, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5561}, {'shards': '0,2', 'objects': 3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:11.718 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:16:11.718 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'9696", 'reported_seq': 9814, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:08.670323+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:16:08.670323+0000', 'last_peered': '2024-09-11T01:16:08.670323+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:16:08.670323+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 9696, 'log_dups_size': 0, 'ondisk_log_size': 9696, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39694336, 'num_objects': 9691, 'num_object_clones': 0, 'num_object_copies': 19382, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9691, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7513, 'num_write_kb': 30052, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4023}, {'shards': '2', 'objects': 5667}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:11.718 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:16:11.718 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'9745", 'reported_seq': 9876, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:09.904499+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:16:09.904499+0000', 'last_peered': '2024-09-11T01:16:09.904499+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:16:09.904499+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 9745, 'log_dups_size': 0, 'ondisk_log_size': 9745, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39907328, 'num_objects': 9743, 'num_object_clones': 0, 'num_object_copies': 19486, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9743, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6482, 'num_write_kb': 25928, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5793}, {'shards': '0,2', 'objects': 3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:11.718 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:16:11.718 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'9548", 'reported_seq': 11573, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:16:10.050185+0000', 'last_change': '2024-09-11T01:15:19.913288+0000', 'last_active': '2024-09-11T01:16:10.050185+0000', 'last_peered': '2024-09-11T01:16:10.050185+0000', 'last_clean': '2024-09-11T01:13:07.624565+0000', 'last_became_active': '2024-09-11T01:13:10.199795+0000', 'last_became_peered': '2024-09-11T01:13:10.199795+0000', 'last_unstale': '2024-09-11T01:16:10.050185+0000', 'last_undegraded': '2024-09-11T01:16:10.050185+0000', 'last_fullsized': '2024-09-11T01:13:09.652311+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3415", 'last_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'objects_scrubbed': 3337, 'log_size': 9548, 'log_dups_size': 0, 'ondisk_log_size': 9548, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1975, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39075840, 'num_objects': 9540, 'num_object_clones': 0, 'num_object_copies': 19080, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2995, 'num_objects_unfound': 0, 'num_objects_dirty': 9540, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6203, 'num_write_kb': 24812, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 944, 'num_bytes_recovered': 3866624, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6545}, {'shards': '0,2', 'objects': 2995}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:11.719 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:16:11.719 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'9679", 'reported_seq': 9813, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:09.996934+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:16:09.996934+0000', 'last_peered': '2024-09-11T01:16:09.996934+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:16:09.996934+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 9679, 'log_dups_size': 0, 'ondisk_log_size': 9679, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39628800, 'num_objects': 9675, 'num_object_clones': 0, 'num_object_copies': 19350, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9675, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6288, 'num_write_kb': 25152, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5636}, {'shards': '0,2', 'objects': 4039}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:11.719 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:16:11.719 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'9610", 'reported_seq': 9743, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:10.025440+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:16:10.025440+0000', 'last_peered': '2024-09-11T01:16:10.025440+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:16:10.025440+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 9610, 'log_dups_size': 0, 'ondisk_log_size': 9610, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39314025, 'num_objects': 9599, 'num_object_clones': 0, 'num_object_copies': 19198, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9599, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6680, 'num_write_kb': 26720, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5682}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:11.719 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:16:12.439 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 228 20 155547 155527 2.66344 1.5625 0.0316787 0.0293255 2024-09-11T01:16:12.903 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:16:12.903 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39769/308672 objects degraded (12.884%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":309,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":154336,"data_bytes":632148604,"bytes_used":2140123136,"bytes_avail":287770169344,"bytes_total":289910292480,"degraded_objects":39769,"degraded_total":308672,"degraded_ratio":0.12883902653949825,"misplaced_objects":2995,"misplaced_total":308672,"misplaced_ratio":0.0097028561061579931,"recovering_objects_per_sec":16,"recovering_bytes_per_sec":67879,"recovering_keys_per_sec":0,"num_objects_recovered":199,"num_bytes_recovered":815104,"num_keys_recovered":0,"write_bytes_sec":1514153,"read_op_per_sec":0,"write_op_per_sec":369},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:16:12.995 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:16:12.995 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:13.441 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 229 20 155975 155955 2.65911 1.67188 0.0337715 0.0293711 2024-09-11T01:16:13.695 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:16:13.695 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:16:13.756 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":193,"stamp":"2024-09-11T01:16:12.638261+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":632148604,"num_objects":154336,"num_object_clones":0,"num_object_copies":308672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2995,"num_objects_unfound":0,"num_objects_dirty":154336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115267,"num_write_kb":461062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4873,"num_bytes_recovered":19947539,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154418,"ondisk_log_size":154418,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2089964,"kb_used_data":1252348,"kb_used_omap":12,"kb_used_meta":837555,"kb_avail":281025556,"statfs":{"total":289910292480,"available":287770169344,"internally_reserved":0,"allocated":1282404352,"data_stored":1281561207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":857656324},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[7,2,0,4,11,8,14,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":18182144,"num_objects":4439,"num_object_clones":0,"num_object_copies":8878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-199,"num_objects_unfound":0,"num_objects_dirty":4439,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4439,"num_write_kb":17756,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":199,"num_bytes_recovered":815104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008128"},"pg_stats":[{"pgid":"2.f","version":"45'9740","reported_seq":9814,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:10.046430+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:16:10.046430+0000","last_peered":"2024-09-11T01:16:10.046430+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:16:10.046430+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":9740,"log_dups_size":0,"ondisk_log_size":9740,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39895040,"num_objects":9740,"num_object_clones":0,"num_object_copies":19480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9740,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9740,"num_write_kb":38960,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5668},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9761","reported_seq":9862,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:08.746577+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:16:08.746577+0000","last_peered":"2024-09-11T01:16:08.746577+0000","last_clean":"2024-09-11T01:16:08.746577+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:16:08.746577+0000","last_undegraded":"2024-09-11T01:16:08.746577+0000","last_fullsized":"2024-09-11T01:16:08.746577+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":9761,"log_dups_size":0,"ondisk_log_size":9761,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39956480,"num_objects":9755,"num_object_clones":0,"num_object_copies":19510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8000,"num_write_kb":32000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9663","reported_seq":9790,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:08.818118+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:16:08.818118+0000","last_peered":"2024-09-11T01:16:08.818118+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:16:08.818118+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":9663,"log_dups_size":0,"ondisk_log_size":9663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39563264,"num_objects":9659,"num_object_clones":0,"num_object_copies":19318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9659,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7964,"num_write_kb":31856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":5763}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9668","reported_seq":9737,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:08.796588+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:16:08.796588+0000","last_peered":"2024-09-11T01:16:08.796588+0000","last_clean":"2024-09-11T01:16:08.796588+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:16:08.796588+0000","last_undegraded":"2024-09-11T01:16:08.796588+0000","last_fullsized":"2024-09-11T01:16:08.796588+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":9668,"log_dups_size":0,"ondisk_log_size":9668,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39600128,"num_objects":9668,"num_object_clones":0,"num_object_copies":19336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9668,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9668,"num_write_kb":38672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9638","reported_seq":9765,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:09.996104+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:16:09.996104+0000","last_peered":"2024-09-11T01:16:09.996104+0000","last_clean":"2024-09-11T01:16:09.996104+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:16:09.996104+0000","last_undegraded":"2024-09-11T01:16:09.996104+0000","last_fullsized":"2024-09-11T01:16:09.996104+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":9638,"log_dups_size":0,"ondisk_log_size":9638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39448576,"num_objects":9631,"num_object_clones":0,"num_object_copies":19262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6643,"num_write_kb":26572,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9707","reported_seq":9877,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:09.918905+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:16:09.918905+0000","last_peered":"2024-09-11T01:16:09.918905+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:16:09.918905+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":9707,"log_dups_size":0,"ondisk_log_size":9707,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39739392,"num_objects":9702,"num_object_clones":0,"num_object_copies":19404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9702,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6048,"num_write_kb":24192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5751},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9572","reported_seq":9702,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:10.021363+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:16:10.021363+0000","last_peered":"2024-09-11T01:16:10.021363+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:16:10.021363+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":9572,"log_dups_size":0,"ondisk_log_size":9572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39182336,"num_objects":9566,"num_object_clones":0,"num_object_copies":19132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9566,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6429,"num_write_kb":25716,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5580},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9717","reported_seq":9846,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:09.887440+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:16:09.887440+0000","last_peered":"2024-09-11T01:16:09.887440+0000","last_clean":"2024-09-11T01:16:09.887440+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:16:09.887440+0000","last_undegraded":"2024-09-11T01:16:09.887440+0000","last_fullsized":"2024-09-11T01:16:09.887440+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":9717,"log_dups_size":0,"ondisk_log_size":9717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39792640,"num_objects":9715,"num_object_clones":0,"num_object_copies":19430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6593,"num_write_kb":26372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9599","reported_seq":9726,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:08.712129+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:16:08.712129+0000","last_peered":"2024-09-11T01:16:08.712129+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:16:08.712129+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":9599,"log_dups_size":0,"ondisk_log_size":9599,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39288832,"num_objects":9592,"num_object_clones":0,"num_object_copies":19184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7570,"num_write_kb":30280,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":5659}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692287+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692287+0000","last_peered":"2024-09-11T01:15:21.692287+0000","last_clean":"2024-09-11T01:15:21.692287+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692287+0000","last_undegraded":"2024-09-11T01:15:21.692287+0000","last_fullsized":"2024-09-11T01:15:21.692287+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9553","reported_seq":9681,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:09.999300+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:16:09.999300+0000","last_peered":"2024-09-11T01:16:09.999300+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:16:09.999300+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":9553,"log_dups_size":0,"ondisk_log_size":9553,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39096320,"num_objects":9545,"num_object_clones":0,"num_object_copies":19090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9545,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6140,"num_write_kb":24560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5561},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9696","reported_seq":9814,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:08.670323+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:16:08.670323+0000","last_peered":"2024-09-11T01:16:08.670323+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:16:08.670323+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":9696,"log_dups_size":0,"ondisk_log_size":9696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39694336,"num_objects":9691,"num_object_clones":0,"num_object_copies":19382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7513,"num_write_kb":30052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":5667}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9519","reported_seq":17500,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:08.794718+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:16:08.794718+0000","last_peered":"2024-09-11T01:16:08.794718+0000","last_clean":"2024-09-11T01:16:08.794718+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:16:08.794718+0000","last_undegraded":"2024-09-11T01:16:08.794718+0000","last_fullsized":"2024-09-11T01:16:08.794718+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":9519,"log_dups_size":0,"ondisk_log_size":9519,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38965248,"num_objects":9513,"num_object_clones":0,"num_object_copies":19026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9513,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7304,"num_write_kb":29216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9745","reported_seq":9876,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:09.904499+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:16:09.904499+0000","last_peered":"2024-09-11T01:16:09.904499+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:16:09.904499+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":9745,"log_dups_size":0,"ondisk_log_size":9745,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39907328,"num_objects":9743,"num_object_clones":0,"num_object_copies":19486,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9743,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6482,"num_write_kb":25928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5793},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9548","reported_seq":11573,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:10.050185+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:16:10.050185+0000","last_peered":"2024-09-11T01:16:10.050185+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:16:10.050185+0000","last_undegraded":"2024-09-11T01:16:10.050185+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":9548,"log_dups_size":0,"ondisk_log_size":9548,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39075840,"num_objects":9540,"num_object_clones":0,"num_object_copies":19080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2995,"num_objects_unfound":0,"num_objects_dirty":9540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6203,"num_write_kb":24812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":944,"num_bytes_recovered":3866624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6545},{"shards":"0,2","objects":2995}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694344+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694344+0000","last_peered":"2024-09-11T01:15:21.694344+0000","last_clean":"2024-09-11T01:15:21.694344+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694344+0000","last_undegraded":"2024-09-11T01:15:21.694344+0000","last_fullsized":"2024-09-11T01:15:21.694344+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9679","reported_seq":9813,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:09.996934+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:16:09.996934+0000","last_peered":"2024-09-11T01:16:09.996934+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:16:09.996934+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":9679,"log_dups_size":0,"ondisk_log_size":9679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39628800,"num_objects":9675,"num_object_clones":0,"num_object_copies":19350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9675,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6288,"num_write_kb":25152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5636},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9610","reported_seq":9743,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:10.025440+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:16:10.025440+0000","last_peered":"2024-09-11T01:16:10.025440+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:16:10.025440+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":9610,"log_dups_size":0,"ondisk_log_size":9610,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39314025,"num_objects":9599,"num_object_clones":0,"num_object_copies":19198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9599,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6680,"num_write_kb":26720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5682},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":632148585,"num_objects":154334,"num_object_clones":0,"num_object_copies":308668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2995,"num_objects_unfound":0,"num_objects_dirty":154334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":115265,"num_write_kb":461060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4872,"num_bytes_recovered":19947520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1282732032,"data_stored":1282652015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":154415,"ondisk_log_size":154415,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182456,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":859840,"kb_used_data":526960,"kb_used_omap":4,"kb_used_meta":332859,"kb_avail":93512000,"statfs":{"total":96636764160,"available":95756288000,"internally_reserved":0,"allocated":539607040,"data_stored":539347826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":340848299},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,1,0,1,6,2,8,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986228,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":706588,"kb_used_data":436172,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93665252,"statfs":{"total":96636764160,"available":95913218048,"internally_reserved":0,"allocated":446640128,"data_stored":446384991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276885162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,0,1,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659173,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":523536,"kb_used_data":289216,"kb_used_omap":4,"kb_used_meta":234299,"kb_avail":93848304,"statfs":{"total":96636764160,"available":96100663296,"internally_reserved":0,"allocated":296157184,"data_stored":295828390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":239922863},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":539570176,"data_stored":539566697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":296714240,"data_stored":296641181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":446447616,"data_stored":446444137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:14.443 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 230 20 156428 156408 2.6552 1.76953 0.0176104 0.0294147 2024-09-11T01:16:15.444 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 231 20 156866 156846 2.65111 1.71094 0.0612094 0.0294613 2024-09-11T01:16:16.443 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 232 20 157302 157282 2.64703 1.70312 0.0541165 0.0295062 2024-09-11T01:16:16.760 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:17.443 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 233 20 157723 157703 2.64273 1.64453 0.0315277 0.0295541 2024-09-11T01:16:17.468 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:16:17.470 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:16:17.522 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2024-09-11T01:16:16.639167+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":641102460,"num_objects":156522,"num_object_clones":0,"num_object_copies":313044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2896,"num_objects_unfound":0,"num_objects_dirty":156522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":117453,"num_write_kb":469806,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4972,"num_bytes_recovered":20353043,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156604,"ondisk_log_size":156604,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2103100,"kb_used_data":1269644,"kb_used_omap":12,"kb_used_meta":833395,"kb_avail":281012420,"statfs":{"total":289910292480,"available":287756718080,"internally_reserved":0,"allocated":1300115456,"data_stored":1299272311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":853396484},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,2,4,5,10,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23662592,"num_objects":5777,"num_object_clones":0,"num_object_copies":11554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-298,"num_objects_unfound":0,"num_objects_dirty":5777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5777,"num_write_kb":23108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":298,"num_bytes_recovered":1220608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002779"},"pg_stats":[{"pgid":"2.f","version":"45'9863","reported_seq":9940,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:14.988418+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:16:14.988418+0000","last_peered":"2024-09-11T01:16:14.988418+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:16:14.988418+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":9863,"log_dups_size":0,"ondisk_log_size":9863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40398848,"num_objects":9863,"num_object_clones":0,"num_object_copies":19726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9863,"num_write_kb":39452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5791},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9909","reported_seq":10010,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:13.914030+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:16:13.914030+0000","last_peered":"2024-09-11T01:16:13.914030+0000","last_clean":"2024-09-11T01:16:13.914030+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:16:13.914030+0000","last_undegraded":"2024-09-11T01:16:13.914030+0000","last_fullsized":"2024-09-11T01:16:13.914030+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":9909,"log_dups_size":0,"ondisk_log_size":9909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40562688,"num_objects":9903,"num_object_clones":0,"num_object_copies":19806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8148,"num_write_kb":32592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9807","reported_seq":9934,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:13.951470+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:16:13.951470+0000","last_peered":"2024-09-11T01:16:13.951470+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:16:13.951470+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":9807,"log_dups_size":0,"ondisk_log_size":9807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40153088,"num_objects":9803,"num_object_clones":0,"num_object_copies":19606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8108,"num_write_kb":32432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":5907}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9814","reported_seq":9883,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:13.940879+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:16:13.940879+0000","last_peered":"2024-09-11T01:16:13.940879+0000","last_clean":"2024-09-11T01:16:13.940879+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:16:13.940879+0000","last_undegraded":"2024-09-11T01:16:13.940879+0000","last_fullsized":"2024-09-11T01:16:13.940879+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":9814,"log_dups_size":0,"ondisk_log_size":9814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40198144,"num_objects":9814,"num_object_clones":0,"num_object_copies":19628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9814,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9814,"num_write_kb":39256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9776","reported_seq":9903,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:15.058365+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:16:15.058365+0000","last_peered":"2024-09-11T01:16:15.058365+0000","last_clean":"2024-09-11T01:16:15.058365+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:16:15.058365+0000","last_undegraded":"2024-09-11T01:16:15.058365+0000","last_fullsized":"2024-09-11T01:16:15.058365+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":9776,"log_dups_size":0,"ondisk_log_size":9776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40013824,"num_objects":9769,"num_object_clones":0,"num_object_copies":19538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6781,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9857","reported_seq":10027,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:15.088581+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:16:15.088581+0000","last_peered":"2024-09-11T01:16:15.088581+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:16:15.088581+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":9857,"log_dups_size":0,"ondisk_log_size":9857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40353792,"num_objects":9852,"num_object_clones":0,"num_object_copies":19704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6198,"num_write_kb":24792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5901},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9700","reported_seq":9830,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:14.933665+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:16:14.933665+0000","last_peered":"2024-09-11T01:16:14.933665+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:16:14.933665+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":9700,"log_dups_size":0,"ondisk_log_size":9700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39706624,"num_objects":9694,"num_object_clones":0,"num_object_copies":19388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6557,"num_write_kb":26228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5708},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9867","reported_seq":9994,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:15.067323+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:16:15.067323+0000","last_peered":"2024-09-11T01:16:15.067323+0000","last_clean":"2024-09-11T01:16:15.067323+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:16:15.067323+0000","last_undegraded":"2024-09-11T01:16:15.067323+0000","last_fullsized":"2024-09-11T01:16:15.067323+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":9867,"log_dups_size":0,"ondisk_log_size":9867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40407040,"num_objects":9865,"num_object_clones":0,"num_object_copies":19730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6743,"num_write_kb":26972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9725","reported_seq":9852,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:13.813981+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:16:13.813981+0000","last_peered":"2024-09-11T01:16:13.813981+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:16:13.813981+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":9725,"log_dups_size":0,"ondisk_log_size":9725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39804928,"num_objects":9718,"num_object_clones":0,"num_object_copies":19436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7696,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":5785}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692287+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692287+0000","last_peered":"2024-09-11T01:15:21.692287+0000","last_clean":"2024-09-11T01:15:21.692287+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692287+0000","last_undegraded":"2024-09-11T01:15:21.692287+0000","last_fullsized":"2024-09-11T01:15:21.692287+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9700","reported_seq":9828,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:15.105220+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:16:15.105220+0000","last_peered":"2024-09-11T01:16:15.105220+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:16:15.105220+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":9700,"log_dups_size":0,"ondisk_log_size":9700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39698432,"num_objects":9692,"num_object_clones":0,"num_object_copies":19384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6287,"num_write_kb":25148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5708},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9814","reported_seq":9932,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:13.750228+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:16:13.750228+0000","last_peered":"2024-09-11T01:16:13.750228+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:16:13.750228+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":9814,"log_dups_size":0,"ondisk_log_size":9814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40177664,"num_objects":9809,"num_object_clones":0,"num_object_copies":19618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9809,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7631,"num_write_kb":30524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":5785}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9657","reported_seq":17637,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:13.885559+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:16:13.885559+0000","last_peered":"2024-09-11T01:16:13.885559+0000","last_clean":"2024-09-11T01:16:13.885559+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:16:13.885559+0000","last_undegraded":"2024-09-11T01:16:13.885559+0000","last_fullsized":"2024-09-11T01:16:13.885559+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":9657,"log_dups_size":0,"ondisk_log_size":9657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39530496,"num_objects":9651,"num_object_clones":0,"num_object_copies":19302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7442,"num_write_kb":29768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9879","reported_seq":10010,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:15.061916+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:16:15.061916+0000","last_peered":"2024-09-11T01:16:15.061916+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:16:15.061916+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":9879,"log_dups_size":0,"ondisk_log_size":9879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40456192,"num_objects":9877,"num_object_clones":0,"num_object_copies":19754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9877,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6616,"num_write_kb":26464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5927},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9678","reported_seq":11901,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:15.111025+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:16:15.111025+0000","last_peered":"2024-09-11T01:16:15.111025+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:16:15.111025+0000","last_undegraded":"2024-09-11T01:16:15.111025+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":9678,"log_dups_size":0,"ondisk_log_size":9678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39608320,"num_objects":9670,"num_object_clones":0,"num_object_copies":19340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2896,"num_objects_unfound":0,"num_objects_dirty":9670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6333,"num_write_kb":25332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1043,"num_bytes_recovered":4272128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6774},{"shards":"0,2","objects":2896}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694344+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694344+0000","last_peered":"2024-09-11T01:15:21.694344+0000","last_clean":"2024-09-11T01:15:21.694344+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694344+0000","last_undegraded":"2024-09-11T01:15:21.694344+0000","last_fullsized":"2024-09-11T01:15:21.694344+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9816","reported_seq":9950,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:15.067865+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:16:15.067865+0000","last_peered":"2024-09-11T01:16:15.067865+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:16:15.067865+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":9816,"log_dups_size":0,"ondisk_log_size":9816,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40189952,"num_objects":9812,"num_object_clones":0,"num_object_copies":19624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9812,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6425,"num_write_kb":25700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5773},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9739","reported_seq":9873,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:15.117342+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:16:15.117342+0000","last_peered":"2024-09-11T01:16:15.117342+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:16:15.117342+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":9739,"log_dups_size":0,"ondisk_log_size":9739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39842409,"num_objects":9728,"num_object_clones":0,"num_object_copies":19456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6809,"num_write_kb":27236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5811},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":641102441,"num_objects":156520,"num_object_clones":0,"num_object_copies":313040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2896,"num_objects_unfound":0,"num_objects_dirty":156520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":117451,"num_write_kb":469804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4971,"num_bytes_recovered":20353024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1300901888,"data_stored":1300821871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156601,"ondisk_log_size":156601,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182457,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":853772,"kb_used_data":533372,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93518068,"statfs":{"total":96636764160,"available":95762501632,"internally_reserved":0,"allocated":546172928,"data_stored":545913714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":328068779},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,0,2,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986229,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":715072,"kb_used_data":440496,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93656768,"statfs":{"total":96636764160,"available":95904530432,"internally_reserved":0,"allocated":451067904,"data_stored":450812767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,1,2,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659174,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":534256,"kb_used_data":295776,"kb_used_omap":4,"kb_used_meta":238459,"kb_avail":93837584,"statfs":{"total":96636764160,"available":96089686016,"internally_reserved":0,"allocated":302874624,"data_stored":302545830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":244182703},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":546328576,"data_stored":546325097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":303595520,"data_stored":303522461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":450977792,"data_stored":450974313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:17.524 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:18.334 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:16:18.335 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:16:18.392 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":195,"stamp":"2024-09-11T01:16:16.639167+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":641102460,"num_objects":156522,"num_object_clones":0,"num_object_copies":313044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2896,"num_objects_unfound":0,"num_objects_dirty":156522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":117453,"num_write_kb":469806,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4972,"num_bytes_recovered":20353043,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156604,"ondisk_log_size":156604,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2103100,"kb_used_data":1269644,"kb_used_omap":12,"kb_used_meta":833395,"kb_avail":281012420,"statfs":{"total":289910292480,"available":287756718080,"internally_reserved":0,"allocated":1300115456,"data_stored":1299272311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":853396484},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,1,2,4,5,10,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":23662592,"num_objects":5777,"num_object_clones":0,"num_object_copies":11554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-298,"num_objects_unfound":0,"num_objects_dirty":5777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5777,"num_write_kb":23108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":298,"num_bytes_recovered":1220608,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002779"},"pg_stats":[{"pgid":"2.f","version":"45'9863","reported_seq":9940,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:14.988418+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:16:14.988418+0000","last_peered":"2024-09-11T01:16:14.988418+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:16:14.988418+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":9863,"log_dups_size":0,"ondisk_log_size":9863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40398848,"num_objects":9863,"num_object_clones":0,"num_object_copies":19726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9863,"num_write_kb":39452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5791},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9909","reported_seq":10010,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:13.914030+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:16:13.914030+0000","last_peered":"2024-09-11T01:16:13.914030+0000","last_clean":"2024-09-11T01:16:13.914030+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:16:13.914030+0000","last_undegraded":"2024-09-11T01:16:13.914030+0000","last_fullsized":"2024-09-11T01:16:13.914030+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":9909,"log_dups_size":0,"ondisk_log_size":9909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40562688,"num_objects":9903,"num_object_clones":0,"num_object_copies":19806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8148,"num_write_kb":32592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9807","reported_seq":9934,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:13.951470+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:16:13.951470+0000","last_peered":"2024-09-11T01:16:13.951470+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:16:13.951470+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":9807,"log_dups_size":0,"ondisk_log_size":9807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40153088,"num_objects":9803,"num_object_clones":0,"num_object_copies":19606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8108,"num_write_kb":32432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":5907}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9814","reported_seq":9883,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:13.940879+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:16:13.940879+0000","last_peered":"2024-09-11T01:16:13.940879+0000","last_clean":"2024-09-11T01:16:13.940879+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:16:13.940879+0000","last_undegraded":"2024-09-11T01:16:13.940879+0000","last_fullsized":"2024-09-11T01:16:13.940879+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":9814,"log_dups_size":0,"ondisk_log_size":9814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40198144,"num_objects":9814,"num_object_clones":0,"num_object_copies":19628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9814,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9814,"num_write_kb":39256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9776","reported_seq":9903,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:15.058365+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:16:15.058365+0000","last_peered":"2024-09-11T01:16:15.058365+0000","last_clean":"2024-09-11T01:16:15.058365+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:16:15.058365+0000","last_undegraded":"2024-09-11T01:16:15.058365+0000","last_fullsized":"2024-09-11T01:16:15.058365+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":9776,"log_dups_size":0,"ondisk_log_size":9776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40013824,"num_objects":9769,"num_object_clones":0,"num_object_copies":19538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6781,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9857","reported_seq":10027,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:15.088581+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:16:15.088581+0000","last_peered":"2024-09-11T01:16:15.088581+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:16:15.088581+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":9857,"log_dups_size":0,"ondisk_log_size":9857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40353792,"num_objects":9852,"num_object_clones":0,"num_object_copies":19704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6198,"num_write_kb":24792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5901},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9700","reported_seq":9830,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:14.933665+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:16:14.933665+0000","last_peered":"2024-09-11T01:16:14.933665+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:16:14.933665+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":9700,"log_dups_size":0,"ondisk_log_size":9700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39706624,"num_objects":9694,"num_object_clones":0,"num_object_copies":19388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6557,"num_write_kb":26228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5708},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9867","reported_seq":9994,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:15.067323+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:16:15.067323+0000","last_peered":"2024-09-11T01:16:15.067323+0000","last_clean":"2024-09-11T01:16:15.067323+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:16:15.067323+0000","last_undegraded":"2024-09-11T01:16:15.067323+0000","last_fullsized":"2024-09-11T01:16:15.067323+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":9867,"log_dups_size":0,"ondisk_log_size":9867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40407040,"num_objects":9865,"num_object_clones":0,"num_object_copies":19730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6743,"num_write_kb":26972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9725","reported_seq":9852,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:13.813981+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:16:13.813981+0000","last_peered":"2024-09-11T01:16:13.813981+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:16:13.813981+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":9725,"log_dups_size":0,"ondisk_log_size":9725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39804928,"num_objects":9718,"num_object_clones":0,"num_object_copies":19436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7696,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":5785}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692287+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692287+0000","last_peered":"2024-09-11T01:15:21.692287+0000","last_clean":"2024-09-11T01:15:21.692287+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692287+0000","last_undegraded":"2024-09-11T01:15:21.692287+0000","last_fullsized":"2024-09-11T01:15:21.692287+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9700","reported_seq":9828,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:15.105220+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:16:15.105220+0000","last_peered":"2024-09-11T01:16:15.105220+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:16:15.105220+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":9700,"log_dups_size":0,"ondisk_log_size":9700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39698432,"num_objects":9692,"num_object_clones":0,"num_object_copies":19384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6287,"num_write_kb":25148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5708},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9814","reported_seq":9932,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:13.750228+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:16:13.750228+0000","last_peered":"2024-09-11T01:16:13.750228+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:16:13.750228+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":9814,"log_dups_size":0,"ondisk_log_size":9814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40177664,"num_objects":9809,"num_object_clones":0,"num_object_copies":19618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9809,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7631,"num_write_kb":30524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":5785}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9657","reported_seq":17637,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:13.885559+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:16:13.885559+0000","last_peered":"2024-09-11T01:16:13.885559+0000","last_clean":"2024-09-11T01:16:13.885559+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:16:13.885559+0000","last_undegraded":"2024-09-11T01:16:13.885559+0000","last_fullsized":"2024-09-11T01:16:13.885559+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":9657,"log_dups_size":0,"ondisk_log_size":9657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39530496,"num_objects":9651,"num_object_clones":0,"num_object_copies":19302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7442,"num_write_kb":29768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9879","reported_seq":10010,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:15.061916+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:16:15.061916+0000","last_peered":"2024-09-11T01:16:15.061916+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:16:15.061916+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":9879,"log_dups_size":0,"ondisk_log_size":9879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40456192,"num_objects":9877,"num_object_clones":0,"num_object_copies":19754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9877,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6616,"num_write_kb":26464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5927},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9678","reported_seq":11901,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:15.111025+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:16:15.111025+0000","last_peered":"2024-09-11T01:16:15.111025+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:16:15.111025+0000","last_undegraded":"2024-09-11T01:16:15.111025+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":9678,"log_dups_size":0,"ondisk_log_size":9678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39608320,"num_objects":9670,"num_object_clones":0,"num_object_copies":19340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2896,"num_objects_unfound":0,"num_objects_dirty":9670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6333,"num_write_kb":25332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1043,"num_bytes_recovered":4272128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6774},{"shards":"0,2","objects":2896}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694344+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694344+0000","last_peered":"2024-09-11T01:15:21.694344+0000","last_clean":"2024-09-11T01:15:21.694344+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694344+0000","last_undegraded":"2024-09-11T01:15:21.694344+0000","last_fullsized":"2024-09-11T01:15:21.694344+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9816","reported_seq":9950,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:15.067865+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:16:15.067865+0000","last_peered":"2024-09-11T01:16:15.067865+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:16:15.067865+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":9816,"log_dups_size":0,"ondisk_log_size":9816,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40189952,"num_objects":9812,"num_object_clones":0,"num_object_copies":19624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9812,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6425,"num_write_kb":25700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5773},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9739","reported_seq":9873,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:15.117342+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:16:15.117342+0000","last_peered":"2024-09-11T01:16:15.117342+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:16:15.117342+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":9739,"log_dups_size":0,"ondisk_log_size":9739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39842409,"num_objects":9728,"num_object_clones":0,"num_object_copies":19456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6809,"num_write_kb":27236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5811},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":641102441,"num_objects":156520,"num_object_clones":0,"num_object_copies":313040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2896,"num_objects_unfound":0,"num_objects_dirty":156520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":117451,"num_write_kb":469804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4971,"num_bytes_recovered":20353024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1300901888,"data_stored":1300821871,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156601,"ondisk_log_size":156601,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182457,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":853772,"kb_used_data":533372,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93518068,"statfs":{"total":96636764160,"available":95762501632,"internally_reserved":0,"allocated":546172928,"data_stored":545913714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":328068779},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,0,2,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986229,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":715072,"kb_used_data":440496,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93656768,"statfs":{"total":96636764160,"available":95904530432,"internally_reserved":0,"allocated":451067904,"data_stored":450812767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,1,2,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659174,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":534256,"kb_used_data":295776,"kb_used_omap":4,"kb_used_meta":238459,"kb_avail":93837584,"statfs":{"total":96636764160,"available":96089686016,"internally_reserved":0,"allocated":302874624,"data_stored":302545830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":244182703},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":546328576,"data_stored":546325097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":303595520,"data_stored":303522461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":450977792,"data_stored":450974313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:18.394 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:16:18.394 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'9863", 'reported_seq': 9940, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:14.988418+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:16:14.988418+0000', 'last_peered': '2024-09-11T01:16:14.988418+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:16:14.988418+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 9863, 'log_dups_size': 0, 'ondisk_log_size': 9863, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40398848, 'num_objects': 9863, 'num_object_clones': 0, 'num_object_copies': 19726, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9863, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9863, 'num_write_kb': 39452, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5791}, {'shards': '0,2', 'objects': 4072}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:18.394 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:16:18.394 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'9807", 'reported_seq': 9934, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:13.951470+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:16:13.951470+0000', 'last_peered': '2024-09-11T01:16:13.951470+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:16:13.951470+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 9807, 'log_dups_size': 0, 'ondisk_log_size': 9807, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40153088, 'num_objects': 9803, 'num_object_clones': 0, 'num_object_copies': 19606, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9803, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8108, 'num_write_kb': 32432, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3896}, {'shards': '2', 'objects': 5907}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:18.394 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:16:18.394 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'9857", 'reported_seq': 10027, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:15.088581+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:16:15.088581+0000', 'last_peered': '2024-09-11T01:16:15.088581+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:16:15.088581+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 9857, 'log_dups_size': 0, 'ondisk_log_size': 9857, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40353792, 'num_objects': 9852, 'num_object_clones': 0, 'num_object_copies': 19704, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9852, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6198, 'num_write_kb': 24792, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5901}, {'shards': '0,2', 'objects': 3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:18.394 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:16:18.395 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'9700", 'reported_seq': 9830, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:14.933665+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:16:14.933665+0000', 'last_peered': '2024-09-11T01:16:14.933665+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:16:14.933665+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 9700, 'log_dups_size': 0, 'ondisk_log_size': 9700, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39706624, 'num_objects': 9694, 'num_object_clones': 0, 'num_object_copies': 19388, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9694, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6557, 'num_write_kb': 26228, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5708}, {'shards': '0,2', 'objects': 3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:18.395 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:16:18.395 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'9725", 'reported_seq': 9852, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:13.813981+0000', 'last_change': '2024-09-11T01:13:10.159069+0000', 'last_active': '2024-09-11T01:16:13.813981+0000', 'last_peered': '2024-09-11T01:16:13.813981+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:16:13.813981+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 9725, 'log_dups_size': 0, 'ondisk_log_size': 9725, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39804928, 'num_objects': 9718, 'num_object_clones': 0, 'num_object_copies': 19436, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9718, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7696, 'num_write_kb': 30784, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3932}, {'shards': '2', 'objects': 5785}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:18.395 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:16:18.395 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'9700", 'reported_seq': 9828, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:15.105220+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:16:15.105220+0000', 'last_peered': '2024-09-11T01:16:15.105220+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:16:15.105220+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 9700, 'log_dups_size': 0, 'ondisk_log_size': 9700, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39698432, 'num_objects': 9692, 'num_object_clones': 0, 'num_object_copies': 19384, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9692, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6287, 'num_write_kb': 25148, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5708}, {'shards': '0,2', 'objects': 3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:18.395 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:16:18.395 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'9814", 'reported_seq': 9932, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:13.750228+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:16:13.750228+0000', 'last_peered': '2024-09-11T01:16:13.750228+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:16:13.750228+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 9814, 'log_dups_size': 0, 'ondisk_log_size': 9814, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40177664, 'num_objects': 9809, 'num_object_clones': 0, 'num_object_copies': 19618, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9809, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7631, 'num_write_kb': 30524, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4023}, {'shards': '2', 'objects': 5785}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:18.395 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:16:18.395 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'9879", 'reported_seq': 10010, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:15.061916+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:16:15.061916+0000', 'last_peered': '2024-09-11T01:16:15.061916+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:16:15.061916+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 9879, 'log_dups_size': 0, 'ondisk_log_size': 9879, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40456192, 'num_objects': 9877, 'num_object_clones': 0, 'num_object_copies': 19754, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9877, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6616, 'num_write_kb': 26464, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5927}, {'shards': '0,2', 'objects': 3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:18.395 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:16:18.395 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'9678", 'reported_seq': 11901, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:16:15.111025+0000', 'last_change': '2024-09-11T01:15:19.913288+0000', 'last_active': '2024-09-11T01:16:15.111025+0000', 'last_peered': '2024-09-11T01:16:15.111025+0000', 'last_clean': '2024-09-11T01:13:07.624565+0000', 'last_became_active': '2024-09-11T01:13:10.199795+0000', 'last_became_peered': '2024-09-11T01:13:10.199795+0000', 'last_unstale': '2024-09-11T01:16:15.111025+0000', 'last_undegraded': '2024-09-11T01:16:15.111025+0000', 'last_fullsized': '2024-09-11T01:13:09.652311+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3415", 'last_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'objects_scrubbed': 3337, 'log_size': 9678, 'log_dups_size': 0, 'ondisk_log_size': 9678, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1975, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39608320, 'num_objects': 9670, 'num_object_clones': 0, 'num_object_copies': 19340, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2896, 'num_objects_unfound': 0, 'num_objects_dirty': 9670, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6333, 'num_write_kb': 25332, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1043, 'num_bytes_recovered': 4272128, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6774}, {'shards': '0,2', 'objects': 2896}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:18.396 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:16:18.396 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'9816", 'reported_seq': 9950, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:15.067865+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:16:15.067865+0000', 'last_peered': '2024-09-11T01:16:15.067865+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:16:15.067865+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 9816, 'log_dups_size': 0, 'ondisk_log_size': 9816, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40189952, 'num_objects': 9812, 'num_object_clones': 0, 'num_object_copies': 19624, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9812, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6425, 'num_write_kb': 25700, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5773}, {'shards': '0,2', 'objects': 4039}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:18.396 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:16:18.396 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'9739", 'reported_seq': 9873, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:15.117342+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:16:15.117342+0000', 'last_peered': '2024-09-11T01:16:15.117342+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:16:15.117342+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 9739, 'log_dups_size': 0, 'ondisk_log_size': 9739, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39842409, 'num_objects': 9728, 'num_object_clones': 0, 'num_object_copies': 19456, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9728, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6809, 'num_write_kb': 27236, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5811}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:18.396 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:16:18.443 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 234 20 158138 158118 2.63836 1.62109 0.0717823 0.0296041 2024-09-11T01:16:19.448 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 235 20 158558 158538 2.63408 1.64062 0.013069 0.0296514 2024-09-11T01:16:19.448 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:16:19.449 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39769/313044 objects degraded (12.704%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":315,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":156522,"data_bytes":641102460,"bytes_used":2152304640,"bytes_avail":287757987840,"bytes_total":289910292480,"degraded_objects":39769,"degraded_total":313044,"degraded_ratio":0.12703964937836215,"misplaced_objects":2896,"misplaced_total":313044,"misplaced_ratio":0.00925109569261829,"recovering_objects_per_sec":16,"recovering_bytes_per_sec":67568,"recovering_keys_per_sec":0,"num_objects_recovered":198,"num_bytes_recovered":811008,"num_keys_recovered":0,"write_bytes_sec":1494355,"read_op_per_sec":0,"write_op_per_sec":364},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:16:19.536 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:16:19.537 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:20.447 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 236 20 158988 158968 2.63003 1.67969 0.0217348 0.0296964 2024-09-11T01:16:20.555 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:16:20.555 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:16:20.632 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":196,"stamp":"2024-09-11T01:16:18.639630+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":641102460,"num_objects":156522,"num_object_clones":0,"num_object_copies":313044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2896,"num_objects_unfound":0,"num_objects_dirty":156522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":117453,"num_write_kb":469806,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4972,"num_bytes_recovered":20353043,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156604,"ondisk_log_size":156604,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2101860,"kb_used_data":1276724,"kb_used_omap":12,"kb_used_meta":825075,"kb_avail":281013660,"statfs":{"total":289910292480,"available":287757987840,"internally_reserved":0,"allocated":1307365376,"data_stored":1306522231,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":844876804},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,3,6,4,8,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17936384,"num_objects":4379,"num_object_clones":0,"num_object_copies":8758,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-198,"num_objects_unfound":0,"num_objects_dirty":4379,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4379,"num_write_kb":17516,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":198,"num_bytes_recovered":811008,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002759"},"pg_stats":[{"pgid":"2.f","version":"45'9863","reported_seq":9940,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:14.988418+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:16:14.988418+0000","last_peered":"2024-09-11T01:16:14.988418+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:16:14.988418+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":9863,"log_dups_size":0,"ondisk_log_size":9863,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40398848,"num_objects":9863,"num_object_clones":0,"num_object_copies":19726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9863,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9863,"num_write_kb":39452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5791},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'9909","reported_seq":10010,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:13.914030+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:16:13.914030+0000","last_peered":"2024-09-11T01:16:13.914030+0000","last_clean":"2024-09-11T01:16:13.914030+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:16:13.914030+0000","last_undegraded":"2024-09-11T01:16:13.914030+0000","last_fullsized":"2024-09-11T01:16:13.914030+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":9909,"log_dups_size":0,"ondisk_log_size":9909,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40562688,"num_objects":9903,"num_object_clones":0,"num_object_copies":19806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8148,"num_write_kb":32592,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9807","reported_seq":9934,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:13.951470+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:16:13.951470+0000","last_peered":"2024-09-11T01:16:13.951470+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:16:13.951470+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":9807,"log_dups_size":0,"ondisk_log_size":9807,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40153088,"num_objects":9803,"num_object_clones":0,"num_object_copies":19606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8108,"num_write_kb":32432,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":5907}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9814","reported_seq":9883,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:13.940879+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:16:13.940879+0000","last_peered":"2024-09-11T01:16:13.940879+0000","last_clean":"2024-09-11T01:16:13.940879+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:16:13.940879+0000","last_undegraded":"2024-09-11T01:16:13.940879+0000","last_fullsized":"2024-09-11T01:16:13.940879+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":9814,"log_dups_size":0,"ondisk_log_size":9814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40198144,"num_objects":9814,"num_object_clones":0,"num_object_copies":19628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9814,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9814,"num_write_kb":39256,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9776","reported_seq":9903,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:15.058365+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:16:15.058365+0000","last_peered":"2024-09-11T01:16:15.058365+0000","last_clean":"2024-09-11T01:16:15.058365+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:16:15.058365+0000","last_undegraded":"2024-09-11T01:16:15.058365+0000","last_fullsized":"2024-09-11T01:16:15.058365+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":9776,"log_dups_size":0,"ondisk_log_size":9776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40013824,"num_objects":9769,"num_object_clones":0,"num_object_copies":19538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6781,"num_write_kb":27124,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'9857","reported_seq":10027,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:15.088581+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:16:15.088581+0000","last_peered":"2024-09-11T01:16:15.088581+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:16:15.088581+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":9857,"log_dups_size":0,"ondisk_log_size":9857,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40353792,"num_objects":9852,"num_object_clones":0,"num_object_copies":19704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9852,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6198,"num_write_kb":24792,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5901},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9700","reported_seq":9830,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:14.933665+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:16:14.933665+0000","last_peered":"2024-09-11T01:16:14.933665+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:16:14.933665+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":9700,"log_dups_size":0,"ondisk_log_size":9700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39706624,"num_objects":9694,"num_object_clones":0,"num_object_copies":19388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9694,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6557,"num_write_kb":26228,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5708},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9867","reported_seq":9994,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:15.067323+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:16:15.067323+0000","last_peered":"2024-09-11T01:16:15.067323+0000","last_clean":"2024-09-11T01:16:15.067323+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:16:15.067323+0000","last_undegraded":"2024-09-11T01:16:15.067323+0000","last_fullsized":"2024-09-11T01:16:15.067323+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":9867,"log_dups_size":0,"ondisk_log_size":9867,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40407040,"num_objects":9865,"num_object_clones":0,"num_object_copies":19730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6743,"num_write_kb":26972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9725","reported_seq":9852,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:13.813981+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:16:13.813981+0000","last_peered":"2024-09-11T01:16:13.813981+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:16:13.813981+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":9725,"log_dups_size":0,"ondisk_log_size":9725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39804928,"num_objects":9718,"num_object_clones":0,"num_object_copies":19436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7696,"num_write_kb":30784,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":5785}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692287+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692287+0000","last_peered":"2024-09-11T01:15:21.692287+0000","last_clean":"2024-09-11T01:15:21.692287+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692287+0000","last_undegraded":"2024-09-11T01:15:21.692287+0000","last_fullsized":"2024-09-11T01:15:21.692287+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9700","reported_seq":9828,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:15.105220+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:16:15.105220+0000","last_peered":"2024-09-11T01:16:15.105220+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:16:15.105220+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":9700,"log_dups_size":0,"ondisk_log_size":9700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39698432,"num_objects":9692,"num_object_clones":0,"num_object_copies":19384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6287,"num_write_kb":25148,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5708},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9814","reported_seq":9932,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:13.750228+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:16:13.750228+0000","last_peered":"2024-09-11T01:16:13.750228+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:16:13.750228+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":9814,"log_dups_size":0,"ondisk_log_size":9814,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40177664,"num_objects":9809,"num_object_clones":0,"num_object_copies":19618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9809,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7631,"num_write_kb":30524,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":5785}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9657","reported_seq":17637,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:13.885559+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:16:13.885559+0000","last_peered":"2024-09-11T01:16:13.885559+0000","last_clean":"2024-09-11T01:16:13.885559+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:16:13.885559+0000","last_undegraded":"2024-09-11T01:16:13.885559+0000","last_fullsized":"2024-09-11T01:16:13.885559+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":9657,"log_dups_size":0,"ondisk_log_size":9657,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39530496,"num_objects":9651,"num_object_clones":0,"num_object_copies":19302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7442,"num_write_kb":29768,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'9879","reported_seq":10010,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:15.061916+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:16:15.061916+0000","last_peered":"2024-09-11T01:16:15.061916+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:16:15.061916+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":9879,"log_dups_size":0,"ondisk_log_size":9879,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40456192,"num_objects":9877,"num_object_clones":0,"num_object_copies":19754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9877,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6616,"num_write_kb":26464,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5927},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9678","reported_seq":11901,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:15.111025+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:16:15.111025+0000","last_peered":"2024-09-11T01:16:15.111025+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:16:15.111025+0000","last_undegraded":"2024-09-11T01:16:15.111025+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":9678,"log_dups_size":0,"ondisk_log_size":9678,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39608320,"num_objects":9670,"num_object_clones":0,"num_object_copies":19340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2896,"num_objects_unfound":0,"num_objects_dirty":9670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6333,"num_write_kb":25332,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1043,"num_bytes_recovered":4272128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6774},{"shards":"0,2","objects":2896}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694344+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694344+0000","last_peered":"2024-09-11T01:15:21.694344+0000","last_clean":"2024-09-11T01:15:21.694344+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694344+0000","last_undegraded":"2024-09-11T01:15:21.694344+0000","last_fullsized":"2024-09-11T01:15:21.694344+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9816","reported_seq":9950,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:15.067865+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:16:15.067865+0000","last_peered":"2024-09-11T01:16:15.067865+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:16:15.067865+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":9816,"log_dups_size":0,"ondisk_log_size":9816,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40189952,"num_objects":9812,"num_object_clones":0,"num_object_copies":19624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9812,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6425,"num_write_kb":25700,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5773},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9739","reported_seq":9873,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:15.117342+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:16:15.117342+0000","last_peered":"2024-09-11T01:16:15.117342+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:16:15.117342+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":9739,"log_dups_size":0,"ondisk_log_size":9739,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39842409,"num_objects":9728,"num_object_clones":0,"num_object_copies":19456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9728,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6809,"num_write_kb":27236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5811},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":641102441,"num_objects":156520,"num_object_clones":0,"num_object_copies":313040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2896,"num_objects_unfound":0,"num_objects_dirty":156520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":117451,"num_write_kb":469804,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4971,"num_bytes_recovered":20353024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1308106752,"data_stored":1308026735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156601,"ondisk_log_size":156601,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182457,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":853772,"kb_used_data":533372,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93518068,"statfs":{"total":96636764160,"available":95762501632,"internally_reserved":0,"allocated":546172928,"data_stored":545913714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":328068779},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,0,2,6,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986229,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":715072,"kb_used_data":440496,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93656768,"statfs":{"total":96636764160,"available":95904530432,"internally_reserved":0,"allocated":451067904,"data_stored":450812767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,1,2,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659175,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":533016,"kb_used_data":302856,"kb_used_omap":4,"kb_used_meta":230139,"kb_avail":93838824,"statfs":{"total":96636764160,"available":96090955776,"internally_reserved":0,"allocated":310124544,"data_stored":309795750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":235663023},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":546328576,"data_stored":546325097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":310800384,"data_stored":310727325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":450977792,"data_stored":450974313,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:21.447 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 237 20 159421 159401 2.62607 1.69141 0.0261374 0.0297421 2024-09-11T01:16:22.447 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 238 20 159862 159842 2.62228 1.72266 0.0230884 0.0297829 2024-09-11T01:16:23.455 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout: 239 20 160305 160285 2.61851 1.73047 0.0201169 0.0298285 2024-09-11T01:16:23.634 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:24.419 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:16:24.420 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:16:24.436 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":198,"stamp":"2024-09-11T01:16:22.640626+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650064508,"num_objects":158710,"num_object_clones":0,"num_object_copies":317420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2795,"num_objects_unfound":0,"num_objects_dirty":158710,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119641,"num_write_kb":478558,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5073,"num_bytes_recovered":20766739,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158792,"ondisk_log_size":158792,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2104056,"kb_used_data":1287240,"kb_used_omap":12,"kb_used_meta":816755,"kb_avail":281011464,"statfs":{"total":289910292480,"available":287755739136,"internally_reserved":0,"allocated":1318133760,"data_stored":1317290615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":836357124},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,4,4,7,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17915904,"num_objects":4374,"num_object_clones":0,"num_object_copies":8748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-200,"num_objects_unfound":0,"num_objects_dirty":4374,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4374,"num_write_kb":17496,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":200,"num_bytes_recovered":819200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002852"},"pg_stats":[{"pgid":"2.f","version":"45'10017","reported_seq":10094,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.230418+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:16:20.230418+0000","last_peered":"2024-09-11T01:16:20.230418+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:16:20.230418+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41029632,"num_objects":10017,"num_object_clones":0,"num_object_copies":20034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10017,"num_write_kb":40068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5945},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10047","reported_seq":10148,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:18.854513+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:16:18.854513+0000","last_peered":"2024-09-11T01:16:18.854513+0000","last_clean":"2024-09-11T01:16:18.854513+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:16:18.854513+0000","last_undegraded":"2024-09-11T01:16:18.854513+0000","last_fullsized":"2024-09-11T01:16:18.854513+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41127936,"num_objects":10041,"num_object_clones":0,"num_object_copies":20082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8286,"num_write_kb":33144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'9929","reported_seq":10056,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:18.840868+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:16:18.840868+0000","last_peered":"2024-09-11T01:16:18.840868+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:16:18.840868+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":9929,"log_dups_size":0,"ondisk_log_size":9929,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40652800,"num_objects":9925,"num_object_clones":0,"num_object_copies":19850,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9925,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8230,"num_write_kb":32920,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":6029}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'9940","reported_seq":10009,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:18.866858+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:16:18.866858+0000","last_peered":"2024-09-11T01:16:18.866858+0000","last_clean":"2024-09-11T01:16:18.866858+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:16:18.866858+0000","last_undegraded":"2024-09-11T01:16:18.866858+0000","last_fullsized":"2024-09-11T01:16:18.866858+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":9940,"log_dups_size":0,"ondisk_log_size":9940,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40714240,"num_objects":9940,"num_object_clones":0,"num_object_copies":19880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9940,"num_write_kb":39760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9891","reported_seq":10018,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:20.279677+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:16:20.279677+0000","last_peered":"2024-09-11T01:16:20.279677+0000","last_clean":"2024-09-11T01:16:20.279677+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:16:20.279677+0000","last_undegraded":"2024-09-11T01:16:20.279677+0000","last_fullsized":"2024-09-11T01:16:20.279677+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":9891,"log_dups_size":0,"ondisk_log_size":9891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40484864,"num_objects":9884,"num_object_clones":0,"num_object_copies":19768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6896,"num_write_kb":27584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10006","reported_seq":10176,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.217167+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:16:20.217167+0000","last_peered":"2024-09-11T01:16:20.217167+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:16:20.217167+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40964096,"num_objects":10001,"num_object_clones":0,"num_object_copies":20002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6347,"num_write_kb":25388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6050},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9850","reported_seq":9980,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.191482+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:16:20.191482+0000","last_peered":"2024-09-11T01:16:20.191482+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:16:20.191482+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":9850,"log_dups_size":0,"ondisk_log_size":9850,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40321024,"num_objects":9844,"num_object_clones":0,"num_object_copies":19688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6707,"num_write_kb":26828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5858},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9994","reported_seq":10124,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:20.039059+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:16:20.039059+0000","last_peered":"2024-09-11T01:16:20.039059+0000","last_clean":"2024-09-11T01:16:20.039059+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:16:20.039059+0000","last_undegraded":"2024-09-11T01:16:20.039059+0000","last_fullsized":"2024-09-11T01:16:20.039059+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":9994,"log_dups_size":0,"ondisk_log_size":9994,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40927232,"num_objects":9992,"num_object_clones":0,"num_object_copies":19984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9992,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6870,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9864","reported_seq":9991,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:18.961579+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:16:18.961579+0000","last_peered":"2024-09-11T01:16:18.961579+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:16:18.961579+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":9864,"log_dups_size":0,"ondisk_log_size":9864,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40374272,"num_objects":9857,"num_object_clones":0,"num_object_copies":19714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7835,"num_write_kb":31340,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":5924}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9828","reported_seq":9956,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.183350+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:16:20.183350+0000","last_peered":"2024-09-11T01:16:20.183350+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:16:20.183350+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":9828,"log_dups_size":0,"ondisk_log_size":9828,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40222720,"num_objects":9820,"num_object_clones":0,"num_object_copies":19640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9820,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6415,"num_write_kb":25660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5836},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'9963","reported_seq":10082,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:18.959360+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:16:18.959360+0000","last_peered":"2024-09-11T01:16:18.959360+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:16:18.959360+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":9963,"log_dups_size":0,"ondisk_log_size":9963,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40787968,"num_objects":9958,"num_object_clones":0,"num_object_copies":19916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7780,"num_write_kb":31120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":5934}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9776","reported_seq":17756,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:18.977201+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:16:18.977201+0000","last_peered":"2024-09-11T01:16:18.977201+0000","last_clean":"2024-09-11T01:16:18.977201+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:16:18.977201+0000","last_undegraded":"2024-09-11T01:16:18.977201+0000","last_fullsized":"2024-09-11T01:16:18.977201+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":9776,"log_dups_size":0,"ondisk_log_size":9776,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40017920,"num_objects":9770,"num_object_clones":0,"num_object_copies":19540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7561,"num_write_kb":30244,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10010","reported_seq":10141,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.258764+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:16:20.258764+0000","last_peered":"2024-09-11T01:16:20.258764+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:16:20.258764+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40992768,"num_objects":10008,"num_object_clones":0,"num_object_copies":20016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6747,"num_write_kb":26988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6058},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9831","reported_seq":12255,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:20.269866+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:16:20.269866+0000","last_peered":"2024-09-11T01:16:20.269866+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:16:20.269866+0000","last_undegraded":"2024-09-11T01:16:20.269866+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":9831,"log_dups_size":0,"ondisk_log_size":9831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40235008,"num_objects":9823,"num_object_clones":0,"num_object_copies":19646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2795,"num_objects_unfound":0,"num_objects_dirty":9823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6486,"num_write_kb":25944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1144,"num_bytes_recovered":4685824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7028},{"shards":"0,2","objects":2795}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9982","reported_seq":10117,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.259917+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:16:20.259917+0000","last_peered":"2024-09-11T01:16:20.259917+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:16:20.259917+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":9982,"log_dups_size":0,"ondisk_log_size":9982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40869888,"num_objects":9978,"num_object_clones":0,"num_object_copies":19956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6591,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5939},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9861","reported_seq":9995,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.225016+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:16:20.225016+0000","last_peered":"2024-09-11T01:16:20.225016+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:16:20.225016+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":9861,"log_dups_size":0,"ondisk_log_size":9861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40342121,"num_objects":9850,"num_object_clones":0,"num_object_copies":19700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6931,"num_write_kb":27724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5933},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":650064489,"num_objects":158708,"num_object_clones":0,"num_object_copies":317416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2795,"num_objects_unfound":0,"num_objects_dirty":158708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":119639,"num_write_kb":478556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5072,"num_bytes_recovered":20766720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1319284736,"data_stored":1319204719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158789,"ondisk_log_size":158789,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182458,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":864360,"kb_used_data":539800,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93507480,"statfs":{"total":96636764160,"available":95751659520,"internally_reserved":0,"allocated":552755200,"data_stored":552495986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":332328619},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,1,7,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986230,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":706680,"kb_used_data":444584,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93665160,"statfs":{"total":96636764160,"available":95913123840,"internally_reserved":0,"allocated":455254016,"data_stored":454998879,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268365482},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659175,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":533016,"kb_used_data":302856,"kb_used_omap":4,"kb_used_meta":230139,"kb_avail":93838824,"statfs":{"total":96636764160,"available":96090955776,"internally_reserved":0,"allocated":310124544,"data_stored":309795750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":235663023},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,1],"upper_bound":32},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":553242624,"data_stored":553239145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":310800384,"data_stored":310727325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":455241728,"data_stored":455238249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:24.437 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:24.451 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout:Total time run: 240.032 2024-09-11T01:16:24.451 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout:Total writes made: 160676 2024-09-11T01:16:24.451 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout:Write size: 4096 2024-09-11T01:16:24.451 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout:Object size: 4096 2024-09-11T01:16:24.451 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout:Bandwidth (MB/sec): 2.61483 2024-09-11T01:16:24.451 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout:Stddev Bandwidth: 1.78023 2024-09-11T01:16:24.451 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout:Max bandwidth (MB/sec): 11.0234 2024-09-11T01:16:24.451 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout:Min bandwidth (MB/sec): 0 2024-09-11T01:16:24.452 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout:Average IOPS: 669 2024-09-11T01:16:24.452 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout:Stddev IOPS: 455.741 2024-09-11T01:16:24.452 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout:Max IOPS: 2822 2024-09-11T01:16:24.452 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout:Min IOPS: 0 2024-09-11T01:16:24.452 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout:Average Latency(s): 0.0298721 2024-09-11T01:16:24.452 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout:Stddev Latency(s): 0.0451584 2024-09-11T01:16:24.452 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout:Max latency(s): 2.69211 2024-09-11T01:16:24.452 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout:Min latency(s): 0.00219431 2024-09-11T01:16:24.462 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout:Cleaning up (deleting benchmark objects) 2024-09-11T01:16:25.294 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:16:25.295 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:16:25.356 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":199,"stamp":"2024-09-11T01:16:24.641044+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":653128316,"num_objects":159458,"num_object_clones":0,"num_object_copies":318916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2795,"num_objects_unfound":0,"num_objects_dirty":159458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120389,"num_write_kb":481550,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5073,"num_bytes_recovered":20766739,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159440,"ondisk_log_size":159440,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2123284,"kb_used_data":1298148,"kb_used_omap":12,"kb_used_meta":825075,"kb_avail":280992236,"statfs":{"total":289910292480,"available":287736049664,"internally_reserved":0,"allocated":1329303552,"data_stored":1328460407,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":844876804},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,4,7,11],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20979712,"num_objects":5122,"num_object_clones":0,"num_object_copies":10244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-200,"num_objects_unfound":0,"num_objects_dirty":5122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5122,"num_write_kb":20488,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":200,"num_bytes_recovered":819200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002782"},"pg_stats":[{"pgid":"2.f","version":"45'10017","reported_seq":10094,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.230418+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:16:20.230418+0000","last_peered":"2024-09-11T01:16:20.230418+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:16:20.230418+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41029632,"num_objects":10017,"num_object_clones":0,"num_object_copies":20034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10017,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10017,"num_write_kb":40068,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5945},{"shards":"0,2","objects":4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10170","reported_seq":10271,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:23.956708+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:16:23.956708+0000","last_peered":"2024-09-11T01:16:23.956708+0000","last_clean":"2024-09-11T01:16:23.956708+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:16:23.956708+0000","last_undegraded":"2024-09-11T01:16:23.956708+0000","last_fullsized":"2024-09-11T01:16:23.956708+0000","mapping_epoch":35,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41631744,"num_objects":10164,"num_object_clones":0,"num_object_copies":20328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8409,"num_write_kb":33636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10063","reported_seq":10190,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:23.997936+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:16:23.997936+0000","last_peered":"2024-09-11T01:16:23.997936+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:16:23.997936+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41201664,"num_objects":10059,"num_object_clones":0,"num_object_copies":20118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10059,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8364,"num_write_kb":33456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":6163}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10063","reported_seq":10132,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:23.932512+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:16:23.932512+0000","last_peered":"2024-09-11T01:16:23.932512+0000","last_clean":"2024-09-11T01:16:23.932512+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:16:23.932512+0000","last_undegraded":"2024-09-11T01:16:23.932512+0000","last_fullsized":"2024-09-11T01:16:23.932512+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41218048,"num_objects":10063,"num_object_clones":0,"num_object_copies":20126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10063,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10063,"num_write_kb":40252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'9891","reported_seq":10018,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:20.279677+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:16:20.279677+0000","last_peered":"2024-09-11T01:16:20.279677+0000","last_clean":"2024-09-11T01:16:20.279677+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:16:20.279677+0000","last_undegraded":"2024-09-11T01:16:20.279677+0000","last_fullsized":"2024-09-11T01:16:20.279677+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":9891,"log_dups_size":0,"ondisk_log_size":9891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40484864,"num_objects":9884,"num_object_clones":0,"num_object_copies":19768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9884,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6896,"num_write_kb":27584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10006","reported_seq":10176,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.217167+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:16:20.217167+0000","last_peered":"2024-09-11T01:16:20.217167+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:16:20.217167+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40964096,"num_objects":10001,"num_object_clones":0,"num_object_copies":20002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6347,"num_write_kb":25388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6050},{"shards":"0,2","objects":3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9850","reported_seq":9980,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.191482+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:16:20.191482+0000","last_peered":"2024-09-11T01:16:20.191482+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:16:20.191482+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":9850,"log_dups_size":0,"ondisk_log_size":9850,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40321024,"num_objects":9844,"num_object_clones":0,"num_object_copies":19688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9844,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6707,"num_write_kb":26828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5858},{"shards":"0,2","objects":3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'9994","reported_seq":10124,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:20.039059+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:16:20.039059+0000","last_peered":"2024-09-11T01:16:20.039059+0000","last_clean":"2024-09-11T01:16:20.039059+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:16:20.039059+0000","last_undegraded":"2024-09-11T01:16:20.039059+0000","last_fullsized":"2024-09-11T01:16:20.039059+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":9994,"log_dups_size":0,"ondisk_log_size":9994,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40927232,"num_objects":9992,"num_object_clones":0,"num_object_copies":19984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9992,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6870,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9983","reported_seq":10110,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:23.941562+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:16:23.941562+0000","last_peered":"2024-09-11T01:16:23.941562+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:16:23.941562+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":9983,"log_dups_size":0,"ondisk_log_size":9983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40861696,"num_objects":9976,"num_object_clones":0,"num_object_copies":19952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7954,"num_write_kb":31816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":6043}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9828","reported_seq":9956,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.183350+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:16:20.183350+0000","last_peered":"2024-09-11T01:16:20.183350+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:16:20.183350+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":9828,"log_dups_size":0,"ondisk_log_size":9828,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40222720,"num_objects":9820,"num_object_clones":0,"num_object_copies":19640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9820,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6415,"num_write_kb":25660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5836},{"shards":"0,2","objects":3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10097","reported_seq":10216,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:23.987182+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:16:23.987182+0000","last_peered":"2024-09-11T01:16:23.987182+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:16:23.987182+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41336832,"num_objects":10092,"num_object_clones":0,"num_object_copies":20184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10092,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7914,"num_write_kb":31656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":6068}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9891","reported_seq":17872,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:23.881211+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:16:23.881211+0000","last_peered":"2024-09-11T01:16:23.881211+0000","last_clean":"2024-09-11T01:16:23.881211+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:16:23.881211+0000","last_undegraded":"2024-09-11T01:16:23.881211+0000","last_fullsized":"2024-09-11T01:16:23.881211+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":9891,"log_dups_size":0,"ondisk_log_size":9891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40488960,"num_objects":9885,"num_object_clones":0,"num_object_copies":19770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7676,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10010","reported_seq":10141,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.258764+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:16:20.258764+0000","last_peered":"2024-09-11T01:16:20.258764+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:16:20.258764+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40992768,"num_objects":10008,"num_object_clones":0,"num_object_copies":20016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10008,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6747,"num_write_kb":26988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6058},{"shards":"0,2","objects":3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9831","reported_seq":12255,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:20.269866+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:16:20.269866+0000","last_peered":"2024-09-11T01:16:20.269866+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:16:20.269866+0000","last_undegraded":"2024-09-11T01:16:20.269866+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":9831,"log_dups_size":0,"ondisk_log_size":9831,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40235008,"num_objects":9823,"num_object_clones":0,"num_object_copies":19646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2795,"num_objects_unfound":0,"num_objects_dirty":9823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6486,"num_write_kb":25944,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1144,"num_bytes_recovered":4685824,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7028},{"shards":"0,2","objects":2795}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'9982","reported_seq":10117,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.259917+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:16:20.259917+0000","last_peered":"2024-09-11T01:16:20.259917+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:16:20.259917+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":9982,"log_dups_size":0,"ondisk_log_size":9982,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40869888,"num_objects":9978,"num_object_clones":0,"num_object_copies":19956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6591,"num_write_kb":26364,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5939},{"shards":"0,2","objects":4039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'9861","reported_seq":9995,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:20.225016+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:16:20.225016+0000","last_peered":"2024-09-11T01:16:20.225016+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:16:20.225016+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":9861,"log_dups_size":0,"ondisk_log_size":9861,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40342121,"num_objects":9850,"num_object_clones":0,"num_object_copies":19700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6931,"num_write_kb":27724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5933},{"shards":"0,2","objects":3917}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":653128297,"num_objects":159456,"num_object_clones":0,"num_object_copies":318912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2795,"num_objects_unfound":0,"num_objects_dirty":159456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":120387,"num_write_kb":481548,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5072,"num_bytes_recovered":20766720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1330761728,"data_stored":1330681711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159437,"ondisk_log_size":159437,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182458,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":864360,"kb_used_data":539800,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93507480,"statfs":{"total":96636764160,"available":95751659520,"internally_reserved":0,"allocated":552755200,"data_stored":552495986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":332328619},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,1,7,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986231,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":714904,"kb_used_data":448648,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93656936,"statfs":{"total":96636764160,"available":95904702464,"internally_reserved":0,"allocated":459415552,"data_stored":459160415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,0,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659176,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":544020,"kb_used_data":309700,"kb_used_omap":4,"kb_used_meta":234299,"kb_avail":93827820,"statfs":{"total":96636764160,"available":96079687680,"internally_reserved":0,"allocated":317132800,"data_stored":316804006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":239922863},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":553242624,"data_stored":553239145,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":317997056,"data_stored":317923997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":459522048,"data_stored":459518569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:25.357 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:16:25.358 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'10017", 'reported_seq': 10094, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:20.230418+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:16:20.230418+0000', 'last_peered': '2024-09-11T01:16:20.230418+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:16:20.230418+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41029632, 'num_objects': 10017, 'num_object_clones': 0, 'num_object_copies': 20034, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10017, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10017, 'num_write_kb': 40068, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5945}, {'shards': '0,2', 'objects': 4072}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:25.358 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:16:25.358 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'10063", 'reported_seq': 10190, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:23.997936+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:16:23.997936+0000', 'last_peered': '2024-09-11T01:16:23.997936+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:16:23.997936+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41201664, 'num_objects': 10059, 'num_object_clones': 0, 'num_object_copies': 20118, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10059, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8364, 'num_write_kb': 33456, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3896}, {'shards': '2', 'objects': 6163}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:25.358 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:16:25.358 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'10006", 'reported_seq': 10176, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:20.217167+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:16:20.217167+0000', 'last_peered': '2024-09-11T01:16:20.217167+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:16:20.217167+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40964096, 'num_objects': 10001, 'num_object_clones': 0, 'num_object_copies': 20002, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10001, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6347, 'num_write_kb': 25388, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6050}, {'shards': '0,2', 'objects': 3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:25.358 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:16:25.358 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'9850", 'reported_seq': 9980, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:20.191482+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:16:20.191482+0000', 'last_peered': '2024-09-11T01:16:20.191482+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:16:20.191482+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 9850, 'log_dups_size': 0, 'ondisk_log_size': 9850, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40321024, 'num_objects': 9844, 'num_object_clones': 0, 'num_object_copies': 19688, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9844, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6707, 'num_write_kb': 26828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5858}, {'shards': '0,2', 'objects': 3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:25.358 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:16:25.358 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'9983", 'reported_seq': 10110, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:23.941562+0000', 'last_change': '2024-09-11T01:13:10.159069+0000', 'last_active': '2024-09-11T01:16:23.941562+0000', 'last_peered': '2024-09-11T01:16:23.941562+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:16:23.941562+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 9983, 'log_dups_size': 0, 'ondisk_log_size': 9983, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40861696, 'num_objects': 9976, 'num_object_clones': 0, 'num_object_copies': 19952, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9976, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7954, 'num_write_kb': 31816, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3932}, {'shards': '2', 'objects': 6043}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:25.358 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:16:25.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'9828", 'reported_seq': 9956, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:20.183350+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:16:20.183350+0000', 'last_peered': '2024-09-11T01:16:20.183350+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:16:20.183350+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 9828, 'log_dups_size': 0, 'ondisk_log_size': 9828, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40222720, 'num_objects': 9820, 'num_object_clones': 0, 'num_object_copies': 19640, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9820, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6415, 'num_write_kb': 25660, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5836}, {'shards': '0,2', 'objects': 3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:25.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:16:25.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'10097", 'reported_seq': 10216, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:23.987182+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:16:23.987182+0000', 'last_peered': '2024-09-11T01:16:23.987182+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:16:23.987182+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 41336832, 'num_objects': 10092, 'num_object_clones': 0, 'num_object_copies': 20184, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10092, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7914, 'num_write_kb': 31656, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 4023}, {'shards': '2', 'objects': 6068}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:25.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:16:25.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'10010", 'reported_seq': 10141, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:20.258764+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:16:20.258764+0000', 'last_peered': '2024-09-11T01:16:20.258764+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:16:20.258764+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40992768, 'num_objects': 10008, 'num_object_clones': 0, 'num_object_copies': 20016, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 10008, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6747, 'num_write_kb': 26988, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6058}, {'shards': '0,2', 'objects': 3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:25.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:16:25.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'9831", 'reported_seq': 12255, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:16:20.269866+0000', 'last_change': '2024-09-11T01:15:19.913288+0000', 'last_active': '2024-09-11T01:16:20.269866+0000', 'last_peered': '2024-09-11T01:16:20.269866+0000', 'last_clean': '2024-09-11T01:13:07.624565+0000', 'last_became_active': '2024-09-11T01:13:10.199795+0000', 'last_became_peered': '2024-09-11T01:13:10.199795+0000', 'last_unstale': '2024-09-11T01:16:20.269866+0000', 'last_undegraded': '2024-09-11T01:16:20.269866+0000', 'last_fullsized': '2024-09-11T01:13:09.652311+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3415", 'last_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'objects_scrubbed': 3337, 'log_size': 9831, 'log_dups_size': 0, 'ondisk_log_size': 9831, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1975, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40235008, 'num_objects': 9823, 'num_object_clones': 0, 'num_object_copies': 19646, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2795, 'num_objects_unfound': 0, 'num_objects_dirty': 9823, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6486, 'num_write_kb': 25944, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1144, 'num_bytes_recovered': 4685824, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7028}, {'shards': '0,2', 'objects': 2795}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:25.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:16:25.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'9982", 'reported_seq': 10117, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:20.259917+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:16:20.259917+0000', 'last_peered': '2024-09-11T01:16:20.259917+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:16:20.259917+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 9982, 'log_dups_size': 0, 'ondisk_log_size': 9982, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40869888, 'num_objects': 9978, 'num_object_clones': 0, 'num_object_copies': 19956, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9978, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6591, 'num_write_kb': 26364, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5939}, {'shards': '0,2', 'objects': 4039}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:25.359 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:16:25.360 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'9861", 'reported_seq': 9995, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:20.225016+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:16:20.225016+0000', 'last_peered': '2024-09-11T01:16:20.225016+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:16:20.225016+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 9861, 'log_dups_size': 0, 'ondisk_log_size': 9861, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40342121, 'num_objects': 9850, 'num_object_clones': 0, 'num_object_copies': 19700, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9850, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6931, 'num_write_kb': 27724, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5933}, {'shards': '0,2', 'objects': 3917}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:25.360 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:16:26.454 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:16:26.454 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39769/318916 objects degraded (12.470%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":322,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":159458,"data_bytes":653128316,"bytes_used":2174242816,"bytes_avail":287736049664,"bytes_total":289910292480,"degraded_objects":39769,"degraded_total":318916,"degraded_ratio":0.12470054810671147,"misplaced_objects":2795,"misplaced_total":318916,"misplaced_ratio":0.008764063264307842,"recovering_objects_per_sec":16,"recovering_bytes_per_sec":68250,"recovering_keys_per_sec":0,"num_objects_recovered":200,"num_bytes_recovered":819200,"num_keys_recovered":0,"write_bytes_sec":1747904,"read_op_per_sec":0,"write_op_per_sec":426},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:16:26.501 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:16:26.502 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:27.282 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:16:27.283 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:16:27.349 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":200,"stamp":"2024-09-11T01:16:26.641550+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656925308,"num_objects":160385,"num_object_clones":0,"num_object_copies":320770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2694,"num_objects_unfound":0,"num_objects_dirty":160385,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121806,"num_write_kb":486238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5174,"num_bytes_recovered":21180435,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160357,"ondisk_log_size":160357,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2155560,"kb_used_data":1304056,"kb_used_omap":12,"kb_used_meta":851443,"kb_avail":280959960,"statfs":{"total":289910292480,"available":287702999040,"internally_reserved":0,"allocated":1335353344,"data_stored":1334510199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":871877636},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,1,4,7,11],"upper_bound":128},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":21417984,"num_objects":5229,"num_object_clones":0,"num_object_copies":10458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-301,"num_objects_unfound":0,"num_objects_dirty":5229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5719,"num_write_kb":21896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":301,"num_bytes_recovered":1232896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002854"},"pg_stats":[{"pgid":"2.f","version":"45'10155","reported_seq":10233,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:25.337689+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:16:25.337689+0000","last_peered":"2024-09-11T01:16:25.337689+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:16:25.337689+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41414656,"num_objects":10111,"num_object_clones":0,"num_object_copies":20222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10155,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6039},{"shards":"0,2","objects":4050}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10170","reported_seq":10271,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:23.956708+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:16:23.956708+0000","last_peered":"2024-09-11T01:16:23.956708+0000","last_clean":"2024-09-11T01:16:23.956708+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:16:23.956708+0000","last_undegraded":"2024-09-11T01:16:23.956708+0000","last_fullsized":"2024-09-11T01:16:23.956708+0000","mapping_epoch":35,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41631744,"num_objects":10164,"num_object_clones":0,"num_object_copies":20328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10164,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8409,"num_write_kb":33636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10063","reported_seq":10190,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:23.997936+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:16:23.997936+0000","last_peered":"2024-09-11T01:16:23.997936+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:16:23.997936+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41201664,"num_objects":10059,"num_object_clones":0,"num_object_copies":20118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10059,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8364,"num_write_kb":33456,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3896},{"shards":"2","objects":6163}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10063","reported_seq":10132,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:23.932512+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:16:23.932512+0000","last_peered":"2024-09-11T01:16:23.932512+0000","last_clean":"2024-09-11T01:16:23.932512+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:16:23.932512+0000","last_undegraded":"2024-09-11T01:16:23.932512+0000","last_fullsized":"2024-09-11T01:16:23.932512+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41218048,"num_objects":10063,"num_object_clones":0,"num_object_copies":20126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10063,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10063,"num_write_kb":40252,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10043","reported_seq":10170,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:25.330248+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:16:25.330248+0000","last_peered":"2024-09-11T01:16:25.330248+0000","last_clean":"2024-09-11T01:16:25.330248+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:16:25.330248+0000","last_undegraded":"2024-09-11T01:16:25.330248+0000","last_fullsized":"2024-09-11T01:16:25.330248+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40820736,"num_objects":9966,"num_object_clones":0,"num_object_copies":19932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7048,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10145","reported_seq":10315,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:25.351498+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:16:25.351498+0000","last_peered":"2024-09-11T01:16:25.351498+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:16:25.351498+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41295872,"num_objects":10082,"num_object_clones":0,"num_object_copies":20164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6486,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6131},{"shards":"0,2","objects":3922}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'9988","reported_seq":10118,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:25.324698+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:16:25.324698+0000","last_peered":"2024-09-11T01:16:25.324698+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:16:25.324698+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":9988,"log_dups_size":0,"ondisk_log_size":9988,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40722432,"num_objects":9942,"num_object_clones":0,"num_object_copies":19884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6845,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5956},{"shards":"0,2","objects":3966}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10146","reported_seq":10274,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:25.298594+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:16:25.298594+0000","last_peered":"2024-09-11T01:16:25.298594+0000","last_clean":"2024-09-11T01:16:25.298594+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:16:25.298594+0000","last_undegraded":"2024-09-11T01:16:25.298594+0000","last_fullsized":"2024-09-11T01:16:25.298594+0000","mapping_epoch":34,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41402368,"num_objects":10108,"num_object_clones":0,"num_object_copies":20216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10108,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7022,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'9983","reported_seq":10110,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:23.941562+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:16:23.941562+0000","last_peered":"2024-09-11T01:16:23.941562+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:16:23.941562+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":9983,"log_dups_size":0,"ondisk_log_size":9983,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40861696,"num_objects":9976,"num_object_clones":0,"num_object_copies":19952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7954,"num_write_kb":31816,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3932},{"shards":"2","objects":6043}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'9968","reported_seq":10093,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:25.361728+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:16:25.361728+0000","last_peered":"2024-09-11T01:16:25.361728+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:16:25.361728+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":9968,"log_dups_size":0,"ondisk_log_size":9968,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40550400,"num_objects":9900,"num_object_clones":0,"num_object_copies":19800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6555,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5916},{"shards":"0,2","objects":3954}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10097","reported_seq":10216,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:23.987182+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:16:23.987182+0000","last_peered":"2024-09-11T01:16:23.987182+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:16:23.987182+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41336832,"num_objects":10092,"num_object_clones":0,"num_object_copies":20184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10092,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7914,"num_write_kb":31656,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":4023},{"shards":"2","objects":6068}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'9891","reported_seq":17872,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:23.881211+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:16:23.881211+0000","last_peered":"2024-09-11T01:16:23.881211+0000","last_clean":"2024-09-11T01:16:23.881211+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:16:23.881211+0000","last_undegraded":"2024-09-11T01:16:23.881211+0000","last_fullsized":"2024-09-11T01:16:23.881211+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":9891,"log_dups_size":0,"ondisk_log_size":9891,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40488960,"num_objects":9885,"num_object_clones":0,"num_object_copies":19770,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9885,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7676,"num_write_kb":30704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10153","reported_seq":10285,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:25.368158+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:16:25.368158+0000","last_peered":"2024-09-11T01:16:25.368158+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:16:25.368158+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41414656,"num_objects":10111,"num_object_clones":0,"num_object_copies":20222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6890,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6161},{"shards":"0,2","objects":3930}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'9972","reported_seq":12600,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:25.345062+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:16:25.345062+0000","last_peered":"2024-09-11T01:16:25.345062+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:16:25.345062+0000","last_undegraded":"2024-09-11T01:16:25.345062+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":9972,"log_dups_size":0,"ondisk_log_size":9972,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40599552,"num_objects":9912,"num_object_clones":0,"num_object_copies":19824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2694,"num_objects_unfound":0,"num_objects_dirty":9912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6627,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1245,"num_bytes_recovered":5099520,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7218},{"shards":"0,2","objects":2694}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10113","reported_seq":10248,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:25.364498+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:16:25.364498+0000","last_peered":"2024-09-11T01:16:25.364498+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:16:25.364498+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41218048,"num_objects":10063,"num_object_clones":0,"num_object_copies":20126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10063,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6722,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6024},{"shards":"0,2","objects":4016}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10004","reported_seq":10138,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:25.308601+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:16:25.308601+0000","last_peered":"2024-09-11T01:16:25.308601+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:16:25.308601+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40747625,"num_objects":9949,"num_object_clones":0,"num_object_copies":19898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9949,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7074,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6032},{"shards":"0,2","objects":3895}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":656925289,"num_objects":160383,"num_object_clones":0,"num_object_copies":320766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2694,"num_objects_unfound":0,"num_objects_dirty":160383,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":121804,"num_write_kb":486236,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5173,"num_bytes_recovered":21180416,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1335025664,"data_stored":1334945647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160354,"ondisk_log_size":160354,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182459,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":896636,"kb_used_data":545708,"kb_used_omap":4,"kb_used_meta":350907,"kb_avail":93475204,"statfs":{"total":96636764160,"available":95718608896,"internally_reserved":0,"allocated":558804992,"data_stored":558545778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":359329451},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,1,7,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986231,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":714904,"kb_used_data":448648,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93656936,"statfs":{"total":96636764160,"available":95904702464,"internally_reserved":0,"allocated":459415552,"data_stored":459160415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,0,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659176,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":544020,"kb_used_data":309700,"kb_used_omap":4,"kb_used_meta":234299,"kb_avail":93827820,"statfs":{"total":96636764160,"available":96079687680,"internally_reserved":0,"allocated":317132800,"data_stored":316804006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":239922863},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":557506560,"data_stored":557503081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":317997056,"data_stored":317923997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":459522048,"data_stored":459518569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:30.351 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:31.166 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:16:31.169 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:16:31.207 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":202,"stamp":"2024-09-11T01:16:30.643033+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":648376980,"num_objects":158299,"num_object_clones":0,"num_object_copies":316598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2583,"num_objects_unfound":0,"num_objects_dirty":158299,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":123992,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5285,"num_bytes_recovered":21635091,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160843,"ondisk_log_size":160843,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2131096,"kb_used_data":1296228,"kb_used_omap":12,"kb_used_meta":834803,"kb_avail":280984424,"statfs":{"total":289910292480,"available":287728050176,"internally_reserved":0,"allocated":1327337472,"data_stored":1326486183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":854838276},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,4,3,10,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7274520,"num_objects":1777,"num_object_clones":0,"num_object_copies":3554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-313,"num_objects_unfound":0,"num_objects_dirty":1777,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6539,"num_write_kb":16629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":1282048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003403"},"pg_stats":[{"pgid":"2.f","version":"45'10283","reported_seq":10361,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.372783+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:16:30.372783+0000","last_peered":"2024-09-11T01:16:30.372783+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:16:30.372783+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40890368,"num_objects":9983,"num_object_clones":0,"num_object_copies":19966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10283,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5911},{"shards":"0,2","objects":3922}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10307","reported_seq":10408,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:28.860786+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:16:28.860786+0000","last_peered":"2024-09-11T01:16:28.860786+0000","last_clean":"2024-09-11T01:16:28.860786+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:16:28.860786+0000","last_undegraded":"2024-09-11T01:16:28.860786+0000","last_fullsized":"2024-09-11T01:16:28.860786+0000","mapping_epoch":35,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41144320,"num_objects":10045,"num_object_clones":0,"num_object_copies":20090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8546,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10206","reported_seq":10334,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:29.022470+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:16:29.022470+0000","last_peered":"2024-09-11T01:16:29.022470+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:16:29.022470+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40701976,"num_objects":9938,"num_object_clones":0,"num_object_copies":19876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9938,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8507,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3764},{"shards":"2","objects":6042}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10170","reported_seq":10239,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:28.946465+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:16:28.946465+0000","last_peered":"2024-09-11T01:16:28.946465+0000","last_clean":"2024-09-11T01:16:28.946465+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:16:28.946465+0000","last_undegraded":"2024-09-11T01:16:28.946465+0000","last_fullsized":"2024-09-11T01:16:28.946465+0000","mapping_epoch":35,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40812544,"num_objects":9964,"num_object_clones":0,"num_object_copies":19928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10170,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10196","reported_seq":10324,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:30.351992+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:16:30.351992+0000","last_peered":"2024-09-11T01:16:30.351992+0000","last_clean":"2024-09-11T01:16:30.351992+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:16:30.351992+0000","last_undegraded":"2024-09-11T01:16:30.351992+0000","last_fullsized":"2024-09-11T01:16:30.351992+0000","mapping_epoch":34,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40194048,"num_objects":9813,"num_object_clones":0,"num_object_copies":19626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7201,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10269","reported_seq":10439,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.368964+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:16:30.368964+0000","last_peered":"2024-09-11T01:16:30.368964+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:16:30.368964+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40787968,"num_objects":9958,"num_object_clones":0,"num_object_copies":19916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6610,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6007},{"shards":"0,2","objects":3798}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10128","reported_seq":10255,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.397654+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:16:30.397654+0000","last_peered":"2024-09-11T01:16:30.397654+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:16:30.397654+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40148992,"num_objects":9802,"num_object_clones":0,"num_object_copies":19604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6985,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5816},{"shards":"0,2","objects":3826}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10296","reported_seq":10426,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:30.286797+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:16:30.286797+0000","last_peered":"2024-09-11T01:16:30.286797+0000","last_clean":"2024-09-11T01:16:30.286797+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:16:30.286797+0000","last_undegraded":"2024-09-11T01:16:30.286797+0000","last_fullsized":"2024-09-11T01:16:30.286797+0000","mapping_epoch":34,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40787968,"num_objects":9958,"num_object_clones":0,"num_object_copies":19916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7172,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10109","reported_seq":10237,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:29.007695+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:16:29.007695+0000","last_peered":"2024-09-11T01:16:29.007695+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:16:29.007695+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40402944,"num_objects":9864,"num_object_clones":0,"num_object_copies":19728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8080,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3813},{"shards":"2","objects":5931}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10117","reported_seq":10245,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.258528+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:16:30.258528+0000","last_peered":"2024-09-11T01:16:30.258528+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:16:30.258528+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39940096,"num_objects":9751,"num_object_clones":0,"num_object_copies":19502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6704,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5767},{"shards":"0,2","objects":3805}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10242","reported_seq":10361,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:28.971804+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:16:28.971804+0000","last_peered":"2024-09-11T01:16:28.971804+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:16:28.971804+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40816640,"num_objects":9965,"num_object_clones":0,"num_object_copies":19930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9965,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8059,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3887},{"shards":"2","objects":5941}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10022","reported_seq":18003,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:28.930043+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:16:28.930043+0000","last_peered":"2024-09-11T01:16:28.930043+0000","last_clean":"2024-09-11T01:16:28.930043+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:16:28.930043+0000","last_undegraded":"2024-09-11T01:16:28.930043+0000","last_fullsized":"2024-09-11T01:16:28.930043+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40034304,"num_objects":9774,"num_object_clones":0,"num_object_copies":19548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7807,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10297","reported_seq":10429,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.338036+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:16:30.338036+0000","last_peered":"2024-09-11T01:16:30.338036+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:16:30.338036+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40824832,"num_objects":9967,"num_object_clones":0,"num_object_copies":19934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7034,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6017},{"shards":"0,2","objects":3786}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10093","reported_seq":12943,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:30.412308+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:16:30.412308+0000","last_peered":"2024-09-11T01:16:30.412308+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:16:30.412308+0000","last_undegraded":"2024-09-11T01:16:30.412308+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40103936,"num_objects":9791,"num_object_clones":0,"num_object_copies":19582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2583,"num_objects_unfound":0,"num_objects_dirty":9791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6748,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1356,"num_bytes_recovered":5554176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7208},{"shards":"0,2","objects":2583}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10250","reported_seq":10384,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.403256+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:16:30.403256+0000","last_peered":"2024-09-11T01:16:30.403256+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:16:30.403256+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40656896,"num_objects":9926,"num_object_clones":0,"num_object_copies":19852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6859,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5887},{"shards":"0,2","objects":3879}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10155","reported_seq":10287,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.312939+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:16:30.312939+0000","last_peered":"2024-09-11T01:16:30.312939+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:16:30.312939+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40129129,"num_objects":9798,"num_object_clones":0,"num_object_copies":19596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7225,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5881},{"shards":"0,2","objects":3744}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":648376961,"num_objects":158297,"num_object_clones":0,"num_object_copies":316594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2583,"num_objects_unfound":0,"num_objects_dirty":158297,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":123990,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5284,"num_bytes_recovered":21635072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1324691456,"data_stored":1324603295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160840,"ondisk_log_size":160840,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182460,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":877572,"kb_used_data":539120,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93494268,"statfs":{"total":96636764160,"available":95738130432,"internally_reserved":0,"allocated":552058880,"data_stored":551799666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":346549931},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,1,7,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986232,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":716284,"kb_used_data":445868,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93655556,"statfs":{"total":96636764160,"available":95903289344,"internally_reserved":0,"allocated":456568832,"data_stored":456309623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276885162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,0,0,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659177,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537240,"kb_used_data":311240,"kb_used_omap":4,"kb_used_meta":225979,"kb_avail":93834600,"statfs":{"total":96636764160,"available":96086630400,"internally_reserved":0,"allocated":318709760,"data_stored":318376894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":231403183},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":550719488,"data_stored":550716009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":318324736,"data_stored":318247605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":455647232,"data_stored":455639681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:31.208 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:31.961 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:16:31.968 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:16:31.999 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":202,"stamp":"2024-09-11T01:16:30.643033+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":648376980,"num_objects":158299,"num_object_clones":0,"num_object_copies":316598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2583,"num_objects_unfound":0,"num_objects_dirty":158299,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":123992,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5285,"num_bytes_recovered":21635091,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160843,"ondisk_log_size":160843,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2131096,"kb_used_data":1296228,"kb_used_omap":12,"kb_used_meta":834803,"kb_avail":280984424,"statfs":{"total":289910292480,"available":287728050176,"internally_reserved":0,"allocated":1327337472,"data_stored":1326486183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":854838276},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,4,3,10,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7274520,"num_objects":1777,"num_object_clones":0,"num_object_copies":3554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-313,"num_objects_unfound":0,"num_objects_dirty":1777,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6539,"num_write_kb":16629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":313,"num_bytes_recovered":1282048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003403"},"pg_stats":[{"pgid":"2.f","version":"45'10283","reported_seq":10361,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.372783+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:16:30.372783+0000","last_peered":"2024-09-11T01:16:30.372783+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:16:30.372783+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40890368,"num_objects":9983,"num_object_clones":0,"num_object_copies":19966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10283,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5911},{"shards":"0,2","objects":3922}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10307","reported_seq":10408,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:28.860786+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:16:28.860786+0000","last_peered":"2024-09-11T01:16:28.860786+0000","last_clean":"2024-09-11T01:16:28.860786+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:16:28.860786+0000","last_undegraded":"2024-09-11T01:16:28.860786+0000","last_fullsized":"2024-09-11T01:16:28.860786+0000","mapping_epoch":35,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41144320,"num_objects":10045,"num_object_clones":0,"num_object_copies":20090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8546,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10206","reported_seq":10334,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:29.022470+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:16:29.022470+0000","last_peered":"2024-09-11T01:16:29.022470+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:16:29.022470+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40701976,"num_objects":9938,"num_object_clones":0,"num_object_copies":19876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9938,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8507,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3764},{"shards":"2","objects":6042}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10170","reported_seq":10239,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:28.946465+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:16:28.946465+0000","last_peered":"2024-09-11T01:16:28.946465+0000","last_clean":"2024-09-11T01:16:28.946465+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:16:28.946465+0000","last_undegraded":"2024-09-11T01:16:28.946465+0000","last_fullsized":"2024-09-11T01:16:28.946465+0000","mapping_epoch":35,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40812544,"num_objects":9964,"num_object_clones":0,"num_object_copies":19928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10170,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10196","reported_seq":10324,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:30.351992+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:16:30.351992+0000","last_peered":"2024-09-11T01:16:30.351992+0000","last_clean":"2024-09-11T01:16:30.351992+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:16:30.351992+0000","last_undegraded":"2024-09-11T01:16:30.351992+0000","last_fullsized":"2024-09-11T01:16:30.351992+0000","mapping_epoch":34,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40194048,"num_objects":9813,"num_object_clones":0,"num_object_copies":19626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7201,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10269","reported_seq":10439,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.368964+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:16:30.368964+0000","last_peered":"2024-09-11T01:16:30.368964+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:16:30.368964+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40787968,"num_objects":9958,"num_object_clones":0,"num_object_copies":19916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6610,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6007},{"shards":"0,2","objects":3798}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10128","reported_seq":10255,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.397654+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:16:30.397654+0000","last_peered":"2024-09-11T01:16:30.397654+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:16:30.397654+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40148992,"num_objects":9802,"num_object_clones":0,"num_object_copies":19604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6985,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5816},{"shards":"0,2","objects":3826}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10296","reported_seq":10426,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:30.286797+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:16:30.286797+0000","last_peered":"2024-09-11T01:16:30.286797+0000","last_clean":"2024-09-11T01:16:30.286797+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:16:30.286797+0000","last_undegraded":"2024-09-11T01:16:30.286797+0000","last_fullsized":"2024-09-11T01:16:30.286797+0000","mapping_epoch":34,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40787968,"num_objects":9958,"num_object_clones":0,"num_object_copies":19916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7172,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10109","reported_seq":10237,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:29.007695+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:16:29.007695+0000","last_peered":"2024-09-11T01:16:29.007695+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:16:29.007695+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40402944,"num_objects":9864,"num_object_clones":0,"num_object_copies":19728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8080,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3813},{"shards":"2","objects":5931}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10117","reported_seq":10245,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.258528+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:16:30.258528+0000","last_peered":"2024-09-11T01:16:30.258528+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:16:30.258528+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39940096,"num_objects":9751,"num_object_clones":0,"num_object_copies":19502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6704,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5767},{"shards":"0,2","objects":3805}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10242","reported_seq":10361,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:28.971804+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:16:28.971804+0000","last_peered":"2024-09-11T01:16:28.971804+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:16:28.971804+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40816640,"num_objects":9965,"num_object_clones":0,"num_object_copies":19930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9965,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8059,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3887},{"shards":"2","objects":5941}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10022","reported_seq":18003,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:28.930043+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:16:28.930043+0000","last_peered":"2024-09-11T01:16:28.930043+0000","last_clean":"2024-09-11T01:16:28.930043+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:16:28.930043+0000","last_undegraded":"2024-09-11T01:16:28.930043+0000","last_fullsized":"2024-09-11T01:16:28.930043+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40034304,"num_objects":9774,"num_object_clones":0,"num_object_copies":19548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7807,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10297","reported_seq":10429,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.338036+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:16:30.338036+0000","last_peered":"2024-09-11T01:16:30.338036+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:16:30.338036+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40824832,"num_objects":9967,"num_object_clones":0,"num_object_copies":19934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7034,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6017},{"shards":"0,2","objects":3786}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10093","reported_seq":12943,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:30.412308+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:16:30.412308+0000","last_peered":"2024-09-11T01:16:30.412308+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:16:30.412308+0000","last_undegraded":"2024-09-11T01:16:30.412308+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40103936,"num_objects":9791,"num_object_clones":0,"num_object_copies":19582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2583,"num_objects_unfound":0,"num_objects_dirty":9791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6748,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1356,"num_bytes_recovered":5554176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7208},{"shards":"0,2","objects":2583}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10250","reported_seq":10384,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.403256+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:16:30.403256+0000","last_peered":"2024-09-11T01:16:30.403256+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:16:30.403256+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40656896,"num_objects":9926,"num_object_clones":0,"num_object_copies":19852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6859,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5887},{"shards":"0,2","objects":3879}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10155","reported_seq":10287,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.312939+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:16:30.312939+0000","last_peered":"2024-09-11T01:16:30.312939+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:16:30.312939+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40129129,"num_objects":9798,"num_object_clones":0,"num_object_copies":19596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7225,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5881},{"shards":"0,2","objects":3744}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":648376961,"num_objects":158297,"num_object_clones":0,"num_object_copies":316594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2583,"num_objects_unfound":0,"num_objects_dirty":158297,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":123990,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5284,"num_bytes_recovered":21635072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1324691456,"data_stored":1324603295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160840,"ondisk_log_size":160840,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182460,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":877572,"kb_used_data":539120,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93494268,"statfs":{"total":96636764160,"available":95738130432,"internally_reserved":0,"allocated":552058880,"data_stored":551799666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":346549931},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,1,7,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986232,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":716284,"kb_used_data":445868,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93655556,"statfs":{"total":96636764160,"available":95903289344,"internally_reserved":0,"allocated":456568832,"data_stored":456309623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276885162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,0,0,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659177,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537240,"kb_used_data":311240,"kb_used_omap":4,"kb_used_meta":225979,"kb_avail":93834600,"statfs":{"total":96636764160,"available":96086630400,"internally_reserved":0,"allocated":318709760,"data_stored":318376894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":231403183},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":550719488,"data_stored":550716009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":318324736,"data_stored":318247605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":455647232,"data_stored":455639681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:32.001 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:16:32.001 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'10283", 'reported_seq': 10361, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:30.372783+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:16:30.372783+0000', 'last_peered': '2024-09-11T01:16:30.372783+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:16:30.372783+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40890368, 'num_objects': 9983, 'num_object_clones': 0, 'num_object_copies': 19966, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9983, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10283, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5911}, {'shards': '0,2', 'objects': 3922}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:32.001 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:16:32.001 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'10206", 'reported_seq': 10334, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:29.022470+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:16:29.022470+0000', 'last_peered': '2024-09-11T01:16:29.022470+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:16:29.022470+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40701976, 'num_objects': 9938, 'num_object_clones': 0, 'num_object_copies': 19876, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9938, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8507, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3764}, {'shards': '2', 'objects': 6042}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:32.001 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:16:32.001 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'10269", 'reported_seq': 10439, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:30.368964+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:16:30.368964+0000', 'last_peered': '2024-09-11T01:16:30.368964+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:16:30.368964+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40787968, 'num_objects': 9958, 'num_object_clones': 0, 'num_object_copies': 19916, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9958, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6610, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6007}, {'shards': '0,2', 'objects': 3798}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:32.002 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:16:32.002 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'10128", 'reported_seq': 10255, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:30.397654+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:16:30.397654+0000', 'last_peered': '2024-09-11T01:16:30.397654+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:16:30.397654+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40148992, 'num_objects': 9802, 'num_object_clones': 0, 'num_object_copies': 19604, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9802, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6985, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5816}, {'shards': '0,2', 'objects': 3826}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:32.002 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:16:32.002 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'10109", 'reported_seq': 10237, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:29.007695+0000', 'last_change': '2024-09-11T01:13:10.159069+0000', 'last_active': '2024-09-11T01:16:29.007695+0000', 'last_peered': '2024-09-11T01:16:29.007695+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:16:29.007695+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40402944, 'num_objects': 9864, 'num_object_clones': 0, 'num_object_copies': 19728, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9864, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8080, 'num_write_kb': 31844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3813}, {'shards': '2', 'objects': 5931}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:32.002 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:16:32.002 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'10117", 'reported_seq': 10245, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:30.258528+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:16:30.258528+0000', 'last_peered': '2024-09-11T01:16:30.258528+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:16:30.258528+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39940096, 'num_objects': 9751, 'num_object_clones': 0, 'num_object_copies': 19502, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9751, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6704, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5767}, {'shards': '0,2', 'objects': 3805}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:32.002 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:16:32.002 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'10242", 'reported_seq': 10361, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:28.971804+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:16:28.971804+0000', 'last_peered': '2024-09-11T01:16:28.971804+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:16:28.971804+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40816640, 'num_objects': 9965, 'num_object_clones': 0, 'num_object_copies': 19930, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9965, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8059, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3887}, {'shards': '2', 'objects': 5941}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:32.002 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:16:32.002 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'10297", 'reported_seq': 10429, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:30.338036+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:16:30.338036+0000', 'last_peered': '2024-09-11T01:16:30.338036+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:16:30.338036+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40824832, 'num_objects': 9967, 'num_object_clones': 0, 'num_object_copies': 19934, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9967, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7034, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 6017}, {'shards': '0,2', 'objects': 3786}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:32.003 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:16:32.003 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'10093", 'reported_seq': 12943, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:16:30.412308+0000', 'last_change': '2024-09-11T01:15:19.913288+0000', 'last_active': '2024-09-11T01:16:30.412308+0000', 'last_peered': '2024-09-11T01:16:30.412308+0000', 'last_clean': '2024-09-11T01:13:07.624565+0000', 'last_became_active': '2024-09-11T01:13:10.199795+0000', 'last_became_peered': '2024-09-11T01:13:10.199795+0000', 'last_unstale': '2024-09-11T01:16:30.412308+0000', 'last_undegraded': '2024-09-11T01:16:30.412308+0000', 'last_fullsized': '2024-09-11T01:13:09.652311+0000', 'mapping_epoch': 42, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3415", 'last_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'objects_scrubbed': 3337, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1975, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40103936, 'num_objects': 9791, 'num_object_clones': 0, 'num_object_copies': 19582, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2583, 'num_objects_unfound': 0, 'num_objects_dirty': 9791, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6748, 'num_write_kb': 26404, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1356, 'num_bytes_recovered': 5554176, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7208}, {'shards': '0,2', 'objects': 2583}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:32.003 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:16:32.003 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'10250", 'reported_seq': 10384, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:30.403256+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:16:30.403256+0000', 'last_peered': '2024-09-11T01:16:30.403256+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:16:30.403256+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40656896, 'num_objects': 9926, 'num_object_clones': 0, 'num_object_copies': 19852, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9926, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6859, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5887}, {'shards': '0,2', 'objects': 3879}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:32.003 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:16:32.003 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'10155", 'reported_seq': 10287, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:30.312939+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:16:30.312939+0000', 'last_peered': '2024-09-11T01:16:30.312939+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:16:30.312939+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "36'100", 'ondisk_log_start': "36'100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40129129, 'num_objects': 9798, 'num_object_clones': 0, 'num_object_copies': 19596, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9798, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7225, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5881}, {'shards': '0,2', 'objects': 3744}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:32.003 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:16:32.939 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:16:32.939 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39769/316598 objects degraded (12.561%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":329,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":158299,"data_bytes":648376980,"bytes_used":2182242304,"bytes_avail":287728050176,"bytes_total":289910292480,"degraded_objects":39769,"degraded_total":316598,"degraded_ratio":0.12561355409699365,"misplaced_objects":2583,"misplaced_total":316598,"misplaced_ratio":0.0081586112356995307,"recovering_objects_per_sec":17,"recovering_bytes_per_sec":72341,"recovering_keys_per_sec":0,"num_objects_recovered":212,"num_bytes_recovered":868352,"num_keys_recovered":0,"read_bytes_sec":85,"write_bytes_sec":671976,"read_op_per_sec":0,"write_op_per_sec":362},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:16:33.005 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:16:33.005 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:33.714 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:16:33.714 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:16:33.781 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":203,"stamp":"2024-09-11T01:16:32.643526+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":648376980,"num_objects":158299,"num_object_clones":0,"num_object_copies":316598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2583,"num_objects_unfound":0,"num_objects_dirty":158299,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":123992,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5285,"num_bytes_recovered":21635091,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160843,"ondisk_log_size":160843,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2131096,"kb_used_data":1296228,"kb_used_omap":12,"kb_used_meta":834803,"kb_avail":280984424,"statfs":{"total":289910292480,"available":287728050176,"internally_reserved":0,"allocated":1327337472,"data_stored":1326486183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":854838276},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,4,3,10,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1687528,"num_objects":-411,"num_object_clones":0,"num_object_copies":-822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-212,"num_objects_unfound":0,"num_objects_dirty":-411,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4351,"num_write_kb":7877,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":212,"num_bytes_recovered":868352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003458"},"pg_stats":[{"pgid":"2.f","version":"45'10283","reported_seq":10361,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.372783+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:16:30.372783+0000","last_peered":"2024-09-11T01:16:30.372783+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:16:30.372783+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40890368,"num_objects":9983,"num_object_clones":0,"num_object_copies":19966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10283,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5911},{"shards":"0,2","objects":3922}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10307","reported_seq":10408,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:28.860786+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:16:28.860786+0000","last_peered":"2024-09-11T01:16:28.860786+0000","last_clean":"2024-09-11T01:16:28.860786+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:16:28.860786+0000","last_undegraded":"2024-09-11T01:16:28.860786+0000","last_fullsized":"2024-09-11T01:16:28.860786+0000","mapping_epoch":35,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":41144320,"num_objects":10045,"num_object_clones":0,"num_object_copies":20090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8546,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10206","reported_seq":10334,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:29.022470+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:16:29.022470+0000","last_peered":"2024-09-11T01:16:29.022470+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:16:29.022470+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40701976,"num_objects":9938,"num_object_clones":0,"num_object_copies":19876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9938,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8507,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3764},{"shards":"2","objects":6042}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10170","reported_seq":10239,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:28.946465+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:16:28.946465+0000","last_peered":"2024-09-11T01:16:28.946465+0000","last_clean":"2024-09-11T01:16:28.946465+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:16:28.946465+0000","last_undegraded":"2024-09-11T01:16:28.946465+0000","last_fullsized":"2024-09-11T01:16:28.946465+0000","mapping_epoch":35,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40812544,"num_objects":9964,"num_object_clones":0,"num_object_copies":19928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10170,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10196","reported_seq":10324,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:30.351992+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:16:30.351992+0000","last_peered":"2024-09-11T01:16:30.351992+0000","last_clean":"2024-09-11T01:16:30.351992+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:16:30.351992+0000","last_undegraded":"2024-09-11T01:16:30.351992+0000","last_fullsized":"2024-09-11T01:16:30.351992+0000","mapping_epoch":34,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40194048,"num_objects":9813,"num_object_clones":0,"num_object_copies":19626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7201,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10269","reported_seq":10439,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.368964+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:16:30.368964+0000","last_peered":"2024-09-11T01:16:30.368964+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:16:30.368964+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40787968,"num_objects":9958,"num_object_clones":0,"num_object_copies":19916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6610,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6007},{"shards":"0,2","objects":3798}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10128","reported_seq":10255,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.397654+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:16:30.397654+0000","last_peered":"2024-09-11T01:16:30.397654+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:16:30.397654+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40148992,"num_objects":9802,"num_object_clones":0,"num_object_copies":19604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6985,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5816},{"shards":"0,2","objects":3826}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10296","reported_seq":10426,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:30.286797+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:16:30.286797+0000","last_peered":"2024-09-11T01:16:30.286797+0000","last_clean":"2024-09-11T01:16:30.286797+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:16:30.286797+0000","last_undegraded":"2024-09-11T01:16:30.286797+0000","last_fullsized":"2024-09-11T01:16:30.286797+0000","mapping_epoch":34,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40787968,"num_objects":9958,"num_object_clones":0,"num_object_copies":19916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7172,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10109","reported_seq":10237,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:29.007695+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:16:29.007695+0000","last_peered":"2024-09-11T01:16:29.007695+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:16:29.007695+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40402944,"num_objects":9864,"num_object_clones":0,"num_object_copies":19728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8080,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3813},{"shards":"2","objects":5931}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10117","reported_seq":10245,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.258528+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:16:30.258528+0000","last_peered":"2024-09-11T01:16:30.258528+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:16:30.258528+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39940096,"num_objects":9751,"num_object_clones":0,"num_object_copies":19502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6704,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5767},{"shards":"0,2","objects":3805}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10242","reported_seq":10361,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:28.971804+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:16:28.971804+0000","last_peered":"2024-09-11T01:16:28.971804+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:16:28.971804+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40816640,"num_objects":9965,"num_object_clones":0,"num_object_copies":19930,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9965,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8059,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3887},{"shards":"2","objects":5941}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10022","reported_seq":18003,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:28.930043+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:16:28.930043+0000","last_peered":"2024-09-11T01:16:28.930043+0000","last_clean":"2024-09-11T01:16:28.930043+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:16:28.930043+0000","last_undegraded":"2024-09-11T01:16:28.930043+0000","last_fullsized":"2024-09-11T01:16:28.930043+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40034304,"num_objects":9774,"num_object_clones":0,"num_object_copies":19548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7807,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10297","reported_seq":10429,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.338036+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:16:30.338036+0000","last_peered":"2024-09-11T01:16:30.338036+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:16:30.338036+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40824832,"num_objects":9967,"num_object_clones":0,"num_object_copies":19934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9967,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7034,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":6017},{"shards":"0,2","objects":3786}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10093","reported_seq":12943,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:30.412308+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:16:30.412308+0000","last_peered":"2024-09-11T01:16:30.412308+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:16:30.412308+0000","last_undegraded":"2024-09-11T01:16:30.412308+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40103936,"num_objects":9791,"num_object_clones":0,"num_object_copies":19582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2583,"num_objects_unfound":0,"num_objects_dirty":9791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6748,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1356,"num_bytes_recovered":5554176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7208},{"shards":"0,2","objects":2583}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10250","reported_seq":10384,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.403256+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:16:30.403256+0000","last_peered":"2024-09-11T01:16:30.403256+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:16:30.403256+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40656896,"num_objects":9926,"num_object_clones":0,"num_object_copies":19852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6859,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5887},{"shards":"0,2","objects":3879}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10155","reported_seq":10287,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:30.312939+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:16:30.312939+0000","last_peered":"2024-09-11T01:16:30.312939+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:16:30.312939+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40129129,"num_objects":9798,"num_object_clones":0,"num_object_copies":19596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9798,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7225,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5881},{"shards":"0,2","objects":3744}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":648376961,"num_objects":158297,"num_object_clones":0,"num_object_copies":316594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2583,"num_objects_unfound":0,"num_objects_dirty":158297,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":123990,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5284,"num_bytes_recovered":21635072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1324691456,"data_stored":1324603295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160840,"ondisk_log_size":160840,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182460,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":877572,"kb_used_data":539120,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93494268,"statfs":{"total":96636764160,"available":95738130432,"internally_reserved":0,"allocated":552058880,"data_stored":551799666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":346549931},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,1,7,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986232,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":716284,"kb_used_data":445868,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93655556,"statfs":{"total":96636764160,"available":95903289344,"internally_reserved":0,"allocated":456568832,"data_stored":456309623,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":276885162},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,1,0,0,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659177,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537240,"kb_used_data":311240,"kb_used_omap":4,"kb_used_meta":225979,"kb_avail":93834600,"statfs":{"total":96636764160,"available":96086630400,"internally_reserved":0,"allocated":318709760,"data_stored":318376894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":231403183},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":550719488,"data_stored":550716009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":318324736,"data_stored":318247605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":455647232,"data_stored":455639681,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:36.784 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:37.740 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:16:37.741 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:16:37.796 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":205,"stamp":"2024-09-11T01:16:36.644455+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":639398548,"num_objects":156107,"num_object_clones":0,"num_object_copies":312214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2477,"num_objects_unfound":0,"num_objects_dirty":156107,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":126184,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5391,"num_bytes_recovered":22069267,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160735,"ondisk_log_size":160735,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2140376,"kb_used_data":1284712,"kb_used_omap":12,"kb_used_meta":855603,"kb_avail":280975144,"statfs":{"total":289910292480,"available":287718547456,"internally_reserved":0,"allocated":1315545088,"data_stored":1314693799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":876137476},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,3,5,6,11,11,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-13729768,"num_objects":-3351,"num_object_clones":0,"num_object_copies":-6702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-318,"num_objects_unfound":0,"num_objects_dirty":-3351,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5795,"num_write_kb":4885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":318,"num_bytes_recovered":1302528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003411"},"pg_stats":[{"pgid":"2.f","version":"45'10421","reported_seq":10499,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:35.371532+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:16:35.371532+0000","last_peered":"2024-09-11T01:16:35.371532+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:16:35.371532+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40325120,"num_objects":9845,"num_object_clones":0,"num_object_copies":19690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10421,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5773},{"shards":"0,2","objects":3784}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10466","reported_seq":10567,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:34.014978+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:16:34.014978+0000","last_peered":"2024-09-11T01:16:34.014978+0000","last_clean":"2024-09-11T01:16:34.014978+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:16:34.014978+0000","last_undegraded":"2024-09-11T01:16:34.014978+0000","last_fullsized":"2024-09-11T01:16:34.014978+0000","mapping_epoch":35,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40493056,"num_objects":9886,"num_object_clones":0,"num_object_copies":19772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9886,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8705,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10327","reported_seq":10455,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:34.053703+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:16:34.053703+0000","last_peered":"2024-09-11T01:16:34.053703+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:16:34.053703+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40206360,"num_objects":9817,"num_object_clones":0,"num_object_copies":19634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9817,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8628,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3643},{"shards":"2","objects":5921}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10316","reported_seq":10384,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:34.028931+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:16:34.028931+0000","last_peered":"2024-09-11T01:16:34.028931+0000","last_clean":"2024-09-11T01:16:34.028931+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:16:34.028931+0000","last_undegraded":"2024-09-11T01:16:34.028931+0000","last_fullsized":"2024-09-11T01:16:34.028931+0000","mapping_epoch":35,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40214528,"num_objects":9818,"num_object_clones":0,"num_object_copies":19636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9818,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10325","reported_seq":10453,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:35.464309+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:16:35.464309+0000","last_peered":"2024-09-11T01:16:35.464309+0000","last_clean":"2024-09-11T01:16:35.464309+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:16:35.464309+0000","last_undegraded":"2024-09-11T01:16:35.464309+0000","last_fullsized":"2024-09-11T01:16:35.464309+0000","mapping_epoch":34,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39665664,"num_objects":9684,"num_object_clones":0,"num_object_copies":19368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9684,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7330,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10416","reported_seq":10586,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:35.429927+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:16:35.429927+0000","last_peered":"2024-09-11T01:16:35.429927+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:16:35.429927+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40185856,"num_objects":9811,"num_object_clones":0,"num_object_copies":19622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6757,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5860},{"shards":"0,2","objects":3651}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10264","reported_seq":10394,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:35.431091+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:16:35.431091+0000","last_peered":"2024-09-11T01:16:35.431091+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:16:35.431091+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39591936,"num_objects":9666,"num_object_clones":0,"num_object_copies":19332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7121,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5680},{"shards":"0,2","objects":3690}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10426","reported_seq":10556,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:35.381890+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:16:35.381890+0000","last_peered":"2024-09-11T01:16:35.381890+0000","last_clean":"2024-09-11T01:16:35.381890+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:16:35.381890+0000","last_undegraded":"2024-09-11T01:16:35.381890+0000","last_fullsized":"2024-09-11T01:16:35.381890+0000","mapping_epoch":34,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40255488,"num_objects":9828,"num_object_clones":0,"num_object_copies":19656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7302,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10261","reported_seq":10388,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:33.997638+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:16:33.997638+0000","last_peered":"2024-09-11T01:16:33.997638+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:16:33.997638+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39780352,"num_objects":9712,"num_object_clones":0,"num_object_copies":19424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9712,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8232,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3661},{"shards":"2","objects":5779}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10249","reported_seq":10377,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:35.431430+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:16:35.431430+0000","last_peered":"2024-09-11T01:16:35.431430+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:16:35.431430+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39399424,"num_objects":9619,"num_object_clones":0,"num_object_copies":19238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6836,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5635},{"shards":"0,2","objects":3673}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10381","reported_seq":10499,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:34.015385+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:16:34.015385+0000","last_peered":"2024-09-11T01:16:34.015385+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:16:34.015385+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40247296,"num_objects":9826,"num_object_clones":0,"num_object_copies":19652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8198,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3748},{"shards":"2","objects":5802}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10156","reported_seq":18136,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:33.961430+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:16:33.961430+0000","last_peered":"2024-09-11T01:16:33.961430+0000","last_clean":"2024-09-11T01:16:33.961430+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:16:33.961430+0000","last_undegraded":"2024-09-11T01:16:33.961430+0000","last_fullsized":"2024-09-11T01:16:33.961430+0000","mapping_epoch":44,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39485440,"num_objects":9640,"num_object_clones":0,"num_object_copies":19280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7941,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10444","reported_seq":10576,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:35.503616+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:16:35.503616+0000","last_peered":"2024-09-11T01:16:35.503616+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:16:35.503616+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40222720,"num_objects":9820,"num_object_clones":0,"num_object_copies":19640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9820,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7181,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5870},{"shards":"0,2","objects":3639}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10240","reported_seq":13302,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:35.468932+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:16:35.468932+0000","last_peered":"2024-09-11T01:16:35.468932+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:16:35.468932+0000","last_undegraded":"2024-09-11T01:16:35.468932+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39501824,"num_objects":9644,"num_object_clones":0,"num_object_copies":19288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2477,"num_objects_unfound":0,"num_objects_dirty":9644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6895,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1462,"num_bytes_recovered":5988352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7167},{"shards":"0,2","objects":2477}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10382","reported_seq":10517,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:35.445897+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:16:35.445897+0000","last_peered":"2024-09-11T01:16:35.445897+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:16:35.445897+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40116224,"num_objects":9794,"num_object_clones":0,"num_object_copies":19588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9794,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6991,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5755},{"shards":"0,2","objects":3747}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10258","reported_seq":10392,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:35.368404+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:16:35.368404+0000","last_peered":"2024-09-11T01:16:35.368404+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:16:35.368404+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39707241,"num_objects":9695,"num_object_clones":0,"num_object_copies":19390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7328,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5778},{"shards":"0,2","objects":3641}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":639398529,"num_objects":156105,"num_object_clones":0,"num_object_copies":312210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2477,"num_objects_unfound":0,"num_objects_dirty":156105,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":126182,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5390,"num_bytes_recovered":22069248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1312677888,"data_stored":1312589727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160732,"ondisk_log_size":160732,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182461,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":879388,"kb_used_data":532620,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93492452,"statfs":{"total":96636764160,"available":95736270848,"internally_reserved":0,"allocated":545402880,"data_stored":545143666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355069611},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,3,5,6,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986233,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":715988,"kb_used_data":441412,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93655852,"statfs":{"total":96636764160,"available":95903592448,"internally_reserved":0,"allocated":452005888,"data_stored":451746679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,1,0,3,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659178,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545000,"kb_used_data":310680,"kb_used_omap":4,"kb_used_meta":234299,"kb_avail":93826840,"statfs":{"total":96636764160,"available":96078684160,"internally_reserved":0,"allocated":318136320,"data_stored":317803454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":239922863},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":543936512,"data_stored":543933033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":317739008,"data_stored":317661877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":451002368,"data_stored":450994817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:37.798 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:38.568 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:16:38.569 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:16:38.660 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":205,"stamp":"2024-09-11T01:16:36.644455+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":639398548,"num_objects":156107,"num_object_clones":0,"num_object_copies":312214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2477,"num_objects_unfound":0,"num_objects_dirty":156107,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":126184,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5391,"num_bytes_recovered":22069267,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160735,"ondisk_log_size":160735,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2140376,"kb_used_data":1284712,"kb_used_omap":12,"kb_used_meta":855603,"kb_avail":280975144,"statfs":{"total":289910292480,"available":287718547456,"internally_reserved":0,"allocated":1315545088,"data_stored":1314693799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":876137476},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,3,5,6,11,11,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-13729768,"num_objects":-3351,"num_object_clones":0,"num_object_copies":-6702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-318,"num_objects_unfound":0,"num_objects_dirty":-3351,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5795,"num_write_kb":4885,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":318,"num_bytes_recovered":1302528,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003411"},"pg_stats":[{"pgid":"2.f","version":"45'10421","reported_seq":10499,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:35.371532+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:16:35.371532+0000","last_peered":"2024-09-11T01:16:35.371532+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:16:35.371532+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40325120,"num_objects":9845,"num_object_clones":0,"num_object_copies":19690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10421,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5773},{"shards":"0,2","objects":3784}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10466","reported_seq":10567,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:34.014978+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:16:34.014978+0000","last_peered":"2024-09-11T01:16:34.014978+0000","last_clean":"2024-09-11T01:16:34.014978+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:16:34.014978+0000","last_undegraded":"2024-09-11T01:16:34.014978+0000","last_fullsized":"2024-09-11T01:16:34.014978+0000","mapping_epoch":35,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40493056,"num_objects":9886,"num_object_clones":0,"num_object_copies":19772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9886,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8705,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10327","reported_seq":10455,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:34.053703+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:16:34.053703+0000","last_peered":"2024-09-11T01:16:34.053703+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:16:34.053703+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40206360,"num_objects":9817,"num_object_clones":0,"num_object_copies":19634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9817,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8628,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3643},{"shards":"2","objects":5921}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10316","reported_seq":10384,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:34.028931+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:16:34.028931+0000","last_peered":"2024-09-11T01:16:34.028931+0000","last_clean":"2024-09-11T01:16:34.028931+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:16:34.028931+0000","last_undegraded":"2024-09-11T01:16:34.028931+0000","last_fullsized":"2024-09-11T01:16:34.028931+0000","mapping_epoch":35,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40214528,"num_objects":9818,"num_object_clones":0,"num_object_copies":19636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9818,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10316,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10325","reported_seq":10453,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:35.464309+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:16:35.464309+0000","last_peered":"2024-09-11T01:16:35.464309+0000","last_clean":"2024-09-11T01:16:35.464309+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:16:35.464309+0000","last_undegraded":"2024-09-11T01:16:35.464309+0000","last_fullsized":"2024-09-11T01:16:35.464309+0000","mapping_epoch":34,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39665664,"num_objects":9684,"num_object_clones":0,"num_object_copies":19368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9684,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7330,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10416","reported_seq":10586,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:35.429927+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:16:35.429927+0000","last_peered":"2024-09-11T01:16:35.429927+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:16:35.429927+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40185856,"num_objects":9811,"num_object_clones":0,"num_object_copies":19622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6757,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5860},{"shards":"0,2","objects":3651}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10264","reported_seq":10394,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:35.431091+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:16:35.431091+0000","last_peered":"2024-09-11T01:16:35.431091+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:16:35.431091+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39591936,"num_objects":9666,"num_object_clones":0,"num_object_copies":19332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7121,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5680},{"shards":"0,2","objects":3690}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10426","reported_seq":10556,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:35.381890+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:16:35.381890+0000","last_peered":"2024-09-11T01:16:35.381890+0000","last_clean":"2024-09-11T01:16:35.381890+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:16:35.381890+0000","last_undegraded":"2024-09-11T01:16:35.381890+0000","last_fullsized":"2024-09-11T01:16:35.381890+0000","mapping_epoch":34,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40255488,"num_objects":9828,"num_object_clones":0,"num_object_copies":19656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7302,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10261","reported_seq":10388,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:33.997638+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:16:33.997638+0000","last_peered":"2024-09-11T01:16:33.997638+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:16:33.997638+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39780352,"num_objects":9712,"num_object_clones":0,"num_object_copies":19424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9712,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8232,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3661},{"shards":"2","objects":5779}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10249","reported_seq":10377,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:35.431430+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:16:35.431430+0000","last_peered":"2024-09-11T01:16:35.431430+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:16:35.431430+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39399424,"num_objects":9619,"num_object_clones":0,"num_object_copies":19238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9619,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6836,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5635},{"shards":"0,2","objects":3673}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10381","reported_seq":10499,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:34.015385+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:16:34.015385+0000","last_peered":"2024-09-11T01:16:34.015385+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:16:34.015385+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40247296,"num_objects":9826,"num_object_clones":0,"num_object_copies":19652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8198,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3748},{"shards":"2","objects":5802}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10156","reported_seq":18136,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:33.961430+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:16:33.961430+0000","last_peered":"2024-09-11T01:16:33.961430+0000","last_clean":"2024-09-11T01:16:33.961430+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:16:33.961430+0000","last_undegraded":"2024-09-11T01:16:33.961430+0000","last_fullsized":"2024-09-11T01:16:33.961430+0000","mapping_epoch":44,"log_start":"36'100","ondisk_log_start":"36'100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39485440,"num_objects":9640,"num_object_clones":0,"num_object_copies":19280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7941,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10444","reported_seq":10576,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:35.503616+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:16:35.503616+0000","last_peered":"2024-09-11T01:16:35.503616+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:16:35.503616+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40222720,"num_objects":9820,"num_object_clones":0,"num_object_copies":19640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9820,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7181,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5870},{"shards":"0,2","objects":3639}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10240","reported_seq":13302,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:35.468932+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:16:35.468932+0000","last_peered":"2024-09-11T01:16:35.468932+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:16:35.468932+0000","last_undegraded":"2024-09-11T01:16:35.468932+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39501824,"num_objects":9644,"num_object_clones":0,"num_object_copies":19288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2477,"num_objects_unfound":0,"num_objects_dirty":9644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6895,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1462,"num_bytes_recovered":5988352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7167},{"shards":"0,2","objects":2477}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10382","reported_seq":10517,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:35.445897+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:16:35.445897+0000","last_peered":"2024-09-11T01:16:35.445897+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:16:35.445897+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":40116224,"num_objects":9794,"num_object_clones":0,"num_object_copies":19588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9794,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6991,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5755},{"shards":"0,2","objects":3747}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10258","reported_seq":10392,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:35.368404+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:16:35.368404+0000","last_peered":"2024-09-11T01:16:35.368404+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:16:35.368404+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39707241,"num_objects":9695,"num_object_clones":0,"num_object_copies":19390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7328,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5778},{"shards":"0,2","objects":3641}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":639398529,"num_objects":156105,"num_object_clones":0,"num_object_copies":312210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2477,"num_objects_unfound":0,"num_objects_dirty":156105,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":126182,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5390,"num_bytes_recovered":22069248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1312677888,"data_stored":1312589727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160732,"ondisk_log_size":160732,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182461,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":879388,"kb_used_data":532620,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93492452,"statfs":{"total":96636764160,"available":95736270848,"internally_reserved":0,"allocated":545402880,"data_stored":545143666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355069611},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,3,5,6,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986233,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":715988,"kb_used_data":441412,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93655852,"statfs":{"total":96636764160,"available":95903592448,"internally_reserved":0,"allocated":452005888,"data_stored":451746679,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":281145002},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,1,0,3,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659178,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545000,"kb_used_data":310680,"kb_used_omap":4,"kb_used_meta":234299,"kb_avail":93826840,"statfs":{"total":96636764160,"available":96078684160,"internally_reserved":0,"allocated":318136320,"data_stored":317803454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":239922863},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":543936512,"data_stored":543933033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":317739008,"data_stored":317661877,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":451002368,"data_stored":450994817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:38.661 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:16:38.661 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'10421", 'reported_seq': 10499, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:35.371532+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:16:35.371532+0000', 'last_peered': '2024-09-11T01:16:35.371532+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:16:35.371532+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40325120, 'num_objects': 9845, 'num_object_clones': 0, 'num_object_copies': 19690, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9845, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10421, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5773}, {'shards': '0,2', 'objects': 3784}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:38.661 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:16:38.661 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'10327", 'reported_seq': 10455, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:34.053703+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:16:34.053703+0000', 'last_peered': '2024-09-11T01:16:34.053703+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:16:34.053703+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40206360, 'num_objects': 9817, 'num_object_clones': 0, 'num_object_copies': 19634, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9817, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8628, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3643}, {'shards': '2', 'objects': 5921}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:38.661 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:16:38.662 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'10416", 'reported_seq': 10586, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:35.429927+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:16:35.429927+0000', 'last_peered': '2024-09-11T01:16:35.429927+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:16:35.429927+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40185856, 'num_objects': 9811, 'num_object_clones': 0, 'num_object_copies': 19622, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9811, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6757, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5860}, {'shards': '0,2', 'objects': 3651}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:38.662 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:16:38.662 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'10264", 'reported_seq': 10394, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:35.431091+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:16:35.431091+0000', 'last_peered': '2024-09-11T01:16:35.431091+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:16:35.431091+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39591936, 'num_objects': 9666, 'num_object_clones': 0, 'num_object_copies': 19332, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9666, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7121, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5680}, {'shards': '0,2', 'objects': 3690}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:38.662 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:16:38.662 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'10261", 'reported_seq': 10388, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:33.997638+0000', 'last_change': '2024-09-11T01:13:10.159069+0000', 'last_active': '2024-09-11T01:16:33.997638+0000', 'last_peered': '2024-09-11T01:16:33.997638+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:16:33.997638+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39780352, 'num_objects': 9712, 'num_object_clones': 0, 'num_object_copies': 19424, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9712, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8232, 'num_write_kb': 31844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3661}, {'shards': '2', 'objects': 5779}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:38.662 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:16:38.662 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'10249", 'reported_seq': 10377, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:35.431430+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:16:35.431430+0000', 'last_peered': '2024-09-11T01:16:35.431430+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:16:35.431430+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39399424, 'num_objects': 9619, 'num_object_clones': 0, 'num_object_copies': 19238, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9619, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6836, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5635}, {'shards': '0,2', 'objects': 3673}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:38.662 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:16:38.662 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'10381", 'reported_seq': 10499, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:34.015385+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:16:34.015385+0000', 'last_peered': '2024-09-11T01:16:34.015385+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:16:34.015385+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40247296, 'num_objects': 9826, 'num_object_clones': 0, 'num_object_copies': 19652, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9826, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8198, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3748}, {'shards': '2', 'objects': 5802}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:38.663 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:16:38.663 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'10444", 'reported_seq': 10576, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:35.503616+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:16:35.503616+0000', 'last_peered': '2024-09-11T01:16:35.503616+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:16:35.503616+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40222720, 'num_objects': 9820, 'num_object_clones': 0, 'num_object_copies': 19640, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9820, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7181, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5870}, {'shards': '0,2', 'objects': 3639}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:38.663 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:16:38.663 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'10240", 'reported_seq': 13302, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:16:35.468932+0000', 'last_change': '2024-09-11T01:15:19.913288+0000', 'last_active': '2024-09-11T01:16:35.468932+0000', 'last_peered': '2024-09-11T01:16:35.468932+0000', 'last_clean': '2024-09-11T01:13:07.624565+0000', 'last_became_active': '2024-09-11T01:13:10.199795+0000', 'last_became_peered': '2024-09-11T01:13:10.199795+0000', 'last_unstale': '2024-09-11T01:16:35.468932+0000', 'last_undegraded': '2024-09-11T01:16:35.468932+0000', 'last_fullsized': '2024-09-11T01:13:09.652311+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3415", 'last_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'objects_scrubbed': 3337, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1975, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39501824, 'num_objects': 9644, 'num_object_clones': 0, 'num_object_copies': 19288, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2477, 'num_objects_unfound': 0, 'num_objects_dirty': 9644, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6895, 'num_write_kb': 26404, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1462, 'num_bytes_recovered': 5988352, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7167}, {'shards': '0,2', 'objects': 2477}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:38.663 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:16:38.663 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'10382", 'reported_seq': 10517, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:35.445897+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:16:35.445897+0000', 'last_peered': '2024-09-11T01:16:35.445897+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:16:35.445897+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 40116224, 'num_objects': 9794, 'num_object_clones': 0, 'num_object_copies': 19588, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9794, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6991, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5755}, {'shards': '0,2', 'objects': 3747}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:38.663 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:16:38.663 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'10258", 'reported_seq': 10392, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:35.368404+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:16:35.368404+0000', 'last_peered': '2024-09-11T01:16:35.368404+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:16:35.368404+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "36'200", 'ondisk_log_start': "36'200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39707241, 'num_objects': 9695, 'num_object_clones': 0, 'num_object_copies': 19390, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9695, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7328, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5778}, {'shards': '0,2', 'objects': 3641}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:38.663 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:16:39.669 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:16:39.670 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39769/312214 objects degraded (12.738%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":336,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":156107,"data_bytes":639398548,"bytes_used":2191745024,"bytes_avail":287718547456,"bytes_total":289910292480,"degraded_objects":39769,"degraded_total":312214,"degraded_ratio":0.12737737577430866,"misplaced_objects":2477,"misplaced_total":312214,"misplaced_ratio":0.0079336608864432733,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":74048,"recovering_keys_per_sec":0,"num_objects_recovered":217,"num_bytes_recovered":888832,"num_keys_recovered":0,"read_bytes_sec":85,"write_bytes_sec":16805,"read_op_per_sec":0,"write_op_per_sec":364},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:16:39.753 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:16:39.753 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:40.763 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:16:40.764 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:16:40.843 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":207,"stamp":"2024-09-11T01:16:40.645457+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":630289044,"num_objects":153883,"num_object_clones":0,"num_object_copies":307766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2364,"num_objects_unfound":0,"num_objects_dirty":153883,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":128408,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5504,"num_bytes_recovered":22532115,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160959,"ondisk_log_size":160959,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2107752,"kb_used_data":1272884,"kb_used_omap":12,"kb_used_meta":834803,"kb_avail":281007768,"statfs":{"total":289910292480,"available":287751954432,"internally_reserved":0,"allocated":1303433216,"data_stored":1302581927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":854838276},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,3,9,12,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26636264,"num_objects":-6502,"num_object_clones":0,"num_object_copies":-13004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-330,"num_objects_unfound":0,"num_objects_dirty":-6502,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":6602,"num_write_kb":197,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":330,"num_bytes_recovered":1351680,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003388"},"pg_stats":[{"pgid":"2.f","version":"45'10558","reported_seq":10636,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.472097+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:16:40.472097+0000","last_peered":"2024-09-11T01:16:40.472097+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:16:40.472097+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39763968,"num_objects":9708,"num_object_clones":0,"num_object_copies":19416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10558,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5636},{"shards":"0,2","objects":3647}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10590","reported_seq":10691,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:39.020610+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:16:39.020610+0000","last_peered":"2024-09-11T01:16:39.020610+0000","last_clean":"2024-09-11T01:16:39.020610+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:16:39.020610+0000","last_undegraded":"2024-09-11T01:16:39.020610+0000","last_fullsized":"2024-09-11T01:16:39.020610+0000","mapping_epoch":35,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39985152,"num_objects":9762,"num_object_clones":0,"num_object_copies":19524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8829,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10465","reported_seq":10593,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:39.010222+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:16:39.010222+0000","last_peered":"2024-09-11T01:16:39.010222+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:16:39.010222+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39641112,"num_objects":9679,"num_object_clones":0,"num_object_copies":19358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9679,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8766,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3505},{"shards":"2","objects":5783}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10468","reported_seq":10537,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:39.051033+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:16:39.051033+0000","last_peered":"2024-09-11T01:16:39.051033+0000","last_clean":"2024-09-11T01:16:39.051033+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:16:39.051033+0000","last_undegraded":"2024-09-11T01:16:39.051033+0000","last_fullsized":"2024-09-11T01:16:39.051033+0000","mapping_epoch":35,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39591936,"num_objects":9666,"num_object_clones":0,"num_object_copies":19332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10468,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10453","reported_seq":10581,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:40.504083+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:16:40.504083+0000","last_peered":"2024-09-11T01:16:40.504083+0000","last_clean":"2024-09-11T01:16:40.504083+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:16:40.504083+0000","last_undegraded":"2024-09-11T01:16:40.504083+0000","last_fullsized":"2024-09-11T01:16:40.504083+0000","mapping_epoch":34,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39141376,"num_objects":9556,"num_object_clones":0,"num_object_copies":19112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7458,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10554","reported_seq":10724,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.465391+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:16:40.465391+0000","last_peered":"2024-09-11T01:16:40.465391+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:16:40.465391+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39620608,"num_objects":9673,"num_object_clones":0,"num_object_copies":19346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6895,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5722},{"shards":"0,2","objects":3513}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10388","reported_seq":10514,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.421365+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:16:40.421365+0000","last_peered":"2024-09-11T01:16:40.421365+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:16:40.421365+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39084032,"num_objects":9542,"num_object_clones":0,"num_object_copies":19084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7245,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5556},{"shards":"0,2","objects":3566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10570","reported_seq":10700,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:40.461221+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:16:40.461221+0000","last_peered":"2024-09-11T01:16:40.461221+0000","last_clean":"2024-09-11T01:16:40.461221+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:16:40.461221+0000","last_undegraded":"2024-09-11T01:16:40.461221+0000","last_fullsized":"2024-09-11T01:16:40.461221+0000","mapping_epoch":34,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39665664,"num_objects":9684,"num_object_clones":0,"num_object_copies":19368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9684,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7446,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10390","reported_seq":10518,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:38.978949+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:16:38.978949+0000","last_peered":"2024-09-11T01:16:38.978949+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:16:38.978949+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39251968,"num_objects":9583,"num_object_clones":0,"num_object_copies":19166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8361,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3532},{"shards":"2","objects":5650}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10421","reported_seq":10549,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.537820+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:16:40.537820+0000","last_peered":"2024-09-11T01:16:40.537820+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:16:40.537820+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38694912,"num_objects":9447,"num_object_clones":0,"num_object_copies":18894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7008,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5463},{"shards":"0,2","objects":3501}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10516","reported_seq":10633,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:39.024373+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:16:39.024373+0000","last_peered":"2024-09-11T01:16:39.024373+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:16:39.024373+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39694336,"num_objects":9691,"num_object_clones":0,"num_object_copies":19382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8333,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3613},{"shards":"2","objects":5667}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10296","reported_seq":18277,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:38.973428+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:16:38.973428+0000","last_peered":"2024-09-11T01:16:38.973428+0000","last_clean":"2024-09-11T01:16:38.973428+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:16:38.973428+0000","last_undegraded":"2024-09-11T01:16:38.973428+0000","last_fullsized":"2024-09-11T01:16:38.973428+0000","mapping_epoch":44,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38912000,"num_objects":9500,"num_object_clones":0,"num_object_copies":19000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8081,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10573","reported_seq":10705,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.471470+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:16:40.471470+0000","last_peered":"2024-09-11T01:16:40.471470+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:16:40.471470+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39694336,"num_objects":9691,"num_object_clones":0,"num_object_copies":19382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7310,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5741},{"shards":"0,2","objects":3510}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10384","reported_seq":13671,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:40.539382+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:16:40.539382+0000","last_peered":"2024-09-11T01:16:40.539382+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:16:40.539382+0000","last_undegraded":"2024-09-11T01:16:40.539382+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38912000,"num_objects":9500,"num_object_clones":0,"num_object_copies":19000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2364,"num_objects_unfound":0,"num_objects_dirty":9500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7039,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1575,"num_bytes_recovered":6451200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7136},{"shards":"0,2","objects":2364}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10527","reported_seq":10662,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.461746+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:16:40.461746+0000","last_peered":"2024-09-11T01:16:40.461746+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:16:40.461746+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39522304,"num_objects":9649,"num_object_clones":0,"num_object_copies":19298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7136,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5610},{"shards":"0,2","objects":3602}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10403","reported_seq":10537,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.492158+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:16:40.492158+0000","last_peered":"2024-09-11T01:16:40.492158+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:16:40.492158+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39113321,"num_objects":9550,"num_object_clones":0,"num_object_copies":19100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9550,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7473,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5633},{"shards":"0,2","objects":3496}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":630289025,"num_objects":153881,"num_object_clones":0,"num_object_copies":307762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2364,"num_objects_unfound":0,"num_objects_dirty":153881,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":128406,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5503,"num_bytes_recovered":22532096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1300623360,"data_stored":1300535199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160956,"ondisk_log_size":160956,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182462,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":860280,"kb_used_data":525992,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93511560,"statfs":{"total":96636764160,"available":95755837440,"internally_reserved":0,"allocated":538615808,"data_stored":538356594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":342290091},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,5,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986234,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":699028,"kb_used_data":436928,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93672812,"statfs":{"total":96636764160,"available":95920959488,"internally_reserved":0,"allocated":447414272,"data_stored":447155063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268365482},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,0,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659179,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548444,"kb_used_data":309964,"kb_used_omap":4,"kb_used_meta":238459,"kb_avail":93823396,"statfs":{"total":96636764160,"available":96075157504,"internally_reserved":0,"allocated":317403136,"data_stored":317070270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":244182703},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":537026560,"data_stored":537023081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":316989440,"data_stored":316912309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":446607360,"data_stored":446599809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:43.845 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:44.627 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:16:44.628 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:16:44.701 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":208,"stamp":"2024-09-11T01:16:42.645996+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":630289044,"num_objects":153883,"num_object_clones":0,"num_object_copies":307766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2364,"num_objects_unfound":0,"num_objects_dirty":153883,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":128408,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5504,"num_bytes_recovered":22532115,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160959,"ondisk_log_size":160959,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2107752,"kb_used_data":1272884,"kb_used_omap":12,"kb_used_meta":834803,"kb_avail":281007768,"statfs":{"total":289910292480,"available":287751954432,"internally_reserved":0,"allocated":1303433216,"data_stored":1302581927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":854838276},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,3,9,12,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18087936,"num_objects":-4416,"num_object_clones":0,"num_object_copies":-8832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-219,"num_objects_unfound":0,"num_objects_dirty":-4416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4416,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":219,"num_bytes_recovered":897024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002963"},"pg_stats":[{"pgid":"2.f","version":"45'10558","reported_seq":10636,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.472097+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:16:40.472097+0000","last_peered":"2024-09-11T01:16:40.472097+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:16:40.472097+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39763968,"num_objects":9708,"num_object_clones":0,"num_object_copies":19416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10558,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5636},{"shards":"0,2","objects":3647}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10590","reported_seq":10691,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:39.020610+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:16:39.020610+0000","last_peered":"2024-09-11T01:16:39.020610+0000","last_clean":"2024-09-11T01:16:39.020610+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:16:39.020610+0000","last_undegraded":"2024-09-11T01:16:39.020610+0000","last_fullsized":"2024-09-11T01:16:39.020610+0000","mapping_epoch":35,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39985152,"num_objects":9762,"num_object_clones":0,"num_object_copies":19524,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9762,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8829,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10465","reported_seq":10593,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:39.010222+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:16:39.010222+0000","last_peered":"2024-09-11T01:16:39.010222+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:16:39.010222+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39641112,"num_objects":9679,"num_object_clones":0,"num_object_copies":19358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9679,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8766,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3505},{"shards":"2","objects":5783}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10468","reported_seq":10537,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:39.051033+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:16:39.051033+0000","last_peered":"2024-09-11T01:16:39.051033+0000","last_clean":"2024-09-11T01:16:39.051033+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:16:39.051033+0000","last_undegraded":"2024-09-11T01:16:39.051033+0000","last_fullsized":"2024-09-11T01:16:39.051033+0000","mapping_epoch":35,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39591936,"num_objects":9666,"num_object_clones":0,"num_object_copies":19332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9666,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10468,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10453","reported_seq":10581,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:40.504083+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:16:40.504083+0000","last_peered":"2024-09-11T01:16:40.504083+0000","last_clean":"2024-09-11T01:16:40.504083+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:16:40.504083+0000","last_undegraded":"2024-09-11T01:16:40.504083+0000","last_fullsized":"2024-09-11T01:16:40.504083+0000","mapping_epoch":34,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39141376,"num_objects":9556,"num_object_clones":0,"num_object_copies":19112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7458,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10554","reported_seq":10724,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.465391+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:16:40.465391+0000","last_peered":"2024-09-11T01:16:40.465391+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:16:40.465391+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39620608,"num_objects":9673,"num_object_clones":0,"num_object_copies":19346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6895,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5722},{"shards":"0,2","objects":3513}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10388","reported_seq":10514,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.421365+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:16:40.421365+0000","last_peered":"2024-09-11T01:16:40.421365+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:16:40.421365+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39084032,"num_objects":9542,"num_object_clones":0,"num_object_copies":19084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7245,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5556},{"shards":"0,2","objects":3566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10570","reported_seq":10700,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:40.461221+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:16:40.461221+0000","last_peered":"2024-09-11T01:16:40.461221+0000","last_clean":"2024-09-11T01:16:40.461221+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:16:40.461221+0000","last_undegraded":"2024-09-11T01:16:40.461221+0000","last_fullsized":"2024-09-11T01:16:40.461221+0000","mapping_epoch":34,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39665664,"num_objects":9684,"num_object_clones":0,"num_object_copies":19368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9684,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7446,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10390","reported_seq":10518,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:38.978949+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:16:38.978949+0000","last_peered":"2024-09-11T01:16:38.978949+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:16:38.978949+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39251968,"num_objects":9583,"num_object_clones":0,"num_object_copies":19166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9583,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8361,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3532},{"shards":"2","objects":5650}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10421","reported_seq":10549,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.537820+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:16:40.537820+0000","last_peered":"2024-09-11T01:16:40.537820+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:16:40.537820+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38694912,"num_objects":9447,"num_object_clones":0,"num_object_copies":18894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7008,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5463},{"shards":"0,2","objects":3501}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10516","reported_seq":10633,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:39.024373+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:16:39.024373+0000","last_peered":"2024-09-11T01:16:39.024373+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:16:39.024373+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39694336,"num_objects":9691,"num_object_clones":0,"num_object_copies":19382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8333,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3613},{"shards":"2","objects":5667}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10296","reported_seq":18277,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:38.973428+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:16:38.973428+0000","last_peered":"2024-09-11T01:16:38.973428+0000","last_clean":"2024-09-11T01:16:38.973428+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:16:38.973428+0000","last_undegraded":"2024-09-11T01:16:38.973428+0000","last_fullsized":"2024-09-11T01:16:38.973428+0000","mapping_epoch":44,"log_start":"36'200","ondisk_log_start":"36'200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38912000,"num_objects":9500,"num_object_clones":0,"num_object_copies":19000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8081,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10573","reported_seq":10705,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.471470+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:16:40.471470+0000","last_peered":"2024-09-11T01:16:40.471470+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:16:40.471470+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39694336,"num_objects":9691,"num_object_clones":0,"num_object_copies":19382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7310,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5741},{"shards":"0,2","objects":3510}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10384","reported_seq":13671,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:40.539382+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:16:40.539382+0000","last_peered":"2024-09-11T01:16:40.539382+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:16:40.539382+0000","last_undegraded":"2024-09-11T01:16:40.539382+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38912000,"num_objects":9500,"num_object_clones":0,"num_object_copies":19000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2364,"num_objects_unfound":0,"num_objects_dirty":9500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7039,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1575,"num_bytes_recovered":6451200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7136},{"shards":"0,2","objects":2364}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10527","reported_seq":10662,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.461746+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:16:40.461746+0000","last_peered":"2024-09-11T01:16:40.461746+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:16:40.461746+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39522304,"num_objects":9649,"num_object_clones":0,"num_object_copies":19298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7136,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5610},{"shards":"0,2","objects":3602}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10403","reported_seq":10537,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.492158+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:16:40.492158+0000","last_peered":"2024-09-11T01:16:40.492158+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:16:40.492158+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39113321,"num_objects":9550,"num_object_clones":0,"num_object_copies":19100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9550,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7473,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5633},{"shards":"0,2","objects":3496}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":630289025,"num_objects":153881,"num_object_clones":0,"num_object_copies":307762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2364,"num_objects_unfound":0,"num_objects_dirty":153881,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":128406,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5503,"num_bytes_recovered":22532096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1300623360,"data_stored":1300535199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160956,"ondisk_log_size":160956,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182462,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":860280,"kb_used_data":525992,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93511560,"statfs":{"total":96636764160,"available":95755837440,"internally_reserved":0,"allocated":538615808,"data_stored":538356594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":342290091},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,5,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986234,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":699028,"kb_used_data":436928,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93672812,"statfs":{"total":96636764160,"available":95920959488,"internally_reserved":0,"allocated":447414272,"data_stored":447155063,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":268365482},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,0,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659179,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":548444,"kb_used_data":309964,"kb_used_omap":4,"kb_used_meta":238459,"kb_avail":93823396,"statfs":{"total":96636764160,"available":96075157504,"internally_reserved":0,"allocated":317403136,"data_stored":317070270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":244182703},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":537026560,"data_stored":537023081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":316989440,"data_stored":316912309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":446607360,"data_stored":446599809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:44.703 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:45.543 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:16:45.545 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:16:45.635 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":209,"stamp":"2024-09-11T01:16:44.646418+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":626975380,"num_objects":153074,"num_object_clones":0,"num_object_copies":306148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2364,"num_objects_unfound":0,"num_objects_dirty":153074,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":129217,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5504,"num_bytes_recovered":22532115,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160668,"ondisk_log_size":160668,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2134988,"kb_used_data":1268184,"kb_used_omap":12,"kb_used_meta":866739,"kb_avail":280980532,"statfs":{"total":289910292480,"available":287724064768,"internally_reserved":0,"allocated":1298620416,"data_stored":1297769127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":887540740},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,9,3,10,13,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21401600,"num_objects":-5225,"num_object_clones":0,"num_object_copies":-10450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-219,"num_objects_unfound":0,"num_objects_dirty":-5225,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5225,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":219,"num_bytes_recovered":897024,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002891"},"pg_stats":[{"pgid":"2.f","version":"45'10558","reported_seq":10636,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.472097+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:16:40.472097+0000","last_peered":"2024-09-11T01:16:40.472097+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:16:40.472097+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39763968,"num_objects":9708,"num_object_clones":0,"num_object_copies":19416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10558,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5636},{"shards":"0,2","objects":3647}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10725","reported_seq":10827,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:43.844518+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:16:43.844518+0000","last_peered":"2024-09-11T01:16:43.844518+0000","last_clean":"2024-09-11T01:16:43.844518+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:16:43.844518+0000","last_undegraded":"2024-09-11T01:16:43.844518+0000","last_fullsized":"2024-09-11T01:16:43.844518+0000","mapping_epoch":35,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39432192,"num_objects":9627,"num_object_clones":0,"num_object_copies":19254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8964,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10617","reported_seq":10744,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:44.030360+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:16:44.030360+0000","last_peered":"2024-09-11T01:16:44.030360+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:16:44.030360+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39018520,"num_objects":9527,"num_object_clones":0,"num_object_copies":19054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9527,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8918,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3353},{"shards":"2","objects":5631}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10602","reported_seq":10672,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:44.032905+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:16:44.032905+0000","last_peered":"2024-09-11T01:16:44.032905+0000","last_clean":"2024-09-11T01:16:44.032905+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:16:44.032905+0000","last_undegraded":"2024-09-11T01:16:44.032905+0000","last_fullsized":"2024-09-11T01:16:44.032905+0000","mapping_epoch":35,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39043072,"num_objects":9532,"num_object_clones":0,"num_object_copies":19064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10602,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10453","reported_seq":10581,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:40.504083+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:16:40.504083+0000","last_peered":"2024-09-11T01:16:40.504083+0000","last_clean":"2024-09-11T01:16:40.504083+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:16:40.504083+0000","last_undegraded":"2024-09-11T01:16:40.504083+0000","last_fullsized":"2024-09-11T01:16:40.504083+0000","mapping_epoch":34,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39141376,"num_objects":9556,"num_object_clones":0,"num_object_copies":19112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9556,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7458,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10554","reported_seq":10724,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.465391+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:16:40.465391+0000","last_peered":"2024-09-11T01:16:40.465391+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:16:40.465391+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39620608,"num_objects":9673,"num_object_clones":0,"num_object_copies":19346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6895,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5722},{"shards":"0,2","objects":3513}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10388","reported_seq":10514,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.421365+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:16:40.421365+0000","last_peered":"2024-09-11T01:16:40.421365+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:16:40.421365+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39084032,"num_objects":9542,"num_object_clones":0,"num_object_copies":19084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7245,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5556},{"shards":"0,2","objects":3566}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10570","reported_seq":10700,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:40.461221+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:16:40.461221+0000","last_peered":"2024-09-11T01:16:40.461221+0000","last_clean":"2024-09-11T01:16:40.461221+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:16:40.461221+0000","last_undegraded":"2024-09-11T01:16:40.461221+0000","last_fullsized":"2024-09-11T01:16:40.461221+0000","mapping_epoch":34,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39665664,"num_objects":9684,"num_object_clones":0,"num_object_copies":19368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9684,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7446,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10523","reported_seq":10651,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:44.010019+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:16:44.010019+0000","last_peered":"2024-09-11T01:16:44.010019+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:16:44.010019+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38707200,"num_objects":9450,"num_object_clones":0,"num_object_copies":18900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8494,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3399},{"shards":"2","objects":5517}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10421","reported_seq":10549,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.537820+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:16:40.537820+0000","last_peered":"2024-09-11T01:16:40.537820+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:16:40.537820+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38694912,"num_objects":9447,"num_object_clones":0,"num_object_copies":18894,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9447,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7008,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5463},{"shards":"0,2","objects":3501}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10660","reported_seq":10776,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:44.038425+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:16:44.038425+0000","last_peered":"2024-09-11T01:16:44.038425+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:16:44.038425+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39104512,"num_objects":9547,"num_object_clones":0,"num_object_copies":19094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8477,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3469},{"shards":"2","objects":5523}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10407","reported_seq":18388,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:44.011065+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:16:44.011065+0000","last_peered":"2024-09-11T01:16:44.011065+0000","last_clean":"2024-09-11T01:16:44.011065+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:16:44.011065+0000","last_undegraded":"2024-09-11T01:16:44.011065+0000","last_fullsized":"2024-09-11T01:16:44.011065+0000","mapping_epoch":44,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38457344,"num_objects":9389,"num_object_clones":0,"num_object_copies":18778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8192,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10573","reported_seq":10705,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.471470+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:16:40.471470+0000","last_peered":"2024-09-11T01:16:40.471470+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:16:40.471470+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39694336,"num_objects":9691,"num_object_clones":0,"num_object_copies":19382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7310,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5741},{"shards":"0,2","objects":3510}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10384","reported_seq":13671,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:40.539382+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:16:40.539382+0000","last_peered":"2024-09-11T01:16:40.539382+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:16:40.539382+0000","last_undegraded":"2024-09-11T01:16:40.539382+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"36'300","ondisk_log_start":"36'300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38912000,"num_objects":9500,"num_object_clones":0,"num_object_copies":19000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2364,"num_objects_unfound":0,"num_objects_dirty":9500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7039,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1575,"num_bytes_recovered":6451200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7136},{"shards":"0,2","objects":2364}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10527","reported_seq":10662,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.461746+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:16:40.461746+0000","last_peered":"2024-09-11T01:16:40.461746+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:16:40.461746+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39522304,"num_objects":9649,"num_object_clones":0,"num_object_copies":19298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7136,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5610},{"shards":"0,2","objects":3602}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10403","reported_seq":10537,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:40.492158+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:16:40.492158+0000","last_peered":"2024-09-11T01:16:40.492158+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:16:40.492158+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39113321,"num_objects":9550,"num_object_clones":0,"num_object_copies":19100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9550,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7473,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5633},{"shards":"0,2","objects":3496}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":626975361,"num_objects":153072,"num_object_clones":0,"num_object_copies":306144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2364,"num_objects_unfound":0,"num_objects_dirty":153072,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":129215,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5503,"num_bytes_recovered":22532096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1295613952,"data_stored":1295525791,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160665,"ondisk_log_size":160665,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182462,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":860280,"kb_used_data":525992,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93511560,"statfs":{"total":96636764160,"available":95755837440,"internally_reserved":0,"allocated":538615808,"data_stored":538356594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":342290091},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,4,2,5,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986235,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":698964,"kb_used_data":432708,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93672876,"statfs":{"total":96636764160,"available":95921025024,"internally_reserved":0,"allocated":443092992,"data_stored":442833783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,0,1,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659180,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":575744,"kb_used_data":309484,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93796096,"statfs":{"total":96636764160,"available":96047202304,"internally_reserved":0,"allocated":316911616,"data_stored":316578750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":272625327},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,1,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":537026560,"data_stored":537023081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":316461056,"data_stored":316383925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":442126336,"data_stored":442118785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:45.636 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:16:45.637 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'10558", 'reported_seq': 10636, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:40.472097+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:16:40.472097+0000', 'last_peered': '2024-09-11T01:16:40.472097+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:16:40.472097+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39763968, 'num_objects': 9708, 'num_object_clones': 0, 'num_object_copies': 19416, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9708, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10558, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5636}, {'shards': '0,2', 'objects': 3647}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:45.637 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:16:45.637 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'10617", 'reported_seq': 10744, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:44.030360+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:16:44.030360+0000', 'last_peered': '2024-09-11T01:16:44.030360+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:16:44.030360+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "36'600", 'ondisk_log_start': "36'600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39018520, 'num_objects': 9527, 'num_object_clones': 0, 'num_object_copies': 19054, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9527, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 8918, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3353}, {'shards': '2', 'objects': 5631}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:45.637 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:16:45.637 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'10554", 'reported_seq': 10724, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:40.465391+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:16:40.465391+0000', 'last_peered': '2024-09-11T01:16:40.465391+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:16:40.465391+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39620608, 'num_objects': 9673, 'num_object_clones': 0, 'num_object_copies': 19346, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9673, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 6895, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5722}, {'shards': '0,2', 'objects': 3513}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:45.637 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:16:45.637 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'10388", 'reported_seq': 10514, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:40.421365+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:16:40.421365+0000', 'last_peered': '2024-09-11T01:16:40.421365+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:16:40.421365+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39084032, 'num_objects': 9542, 'num_object_clones': 0, 'num_object_copies': 19084, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9542, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7245, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5556}, {'shards': '0,2', 'objects': 3566}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:45.637 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:16:45.637 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'10523", 'reported_seq': 10651, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:44.010019+0000', 'last_change': '2024-09-11T01:13:10.159069+0000', 'last_active': '2024-09-11T01:16:44.010019+0000', 'last_peered': '2024-09-11T01:16:44.010019+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:16:44.010019+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38707200, 'num_objects': 9450, 'num_object_clones': 0, 'num_object_copies': 18900, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9450, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8494, 'num_write_kb': 31844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3399}, {'shards': '2', 'objects': 5517}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:45.637 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:16:45.638 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'10421", 'reported_seq': 10549, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:40.537820+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:16:40.537820+0000', 'last_peered': '2024-09-11T01:16:40.537820+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:16:40.537820+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38694912, 'num_objects': 9447, 'num_object_clones': 0, 'num_object_copies': 18894, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9447, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7008, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5463}, {'shards': '0,2', 'objects': 3501}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:45.638 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:16:45.638 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'10660", 'reported_seq': 10776, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:44.038425+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:16:44.038425+0000', 'last_peered': '2024-09-11T01:16:44.038425+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:16:44.038425+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "36'600", 'ondisk_log_start': "36'600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39104512, 'num_objects': 9547, 'num_object_clones': 0, 'num_object_copies': 19094, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9547, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8477, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3469}, {'shards': '2', 'objects': 5523}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:45.638 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:16:45.638 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'10573", 'reported_seq': 10705, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:40.471470+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:16:40.471470+0000', 'last_peered': '2024-09-11T01:16:40.471470+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:16:40.471470+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39694336, 'num_objects': 9691, 'num_object_clones': 0, 'num_object_copies': 19382, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9691, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7310, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5741}, {'shards': '0,2', 'objects': 3510}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:45.638 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:16:45.638 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'10384", 'reported_seq': 13671, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:16:40.539382+0000', 'last_change': '2024-09-11T01:15:19.913288+0000', 'last_active': '2024-09-11T01:16:40.539382+0000', 'last_peered': '2024-09-11T01:16:40.539382+0000', 'last_clean': '2024-09-11T01:13:07.624565+0000', 'last_became_active': '2024-09-11T01:13:10.199795+0000', 'last_became_peered': '2024-09-11T01:13:10.199795+0000', 'last_unstale': '2024-09-11T01:16:40.539382+0000', 'last_undegraded': '2024-09-11T01:16:40.539382+0000', 'last_fullsized': '2024-09-11T01:13:09.652311+0000', 'mapping_epoch': 42, 'log_start': "36'300", 'ondisk_log_start': "36'300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3415", 'last_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'objects_scrubbed': 3337, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1975, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38912000, 'num_objects': 9500, 'num_object_clones': 0, 'num_object_copies': 19000, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2364, 'num_objects_unfound': 0, 'num_objects_dirty': 9500, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7039, 'num_write_kb': 26404, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1575, 'num_bytes_recovered': 6451200, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7136}, {'shards': '0,2', 'objects': 2364}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:45.638 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:16:45.638 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'10527", 'reported_seq': 10662, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:40.461746+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:16:40.461746+0000', 'last_peered': '2024-09-11T01:16:40.461746+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:16:40.461746+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39522304, 'num_objects': 9649, 'num_object_clones': 0, 'num_object_copies': 19298, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9649, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7136, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5610}, {'shards': '0,2', 'objects': 3602}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:45.638 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:16:45.638 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'10403", 'reported_seq': 10537, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:40.492158+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:16:40.492158+0000', 'last_peered': '2024-09-11T01:16:40.492158+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:16:40.492158+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "36'400", 'ondisk_log_start': "36'400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39113321, 'num_objects': 9550, 'num_object_clones': 0, 'num_object_copies': 19100, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9550, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7473, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5633}, {'shards': '0,2', 'objects': 3496}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:45.639 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:16:46.531 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:16:46.531 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39769/306148 objects degraded (12.990%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":343,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":153074,"data_bytes":626975380,"bytes_used":2186227712,"bytes_avail":287724064768,"bytes_total":289910292480,"degraded_objects":39769,"degraded_total":306148,"degraded_ratio":0.12990122424448305,"misplaced_objects":2364,"misplaced_total":306148,"misplaced_ratio":0.0077217554908083673,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":74733,"recovering_keys_per_sec":0,"num_objects_recovered":219,"num_bytes_recovered":897024,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":435},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:16:46.573 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:16:46.573 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:47.382 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:16:47.382 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:16:47.461 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":210,"stamp":"2024-09-11T01:16:46.649406+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":621159060,"num_objects":151654,"num_object_clones":0,"num_object_copies":303308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2233,"num_objects_unfound":0,"num_objects_dirty":151654,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":130637,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5635,"num_bytes_recovered":23068691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160588,"ondisk_log_size":160588,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2132332,"kb_used_data":1261368,"kb_used_omap":12,"kb_used_meta":870899,"kb_avail":280983188,"statfs":{"total":289910292480,"available":287726784512,"internally_reserved":0,"allocated":1291640832,"data_stored":1290789543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":891800580},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,5,3,10,7,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23732224,"num_objects":-5794,"num_object_clones":0,"num_object_copies":-11588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-350,"num_objects_unfound":0,"num_objects_dirty":-5794,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5794,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":350,"num_bytes_recovered":1433600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005449"},"pg_stats":[{"pgid":"2.f","version":"45'10684","reported_seq":10762,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.516185+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:16:45.516185+0000","last_peered":"2024-09-11T01:16:45.516185+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:16:45.516185+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39247872,"num_objects":9582,"num_object_clones":0,"num_object_copies":19164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10684,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5510},{"shards":"0,2","objects":3521}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10725","reported_seq":10827,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:43.844518+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:16:43.844518+0000","last_peered":"2024-09-11T01:16:43.844518+0000","last_clean":"2024-09-11T01:16:43.844518+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:16:43.844518+0000","last_undegraded":"2024-09-11T01:16:43.844518+0000","last_fullsized":"2024-09-11T01:16:43.844518+0000","mapping_epoch":35,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39432192,"num_objects":9627,"num_object_clones":0,"num_object_copies":19254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8964,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10617","reported_seq":10744,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:44.030360+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:16:44.030360+0000","last_peered":"2024-09-11T01:16:44.030360+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:16:44.030360+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39018520,"num_objects":9527,"num_object_clones":0,"num_object_copies":19054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9527,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":8918,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3353},{"shards":"2","objects":5631}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10602","reported_seq":10672,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:44.032905+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:16:44.032905+0000","last_peered":"2024-09-11T01:16:44.032905+0000","last_clean":"2024-09-11T01:16:44.032905+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:16:44.032905+0000","last_undegraded":"2024-09-11T01:16:44.032905+0000","last_fullsized":"2024-09-11T01:16:44.032905+0000","mapping_epoch":35,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39043072,"num_objects":9532,"num_object_clones":0,"num_object_copies":19064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9532,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10602,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10605","reported_seq":10733,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:45.541541+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:16:45.541541+0000","last_peered":"2024-09-11T01:16:45.541541+0000","last_clean":"2024-09-11T01:16:45.541541+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:16:45.541541+0000","last_undegraded":"2024-09-11T01:16:45.541541+0000","last_fullsized":"2024-09-11T01:16:45.541541+0000","mapping_epoch":34,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38518784,"num_objects":9404,"num_object_clones":0,"num_object_copies":18808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7610,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10681","reported_seq":10851,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.536724+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:16:45.536724+0000","last_peered":"2024-09-11T01:16:45.536724+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:16:45.536724+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39100416,"num_objects":9546,"num_object_clones":0,"num_object_copies":19092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7022,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5595},{"shards":"0,2","objects":3386}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10540","reported_seq":10670,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.551620+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:16:45.551620+0000","last_peered":"2024-09-11T01:16:45.551620+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:16:45.551620+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38461440,"num_objects":9390,"num_object_clones":0,"num_object_copies":18780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7397,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5404},{"shards":"0,2","objects":3414}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10716","reported_seq":10846,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:45.481747+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:16:45.481747+0000","last_peered":"2024-09-11T01:16:45.481747+0000","last_clean":"2024-09-11T01:16:45.481747+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:16:45.481747+0000","last_undegraded":"2024-09-11T01:16:45.481747+0000","last_fullsized":"2024-09-11T01:16:45.481747+0000","mapping_epoch":34,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39067648,"num_objects":9538,"num_object_clones":0,"num_object_copies":19076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7592,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10523","reported_seq":10651,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:44.010019+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:16:44.010019+0000","last_peered":"2024-09-11T01:16:44.010019+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:16:44.010019+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38707200,"num_objects":9450,"num_object_clones":0,"num_object_copies":18900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9450,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8494,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3399},{"shards":"2","objects":5517}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10558","reported_seq":10686,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.571229+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:16:45.571229+0000","last_peered":"2024-09-11T01:16:45.571229+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:16:45.571229+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38133760,"num_objects":9310,"num_object_clones":0,"num_object_copies":18620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7145,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5326},{"shards":"0,2","objects":3364}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10660","reported_seq":10776,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:44.038425+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:16:44.038425+0000","last_peered":"2024-09-11T01:16:44.038425+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:16:44.038425+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39104512,"num_objects":9547,"num_object_clones":0,"num_object_copies":19094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9547,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8477,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3469},{"shards":"2","objects":5523}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10407","reported_seq":18388,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:44.011065+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:16:44.011065+0000","last_peered":"2024-09-11T01:16:44.011065+0000","last_clean":"2024-09-11T01:16:44.011065+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:16:44.011065+0000","last_undegraded":"2024-09-11T01:16:44.011065+0000","last_fullsized":"2024-09-11T01:16:44.011065+0000","mapping_epoch":44,"log_start":"36'400","ondisk_log_start":"36'400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38457344,"num_objects":9389,"num_object_clones":0,"num_object_copies":18778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8192,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10714","reported_seq":10845,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.621918+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:16:45.621918+0000","last_peered":"2024-09-11T01:16:45.621918+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:16:45.621918+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39116800,"num_objects":9550,"num_object_clones":0,"num_object_copies":19100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9550,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7451,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5600},{"shards":"0,2","objects":3369}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10512","reported_seq":14062,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:45.598554+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:16:45.598554+0000","last_peered":"2024-09-11T01:16:45.598554+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:16:45.598554+0000","last_undegraded":"2024-09-11T01:16:45.598554+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38387712,"num_objects":9372,"num_object_clones":0,"num_object_copies":18744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2233,"num_objects_unfound":0,"num_objects_dirty":9372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7167,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1706,"num_bytes_recovered":6987776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7139},{"shards":"0,2","objects":2233}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10679","reported_seq":10813,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.621215+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:16:45.621215+0000","last_peered":"2024-09-11T01:16:45.621215+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:16:45.621215+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38899712,"num_objects":9497,"num_object_clones":0,"num_object_copies":18994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9497,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7288,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5458},{"shards":"0,2","objects":3450}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10562","reported_seq":10696,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.585193+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:16:45.585193+0000","last_peered":"2024-09-11T01:16:45.585193+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:16:45.585193+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38462057,"num_objects":9391,"num_object_clones":0,"num_object_copies":18782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7632,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5474},{"shards":"0,2","objects":3337}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":621159041,"num_objects":151652,"num_object_clones":0,"num_object_copies":303304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2233,"num_objects_unfound":0,"num_objects_dirty":151652,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":130635,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5634,"num_bytes_recovered":23068672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1288691712,"data_stored":1288603551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160585,"ondisk_log_size":160585,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182463,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":857624,"kb_used_data":519176,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93514216,"statfs":{"total":96636764160,"available":95758557184,"internally_reserved":0,"allocated":531636224,"data_stored":531377010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":346549931},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,5,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986235,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":698964,"kb_used_data":432708,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93672876,"statfs":{"total":96636764160,"available":95921025024,"internally_reserved":0,"allocated":443092992,"data_stored":442833783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":272625322},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,0,1,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659180,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":575744,"kb_used_data":309484,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93796096,"statfs":{"total":96636764160,"available":96047202304,"internally_reserved":0,"allocated":316911616,"data_stored":316578750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":272625327},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,1,4,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":530104320,"data_stored":530100841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":316461056,"data_stored":316383925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":442126336,"data_stored":442118785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:50.464 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:51.225 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:16:51.241 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:16:51.303 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2024-09-11T01:16:50.650321+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":617763476,"num_objects":150825,"num_object_clones":0,"num_object_copies":301650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2233,"num_objects_unfound":0,"num_objects_dirty":150825,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":131466,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5635,"num_bytes_recovered":23068691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160817,"ondisk_log_size":160817,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2153680,"kb_used_data":1256352,"kb_used_omap":12,"kb_used_meta":897267,"kb_avail":280961840,"statfs":{"total":289910292480,"available":287704924160,"internally_reserved":0,"allocated":1286504448,"data_stored":1285653159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":918801412},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,3,13,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21635072,"num_objects":-5282,"num_object_clones":0,"num_object_copies":-10564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-244,"num_objects_unfound":0,"num_objects_dirty":-5282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5282,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":999424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005318"},"pg_stats":[{"pgid":"2.f","version":"45'10684","reported_seq":10762,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.516185+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:16:45.516185+0000","last_peered":"2024-09-11T01:16:45.516185+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:16:45.516185+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39247872,"num_objects":9582,"num_object_clones":0,"num_object_copies":19164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10684,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5510},{"shards":"0,2","objects":3521}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10868","reported_seq":10970,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:49.037809+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:16:49.037809+0000","last_peered":"2024-09-11T01:16:49.037809+0000","last_clean":"2024-09-11T01:16:49.037809+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:16:49.037809+0000","last_undegraded":"2024-09-11T01:16:49.037809+0000","last_fullsized":"2024-09-11T01:16:49.037809+0000","mapping_epoch":35,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38846464,"num_objects":9484,"num_object_clones":0,"num_object_copies":18968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9107,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10740","reported_seq":10866,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:49.109682+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:16:49.109682+0000","last_peered":"2024-09-11T01:16:49.109682+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:16:49.109682+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38514712,"num_objects":9404,"num_object_clones":0,"num_object_copies":18808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9404,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9041,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3230},{"shards":"2","objects":5508}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10745","reported_seq":10815,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:49.020449+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:16:49.020449+0000","last_peered":"2024-09-11T01:16:49.020449+0000","last_clean":"2024-09-11T01:16:49.020449+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:16:49.020449+0000","last_undegraded":"2024-09-11T01:16:49.020449+0000","last_fullsized":"2024-09-11T01:16:49.020449+0000","mapping_epoch":35,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38457344,"num_objects":9389,"num_object_clones":0,"num_object_copies":18778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10745,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10605","reported_seq":10733,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:45.541541+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:16:45.541541+0000","last_peered":"2024-09-11T01:16:45.541541+0000","last_clean":"2024-09-11T01:16:45.541541+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:16:45.541541+0000","last_undegraded":"2024-09-11T01:16:45.541541+0000","last_fullsized":"2024-09-11T01:16:45.541541+0000","mapping_epoch":34,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38518784,"num_objects":9404,"num_object_clones":0,"num_object_copies":18808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7610,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10681","reported_seq":10851,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.536724+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:16:45.536724+0000","last_peered":"2024-09-11T01:16:45.536724+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:16:45.536724+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39100416,"num_objects":9546,"num_object_clones":0,"num_object_copies":19092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7022,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5595},{"shards":"0,2","objects":3386}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10540","reported_seq":10670,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.551620+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:16:45.551620+0000","last_peered":"2024-09-11T01:16:45.551620+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:16:45.551620+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38461440,"num_objects":9390,"num_object_clones":0,"num_object_copies":18780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7397,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5404},{"shards":"0,2","objects":3414}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10716","reported_seq":10846,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:45.481747+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:16:45.481747+0000","last_peered":"2024-09-11T01:16:45.481747+0000","last_clean":"2024-09-11T01:16:45.481747+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:16:45.481747+0000","last_undegraded":"2024-09-11T01:16:45.481747+0000","last_fullsized":"2024-09-11T01:16:45.481747+0000","mapping_epoch":34,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39067648,"num_objects":9538,"num_object_clones":0,"num_object_copies":19076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7592,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10661","reported_seq":10789,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:49.108267+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:16:49.108267+0000","last_peered":"2024-09-11T01:16:49.108267+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:16:49.108267+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38141952,"num_objects":9312,"num_object_clones":0,"num_object_copies":18624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8632,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3261},{"shards":"2","objects":5379}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10558","reported_seq":10686,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.571229+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:16:45.571229+0000","last_peered":"2024-09-11T01:16:45.571229+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:16:45.571229+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38133760,"num_objects":9310,"num_object_clones":0,"num_object_copies":18620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7145,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5326},{"shards":"0,2","objects":3364}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10791","reported_seq":10910,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:49.073073+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:16:49.073073+0000","last_peered":"2024-09-11T01:16:49.073073+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:16:49.073073+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38567936,"num_objects":9416,"num_object_clones":0,"num_object_copies":18832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8608,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":5392}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10558","reported_seq":18538,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:49.109111+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:16:49.109111+0000","last_peered":"2024-09-11T01:16:49.109111+0000","last_clean":"2024-09-11T01:16:49.109111+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:16:49.109111+0000","last_undegraded":"2024-09-11T01:16:49.109111+0000","last_fullsized":"2024-09-11T01:16:49.109111+0000","mapping_epoch":44,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37838848,"num_objects":9238,"num_object_clones":0,"num_object_copies":18476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8343,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10714","reported_seq":10845,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.621918+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:16:45.621918+0000","last_peered":"2024-09-11T01:16:45.621918+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:16:45.621918+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39116800,"num_objects":9550,"num_object_clones":0,"num_object_copies":19100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9550,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7451,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5600},{"shards":"0,2","objects":3369}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10512","reported_seq":14062,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:45.598554+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:16:45.598554+0000","last_peered":"2024-09-11T01:16:45.598554+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:16:45.598554+0000","last_undegraded":"2024-09-11T01:16:45.598554+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38387712,"num_objects":9372,"num_object_clones":0,"num_object_copies":18744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2233,"num_objects_unfound":0,"num_objects_dirty":9372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7167,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1706,"num_bytes_recovered":6987776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7139},{"shards":"0,2","objects":2233}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10679","reported_seq":10813,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.621215+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:16:45.621215+0000","last_peered":"2024-09-11T01:16:45.621215+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:16:45.621215+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38899712,"num_objects":9497,"num_object_clones":0,"num_object_copies":18994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9497,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7288,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5458},{"shards":"0,2","objects":3450}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10562","reported_seq":10696,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.585193+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:16:45.585193+0000","last_peered":"2024-09-11T01:16:45.585193+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:16:45.585193+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38462057,"num_objects":9391,"num_object_clones":0,"num_object_copies":18782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7632,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5474},{"shards":"0,2","objects":3337}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617763457,"num_objects":150823,"num_object_clones":0,"num_object_copies":301646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2233,"num_objects_unfound":0,"num_objects_dirty":150823,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":131464,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5634,"num_bytes_recovered":23068672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1283383296,"data_stored":1283295135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160814,"ondisk_log_size":160814,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182463,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":857624,"kb_used_data":519176,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93514216,"statfs":{"total":96636764160,"available":95758557184,"internally_reserved":0,"allocated":531636224,"data_stored":531377010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":346549931},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,5,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986236,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":716864,"kb_used_data":428400,"kb_used_omap":4,"kb_used_meta":288443,"kb_avail":93654976,"statfs":{"total":96636764160,"available":95902695424,"internally_reserved":0,"allocated":438681600,"data_stored":438422391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":295366314},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659181,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":579192,"kb_used_data":308776,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93792648,"statfs":{"total":96636764160,"available":96043671552,"internally_reserved":0,"allocated":316186624,"data_stored":315853758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":276885167},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":530104320,"data_stored":530100841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":315731968,"data_stored":315654837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":437547008,"data_stored":437539457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:51.304 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:52.279 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:16:52.281 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:16:52.340 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":212,"stamp":"2024-09-11T01:16:50.650321+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":617763476,"num_objects":150825,"num_object_clones":0,"num_object_copies":301650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2233,"num_objects_unfound":0,"num_objects_dirty":150825,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":131466,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5635,"num_bytes_recovered":23068691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160817,"ondisk_log_size":160817,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2153680,"kb_used_data":1256352,"kb_used_omap":12,"kb_used_meta":897267,"kb_avail":280961840,"statfs":{"total":289910292480,"available":287704924160,"internally_reserved":0,"allocated":1286504448,"data_stored":1285653159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":918801412},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,3,13,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21635072,"num_objects":-5282,"num_object_clones":0,"num_object_copies":-10564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-244,"num_objects_unfound":0,"num_objects_dirty":-5282,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5282,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":244,"num_bytes_recovered":999424,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005318"},"pg_stats":[{"pgid":"2.f","version":"45'10684","reported_seq":10762,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.516185+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:16:45.516185+0000","last_peered":"2024-09-11T01:16:45.516185+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:16:45.516185+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39247872,"num_objects":9582,"num_object_clones":0,"num_object_copies":19164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9582,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10684,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5510},{"shards":"0,2","objects":3521}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10868","reported_seq":10970,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:49.037809+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:16:49.037809+0000","last_peered":"2024-09-11T01:16:49.037809+0000","last_clean":"2024-09-11T01:16:49.037809+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:16:49.037809+0000","last_undegraded":"2024-09-11T01:16:49.037809+0000","last_fullsized":"2024-09-11T01:16:49.037809+0000","mapping_epoch":35,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38846464,"num_objects":9484,"num_object_clones":0,"num_object_copies":18968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9107,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10740","reported_seq":10866,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:49.109682+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:16:49.109682+0000","last_peered":"2024-09-11T01:16:49.109682+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:16:49.109682+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38514712,"num_objects":9404,"num_object_clones":0,"num_object_copies":18808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9404,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9041,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3230},{"shards":"2","objects":5508}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10745","reported_seq":10815,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:49.020449+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:16:49.020449+0000","last_peered":"2024-09-11T01:16:49.020449+0000","last_clean":"2024-09-11T01:16:49.020449+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:16:49.020449+0000","last_undegraded":"2024-09-11T01:16:49.020449+0000","last_fullsized":"2024-09-11T01:16:49.020449+0000","mapping_epoch":35,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38457344,"num_objects":9389,"num_object_clones":0,"num_object_copies":18778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10745,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10605","reported_seq":10733,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:45.541541+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:16:45.541541+0000","last_peered":"2024-09-11T01:16:45.541541+0000","last_clean":"2024-09-11T01:16:45.541541+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:16:45.541541+0000","last_undegraded":"2024-09-11T01:16:45.541541+0000","last_fullsized":"2024-09-11T01:16:45.541541+0000","mapping_epoch":34,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38518784,"num_objects":9404,"num_object_clones":0,"num_object_copies":18808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7610,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10681","reported_seq":10851,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.536724+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:16:45.536724+0000","last_peered":"2024-09-11T01:16:45.536724+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:16:45.536724+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39100416,"num_objects":9546,"num_object_clones":0,"num_object_copies":19092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7022,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5595},{"shards":"0,2","objects":3386}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10540","reported_seq":10670,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.551620+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:16:45.551620+0000","last_peered":"2024-09-11T01:16:45.551620+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:16:45.551620+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38461440,"num_objects":9390,"num_object_clones":0,"num_object_copies":18780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7397,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5404},{"shards":"0,2","objects":3414}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10716","reported_seq":10846,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:45.481747+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:16:45.481747+0000","last_peered":"2024-09-11T01:16:45.481747+0000","last_clean":"2024-09-11T01:16:45.481747+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:16:45.481747+0000","last_undegraded":"2024-09-11T01:16:45.481747+0000","last_fullsized":"2024-09-11T01:16:45.481747+0000","mapping_epoch":34,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39067648,"num_objects":9538,"num_object_clones":0,"num_object_copies":19076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7592,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10661","reported_seq":10789,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:49.108267+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:16:49.108267+0000","last_peered":"2024-09-11T01:16:49.108267+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:16:49.108267+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38141952,"num_objects":9312,"num_object_clones":0,"num_object_copies":18624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8632,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3261},{"shards":"2","objects":5379}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10558","reported_seq":10686,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.571229+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:16:45.571229+0000","last_peered":"2024-09-11T01:16:45.571229+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:16:45.571229+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38133760,"num_objects":9310,"num_object_clones":0,"num_object_copies":18620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7145,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5326},{"shards":"0,2","objects":3364}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10791","reported_seq":10910,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:49.073073+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:16:49.073073+0000","last_peered":"2024-09-11T01:16:49.073073+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:16:49.073073+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38567936,"num_objects":9416,"num_object_clones":0,"num_object_copies":18832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8608,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":5392}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10558","reported_seq":18538,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:49.109111+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:16:49.109111+0000","last_peered":"2024-09-11T01:16:49.109111+0000","last_clean":"2024-09-11T01:16:49.109111+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:16:49.109111+0000","last_undegraded":"2024-09-11T01:16:49.109111+0000","last_fullsized":"2024-09-11T01:16:49.109111+0000","mapping_epoch":44,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37838848,"num_objects":9238,"num_object_clones":0,"num_object_copies":18476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8343,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10714","reported_seq":10845,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.621918+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:16:45.621918+0000","last_peered":"2024-09-11T01:16:45.621918+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:16:45.621918+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":39116800,"num_objects":9550,"num_object_clones":0,"num_object_copies":19100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9550,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7451,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5600},{"shards":"0,2","objects":3369}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10512","reported_seq":14062,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:45.598554+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:16:45.598554+0000","last_peered":"2024-09-11T01:16:45.598554+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:16:45.598554+0000","last_undegraded":"2024-09-11T01:16:45.598554+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38387712,"num_objects":9372,"num_object_clones":0,"num_object_copies":18744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2233,"num_objects_unfound":0,"num_objects_dirty":9372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7167,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1706,"num_bytes_recovered":6987776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7139},{"shards":"0,2","objects":2233}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10679","reported_seq":10813,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.621215+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:16:45.621215+0000","last_peered":"2024-09-11T01:16:45.621215+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:16:45.621215+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38899712,"num_objects":9497,"num_object_clones":0,"num_object_copies":18994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9497,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7288,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5458},{"shards":"0,2","objects":3450}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10562","reported_seq":10696,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:45.585193+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:16:45.585193+0000","last_peered":"2024-09-11T01:16:45.585193+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:16:45.585193+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38462057,"num_objects":9391,"num_object_clones":0,"num_object_copies":18782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7632,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5474},{"shards":"0,2","objects":3337}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617763457,"num_objects":150823,"num_object_clones":0,"num_object_copies":301646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2233,"num_objects_unfound":0,"num_objects_dirty":150823,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":131464,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5634,"num_bytes_recovered":23068672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1283383296,"data_stored":1283295135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160814,"ondisk_log_size":160814,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182463,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":857624,"kb_used_data":519176,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93514216,"statfs":{"total":96636764160,"available":95758557184,"internally_reserved":0,"allocated":531636224,"data_stored":531377010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":346549931},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,0,2,5,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986236,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":716864,"kb_used_data":428400,"kb_used_omap":4,"kb_used_meta":288443,"kb_avail":93654976,"statfs":{"total":96636764160,"available":95902695424,"internally_reserved":0,"allocated":438681600,"data_stored":438422391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":295366314},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659181,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":579192,"kb_used_data":308776,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93792648,"statfs":{"total":96636764160,"available":96043671552,"internally_reserved":0,"allocated":316186624,"data_stored":315853758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":276885167},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":530104320,"data_stored":530100841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":315731968,"data_stored":315654837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":437547008,"data_stored":437539457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:52.341 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:16:52.341 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'10684", 'reported_seq': 10762, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:45.516185+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:16:45.516185+0000', 'last_peered': '2024-09-11T01:16:45.516185+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:16:45.516185+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "36'600", 'ondisk_log_start': "36'600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39247872, 'num_objects': 9582, 'num_object_clones': 0, 'num_object_copies': 19164, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9582, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10684, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5510}, {'shards': '0,2', 'objects': 3521}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:52.341 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:16:52.341 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'10740", 'reported_seq': 10866, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:49.109682+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:16:49.109682+0000', 'last_peered': '2024-09-11T01:16:49.109682+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:16:49.109682+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38514712, 'num_objects': 9404, 'num_object_clones': 0, 'num_object_copies': 18808, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9404, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9041, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3230}, {'shards': '2', 'objects': 5508}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:52.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:16:52.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'10681", 'reported_seq': 10851, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:45.536724+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:16:45.536724+0000', 'last_peered': '2024-09-11T01:16:45.536724+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:16:45.536724+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "36'600", 'ondisk_log_start': "36'600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39100416, 'num_objects': 9546, 'num_object_clones': 0, 'num_object_copies': 19092, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9546, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7022, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5595}, {'shards': '0,2', 'objects': 3386}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:52.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:16:52.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'10540", 'reported_seq': 10670, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:45.551620+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:16:45.551620+0000', 'last_peered': '2024-09-11T01:16:45.551620+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:16:45.551620+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38461440, 'num_objects': 9390, 'num_object_clones': 0, 'num_object_copies': 18780, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9390, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7397, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5404}, {'shards': '0,2', 'objects': 3414}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:52.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:16:52.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'10661", 'reported_seq': 10789, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:49.108267+0000', 'last_change': '2024-09-11T01:13:10.159069+0000', 'last_active': '2024-09-11T01:16:49.108267+0000', 'last_peered': '2024-09-11T01:16:49.108267+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:16:49.108267+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "36'600", 'ondisk_log_start': "36'600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38141952, 'num_objects': 9312, 'num_object_clones': 0, 'num_object_copies': 18624, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9312, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8632, 'num_write_kb': 31844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3261}, {'shards': '2', 'objects': 5379}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:52.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:16:52.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'10558", 'reported_seq': 10686, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:45.571229+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:16:45.571229+0000', 'last_peered': '2024-09-11T01:16:45.571229+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:16:45.571229+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38133760, 'num_objects': 9310, 'num_object_clones': 0, 'num_object_copies': 18620, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9310, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7145, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5326}, {'shards': '0,2', 'objects': 3364}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:52.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:16:52.342 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'10791", 'reported_seq': 10910, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:49.073073+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:16:49.073073+0000', 'last_peered': '2024-09-11T01:16:49.073073+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:16:49.073073+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38567936, 'num_objects': 9416, 'num_object_clones': 0, 'num_object_copies': 18832, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9416, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8608, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3338}, {'shards': '2', 'objects': 5392}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:52.343 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:16:52.343 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'10714", 'reported_seq': 10845, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:45.621918+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:16:45.621918+0000', 'last_peered': '2024-09-11T01:16:45.621918+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:16:45.621918+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 39116800, 'num_objects': 9550, 'num_object_clones': 0, 'num_object_copies': 19100, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9550, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7451, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5600}, {'shards': '0,2', 'objects': 3369}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:52.343 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:16:52.343 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'10512", 'reported_seq': 14062, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:16:45.598554+0000', 'last_change': '2024-09-11T01:15:19.913288+0000', 'last_active': '2024-09-11T01:16:45.598554+0000', 'last_peered': '2024-09-11T01:16:45.598554+0000', 'last_clean': '2024-09-11T01:13:07.624565+0000', 'last_became_active': '2024-09-11T01:13:10.199795+0000', 'last_became_peered': '2024-09-11T01:13:10.199795+0000', 'last_unstale': '2024-09-11T01:16:45.598554+0000', 'last_undegraded': '2024-09-11T01:16:45.598554+0000', 'last_fullsized': '2024-09-11T01:13:09.652311+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3415", 'last_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'objects_scrubbed': 3337, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1975, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38387712, 'num_objects': 9372, 'num_object_clones': 0, 'num_object_copies': 18744, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2233, 'num_objects_unfound': 0, 'num_objects_dirty': 9372, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7167, 'num_write_kb': 26404, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1706, 'num_bytes_recovered': 6987776, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7139}, {'shards': '0,2', 'objects': 2233}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:52.343 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:16:52.343 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'10679", 'reported_seq': 10813, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:45.621215+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:16:45.621215+0000', 'last_peered': '2024-09-11T01:16:45.621215+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:16:45.621215+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "36'600", 'ondisk_log_start': "36'600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38899712, 'num_objects': 9497, 'num_object_clones': 0, 'num_object_copies': 18994, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9497, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7288, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5458}, {'shards': '0,2', 'objects': 3450}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:52.343 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:16:52.343 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'10562", 'reported_seq': 10696, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:45.585193+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:16:45.585193+0000', 'last_peered': '2024-09-11T01:16:45.585193+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:16:45.585193+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "36'500", 'ondisk_log_start': "36'500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38462057, 'num_objects': 9391, 'num_object_clones': 0, 'num_object_copies': 18782, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9391, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7632, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5474}, {'shards': '0,2', 'objects': 3337}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:52.344 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:16:53.370 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:16:53.370 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39769/298856 objects degraded (13.307%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":349,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":149428,"data_bytes":612041364,"bytes_used":2206998528,"bytes_avail":287703293952,"bytes_total":289910292480,"degraded_objects":39769,"degraded_total":298856,"degraded_ratio":0.13307077656128705,"misplaced_objects":2126,"misplaced_total":298856,"misplaced_ratio":0.0071137939342024248,"recovering_objects_per_sec":19,"recovering_bytes_per_sec":81190,"recovering_keys_per_sec":0,"num_objects_recovered":238,"num_bytes_recovered":974848,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":371},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:16:53.425 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:16:53.425 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:54.282 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:16:54.283 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:16:54.326 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":213,"stamp":"2024-09-11T01:16:52.652415+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":612041364,"num_objects":149428,"num_object_clones":0,"num_object_copies":298856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2126,"num_objects_unfound":0,"num_objects_dirty":149428,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":132863,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5742,"num_bytes_recovered":23506963,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160914,"ondisk_log_size":160914,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2155272,"kb_used_data":1249620,"kb_used_omap":12,"kb_used_meta":905587,"kb_avail":280960248,"statfs":{"total":289910292480,"available":287703293952,"internally_reserved":0,"allocated":1279610880,"data_stored":1278759591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":927321092},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,4,2,13,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18247680,"num_objects":-4455,"num_object_clones":0,"num_object_copies":-8910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-238,"num_objects_unfound":0,"num_objects_dirty":-4455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4455,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":238,"num_bytes_recovered":974848,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006957"},"pg_stats":[{"pgid":"2.f","version":"45'10814","reported_seq":10892,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:50.678199+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:16:50.678199+0000","last_peered":"2024-09-11T01:16:50.678199+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:16:50.678199+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38715392,"num_objects":9452,"num_object_clones":0,"num_object_copies":18904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10814,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5380},{"shards":"0,2","objects":3391}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'10868","reported_seq":10970,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:49.037809+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:16:49.037809+0000","last_peered":"2024-09-11T01:16:49.037809+0000","last_clean":"2024-09-11T01:16:49.037809+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:16:49.037809+0000","last_undegraded":"2024-09-11T01:16:49.037809+0000","last_fullsized":"2024-09-11T01:16:49.037809+0000","mapping_epoch":35,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38846464,"num_objects":9484,"num_object_clones":0,"num_object_copies":18968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9107,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10740","reported_seq":10866,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:49.109682+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:16:49.109682+0000","last_peered":"2024-09-11T01:16:49.109682+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:16:49.109682+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38514712,"num_objects":9404,"num_object_clones":0,"num_object_copies":18808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9404,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9041,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3230},{"shards":"2","objects":5508}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10745","reported_seq":10815,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:49.020449+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:16:49.020449+0000","last_peered":"2024-09-11T01:16:49.020449+0000","last_clean":"2024-09-11T01:16:49.020449+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:16:49.020449+0000","last_undegraded":"2024-09-11T01:16:49.020449+0000","last_fullsized":"2024-09-11T01:16:49.020449+0000","mapping_epoch":35,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38457344,"num_objects":9389,"num_object_clones":0,"num_object_copies":18778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10745,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10753","reported_seq":10881,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:50.652777+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:16:50.652777+0000","last_peered":"2024-09-11T01:16:50.652777+0000","last_clean":"2024-09-11T01:16:50.652777+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:16:50.652777+0000","last_undegraded":"2024-09-11T01:16:50.652777+0000","last_fullsized":"2024-09-11T01:16:50.652777+0000","mapping_epoch":34,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37912576,"num_objects":9256,"num_object_clones":0,"num_object_copies":18512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7758,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10807","reported_seq":10977,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:50.597635+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:16:50.597635+0000","last_peered":"2024-09-11T01:16:50.597635+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:16:50.597635+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38584320,"num_objects":9420,"num_object_clones":0,"num_object_copies":18840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7148,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5469},{"shards":"0,2","objects":3260}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10680","reported_seq":10810,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:50.522393+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:16:50.522393+0000","last_peered":"2024-09-11T01:16:50.522393+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:16:50.522393+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37888000,"num_objects":9250,"num_object_clones":0,"num_object_copies":18500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7537,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5264},{"shards":"0,2","objects":3274}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'10865","reported_seq":10995,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:50.540726+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:16:50.540726+0000","last_peered":"2024-09-11T01:16:50.540726+0000","last_clean":"2024-09-11T01:16:50.540726+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:16:50.540726+0000","last_undegraded":"2024-09-11T01:16:50.540726+0000","last_fullsized":"2024-09-11T01:16:50.540726+0000","mapping_epoch":34,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38457344,"num_objects":9389,"num_object_clones":0,"num_object_copies":18778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7741,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10661","reported_seq":10789,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:49.108267+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:16:49.108267+0000","last_peered":"2024-09-11T01:16:49.108267+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:16:49.108267+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38141952,"num_objects":9312,"num_object_clones":0,"num_object_copies":18624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8632,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3261},{"shards":"2","objects":5379}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10699","reported_seq":10826,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:50.557687+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:16:50.557687+0000","last_peered":"2024-09-11T01:16:50.557687+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:16:50.557687+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37556224,"num_objects":9169,"num_object_clones":0,"num_object_copies":18338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7286,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5185},{"shards":"0,2","objects":3223}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10791","reported_seq":10910,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:49.073073+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:16:49.073073+0000","last_peered":"2024-09-11T01:16:49.073073+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:16:49.073073+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38567936,"num_objects":9416,"num_object_clones":0,"num_object_copies":18832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8608,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3338},{"shards":"2","objects":5392}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10558","reported_seq":18538,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:49.109111+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:16:49.109111+0000","last_peered":"2024-09-11T01:16:49.109111+0000","last_clean":"2024-09-11T01:16:49.109111+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:16:49.109111+0000","last_undegraded":"2024-09-11T01:16:49.109111+0000","last_fullsized":"2024-09-11T01:16:49.109111+0000","mapping_epoch":44,"log_start":"36'500","ondisk_log_start":"36'500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37838848,"num_objects":9238,"num_object_clones":0,"num_object_copies":18476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9238,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8343,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'10858","reported_seq":10990,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:50.618005+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:16:50.618005+0000","last_peered":"2024-09-11T01:16:50.618005+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:16:50.618005+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38526976,"num_objects":9406,"num_object_clones":0,"num_object_copies":18812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7595,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5456},{"shards":"0,2","objects":3225}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10666","reported_seq":14430,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:50.702994+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:16:50.702994+0000","last_peered":"2024-09-11T01:16:50.702994+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:16:50.702994+0000","last_undegraded":"2024-09-11T01:16:50.702994+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37756928,"num_objects":9218,"num_object_clones":0,"num_object_copies":18436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2126,"num_objects_unfound":0,"num_objects_dirty":9218,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7321,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1813,"num_bytes_recovered":7426048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7092},{"shards":"0,2","objects":2126}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10814","reported_seq":10949,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:50.664329+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:16:50.664329+0000","last_peered":"2024-09-11T01:16:50.664329+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:16:50.664329+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38346752,"num_objects":9362,"num_object_clones":0,"num_object_copies":18724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9362,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7423,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5323},{"shards":"0,2","objects":3315}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10692","reported_seq":10826,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:50.653307+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:16:50.653307+0000","last_peered":"2024-09-11T01:16:50.653307+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:16:50.653307+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37929577,"num_objects":9261,"num_object_clones":0,"num_object_copies":18522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7762,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5344},{"shards":"0,2","objects":3207}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":612041345,"num_objects":149426,"num_object_clones":0,"num_object_copies":298852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2126,"num_objects_unfound":0,"num_objects_dirty":149426,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":132861,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5741,"num_bytes_recovered":23506944,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1276514304,"data_stored":1276426143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160911,"ondisk_log_size":160911,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182464,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":859216,"kb_used_data":512444,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93512624,"statfs":{"total":96636764160,"available":95756926976,"internally_reserved":0,"allocated":524742656,"data_stored":524483442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355069611},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,5,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986236,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":716864,"kb_used_data":428400,"kb_used_omap":4,"kb_used_meta":288443,"kb_avail":93654976,"statfs":{"total":96636764160,"available":95902695424,"internally_reserved":0,"allocated":438681600,"data_stored":438422391,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":295366314},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659181,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":579192,"kb_used_data":308776,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93792648,"statfs":{"total":96636764160,"available":96043671552,"internally_reserved":0,"allocated":316186624,"data_stored":315853758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":276885167},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":523235328,"data_stored":523231849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":315731968,"data_stored":315654837,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":437547008,"data_stored":437539457,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:57.329 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:58.205 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:16:58.206 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:16:58.262 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":215,"stamp":"2024-09-11T01:16:56.653453+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":603116180,"num_objects":147249,"num_object_clones":0,"num_object_copies":294498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2013,"num_objects_unfound":0,"num_objects_dirty":147249,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":135042,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5855,"num_bytes_recovered":23969811,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160693,"ondisk_log_size":160693,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2139364,"kb_used_data":1237864,"kb_used_omap":12,"kb_used_meta":901427,"kb_avail":280976156,"statfs":{"total":289910292480,"available":287719583744,"internally_reserved":0,"allocated":1267572736,"data_stored":1266721447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":923061252},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,5,4,7,10,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23859200,"num_objects":-5825,"num_object_clones":0,"num_object_copies":-11650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-351,"num_objects_unfound":0,"num_objects_dirty":-5825,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5825,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":351,"num_bytes_recovered":1437696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007034"},"pg_stats":[{"pgid":"2.f","version":"45'10943","reported_seq":11021,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.633812+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:16:55.633812+0000","last_peered":"2024-09-11T01:16:55.633812+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:16:55.633812+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38187008,"num_objects":9323,"num_object_clones":0,"num_object_copies":18646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10943,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5251},{"shards":"0,2","objects":3262}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11007","reported_seq":11109,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:54.124612+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:16:54.124612+0000","last_peered":"2024-09-11T01:16:54.124612+0000","last_clean":"2024-09-11T01:16:54.124612+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:16:54.124612+0000","last_undegraded":"2024-09-11T01:16:54.124612+0000","last_fullsized":"2024-09-11T01:16:54.124612+0000","mapping_epoch":35,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38277120,"num_objects":9345,"num_object_clones":0,"num_object_copies":18690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9246,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10866","reported_seq":10995,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:54.096926+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:16:54.096926+0000","last_peered":"2024-09-11T01:16:54.096926+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:16:54.096926+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37998616,"num_objects":9278,"num_object_clones":0,"num_object_copies":18556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9278,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9167,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3104},{"shards":"2","objects":5382}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10875","reported_seq":10945,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:54.076693+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:16:54.076693+0000","last_peered":"2024-09-11T01:16:54.076693+0000","last_clean":"2024-09-11T01:16:54.076693+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:16:54.076693+0000","last_undegraded":"2024-09-11T01:16:54.076693+0000","last_fullsized":"2024-09-11T01:16:54.076693+0000","mapping_epoch":35,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37924864,"num_objects":9259,"num_object_clones":0,"num_object_copies":18518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10875,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10891","reported_seq":11019,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:55.716357+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:16:55.716357+0000","last_peered":"2024-09-11T01:16:55.716357+0000","last_clean":"2024-09-11T01:16:55.716357+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:16:55.716357+0000","last_undegraded":"2024-09-11T01:16:55.716357+0000","last_fullsized":"2024-09-11T01:16:55.716357+0000","mapping_epoch":34,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37347328,"num_objects":9118,"num_object_clones":0,"num_object_copies":18236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7896,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10941","reported_seq":11111,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.583486+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:16:55.583486+0000","last_peered":"2024-09-11T01:16:55.583486+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:16:55.583486+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38035456,"num_objects":9286,"num_object_clones":0,"num_object_copies":18572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7282,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5335},{"shards":"0,2","objects":3126}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10810","reported_seq":10940,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.648823+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:16:55.648823+0000","last_peered":"2024-09-11T01:16:55.648823+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:16:55.648823+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37355520,"num_objects":9120,"num_object_clones":0,"num_object_copies":18240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7667,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5134},{"shards":"0,2","objects":3144}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11010","reported_seq":11136,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:55.681301+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:16:55.681301+0000","last_peered":"2024-09-11T01:16:55.681301+0000","last_clean":"2024-09-11T01:16:55.681301+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:16:55.681301+0000","last_undegraded":"2024-09-11T01:16:55.681301+0000","last_fullsized":"2024-09-11T01:16:55.681301+0000","mapping_epoch":34,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37863424,"num_objects":9244,"num_object_clones":0,"num_object_copies":18488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9244,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7886,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10773","reported_seq":10901,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:53.911914+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:16:53.911914+0000","last_peered":"2024-09-11T01:16:53.911914+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:16:53.911914+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37683200,"num_objects":9200,"num_object_clones":0,"num_object_copies":18400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8744,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3149},{"shards":"2","objects":5267}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10835","reported_seq":10963,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.611935+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:16:55.611935+0000","last_peered":"2024-09-11T01:16:55.611935+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:16:55.611935+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36999168,"num_objects":9033,"num_object_clones":0,"num_object_copies":18066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7422,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5049},{"shards":"0,2","objects":3087}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10948","reported_seq":11067,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:54.093144+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:16:54.093144+0000","last_peered":"2024-09-11T01:16:54.093144+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:16:54.093144+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37924864,"num_objects":9259,"num_object_clones":0,"num_object_copies":18518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8765,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3181},{"shards":"2","objects":5235}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10689","reported_seq":18670,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:54.050304+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:16:54.050304+0000","last_peered":"2024-09-11T01:16:54.050304+0000","last_clean":"2024-09-11T01:16:54.050304+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:16:54.050304+0000","last_undegraded":"2024-09-11T01:16:54.050304+0000","last_fullsized":"2024-09-11T01:16:54.050304+0000","mapping_epoch":44,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37302272,"num_objects":9107,"num_object_clones":0,"num_object_copies":18214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8474,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11003","reported_seq":11135,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.695003+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:16:55.695003+0000","last_peered":"2024-09-11T01:16:55.695003+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:16:55.695003+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37933056,"num_objects":9261,"num_object_clones":0,"num_object_copies":18522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7740,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5311},{"shards":"0,2","objects":3080}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10810","reported_seq":14800,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:55.722682+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:16:55.722682+0000","last_peered":"2024-09-11T01:16:55.722682+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:16:55.722682+0000","last_undegraded":"2024-09-11T01:16:55.722682+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37167104,"num_objects":9074,"num_object_clones":0,"num_object_copies":18148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2013,"num_objects_unfound":0,"num_objects_dirty":9074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7465,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1926,"num_bytes_recovered":7888896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7061},{"shards":"0,2","objects":2013}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10962","reported_seq":11097,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.668797+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:16:55.668797+0000","last_peered":"2024-09-11T01:16:55.668797+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:16:55.668797+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37740544,"num_objects":9214,"num_object_clones":0,"num_object_copies":18428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7571,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5175},{"shards":"0,2","objects":3167}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10827","reported_seq":10961,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.681179+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:16:55.681179+0000","last_peered":"2024-09-11T01:16:55.681179+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:16:55.681179+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37376617,"num_objects":9126,"num_object_clones":0,"num_object_copies":18252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7897,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5209},{"shards":"0,2","objects":3072}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":603116161,"num_objects":147247,"num_object_clones":0,"num_object_copies":294494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2013,"num_objects_unfound":0,"num_objects_dirty":147247,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":135040,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5854,"num_bytes_recovered":23969792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1264566272,"data_stored":1264478111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160690,"ondisk_log_size":160690,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182465,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":840120,"kb_used_data":505828,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93531720,"statfs":{"total":96636764160,"available":95776481280,"internally_reserved":0,"allocated":517967872,"data_stored":517708658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":342290091},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,6,3,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986237,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":716588,"kb_used_data":423964,"kb_used_omap":4,"kb_used_meta":292603,"kb_avail":93655252,"statfs":{"total":96636764160,"available":95902978048,"internally_reserved":0,"allocated":434139136,"data_stored":433879927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":299626154},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659182,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":582656,"kb_used_data":308072,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93789184,"statfs":{"total":96636764160,"available":96040124416,"internally_reserved":0,"allocated":315465728,"data_stored":315132862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":281145007},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":516415488,"data_stored":516412009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":315047936,"data_stored":314970805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":433102848,"data_stored":433095297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:58.263 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:16:59.172 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:16:59.174 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:16:59.246 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":216,"stamp":"2024-09-11T01:16:58.653915+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":603116180,"num_objects":147249,"num_object_clones":0,"num_object_copies":294498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2013,"num_objects_unfound":0,"num_objects_dirty":147249,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":135042,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5855,"num_bytes_recovered":23969811,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160693,"ondisk_log_size":160693,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2139364,"kb_used_data":1237864,"kb_used_omap":12,"kb_used_meta":901427,"kb_avail":280976156,"statfs":{"total":289910292480,"available":287719583744,"internally_reserved":0,"allocated":1267572736,"data_stored":1266721447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":923061252},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,5,4,7,10,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18042880,"num_objects":-4405,"num_object_clones":0,"num_object_copies":-8810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-220,"num_objects_unfound":0,"num_objects_dirty":-4405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4405,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":220,"num_bytes_recovered":901120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004509"},"pg_stats":[{"pgid":"2.f","version":"45'10943","reported_seq":11021,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.633812+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:16:55.633812+0000","last_peered":"2024-09-11T01:16:55.633812+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:16:55.633812+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38187008,"num_objects":9323,"num_object_clones":0,"num_object_copies":18646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10943,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5251},{"shards":"0,2","objects":3262}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11007","reported_seq":11109,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:54.124612+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:16:54.124612+0000","last_peered":"2024-09-11T01:16:54.124612+0000","last_clean":"2024-09-11T01:16:54.124612+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:16:54.124612+0000","last_undegraded":"2024-09-11T01:16:54.124612+0000","last_fullsized":"2024-09-11T01:16:54.124612+0000","mapping_epoch":35,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38277120,"num_objects":9345,"num_object_clones":0,"num_object_copies":18690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9246,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10866","reported_seq":10995,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:54.096926+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:16:54.096926+0000","last_peered":"2024-09-11T01:16:54.096926+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:16:54.096926+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37998616,"num_objects":9278,"num_object_clones":0,"num_object_copies":18556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9278,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9167,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3104},{"shards":"2","objects":5382}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'10875","reported_seq":10945,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:54.076693+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:16:54.076693+0000","last_peered":"2024-09-11T01:16:54.076693+0000","last_clean":"2024-09-11T01:16:54.076693+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:16:54.076693+0000","last_undegraded":"2024-09-11T01:16:54.076693+0000","last_fullsized":"2024-09-11T01:16:54.076693+0000","mapping_epoch":35,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37924864,"num_objects":9259,"num_object_clones":0,"num_object_copies":18518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10875,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10891","reported_seq":11019,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:55.716357+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:16:55.716357+0000","last_peered":"2024-09-11T01:16:55.716357+0000","last_clean":"2024-09-11T01:16:55.716357+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:16:55.716357+0000","last_undegraded":"2024-09-11T01:16:55.716357+0000","last_fullsized":"2024-09-11T01:16:55.716357+0000","mapping_epoch":34,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37347328,"num_objects":9118,"num_object_clones":0,"num_object_copies":18236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7896,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10941","reported_seq":11111,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.583486+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:16:55.583486+0000","last_peered":"2024-09-11T01:16:55.583486+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:16:55.583486+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38035456,"num_objects":9286,"num_object_clones":0,"num_object_copies":18572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7282,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5335},{"shards":"0,2","objects":3126}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10810","reported_seq":10940,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.648823+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:16:55.648823+0000","last_peered":"2024-09-11T01:16:55.648823+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:16:55.648823+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37355520,"num_objects":9120,"num_object_clones":0,"num_object_copies":18240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7667,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5134},{"shards":"0,2","objects":3144}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11010","reported_seq":11136,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:55.681301+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:16:55.681301+0000","last_peered":"2024-09-11T01:16:55.681301+0000","last_clean":"2024-09-11T01:16:55.681301+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:16:55.681301+0000","last_undegraded":"2024-09-11T01:16:55.681301+0000","last_fullsized":"2024-09-11T01:16:55.681301+0000","mapping_epoch":34,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37863424,"num_objects":9244,"num_object_clones":0,"num_object_copies":18488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9244,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7886,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10773","reported_seq":10901,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:53.911914+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:16:53.911914+0000","last_peered":"2024-09-11T01:16:53.911914+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:16:53.911914+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'700","ondisk_log_start":"36'700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37683200,"num_objects":9200,"num_object_clones":0,"num_object_copies":18400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8744,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3149},{"shards":"2","objects":5267}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10835","reported_seq":10963,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.611935+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:16:55.611935+0000","last_peered":"2024-09-11T01:16:55.611935+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:16:55.611935+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36999168,"num_objects":9033,"num_object_clones":0,"num_object_copies":18066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7422,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5049},{"shards":"0,2","objects":3087}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'10948","reported_seq":11067,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:54.093144+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:16:54.093144+0000","last_peered":"2024-09-11T01:16:54.093144+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:16:54.093144+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37924864,"num_objects":9259,"num_object_clones":0,"num_object_copies":18518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9259,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8765,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3181},{"shards":"2","objects":5235}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10689","reported_seq":18670,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:54.050304+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:16:54.050304+0000","last_peered":"2024-09-11T01:16:54.050304+0000","last_clean":"2024-09-11T01:16:54.050304+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:16:54.050304+0000","last_undegraded":"2024-09-11T01:16:54.050304+0000","last_fullsized":"2024-09-11T01:16:54.050304+0000","mapping_epoch":44,"log_start":"36'600","ondisk_log_start":"36'600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37302272,"num_objects":9107,"num_object_clones":0,"num_object_copies":18214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8474,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11003","reported_seq":11135,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.695003+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:16:55.695003+0000","last_peered":"2024-09-11T01:16:55.695003+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:16:55.695003+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37933056,"num_objects":9261,"num_object_clones":0,"num_object_copies":18522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7740,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5311},{"shards":"0,2","objects":3080}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10810","reported_seq":14800,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:55.722682+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:16:55.722682+0000","last_peered":"2024-09-11T01:16:55.722682+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:16:55.722682+0000","last_undegraded":"2024-09-11T01:16:55.722682+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37167104,"num_objects":9074,"num_object_clones":0,"num_object_copies":18148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2013,"num_objects_unfound":0,"num_objects_dirty":9074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7465,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1926,"num_bytes_recovered":7888896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7061},{"shards":"0,2","objects":2013}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10962","reported_seq":11097,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.668797+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:16:55.668797+0000","last_peered":"2024-09-11T01:16:55.668797+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:16:55.668797+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37740544,"num_objects":9214,"num_object_clones":0,"num_object_copies":18428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7571,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5175},{"shards":"0,2","objects":3167}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10827","reported_seq":10961,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.681179+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:16:55.681179+0000","last_peered":"2024-09-11T01:16:55.681179+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:16:55.681179+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37376617,"num_objects":9126,"num_object_clones":0,"num_object_copies":18252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7897,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5209},{"shards":"0,2","objects":3072}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":603116161,"num_objects":147247,"num_object_clones":0,"num_object_copies":294494,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2013,"num_objects_unfound":0,"num_objects_dirty":147247,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":135040,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5854,"num_bytes_recovered":23969792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1264566272,"data_stored":1264478111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160690,"ondisk_log_size":160690,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182465,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":840120,"kb_used_data":505828,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93531720,"statfs":{"total":96636764160,"available":95776481280,"internally_reserved":0,"allocated":517967872,"data_stored":517708658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":342290091},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,6,3,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986237,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":716588,"kb_used_data":423964,"kb_used_omap":4,"kb_used_meta":292603,"kb_avail":93655252,"statfs":{"total":96636764160,"available":95902978048,"internally_reserved":0,"allocated":434139136,"data_stored":433879927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":299626154},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,6,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659182,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":582656,"kb_used_data":308072,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93789184,"statfs":{"total":96636764160,"available":96040124416,"internally_reserved":0,"allocated":315465728,"data_stored":315132862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":281145007},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,0,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":516415488,"data_stored":516412009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":315047936,"data_stored":314970805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":433102848,"data_stored":433095297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:16:59.247 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:16:59.247 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'10943", 'reported_seq': 11021, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:55.633812+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:16:55.633812+0000', 'last_peered': '2024-09-11T01:16:55.633812+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:16:55.633812+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "36'900", 'ondisk_log_start': "36'900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38187008, 'num_objects': 9323, 'num_object_clones': 0, 'num_object_copies': 18646, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9323, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10943, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5251}, {'shards': '0,2', 'objects': 3262}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:59.248 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:16:59.248 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'10866", 'reported_seq': 10995, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:54.096926+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:16:54.096926+0000', 'last_peered': '2024-09-11T01:16:54.096926+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:16:54.096926+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37998616, 'num_objects': 9278, 'num_object_clones': 0, 'num_object_copies': 18556, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9278, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9167, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3104}, {'shards': '2', 'objects': 5382}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:59.248 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:16:59.248 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'10941", 'reported_seq': 11111, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:55.583486+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:16:55.583486+0000', 'last_peered': '2024-09-11T01:16:55.583486+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:16:55.583486+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "36'900", 'ondisk_log_start': "36'900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 38035456, 'num_objects': 9286, 'num_object_clones': 0, 'num_object_copies': 18572, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9286, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7282, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5335}, {'shards': '0,2', 'objects': 3126}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:59.248 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:16:59.248 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'10810", 'reported_seq': 10940, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:55.648823+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:16:55.648823+0000', 'last_peered': '2024-09-11T01:16:55.648823+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:16:55.648823+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37355520, 'num_objects': 9120, 'num_object_clones': 0, 'num_object_copies': 18240, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9120, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7667, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5134}, {'shards': '0,2', 'objects': 3144}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:59.248 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:16:59.248 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'10773", 'reported_seq': 10901, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:53.911914+0000', 'last_change': '2024-09-11T01:13:10.159069+0000', 'last_active': '2024-09-11T01:16:53.911914+0000', 'last_peered': '2024-09-11T01:16:53.911914+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:16:53.911914+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "36'700", 'ondisk_log_start': "36'700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37683200, 'num_objects': 9200, 'num_object_clones': 0, 'num_object_copies': 18400, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9200, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8744, 'num_write_kb': 31844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3149}, {'shards': '2', 'objects': 5267}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:59.248 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:16:59.248 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'10835", 'reported_seq': 10963, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:55.611935+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:16:55.611935+0000', 'last_peered': '2024-09-11T01:16:55.611935+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:16:55.611935+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36999168, 'num_objects': 9033, 'num_object_clones': 0, 'num_object_copies': 18066, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9033, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7422, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5049}, {'shards': '0,2', 'objects': 3087}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:59.249 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:16:59.249 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'10948", 'reported_seq': 11067, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:54.093144+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:16:54.093144+0000', 'last_peered': '2024-09-11T01:16:54.093144+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:16:54.093144+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "36'900", 'ondisk_log_start': "36'900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37924864, 'num_objects': 9259, 'num_object_clones': 0, 'num_object_copies': 18518, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9259, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8765, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 3181}, {'shards': '2', 'objects': 5235}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:16:59.249 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:16:59.249 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'11003", 'reported_seq': 11135, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:55.695003+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:16:55.695003+0000', 'last_peered': '2024-09-11T01:16:55.695003+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:16:55.695003+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "36'1000", 'ondisk_log_start': "36'1000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37933056, 'num_objects': 9261, 'num_object_clones': 0, 'num_object_copies': 18522, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9261, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7740, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5311}, {'shards': '0,2', 'objects': 3080}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:59.249 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:16:59.249 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'10810", 'reported_seq': 14800, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:16:55.722682+0000', 'last_change': '2024-09-11T01:15:19.913288+0000', 'last_active': '2024-09-11T01:16:55.722682+0000', 'last_peered': '2024-09-11T01:16:55.722682+0000', 'last_clean': '2024-09-11T01:13:07.624565+0000', 'last_became_active': '2024-09-11T01:13:10.199795+0000', 'last_became_peered': '2024-09-11T01:13:10.199795+0000', 'last_unstale': '2024-09-11T01:16:55.722682+0000', 'last_undegraded': '2024-09-11T01:16:55.722682+0000', 'last_fullsized': '2024-09-11T01:13:09.652311+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3415", 'last_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'objects_scrubbed': 3337, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1975, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37167104, 'num_objects': 9074, 'num_object_clones': 0, 'num_object_copies': 18148, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 2013, 'num_objects_unfound': 0, 'num_objects_dirty': 9074, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7465, 'num_write_kb': 26404, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1926, 'num_bytes_recovered': 7888896, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7061}, {'shards': '0,2', 'objects': 2013}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:59.249 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:16:59.249 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'10962", 'reported_seq': 11097, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:55.668797+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:16:55.668797+0000', 'last_peered': '2024-09-11T01:16:55.668797+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:16:55.668797+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "36'900", 'ondisk_log_start': "36'900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37740544, 'num_objects': 9214, 'num_object_clones': 0, 'num_object_copies': 18428, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9214, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7571, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5175}, {'shards': '0,2', 'objects': 3167}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:59.249 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:16:59.249 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'10827", 'reported_seq': 10961, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:16:55.681179+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:16:55.681179+0000', 'last_peered': '2024-09-11T01:16:55.681179+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:16:55.681179+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "36'800", 'ondisk_log_start': "36'800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37376617, 'num_objects': 9126, 'num_object_clones': 0, 'num_object_copies': 18252, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9126, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7897, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5209}, {'shards': '0,2', 'objects': 3072}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:16:59.250 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:17:00.258 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:17:00.258 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39769/294498 objects degraded (13.504%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":356,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":147249,"data_bytes":603116180,"bytes_used":2190708736,"bytes_avail":287719583744,"bytes_total":289910292480,"degraded_objects":39769,"degraded_total":294498,"degraded_ratio":0.13503996631556073,"misplaced_objects":2013,"misplaced_total":294498,"misplaced_ratio":0.0068353605117861581,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":75065,"recovering_keys_per_sec":0,"num_objects_recovered":220,"num_bytes_recovered":901120,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":366},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:17:00.305 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:17:00.305 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:01.108 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:17:01.109 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:17:01.163 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":217,"stamp":"2024-09-11T01:17:00.654358+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":599675540,"num_objects":146409,"num_object_clones":0,"num_object_copies":292818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2013,"num_objects_unfound":0,"num_objects_dirty":146409,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":135882,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5855,"num_bytes_recovered":23969811,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160633,"ondisk_log_size":160633,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2113564,"kb_used_data":1232868,"kb_used_omap":12,"kb_used_meta":880627,"kb_avail":281001956,"statfs":{"total":289910292480,"available":287746002944,"internally_reserved":0,"allocated":1262456832,"data_stored":1261605543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":901762052},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,5,11,7,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21483520,"num_objects":-5245,"num_object_clones":0,"num_object_copies":-10490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-220,"num_objects_unfound":0,"num_objects_dirty":-5245,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5245,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":220,"num_bytes_recovered":901120,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004471"},"pg_stats":[{"pgid":"2.f","version":"45'10943","reported_seq":11021,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.633812+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:16:55.633812+0000","last_peered":"2024-09-11T01:16:55.633812+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:16:55.633812+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38187008,"num_objects":9323,"num_object_clones":0,"num_object_copies":18646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10943,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5251},{"shards":"0,2","objects":3262}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11154","reported_seq":11254,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:59.144888+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:16:59.144888+0000","last_peered":"2024-09-11T01:16:59.144888+0000","last_clean":"2024-09-11T01:16:59.144888+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:16:59.144888+0000","last_undegraded":"2024-09-11T01:16:59.144888+0000","last_fullsized":"2024-09-11T01:16:59.144888+0000","mapping_epoch":35,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37675008,"num_objects":9198,"num_object_clones":0,"num_object_copies":18396,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9198,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9393,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'10993","reported_seq":11122,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:59.070772+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:16:59.070772+0000","last_peered":"2024-09-11T01:16:59.070772+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:16:59.070772+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37478424,"num_objects":9151,"num_object_clones":0,"num_object_copies":18302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9151,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9294,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2977},{"shards":"2","objects":5255}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11024","reported_seq":11094,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:59.126202+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:16:59.126202+0000","last_peered":"2024-09-11T01:16:59.126202+0000","last_clean":"2024-09-11T01:16:59.126202+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:16:59.126202+0000","last_undegraded":"2024-09-11T01:16:59.126202+0000","last_fullsized":"2024-09-11T01:16:59.126202+0000","mapping_epoch":35,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37314560,"num_objects":9110,"num_object_clones":0,"num_object_copies":18220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11024,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'10891","reported_seq":11019,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:55.716357+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:16:55.716357+0000","last_peered":"2024-09-11T01:16:55.716357+0000","last_clean":"2024-09-11T01:16:55.716357+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:16:55.716357+0000","last_undegraded":"2024-09-11T01:16:55.716357+0000","last_fullsized":"2024-09-11T01:16:55.716357+0000","mapping_epoch":34,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37347328,"num_objects":9118,"num_object_clones":0,"num_object_copies":18236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9118,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7896,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'10941","reported_seq":11111,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.583486+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:16:55.583486+0000","last_peered":"2024-09-11T01:16:55.583486+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:16:55.583486+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":38035456,"num_objects":9286,"num_object_clones":0,"num_object_copies":18572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7282,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5335},{"shards":"0,2","objects":3126}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10810","reported_seq":10940,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.648823+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:16:55.648823+0000","last_peered":"2024-09-11T01:16:55.648823+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:16:55.648823+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37355520,"num_objects":9120,"num_object_clones":0,"num_object_copies":18240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7667,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5134},{"shards":"0,2","objects":3144}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11010","reported_seq":11136,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:55.681301+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:16:55.681301+0000","last_peered":"2024-09-11T01:16:55.681301+0000","last_clean":"2024-09-11T01:16:55.681301+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:16:55.681301+0000","last_undegraded":"2024-09-11T01:16:55.681301+0000","last_fullsized":"2024-09-11T01:16:55.681301+0000","mapping_epoch":34,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37863424,"num_objects":9244,"num_object_clones":0,"num_object_copies":18488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9244,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7886,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'10915","reported_seq":11043,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:59.118735+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:16:59.118735+0000","last_peered":"2024-09-11T01:16:59.118735+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:16:59.118735+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37101568,"num_objects":9058,"num_object_clones":0,"num_object_copies":18116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8886,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3007},{"shards":"2","objects":5125}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10835","reported_seq":10963,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.611935+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:16:55.611935+0000","last_peered":"2024-09-11T01:16:55.611935+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:16:55.611935+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36999168,"num_objects":9033,"num_object_clones":0,"num_object_copies":18066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7422,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5049},{"shards":"0,2","objects":3087}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11080","reported_seq":11198,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:59.126914+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:16:59.126914+0000","last_peered":"2024-09-11T01:16:59.126914+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:16:59.126914+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37384192,"num_objects":9127,"num_object_clones":0,"num_object_copies":18254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8897,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":3049},{"shards":"2","objects":5103}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10832","reported_seq":18813,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:16:59.049922+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:16:59.049922+0000","last_peered":"2024-09-11T01:16:59.049922+0000","last_clean":"2024-09-11T01:16:59.049922+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:16:59.049922+0000","last_undegraded":"2024-09-11T01:16:59.049922+0000","last_fullsized":"2024-09-11T01:16:59.049922+0000","mapping_epoch":44,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36716544,"num_objects":8964,"num_object_clones":0,"num_object_copies":17928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8617,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11003","reported_seq":11135,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.695003+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:16:55.695003+0000","last_peered":"2024-09-11T01:16:55.695003+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:16:55.695003+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37933056,"num_objects":9261,"num_object_clones":0,"num_object_copies":18522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7740,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5311},{"shards":"0,2","objects":3080}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10810","reported_seq":14800,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:16:55.722682+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:16:55.722682+0000","last_peered":"2024-09-11T01:16:55.722682+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:16:55.722682+0000","last_undegraded":"2024-09-11T01:16:55.722682+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37167104,"num_objects":9074,"num_object_clones":0,"num_object_copies":18148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":2013,"num_objects_unfound":0,"num_objects_dirty":9074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7465,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1926,"num_bytes_recovered":7888896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7061},{"shards":"0,2","objects":2013}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'10962","reported_seq":11097,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.668797+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:16:55.668797+0000","last_peered":"2024-09-11T01:16:55.668797+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:16:55.668797+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37740544,"num_objects":9214,"num_object_clones":0,"num_object_copies":18428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7571,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5175},{"shards":"0,2","objects":3167}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10827","reported_seq":10961,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:16:55.681179+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:16:55.681179+0000","last_peered":"2024-09-11T01:16:55.681179+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:16:55.681179+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'800","ondisk_log_start":"36'800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37376617,"num_objects":9126,"num_object_clones":0,"num_object_copies":18252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7897,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5209},{"shards":"0,2","objects":3072}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":599675521,"num_objects":146407,"num_object_clones":0,"num_object_copies":292814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":2013,"num_objects_unfound":0,"num_objects_dirty":146407,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":135880,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5854,"num_bytes_recovered":23969792,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1259368448,"data_stored":1259280287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160630,"ondisk_log_size":160630,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182465,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":840120,"kb_used_data":505828,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93531720,"statfs":{"total":96636764160,"available":95776481280,"internally_reserved":0,"allocated":517967872,"data_stored":517708658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":342290091},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,6,3,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986238,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":699616,"kb_used_data":419468,"kb_used_omap":4,"kb_used_meta":280123,"kb_avail":93672224,"statfs":{"total":96636764160,"available":95920357376,"internally_reserved":0,"allocated":429535232,"data_stored":429276023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":286846634},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,1,2,0,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659183,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":573828,"kb_used_data":307572,"kb_used_omap":4,"kb_used_meta":266235,"kb_avail":93798012,"statfs":{"total":96636764160,"available":96049164288,"internally_reserved":0,"allocated":314953728,"data_stored":314620862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":272625327},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":516415488,"data_stored":516412009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":314474496,"data_stored":314397365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":428478464,"data_stored":428470913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:04.165 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:05.079 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:17:05.081 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:17:05.160 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":219,"stamp":"2024-09-11T01:17:04.655223+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590652052,"num_objects":144206,"num_object_clones":0,"num_object_copies":288412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":1885,"num_objects_unfound":0,"num_objects_dirty":144206,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":138085,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5983,"num_bytes_recovered":24494099,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2122860,"kb_used_data":1221364,"kb_used_omap":12,"kb_used_meta":901427,"kb_avail":280992660,"statfs":{"total":289910292480,"available":287736483840,"internally_reserved":0,"allocated":1250676736,"data_stored":1249825447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":923061252},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,5,4,12,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21389312,"num_objects":-5222,"num_object_clones":0,"num_object_copies":-10444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-241,"num_objects_unfound":0,"num_objects_dirty":-5222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5222,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":987136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002807"},"pg_stats":[{"pgid":"2.f","version":"45'11091","reported_seq":11169,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:00.718393+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:17:00.718393+0000","last_peered":"2024-09-11T01:17:00.718393+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:17:00.718393+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37580800,"num_objects":9175,"num_object_clones":0,"num_object_copies":18350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11091,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5103},{"shards":"0,2","objects":3114}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11314","reported_seq":11416,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:04.132941+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:17:04.132941+0000","last_peered":"2024-09-11T01:17:04.132941+0000","last_clean":"2024-09-11T01:17:04.132941+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:17:04.132941+0000","last_undegraded":"2024-09-11T01:17:04.132941+0000","last_fullsized":"2024-09-11T01:17:04.132941+0000","mapping_epoch":35,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37019648,"num_objects":9038,"num_object_clones":0,"num_object_copies":18076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9553,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11145","reported_seq":11274,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:04.091166+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:17:04.091166+0000","last_peered":"2024-09-11T01:17:04.091166+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:17:04.091166+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36855832,"num_objects":8999,"num_object_clones":0,"num_object_copies":17998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8999,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9446,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2825},{"shards":"2","objects":5103}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11161","reported_seq":11231,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:04.104562+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:17:04.104562+0000","last_peered":"2024-09-11T01:17:04.104562+0000","last_clean":"2024-09-11T01:17:04.104562+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:17:04.104562+0000","last_undegraded":"2024-09-11T01:17:04.104562+0000","last_fullsized":"2024-09-11T01:17:04.104562+0000","mapping_epoch":35,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36753408,"num_objects":8973,"num_object_clones":0,"num_object_copies":17946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11161,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11031","reported_seq":11159,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:00.750466+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:17:00.750466+0000","last_peered":"2024-09-11T01:17:00.750466+0000","last_clean":"2024-09-11T01:17:00.750466+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:17:00.750466+0000","last_undegraded":"2024-09-11T01:17:00.750466+0000","last_fullsized":"2024-09-11T01:17:00.750466+0000","mapping_epoch":34,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36773888,"num_objects":8978,"num_object_clones":0,"num_object_copies":17956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8036,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11071","reported_seq":11241,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:00.759141+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:17:00.759141+0000","last_peered":"2024-09-11T01:17:00.759141+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:17:00.759141+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37502976,"num_objects":9156,"num_object_clones":0,"num_object_copies":18312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7412,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5205},{"shards":"0,2","objects":2996}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10977","reported_seq":11106,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:00.641395+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:17:00.641395+0000","last_peered":"2024-09-11T01:17:00.641395+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:17:00.641395+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36671488,"num_objects":8953,"num_object_clones":0,"num_object_copies":17906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7834,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4967},{"shards":"0,2","objects":2977}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11158","reported_seq":11287,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:00.761511+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:17:00.761511+0000","last_peered":"2024-09-11T01:17:00.761511+0000","last_clean":"2024-09-11T01:17:00.761511+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:17:00.761511+0000","last_undegraded":"2024-09-11T01:17:00.761511+0000","last_fullsized":"2024-09-11T01:17:00.761511+0000","mapping_epoch":34,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37257216,"num_objects":9096,"num_object_clones":0,"num_object_copies":18192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8034,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11033","reported_seq":11161,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:04.080800+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:17:04.080800+0000","last_peered":"2024-09-11T01:17:04.080800+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:17:04.080800+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36618240,"num_objects":8940,"num_object_clones":0,"num_object_copies":17880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9004,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2889},{"shards":"2","objects":5007}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10953","reported_seq":11081,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:00.716295+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:17:00.716295+0000","last_peered":"2024-09-11T01:17:00.716295+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:17:00.716295+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36515840,"num_objects":8915,"num_object_clones":0,"num_object_copies":17830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7540,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4931},{"shards":"0,2","objects":2969}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11231","reported_seq":11349,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:04.072556+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:17:04.072556+0000","last_peered":"2024-09-11T01:17:04.072556+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:17:04.072556+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36765696,"num_objects":8976,"num_object_clones":0,"num_object_copies":17952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9048,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2898},{"shards":"2","objects":4952}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10957","reported_seq":18938,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:04.008784+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:17:04.008784+0000","last_peered":"2024-09-11T01:17:04.008784+0000","last_clean":"2024-09-11T01:17:04.008784+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:17:04.008784+0000","last_undegraded":"2024-09-11T01:17:04.008784+0000","last_fullsized":"2024-09-11T01:17:04.008784+0000","mapping_epoch":44,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36204544,"num_objects":8839,"num_object_clones":0,"num_object_copies":17678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8742,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11138","reported_seq":11270,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:00.570767+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:17:00.570767+0000","last_peered":"2024-09-11T01:17:00.570767+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:17:00.570767+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37380096,"num_objects":9126,"num_object_clones":0,"num_object_copies":18252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7875,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5176},{"shards":"0,2","objects":2945}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10930","reported_seq":15176,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:00.736660+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:17:00.736660+0000","last_peered":"2024-09-11T01:17:00.736660+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:17:00.736660+0000","last_undegraded":"2024-09-11T01:17:00.736660+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36675584,"num_objects":8954,"num_object_clones":0,"num_object_copies":17908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1885,"num_objects_unfound":0,"num_objects_dirty":8954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7585,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2054,"num_bytes_recovered":8413184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7069},{"shards":"0,2","objects":1885}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11079","reported_seq":11213,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:00.774009+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:17:00.774009+0000","last_peered":"2024-09-11T01:17:00.774009+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:17:00.774009+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37261312,"num_objects":9097,"num_object_clones":0,"num_object_copies":18194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9097,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7688,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5058},{"shards":"0,2","objects":3050}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10964","reported_seq":11097,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:00.762612+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:17:00.762612+0000","last_peered":"2024-09-11T01:17:00.762612+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:17:00.762612+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36815465,"num_objects":8989,"num_object_clones":0,"num_object_copies":17978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8034,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5072},{"shards":"0,2","objects":2935}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":590652033,"num_objects":144204,"num_object_clones":0,"num_object_copies":288408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":1885,"num_objects_unfound":0,"num_objects_dirty":144204,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":138083,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5982,"num_bytes_recovered":24494080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1247535104,"data_stored":1247446943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160833,"ondisk_log_size":160833,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182466,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":841892,"kb_used_data":499280,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93529948,"statfs":{"total":96636764160,"available":95774666752,"internally_reserved":0,"allocated":511262720,"data_stored":511003506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":350809771},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,2,4,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986239,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":699352,"kb_used_data":415048,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93672488,"statfs":{"total":96636764160,"available":95920627712,"internally_reserved":0,"allocated":425009152,"data_stored":424749943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":291106474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,4,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659184,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":581616,"kb_used_data":307036,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93790224,"statfs":{"total":96636764160,"available":96041189376,"internally_reserved":0,"allocated":314404864,"data_stored":314071998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":281145007},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":509620224,"data_stored":509616745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":313987072,"data_stored":313909941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":423927808,"data_stored":423920257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:05.161 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:05.854 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:17:05.855 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:17:05.909 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":219,"stamp":"2024-09-11T01:17:04.655223+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590652052,"num_objects":144206,"num_object_clones":0,"num_object_copies":288412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":1885,"num_objects_unfound":0,"num_objects_dirty":144206,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":138085,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5983,"num_bytes_recovered":24494099,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2122860,"kb_used_data":1221364,"kb_used_omap":12,"kb_used_meta":901427,"kb_avail":280992660,"statfs":{"total":289910292480,"available":287736483840,"internally_reserved":0,"allocated":1250676736,"data_stored":1249825447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":923061252},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,5,4,12,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21389312,"num_objects":-5222,"num_object_clones":0,"num_object_copies":-10444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-241,"num_objects_unfound":0,"num_objects_dirty":-5222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5222,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":241,"num_bytes_recovered":987136,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002807"},"pg_stats":[{"pgid":"2.f","version":"45'11091","reported_seq":11169,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:00.718393+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:17:00.718393+0000","last_peered":"2024-09-11T01:17:00.718393+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:17:00.718393+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37580800,"num_objects":9175,"num_object_clones":0,"num_object_copies":18350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11091,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5103},{"shards":"0,2","objects":3114}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11314","reported_seq":11416,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:04.132941+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:17:04.132941+0000","last_peered":"2024-09-11T01:17:04.132941+0000","last_clean":"2024-09-11T01:17:04.132941+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:17:04.132941+0000","last_undegraded":"2024-09-11T01:17:04.132941+0000","last_fullsized":"2024-09-11T01:17:04.132941+0000","mapping_epoch":35,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37019648,"num_objects":9038,"num_object_clones":0,"num_object_copies":18076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9553,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11145","reported_seq":11274,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:04.091166+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:17:04.091166+0000","last_peered":"2024-09-11T01:17:04.091166+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:17:04.091166+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36855832,"num_objects":8999,"num_object_clones":0,"num_object_copies":17998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8999,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9446,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2825},{"shards":"2","objects":5103}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11161","reported_seq":11231,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:04.104562+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:17:04.104562+0000","last_peered":"2024-09-11T01:17:04.104562+0000","last_clean":"2024-09-11T01:17:04.104562+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:17:04.104562+0000","last_undegraded":"2024-09-11T01:17:04.104562+0000","last_fullsized":"2024-09-11T01:17:04.104562+0000","mapping_epoch":35,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36753408,"num_objects":8973,"num_object_clones":0,"num_object_copies":17946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11161,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11031","reported_seq":11159,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:00.750466+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:17:00.750466+0000","last_peered":"2024-09-11T01:17:00.750466+0000","last_clean":"2024-09-11T01:17:00.750466+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:17:00.750466+0000","last_undegraded":"2024-09-11T01:17:00.750466+0000","last_fullsized":"2024-09-11T01:17:00.750466+0000","mapping_epoch":34,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36773888,"num_objects":8978,"num_object_clones":0,"num_object_copies":17956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8036,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11071","reported_seq":11241,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:00.759141+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:17:00.759141+0000","last_peered":"2024-09-11T01:17:00.759141+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:17:00.759141+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37502976,"num_objects":9156,"num_object_clones":0,"num_object_copies":18312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9156,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7412,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5205},{"shards":"0,2","objects":2996}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'10977","reported_seq":11106,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:00.641395+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:17:00.641395+0000","last_peered":"2024-09-11T01:17:00.641395+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:17:00.641395+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36671488,"num_objects":8953,"num_object_clones":0,"num_object_copies":17906,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8953,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7834,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4967},{"shards":"0,2","objects":2977}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11158","reported_seq":11287,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:00.761511+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:17:00.761511+0000","last_peered":"2024-09-11T01:17:00.761511+0000","last_clean":"2024-09-11T01:17:00.761511+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:17:00.761511+0000","last_undegraded":"2024-09-11T01:17:00.761511+0000","last_fullsized":"2024-09-11T01:17:00.761511+0000","mapping_epoch":34,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37257216,"num_objects":9096,"num_object_clones":0,"num_object_copies":18192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8034,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11033","reported_seq":11161,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:04.080800+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:17:04.080800+0000","last_peered":"2024-09-11T01:17:04.080800+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:17:04.080800+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36618240,"num_objects":8940,"num_object_clones":0,"num_object_copies":17880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9004,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2889},{"shards":"2","objects":5007}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'10953","reported_seq":11081,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:00.716295+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:17:00.716295+0000","last_peered":"2024-09-11T01:17:00.716295+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:17:00.716295+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36515840,"num_objects":8915,"num_object_clones":0,"num_object_copies":17830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7540,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4931},{"shards":"0,2","objects":2969}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11231","reported_seq":11349,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:04.072556+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:17:04.072556+0000","last_peered":"2024-09-11T01:17:04.072556+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:17:04.072556+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36765696,"num_objects":8976,"num_object_clones":0,"num_object_copies":17952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9048,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2898},{"shards":"2","objects":4952}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10957","reported_seq":18938,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:04.008784+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:17:04.008784+0000","last_peered":"2024-09-11T01:17:04.008784+0000","last_clean":"2024-09-11T01:17:04.008784+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:17:04.008784+0000","last_undegraded":"2024-09-11T01:17:04.008784+0000","last_fullsized":"2024-09-11T01:17:04.008784+0000","mapping_epoch":44,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36204544,"num_objects":8839,"num_object_clones":0,"num_object_copies":17678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8742,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11138","reported_seq":11270,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:00.570767+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:17:00.570767+0000","last_peered":"2024-09-11T01:17:00.570767+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:17:00.570767+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37380096,"num_objects":9126,"num_object_clones":0,"num_object_copies":18252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7875,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5176},{"shards":"0,2","objects":2945}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'10930","reported_seq":15176,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:00.736660+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:17:00.736660+0000","last_peered":"2024-09-11T01:17:00.736660+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:17:00.736660+0000","last_undegraded":"2024-09-11T01:17:00.736660+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36675584,"num_objects":8954,"num_object_clones":0,"num_object_copies":17908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1885,"num_objects_unfound":0,"num_objects_dirty":8954,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7585,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2054,"num_bytes_recovered":8413184,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7069},{"shards":"0,2","objects":1885}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11079","reported_seq":11213,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:00.774009+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:17:00.774009+0000","last_peered":"2024-09-11T01:17:00.774009+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:17:00.774009+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37261312,"num_objects":9097,"num_object_clones":0,"num_object_copies":18194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9097,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7688,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5058},{"shards":"0,2","objects":3050}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'10964","reported_seq":11097,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:00.762612+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:17:00.762612+0000","last_peered":"2024-09-11T01:17:00.762612+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:17:00.762612+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36815465,"num_objects":8989,"num_object_clones":0,"num_object_copies":17978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8989,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8034,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5072},{"shards":"0,2","objects":2935}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":590652033,"num_objects":144204,"num_object_clones":0,"num_object_copies":288408,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":1885,"num_objects_unfound":0,"num_objects_dirty":144204,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":138083,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5982,"num_bytes_recovered":24494080,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1247535104,"data_stored":1247446943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160833,"ondisk_log_size":160833,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182466,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":841892,"kb_used_data":499280,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93529948,"statfs":{"total":96636764160,"available":95774666752,"internally_reserved":0,"allocated":511262720,"data_stored":511003506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":350809771},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,2,4,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986239,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":699352,"kb_used_data":415048,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93672488,"statfs":{"total":96636764160,"available":95920627712,"internally_reserved":0,"allocated":425009152,"data_stored":424749943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":291106474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,4,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659184,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":581616,"kb_used_data":307036,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93790224,"statfs":{"total":96636764160,"available":96041189376,"internally_reserved":0,"allocated":314404864,"data_stored":314071998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":281145007},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":509620224,"data_stored":509616745,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":313987072,"data_stored":313909941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":423927808,"data_stored":423920257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:05.910 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:17:05.910 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'11091", 'reported_seq': 11169, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:00.718393+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:17:00.718393+0000', 'last_peered': '2024-09-11T01:17:00.718393+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:17:00.718393+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "36'1000", 'ondisk_log_start': "36'1000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37580800, 'num_objects': 9175, 'num_object_clones': 0, 'num_object_copies': 18350, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9175, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11091, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5103}, {'shards': '0,2', 'objects': 3114}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:05.910 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:17:05.910 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'11145", 'reported_seq': 11274, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:04.091166+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:17:04.091166+0000', 'last_peered': '2024-09-11T01:17:04.091166+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:17:04.091166+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36855832, 'num_objects': 8999, 'num_object_clones': 0, 'num_object_copies': 17998, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8999, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9446, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2825}, {'shards': '2', 'objects': 5103}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:05.911 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:17:05.911 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'11071", 'reported_seq': 11241, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:00.759141+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:17:00.759141+0000', 'last_peered': '2024-09-11T01:17:00.759141+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:17:00.759141+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "36'1000", 'ondisk_log_start': "36'1000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37502976, 'num_objects': 9156, 'num_object_clones': 0, 'num_object_copies': 18312, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9156, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7412, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5205}, {'shards': '0,2', 'objects': 2996}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:05.911 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:17:05.911 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'10977", 'reported_seq': 11106, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:00.641395+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:17:00.641395+0000', 'last_peered': '2024-09-11T01:17:00.641395+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:17:00.641395+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "36'900", 'ondisk_log_start': "36'900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36671488, 'num_objects': 8953, 'num_object_clones': 0, 'num_object_copies': 17906, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8953, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7834, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4967}, {'shards': '0,2', 'objects': 2977}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:05.911 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:17:05.911 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'11033", 'reported_seq': 11161, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:04.080800+0000', 'last_change': '2024-09-11T01:13:10.159069+0000', 'last_active': '2024-09-11T01:17:04.080800+0000', 'last_peered': '2024-09-11T01:17:04.080800+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:17:04.080800+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "36'1000", 'ondisk_log_start': "36'1000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36618240, 'num_objects': 8940, 'num_object_clones': 0, 'num_object_copies': 17880, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8940, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9004, 'num_write_kb': 31844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2889}, {'shards': '2', 'objects': 5007}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:05.911 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:17:05.911 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'10953", 'reported_seq': 11081, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:00.716295+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:17:00.716295+0000', 'last_peered': '2024-09-11T01:17:00.716295+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:17:00.716295+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "36'900", 'ondisk_log_start': "36'900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36515840, 'num_objects': 8915, 'num_object_clones': 0, 'num_object_copies': 17830, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8915, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7540, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4931}, {'shards': '0,2', 'objects': 2969}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:05.911 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:17:05.911 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'11231", 'reported_seq': 11349, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:04.072556+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:17:04.072556+0000', 'last_peered': '2024-09-11T01:17:04.072556+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:17:04.072556+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "36'1200", 'ondisk_log_start': "36'1200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36765696, 'num_objects': 8976, 'num_object_clones': 0, 'num_object_copies': 17952, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8976, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9048, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2898}, {'shards': '2', 'objects': 4952}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:05.912 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:17:05.912 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'11138", 'reported_seq': 11270, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:00.570767+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:17:00.570767+0000', 'last_peered': '2024-09-11T01:17:00.570767+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:17:00.570767+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37380096, 'num_objects': 9126, 'num_object_clones': 0, 'num_object_copies': 18252, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9126, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7875, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5176}, {'shards': '0,2', 'objects': 2945}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:05.912 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:17:05.912 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'10930", 'reported_seq': 15176, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:17:00.736660+0000', 'last_change': '2024-09-11T01:15:19.913288+0000', 'last_active': '2024-09-11T01:17:00.736660+0000', 'last_peered': '2024-09-11T01:17:00.736660+0000', 'last_clean': '2024-09-11T01:13:07.624565+0000', 'last_became_active': '2024-09-11T01:13:10.199795+0000', 'last_became_peered': '2024-09-11T01:13:10.199795+0000', 'last_unstale': '2024-09-11T01:17:00.736660+0000', 'last_undegraded': '2024-09-11T01:17:00.736660+0000', 'last_fullsized': '2024-09-11T01:13:09.652311+0000', 'mapping_epoch': 42, 'log_start': "36'900", 'ondisk_log_start': "36'900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3415", 'last_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'objects_scrubbed': 3337, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1975, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36675584, 'num_objects': 8954, 'num_object_clones': 0, 'num_object_copies': 17908, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1885, 'num_objects_unfound': 0, 'num_objects_dirty': 8954, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7585, 'num_write_kb': 26404, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2054, 'num_bytes_recovered': 8413184, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7069}, {'shards': '0,2', 'objects': 1885}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:05.912 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:17:05.912 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'11079", 'reported_seq': 11213, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:00.774009+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:17:00.774009+0000', 'last_peered': '2024-09-11T01:17:00.774009+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:17:00.774009+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "36'1000", 'ondisk_log_start': "36'1000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 37261312, 'num_objects': 9097, 'num_object_clones': 0, 'num_object_copies': 18194, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 9097, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7688, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5058}, {'shards': '0,2', 'objects': 3050}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:05.912 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:17:05.912 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'10964", 'reported_seq': 11097, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:00.762612+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:17:00.762612+0000', 'last_peered': '2024-09-11T01:17:00.762612+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:17:00.762612+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "36'900", 'ondisk_log_start': "36'900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36815465, 'num_objects': 8989, 'num_object_clones': 0, 'num_object_copies': 17978, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8989, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8034, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 5072}, {'shards': '0,2', 'objects': 2935}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:05.913 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:17:06.990 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:17:06.990 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39769/285614 objects degraded (13.924%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":363,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":142807,"data_bytes":584921748,"bytes_used":2171154432,"bytes_avail":287739138048,"bytes_total":289910292480,"degraded_objects":39769,"degraded_total":285614,"degraded_ratio":0.13924037337105324,"misplaced_objects":1743,"misplaced_total":285614,"misplaced_ratio":0.0061026420273516003,"recovering_objects_per_sec":32,"recovering_bytes_per_sec":130676,"recovering_keys_per_sec":0,"num_objects_recovered":386,"num_bytes_recovered":1568768,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":485},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:17:07.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:17:07.037 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:07.885 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:17:07.886 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:17:07.970 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2024-09-11T01:17:06.657972+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":584921748,"num_objects":142807,"num_object_clones":0,"num_object_copies":285614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":1743,"num_objects_unfound":0,"num_objects_dirty":142807,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":139484,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6128,"num_bytes_recovered":25075731,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160735,"ondisk_log_size":160735,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2120268,"kb_used_data":1214612,"kb_used_omap":12,"kb_used_meta":905587,"kb_avail":280995252,"statfs":{"total":289910292480,"available":287739138048,"internally_reserved":0,"allocated":1243762688,"data_stored":1242911399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":927321092},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,5,12,5,3,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23863296,"num_objects":-5826,"num_object_clones":0,"num_object_copies":-11652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-383,"num_objects_unfound":0,"num_objects_dirty":-5826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5826,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":386,"num_bytes_recovered":1568768,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004979"},"pg_stats":[{"pgid":"2.f","version":"45'11235","reported_seq":11310,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:05.750905+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:17:05.750905+0000","last_peered":"2024-09-11T01:17:05.750905+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:17:05.750905+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36990976,"num_objects":9031,"num_object_clones":0,"num_object_copies":18062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11235,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4959},{"shards":"0,2","objects":2970}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11314","reported_seq":11416,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:04.132941+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:17:04.132941+0000","last_peered":"2024-09-11T01:17:04.132941+0000","last_clean":"2024-09-11T01:17:04.132941+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:17:04.132941+0000","last_undegraded":"2024-09-11T01:17:04.132941+0000","last_fullsized":"2024-09-11T01:17:04.132941+0000","mapping_epoch":35,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":37019648,"num_objects":9038,"num_object_clones":0,"num_object_copies":18076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9038,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9553,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11145","reported_seq":11274,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:04.091166+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:17:04.091166+0000","last_peered":"2024-09-11T01:17:04.091166+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:17:04.091166+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36855832,"num_objects":8999,"num_object_clones":0,"num_object_copies":17998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8999,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9446,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2825},{"shards":"2","objects":5103}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11161","reported_seq":11231,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:04.104562+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:17:04.104562+0000","last_peered":"2024-09-11T01:17:04.104562+0000","last_clean":"2024-09-11T01:17:04.104562+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:17:04.104562+0000","last_undegraded":"2024-09-11T01:17:04.104562+0000","last_fullsized":"2024-09-11T01:17:04.104562+0000","mapping_epoch":35,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36753408,"num_objects":8973,"num_object_clones":0,"num_object_copies":17946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11161,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11164","reported_seq":11292,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:05.775176+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:17:05.775176+0000","last_peered":"2024-09-11T01:17:05.775176+0000","last_clean":"2024-09-11T01:17:05.775176+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:17:05.775176+0000","last_undegraded":"2024-09-11T01:17:05.775176+0000","last_fullsized":"2024-09-11T01:17:05.775176+0000","mapping_epoch":34,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36229120,"num_objects":8845,"num_object_clones":0,"num_object_copies":17690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8169,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11213","reported_seq":11383,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:05.784689+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:17:05.784689+0000","last_peered":"2024-09-11T01:17:05.784689+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:17:05.784689+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36921344,"num_objects":9014,"num_object_clones":0,"num_object_copies":18028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7554,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5063},{"shards":"0,2","objects":2854}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11130","reported_seq":11260,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:05.771079+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:17:05.771079+0000","last_peered":"2024-09-11T01:17:05.771079+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:17:05.771079+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36044800,"num_objects":8800,"num_object_clones":0,"num_object_copies":17600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8800,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7987,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4814},{"shards":"0,2","objects":2824}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11298","reported_seq":11428,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:05.750155+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:17:05.750155+0000","last_peered":"2024-09-11T01:17:05.750155+0000","last_clean":"2024-09-11T01:17:05.750155+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:17:05.750155+0000","last_undegraded":"2024-09-11T01:17:05.750155+0000","last_fullsized":"2024-09-11T01:17:05.750155+0000","mapping_epoch":34,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36683776,"num_objects":8956,"num_object_clones":0,"num_object_copies":17912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8956,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8174,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683685+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683685+0000","last_peered":"2024-09-11T01:15:21.683685+0000","last_clean":"2024-09-11T01:15:21.683685+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683685+0000","last_undegraded":"2024-09-11T01:15:21.683685+0000","last_fullsized":"2024-09-11T01:15:21.683685+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11033","reported_seq":11161,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:04.080800+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:17:04.080800+0000","last_peered":"2024-09-11T01:17:04.080800+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:17:04.080800+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36618240,"num_objects":8940,"num_object_clones":0,"num_object_copies":17880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9004,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2889},{"shards":"2","objects":5007}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11069","reported_seq":11197,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:05.719045+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:17:05.719045+0000","last_peered":"2024-09-11T01:17:05.719045+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:17:05.719045+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36040704,"num_objects":8799,"num_object_clones":0,"num_object_copies":17598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8799,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7656,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4815},{"shards":"0,2","objects":2853}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11231","reported_seq":11349,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:04.072556+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:17:04.072556+0000","last_peered":"2024-09-11T01:17:04.072556+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:17:04.072556+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36765696,"num_objects":8976,"num_object_clones":0,"num_object_copies":17952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8976,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9048,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2898},{"shards":"2","objects":4952}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'10957","reported_seq":18938,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:04.008784+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:17:04.008784+0000","last_peered":"2024-09-11T01:17:04.008784+0000","last_clean":"2024-09-11T01:17:04.008784+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:17:04.008784+0000","last_undegraded":"2024-09-11T01:17:04.008784+0000","last_fullsized":"2024-09-11T01:17:04.008784+0000","mapping_epoch":44,"log_start":"36'900","ondisk_log_start":"36'900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36204544,"num_objects":8839,"num_object_clones":0,"num_object_copies":17678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8839,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8742,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11296","reported_seq":11424,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:05.765211+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:17:05.765211+0000","last_peered":"2024-09-11T01:17:05.765211+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:17:05.765211+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36732928,"num_objects":8968,"num_object_clones":0,"num_object_copies":17936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8033,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5018},{"shards":"0,2","objects":2787}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11062","reported_seq":15598,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:05.810139+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:17:05.810139+0000","last_peered":"2024-09-11T01:17:05.810139+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:17:05.810139+0000","last_undegraded":"2024-09-11T01:17:05.810139+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36134912,"num_objects":8822,"num_object_clones":0,"num_object_copies":17644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1743,"num_objects_unfound":0,"num_objects_dirty":8822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7717,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2199,"num_bytes_recovered":8994816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7079},{"shards":"0,2","objects":1743}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11217","reported_seq":11351,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:05.733054+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:17:05.733054+0000","last_peered":"2024-09-11T01:17:05.733054+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:17:05.733054+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36696064,"num_objects":8959,"num_object_clones":0,"num_object_copies":17918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8959,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7826,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4920},{"shards":"0,2","objects":2912}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11107","reported_seq":11241,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:05.716235+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:17:05.716235+0000","last_peered":"2024-09-11T01:17:05.716235+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:17:05.716235+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36229737,"num_objects":8846,"num_object_clones":0,"num_object_copies":17692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8177,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4929},{"shards":"0,2","objects":2792}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":584921729,"num_objects":142805,"num_object_clones":0,"num_object_copies":285610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":1743,"num_objects_unfound":0,"num_objects_dirty":142805,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":139482,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6127,"num_bytes_recovered":25075712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1240686592,"data_stored":1240598431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160732,"ondisk_log_size":160732,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182467,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":839300,"kb_used_data":492528,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93532540,"statfs":{"total":96636764160,"available":95777320960,"internally_reserved":0,"allocated":504348672,"data_stored":504089458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355069611},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,4,1,2,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986239,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":699352,"kb_used_data":415048,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93672488,"statfs":{"total":96636764160,"available":95920627712,"internally_reserved":0,"allocated":425009152,"data_stored":424749943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":291106474},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,4,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659184,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":581616,"kb_used_data":307036,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93790224,"statfs":{"total":96636764160,"available":96041189376,"internally_reserved":0,"allocated":314404864,"data_stored":314071998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":281145007},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":502771712,"data_stored":502768233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":313987072,"data_stored":313909941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":423927808,"data_stored":423920257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:10.973 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:11.869 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:17:11.872 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:17:11.921 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":222,"stamp":"2024-09-11T01:17:10.658891+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":581735060,"num_objects":142029,"num_object_clones":0,"num_object_copies":284058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":1743,"num_objects_unfound":0,"num_objects_dirty":142029,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":140262,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6128,"num_bytes_recovered":25075731,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160913,"ondisk_log_size":160913,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2107380,"kb_used_data":1210048,"kb_used_omap":12,"kb_used_meta":897267,"kb_avail":281008140,"statfs":{"total":289910292480,"available":287752335360,"internally_reserved":0,"allocated":1239089152,"data_stored":1238248300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":918801412},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,7,7,8,5,2,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21381120,"num_objects":-5220,"num_object_clones":0,"num_object_copies":-10440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-270,"num_objects_unfound":0,"num_objects_dirty":-5220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5220,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":273,"num_bytes_recovered":1105920,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004975"},"pg_stats":[{"pgid":"2.f","version":"45'11235","reported_seq":11310,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:05.750905+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:17:05.750905+0000","last_peered":"2024-09-11T01:17:05.750905+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:17:05.750905+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36990976,"num_objects":9031,"num_object_clones":0,"num_object_copies":18062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11235,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4959},{"shards":"0,2","objects":2970}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11465","reported_seq":11566,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:09.153234+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:17:09.153234+0000","last_peered":"2024-09-11T01:17:09.153234+0000","last_clean":"2024-09-11T01:17:09.153234+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:17:09.153234+0000","last_undegraded":"2024-09-11T01:17:09.153234+0000","last_fullsized":"2024-09-11T01:17:09.153234+0000","mapping_epoch":35,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36401152,"num_objects":8887,"num_object_clones":0,"num_object_copies":17774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9704,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11281","reported_seq":11410,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:09.115837+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:17:09.115837+0000","last_peered":"2024-09-11T01:17:09.115837+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:17:09.115837+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36298776,"num_objects":8863,"num_object_clones":0,"num_object_copies":17726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8863,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9582,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2689},{"shards":"2","objects":4967}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11276","reported_seq":11346,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:09.103374+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:17:09.103374+0000","last_peered":"2024-09-11T01:17:09.103374+0000","last_clean":"2024-09-11T01:17:09.103374+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:17:09.103374+0000","last_undegraded":"2024-09-11T01:17:09.103374+0000","last_fullsized":"2024-09-11T01:17:09.103374+0000","mapping_epoch":35,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36282368,"num_objects":8858,"num_object_clones":0,"num_object_copies":17716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11276,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11164","reported_seq":11292,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:05.775176+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:17:05.775176+0000","last_peered":"2024-09-11T01:17:05.775176+0000","last_clean":"2024-09-11T01:17:05.775176+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:17:05.775176+0000","last_undegraded":"2024-09-11T01:17:05.775176+0000","last_fullsized":"2024-09-11T01:17:05.775176+0000","mapping_epoch":34,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36229120,"num_objects":8845,"num_object_clones":0,"num_object_copies":17690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8169,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11213","reported_seq":11383,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:05.784689+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:17:05.784689+0000","last_peered":"2024-09-11T01:17:05.784689+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:17:05.784689+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36921344,"num_objects":9014,"num_object_clones":0,"num_object_copies":18028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7554,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5063},{"shards":"0,2","objects":2854}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11130","reported_seq":11260,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:05.771079+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:17:05.771079+0000","last_peered":"2024-09-11T01:17:05.771079+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:17:05.771079+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36044800,"num_objects":8800,"num_object_clones":0,"num_object_copies":17600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8800,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7987,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4814},{"shards":"0,2","objects":2824}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11298","reported_seq":11428,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:05.750155+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:17:05.750155+0000","last_peered":"2024-09-11T01:17:05.750155+0000","last_clean":"2024-09-11T01:17:05.750155+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:17:05.750155+0000","last_undegraded":"2024-09-11T01:17:05.750155+0000","last_fullsized":"2024-09-11T01:17:05.750155+0000","mapping_epoch":34,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36683776,"num_objects":8956,"num_object_clones":0,"num_object_copies":17912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8956,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8174,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683747+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683747+0000","last_peered":"2024-09-11T01:15:21.683747+0000","last_clean":"2024-09-11T01:15:21.683747+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683747+0000","last_undegraded":"2024-09-11T01:15:21.683747+0000","last_fullsized":"2024-09-11T01:15:21.683747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11147","reported_seq":11275,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:08.938676+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:17:08.938676+0000","last_peered":"2024-09-11T01:17:08.938676+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:17:08.938676+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36151296,"num_objects":8826,"num_object_clones":0,"num_object_copies":17652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9118,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2775},{"shards":"2","objects":4893}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11069","reported_seq":11197,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:05.719045+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:17:05.719045+0000","last_peered":"2024-09-11T01:17:05.719045+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:17:05.719045+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36040704,"num_objects":8799,"num_object_clones":0,"num_object_copies":17598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8799,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7656,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4815},{"shards":"0,2","objects":2853}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11362","reported_seq":11481,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:09.135788+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:17:09.135788+0000","last_peered":"2024-09-11T01:17:09.135788+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:17:09.135788+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36229120,"num_objects":8845,"num_object_clones":0,"num_object_copies":17690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9179,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2767},{"shards":"2","objects":4821}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11088","reported_seq":19069,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:09.136186+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:17:09.136186+0000","last_peered":"2024-09-11T01:17:09.136186+0000","last_clean":"2024-09-11T01:17:09.136186+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:17:09.136186+0000","last_undegraded":"2024-09-11T01:17:09.136186+0000","last_fullsized":"2024-09-11T01:17:09.136186+0000","mapping_epoch":44,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35667968,"num_objects":8708,"num_object_clones":0,"num_object_copies":17416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8873,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11296","reported_seq":11424,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:05.765211+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:17:05.765211+0000","last_peered":"2024-09-11T01:17:05.765211+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:17:05.765211+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36732928,"num_objects":8968,"num_object_clones":0,"num_object_copies":17936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8968,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8033,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":5018},{"shards":"0,2","objects":2787}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11062","reported_seq":15598,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:05.810139+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:17:05.810139+0000","last_peered":"2024-09-11T01:17:05.810139+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:17:05.810139+0000","last_undegraded":"2024-09-11T01:17:05.810139+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36134912,"num_objects":8822,"num_object_clones":0,"num_object_copies":17644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1743,"num_objects_unfound":0,"num_objects_dirty":8822,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7717,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2199,"num_bytes_recovered":8994816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7079},{"shards":"0,2","objects":1743}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11217","reported_seq":11351,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:05.733054+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:17:05.733054+0000","last_peered":"2024-09-11T01:17:05.733054+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:17:05.733054+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36696064,"num_objects":8959,"num_object_clones":0,"num_object_copies":17918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8959,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7826,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4920},{"shards":"0,2","objects":2912}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11107","reported_seq":11241,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:05.716235+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:17:05.716235+0000","last_peered":"2024-09-11T01:17:05.716235+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:17:05.716235+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36229737,"num_objects":8846,"num_object_clones":0,"num_object_copies":17692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8846,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8177,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4929},{"shards":"0,2","objects":2792}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":581735041,"num_objects":142027,"num_object_clones":0,"num_object_copies":284054,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":1743,"num_objects_unfound":0,"num_objects_dirty":142027,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":140260,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6127,"num_bytes_recovered":25075712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1235865600,"data_stored":1235787876,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160910,"ondisk_log_size":160910,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182467,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":839300,"kb_used_data":492528,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93532540,"statfs":{"total":96636764160,"available":95777320960,"internally_reserved":0,"allocated":504348672,"data_stored":504089458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":355069611},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,4,1,2,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986240,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":699396,"kb_used_data":410932,"kb_used_omap":4,"kb_used_meta":288443,"kb_avail":93672444,"statfs":{"total":96636764160,"available":95920582656,"internally_reserved":0,"allocated":420794368,"data_stored":420535159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":295366314},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659185,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":568684,"kb_used_data":306588,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93803156,"statfs":{"total":96636764160,"available":96054431744,"internally_reserved":0,"allocated":313946112,"data_stored":313623683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":268365487},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,3,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":502771712,"data_stored":502768233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":313462784,"data_stored":313396090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":419631104,"data_stored":419623553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:11.922 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:12.715 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:17:12.722 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:17:12.779 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":223,"stamp":"2024-09-11T01:17:12.659362+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":576025236,"num_objects":140635,"num_object_clones":0,"num_object_copies":281270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":1606,"num_objects_unfound":0,"num_objects_dirty":140635,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":141656,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6265,"num_bytes_recovered":25636883,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161007,"ondisk_log_size":161007,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2092256,"kb_used_data":1203248,"kb_used_omap":12,"kb_used_meta":888947,"kb_avail":281023264,"statfs":{"total":289910292480,"available":287767822336,"internally_reserved":0,"allocated":1232125952,"data_stored":1231285100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":910281732},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,8,5,7,10,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23650304,"num_objects":-5774,"num_object_clones":0,"num_object_copies":-11548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-407,"num_objects_unfound":0,"num_objects_dirty":-5774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5774,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":410,"num_bytes_recovered":1667072,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005003"},"pg_stats":[{"pgid":"2.f","version":"45'11396","reported_seq":11474,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:10.839124+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:17:10.839124+0000","last_peered":"2024-09-11T01:17:10.839124+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:17:10.839124+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36331520,"num_objects":8870,"num_object_clones":0,"num_object_copies":17740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11396,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4798},{"shards":"0,2","objects":2809}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11465","reported_seq":11566,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:09.153234+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:17:09.153234+0000","last_peered":"2024-09-11T01:17:09.153234+0000","last_clean":"2024-09-11T01:17:09.153234+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:17:09.153234+0000","last_undegraded":"2024-09-11T01:17:09.153234+0000","last_fullsized":"2024-09-11T01:17:09.153234+0000","mapping_epoch":35,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36401152,"num_objects":8887,"num_object_clones":0,"num_object_copies":17774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9704,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11281","reported_seq":11410,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:09.115837+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:17:09.115837+0000","last_peered":"2024-09-11T01:17:09.115837+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:17:09.115837+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36298776,"num_objects":8863,"num_object_clones":0,"num_object_copies":17726,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8863,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9582,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2689},{"shards":"2","objects":4967}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11276","reported_seq":11346,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:09.103374+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:17:09.103374+0000","last_peered":"2024-09-11T01:17:09.103374+0000","last_clean":"2024-09-11T01:17:09.103374+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:17:09.103374+0000","last_undegraded":"2024-09-11T01:17:09.103374+0000","last_fullsized":"2024-09-11T01:17:09.103374+0000","mapping_epoch":35,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36282368,"num_objects":8858,"num_object_clones":0,"num_object_copies":17716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8858,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11276,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11294","reported_seq":11422,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:10.839471+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:17:10.839471+0000","last_peered":"2024-09-11T01:17:10.839471+0000","last_clean":"2024-09-11T01:17:10.839471+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:17:10.839471+0000","last_undegraded":"2024-09-11T01:17:10.839471+0000","last_fullsized":"2024-09-11T01:17:10.839471+0000","mapping_epoch":34,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35696640,"num_objects":8715,"num_object_clones":0,"num_object_copies":17430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8299,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11374","reported_seq":11542,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:10.852006+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:17:10.852006+0000","last_peered":"2024-09-11T01:17:10.852006+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:17:10.852006+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36261888,"num_objects":8853,"num_object_clones":0,"num_object_copies":17706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7715,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4902},{"shards":"0,2","objects":2693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11261","reported_seq":11390,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:10.821675+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:17:10.821675+0000","last_peered":"2024-09-11T01:17:10.821675+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:17:10.821675+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35508224,"num_objects":8669,"num_object_clones":0,"num_object_copies":17338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8118,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4683},{"shards":"0,2","objects":2693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11428","reported_seq":11558,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:10.838822+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:17:10.838822+0000","last_peered":"2024-09-11T01:17:10.838822+0000","last_clean":"2024-09-11T01:17:10.838822+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:17:10.838822+0000","last_undegraded":"2024-09-11T01:17:10.838822+0000","last_fullsized":"2024-09-11T01:17:10.838822+0000","mapping_epoch":34,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36151296,"num_objects":8826,"num_object_clones":0,"num_object_copies":17652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8304,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683747+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683747+0000","last_peered":"2024-09-11T01:15:21.683747+0000","last_clean":"2024-09-11T01:15:21.683747+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683747+0000","last_undegraded":"2024-09-11T01:15:21.683747+0000","last_fullsized":"2024-09-11T01:15:21.683747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11147","reported_seq":11275,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:08.938676+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:17:08.938676+0000","last_peered":"2024-09-11T01:17:08.938676+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:17:08.938676+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36151296,"num_objects":8826,"num_object_clones":0,"num_object_copies":17652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9118,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2775},{"shards":"2","objects":4893}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11204","reported_seq":11332,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:10.858701+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:17:10.858701+0000","last_peered":"2024-09-11T01:17:10.858701+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:17:10.858701+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35487744,"num_objects":8664,"num_object_clones":0,"num_object_copies":17328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7791,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4680},{"shards":"0,2","objects":2718}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11362","reported_seq":11481,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:09.135788+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:17:09.135788+0000","last_peered":"2024-09-11T01:17:09.135788+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:17:09.135788+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36229120,"num_objects":8845,"num_object_clones":0,"num_object_copies":17690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9179,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2767},{"shards":"2","objects":4821}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11088","reported_seq":19069,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:09.136186+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:17:09.136186+0000","last_peered":"2024-09-11T01:17:09.136186+0000","last_clean":"2024-09-11T01:17:09.136186+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:17:09.136186+0000","last_undegraded":"2024-09-11T01:17:09.136186+0000","last_fullsized":"2024-09-11T01:17:09.136186+0000","mapping_epoch":44,"log_start":"36'1000","ondisk_log_start":"36'1000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35667968,"num_objects":8708,"num_object_clones":0,"num_object_copies":17416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8873,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11441","reported_seq":11573,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:10.669637+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:17:10.669637+0000","last_peered":"2024-09-11T01:17:10.669637+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:17:10.669637+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36139008,"num_objects":8823,"num_object_clones":0,"num_object_copies":17646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8178,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4873},{"shards":"0,2","objects":2642}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11194","reported_seq":16004,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:10.870451+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:17:10.870451+0000","last_peered":"2024-09-11T01:17:10.870451+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:17:10.870451+0000","last_undegraded":"2024-09-11T01:17:10.870451+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35594240,"num_objects":8690,"num_object_clones":0,"num_object_copies":17380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1606,"num_objects_unfound":0,"num_objects_dirty":8690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7849,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2336,"num_bytes_recovered":9555968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7084},{"shards":"0,2","objects":1606}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11365","reported_seq":11498,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:10.815355+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:17:10.815355+0000","last_peered":"2024-09-11T01:17:10.815355+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:17:10.815355+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36089856,"num_objects":8811,"num_object_clones":0,"num_object_copies":17622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7974,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4772},{"shards":"0,2","objects":2764}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11228","reported_seq":11362,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:10.744118+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:17:10.744118+0000","last_peered":"2024-09-11T01:17:10.744118+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:17:10.744118+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35734121,"num_objects":8725,"num_object_clones":0,"num_object_copies":17450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8298,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4808},{"shards":"0,2","objects":2671}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":576025217,"num_objects":140633,"num_object_clones":0,"num_object_copies":281266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":1606,"num_objects_unfound":0,"num_objects_dirty":140633,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":141654,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6264,"num_bytes_recovered":25636864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1228886016,"data_stored":1228808292,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161004,"ondisk_log_size":161004,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182468,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":824176,"kb_used_data":485728,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93547664,"statfs":{"total":96636764160,"available":95792807936,"internally_reserved":0,"allocated":497385472,"data_stored":497126258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":346549931},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,3,6,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986240,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":699396,"kb_used_data":410932,"kb_used_omap":4,"kb_used_meta":288443,"kb_avail":93672444,"statfs":{"total":96636764160,"available":95920582656,"internally_reserved":0,"allocated":420794368,"data_stored":420535159,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":295366314},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659185,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":568684,"kb_used_data":306588,"kb_used_omap":4,"kb_used_meta":262075,"kb_avail":93803156,"statfs":{"total":96636764160,"available":96054431744,"internally_reserved":0,"allocated":313946112,"data_stored":313623683,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":268365487},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,3,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":495792128,"data_stored":495788649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":313462784,"data_stored":313396090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":419631104,"data_stored":419623553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:12.781 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:17:12.781 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'11396", 'reported_seq': 11474, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:10.839124+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:17:10.839124+0000', 'last_peered': '2024-09-11T01:17:10.839124+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:17:10.839124+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36331520, 'num_objects': 8870, 'num_object_clones': 0, 'num_object_copies': 17740, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8870, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11396, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4798}, {'shards': '0,2', 'objects': 2809}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:12.781 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:17:12.781 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'11281", 'reported_seq': 11410, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:09.115837+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:17:09.115837+0000', 'last_peered': '2024-09-11T01:17:09.115837+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:17:09.115837+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "36'1200", 'ondisk_log_start': "36'1200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36298776, 'num_objects': 8863, 'num_object_clones': 0, 'num_object_copies': 17726, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8863, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9582, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2689}, {'shards': '2', 'objects': 4967}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:12.781 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:17:12.781 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'11374", 'reported_seq': 11542, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:10.852006+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:17:10.852006+0000', 'last_peered': '2024-09-11T01:17:10.852006+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:17:10.852006+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36261888, 'num_objects': 8853, 'num_object_clones': 0, 'num_object_copies': 17706, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8853, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7715, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4902}, {'shards': '0,2', 'objects': 2693}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:12.781 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:17:12.781 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'11261", 'reported_seq': 11390, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:10.821675+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:17:10.821675+0000', 'last_peered': '2024-09-11T01:17:10.821675+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:17:10.821675+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "36'1200", 'ondisk_log_start': "36'1200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35508224, 'num_objects': 8669, 'num_object_clones': 0, 'num_object_copies': 17338, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8669, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8118, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4683}, {'shards': '0,2', 'objects': 2693}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:12.782 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:17:12.782 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'11147", 'reported_seq': 11275, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:08.938676+0000', 'last_change': '2024-09-11T01:13:10.159069+0000', 'last_active': '2024-09-11T01:17:08.938676+0000', 'last_peered': '2024-09-11T01:17:08.938676+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:17:08.938676+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36151296, 'num_objects': 8826, 'num_object_clones': 0, 'num_object_copies': 17652, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8826, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9118, 'num_write_kb': 31844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2775}, {'shards': '2', 'objects': 4893}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:12.782 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:17:12.782 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'11204", 'reported_seq': 11332, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:10.858701+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:17:10.858701+0000', 'last_peered': '2024-09-11T01:17:10.858701+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:17:10.858701+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "36'1200", 'ondisk_log_start': "36'1200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35487744, 'num_objects': 8664, 'num_object_clones': 0, 'num_object_copies': 17328, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8664, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7791, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4680}, {'shards': '0,2', 'objects': 2718}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:12.782 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:17:12.782 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'11362", 'reported_seq': 11481, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:09.135788+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:17:09.135788+0000', 'last_peered': '2024-09-11T01:17:09.135788+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:17:09.135788+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36229120, 'num_objects': 8845, 'num_object_clones': 0, 'num_object_copies': 17690, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8845, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9179, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2767}, {'shards': '2', 'objects': 4821}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:12.782 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:17:12.782 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'11441", 'reported_seq': 11573, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:10.669637+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:17:10.669637+0000', 'last_peered': '2024-09-11T01:17:10.669637+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:17:10.669637+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36139008, 'num_objects': 8823, 'num_object_clones': 0, 'num_object_copies': 17646, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8823, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8178, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4873}, {'shards': '0,2', 'objects': 2642}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:12.782 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:17:12.782 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'11194", 'reported_seq': 16004, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:17:10.870451+0000', 'last_change': '2024-09-11T01:15:19.913288+0000', 'last_active': '2024-09-11T01:17:10.870451+0000', 'last_peered': '2024-09-11T01:17:10.870451+0000', 'last_clean': '2024-09-11T01:13:07.624565+0000', 'last_became_active': '2024-09-11T01:13:10.199795+0000', 'last_became_peered': '2024-09-11T01:13:10.199795+0000', 'last_unstale': '2024-09-11T01:17:10.870451+0000', 'last_undegraded': '2024-09-11T01:17:10.870451+0000', 'last_fullsized': '2024-09-11T01:13:09.652311+0000', 'mapping_epoch': 42, 'log_start': "36'1100", 'ondisk_log_start': "36'1100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3415", 'last_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'objects_scrubbed': 3337, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1975, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35594240, 'num_objects': 8690, 'num_object_clones': 0, 'num_object_copies': 17380, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1606, 'num_objects_unfound': 0, 'num_objects_dirty': 8690, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7849, 'num_write_kb': 26404, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2336, 'num_bytes_recovered': 9555968, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7084}, {'shards': '0,2', 'objects': 1606}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:12.783 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:17:12.783 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'11365", 'reported_seq': 11498, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:10.815355+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:17:10.815355+0000', 'last_peered': '2024-09-11T01:17:10.815355+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:17:10.815355+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 36089856, 'num_objects': 8811, 'num_object_clones': 0, 'num_object_copies': 17622, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8811, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7974, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4772}, {'shards': '0,2', 'objects': 2764}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:12.783 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:17:12.783 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'11228", 'reported_seq': 11362, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:10.744118+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:17:10.744118+0000', 'last_peered': '2024-09-11T01:17:10.744118+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:17:10.744118+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "36'1200", 'ondisk_log_start': "36'1200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35734121, 'num_objects': 8725, 'num_object_clones': 0, 'num_object_copies': 17450, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8725, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8298, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4808}, {'shards': '0,2', 'objects': 2671}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:12.783 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:17:13.790 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:17:13.790 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39769/281270 objects degraded (14.139%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":370,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":140635,"data_bytes":576025236,"bytes_used":2142470144,"bytes_avail":287767822336,"bytes_total":289910292480,"degraded_objects":39769,"degraded_total":281270,"degraded_ratio":0.14139083442955167,"misplaced_objects":1606,"misplaced_total":281270,"misplaced_ratio":0.0057098161908486515,"recovering_objects_per_sec":34,"recovering_bytes_per_sec":138864,"recovering_keys_per_sec":0,"num_objects_recovered":410,"num_bytes_recovered":1667072,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":480},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:17:13.838 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:17:13.838 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:14.804 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:17:14.805 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:17:14.869 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":224,"stamp":"2024-09-11T01:17:14.659851+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":572699284,"num_objects":139823,"num_object_clones":0,"num_object_copies":279646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":1606,"num_objects_unfound":0,"num_objects_dirty":139823,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":142468,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6265,"num_bytes_recovered":25636883,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160819,"ondisk_log_size":160819,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2117788,"kb_used_data":1198252,"kb_used_omap":12,"kb_used_meta":919475,"kb_avail":280997732,"statfs":{"total":289910292480,"available":287741677568,"internally_reserved":0,"allocated":1227010048,"data_stored":1226169196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":941542404},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,2,3,4,11,12,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21405696,"num_objects":-5226,"num_object_clones":0,"num_object_copies":-10452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-279,"num_objects_unfound":0,"num_objects_dirty":-5226,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5226,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":282,"num_bytes_recovered":1142784,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005000"},"pg_stats":[{"pgid":"2.f","version":"45'11396","reported_seq":11474,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:10.839124+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:17:10.839124+0000","last_peered":"2024-09-11T01:17:10.839124+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:17:10.839124+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36331520,"num_objects":8870,"num_object_clones":0,"num_object_copies":17740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11396,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4798},{"shards":"0,2","objects":2809}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11604","reported_seq":11706,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:14.203934+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:17:14.203934+0000","last_peered":"2024-09-11T01:17:14.203934+0000","last_clean":"2024-09-11T01:17:14.203934+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:17:14.203934+0000","last_undegraded":"2024-09-11T01:17:14.203934+0000","last_fullsized":"2024-09-11T01:17:14.203934+0000","mapping_epoch":35,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35831808,"num_objects":8748,"num_object_clones":0,"num_object_copies":17496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9843,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11412","reported_seq":11541,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:14.155871+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:17:14.155871+0000","last_peered":"2024-09-11T01:17:14.155871+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:17:14.155871+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35762200,"num_objects":8732,"num_object_clones":0,"num_object_copies":17464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8732,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9713,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2558},{"shards":"2","objects":4836}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11416","reported_seq":11486,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:14.098088+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:17:14.098088+0000","last_peered":"2024-09-11T01:17:14.098088+0000","last_clean":"2024-09-11T01:17:14.098088+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:17:14.098088+0000","last_undegraded":"2024-09-11T01:17:14.098088+0000","last_fullsized":"2024-09-11T01:17:14.098088+0000","mapping_epoch":35,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35708928,"num_objects":8718,"num_object_clones":0,"num_object_copies":17436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11416,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11294","reported_seq":11422,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:10.839471+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:17:10.839471+0000","last_peered":"2024-09-11T01:17:10.839471+0000","last_clean":"2024-09-11T01:17:10.839471+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:17:10.839471+0000","last_undegraded":"2024-09-11T01:17:10.839471+0000","last_fullsized":"2024-09-11T01:17:10.839471+0000","mapping_epoch":34,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35696640,"num_objects":8715,"num_object_clones":0,"num_object_copies":17430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8299,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11374","reported_seq":11542,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:10.852006+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:17:10.852006+0000","last_peered":"2024-09-11T01:17:10.852006+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:17:10.852006+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36261888,"num_objects":8853,"num_object_clones":0,"num_object_copies":17706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7715,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4902},{"shards":"0,2","objects":2693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11261","reported_seq":11390,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:10.821675+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:17:10.821675+0000","last_peered":"2024-09-11T01:17:10.821675+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:17:10.821675+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35508224,"num_objects":8669,"num_object_clones":0,"num_object_copies":17338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8118,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4683},{"shards":"0,2","objects":2693}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11428","reported_seq":11558,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:10.838822+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:17:10.838822+0000","last_peered":"2024-09-11T01:17:10.838822+0000","last_clean":"2024-09-11T01:17:10.838822+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:17:10.838822+0000","last_undegraded":"2024-09-11T01:17:10.838822+0000","last_fullsized":"2024-09-11T01:17:10.838822+0000","mapping_epoch":34,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36151296,"num_objects":8826,"num_object_clones":0,"num_object_copies":17652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8826,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8304,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683747+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683747+0000","last_peered":"2024-09-11T01:15:21.683747+0000","last_clean":"2024-09-11T01:15:21.683747+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683747+0000","last_undegraded":"2024-09-11T01:15:21.683747+0000","last_fullsized":"2024-09-11T01:15:21.683747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11282","reported_seq":11408,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:14.159721+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:17:14.159721+0000","last_peered":"2024-09-11T01:17:14.159721+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:17:14.159721+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35598336,"num_objects":8691,"num_object_clones":0,"num_object_copies":17382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9253,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2640},{"shards":"2","objects":4758}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11204","reported_seq":11332,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:10.858701+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:17:10.858701+0000","last_peered":"2024-09-11T01:17:10.858701+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:17:10.858701+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35487744,"num_objects":8664,"num_object_clones":0,"num_object_copies":17328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7791,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4680},{"shards":"0,2","objects":2718}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11486","reported_seq":11605,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:14.125296+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:17:14.125296+0000","last_peered":"2024-09-11T01:17:14.125296+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:17:14.125296+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35721216,"num_objects":8721,"num_object_clones":0,"num_object_copies":17442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9303,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2643},{"shards":"2","objects":4697}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11231","reported_seq":19212,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:14.171112+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:17:14.171112+0000","last_peered":"2024-09-11T01:17:14.171112+0000","last_clean":"2024-09-11T01:17:14.171112+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:17:14.171112+0000","last_undegraded":"2024-09-11T01:17:14.171112+0000","last_fullsized":"2024-09-11T01:17:14.171112+0000","mapping_epoch":44,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35082240,"num_objects":8565,"num_object_clones":0,"num_object_copies":17130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9016,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11441","reported_seq":11573,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:10.669637+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:17:10.669637+0000","last_peered":"2024-09-11T01:17:10.669637+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:17:10.669637+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36139008,"num_objects":8823,"num_object_clones":0,"num_object_copies":17646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8178,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4873},{"shards":"0,2","objects":2642}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11194","reported_seq":16004,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:10.870451+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:17:10.870451+0000","last_peered":"2024-09-11T01:17:10.870451+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:17:10.870451+0000","last_undegraded":"2024-09-11T01:17:10.870451+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"36'1100","ondisk_log_start":"36'1100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35594240,"num_objects":8690,"num_object_clones":0,"num_object_copies":17380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1606,"num_objects_unfound":0,"num_objects_dirty":8690,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7849,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2336,"num_bytes_recovered":9555968,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7084},{"shards":"0,2","objects":1606}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11365","reported_seq":11498,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:10.815355+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:17:10.815355+0000","last_peered":"2024-09-11T01:17:10.815355+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:17:10.815355+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":36089856,"num_objects":8811,"num_object_clones":0,"num_object_copies":17622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8811,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7974,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4772},{"shards":"0,2","objects":2764}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11228","reported_seq":11362,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:10.744118+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:17:10.744118+0000","last_peered":"2024-09-11T01:17:10.744118+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:17:10.744118+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35734121,"num_objects":8725,"num_object_clones":0,"num_object_copies":17450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8298,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4808},{"shards":"0,2","objects":2671}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":572699265,"num_objects":139821,"num_object_clones":0,"num_object_copies":279642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":1606,"num_objects_unfound":0,"num_objects_dirty":139821,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":142466,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6264,"num_bytes_recovered":25636864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1223966720,"data_stored":1223888996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160816,"ondisk_log_size":160816,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182468,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":824176,"kb_used_data":485728,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93547664,"statfs":{"total":96636764160,"available":95792807936,"internally_reserved":0,"allocated":497385472,"data_stored":497126258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":346549931},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,3,6,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986241,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":717152,"kb_used_data":406480,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93654688,"statfs":{"total":96636764160,"available":95902400512,"internally_reserved":0,"allocated":416235520,"data_stored":415976311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659186,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576460,"kb_used_data":306044,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93795380,"statfs":{"total":96636764160,"available":96046469120,"internally_reserved":0,"allocated":313389056,"data_stored":313066627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":276885167},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,1,1,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":495792128,"data_stored":495788649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":312950784,"data_stored":312884090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":415223808,"data_stored":415216257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:17.872 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:18.648 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:17:18.649 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:17:18.689 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":225,"stamp":"2024-09-11T01:17:16.660333+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":566915732,"num_objects":138411,"num_object_clones":0,"num_object_copies":276822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":1463,"num_objects_unfound":0,"num_objects_dirty":138411,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":143880,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6408,"num_bytes_recovered":26222611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160731,"ondisk_log_size":160731,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2115396,"kb_used_data":1191696,"kb_used_omap":12,"kb_used_meta":923635,"kb_avail":281000124,"statfs":{"total":289910292480,"available":287744126976,"internally_reserved":0,"allocated":1220296704,"data_stored":1219455852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":945802244},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,3,3,7,11,8,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23736320,"num_objects":-5795,"num_object_clones":0,"num_object_copies":-11590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-422,"num_objects_unfound":0,"num_objects_dirty":-5795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5795,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":425,"num_bytes_recovered":1728512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005110"},"pg_stats":[{"pgid":"2.f","version":"45'11534","reported_seq":11611,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.904626+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:17:15.904626+0000","last_peered":"2024-09-11T01:17:15.904626+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:17:15.904626+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35766272,"num_objects":8732,"num_object_clones":0,"num_object_copies":17464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11534,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4660},{"shards":"0,2","objects":2671}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11604","reported_seq":11706,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:14.203934+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:17:14.203934+0000","last_peered":"2024-09-11T01:17:14.203934+0000","last_clean":"2024-09-11T01:17:14.203934+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:17:14.203934+0000","last_undegraded":"2024-09-11T01:17:14.203934+0000","last_fullsized":"2024-09-11T01:17:14.203934+0000","mapping_epoch":35,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35831808,"num_objects":8748,"num_object_clones":0,"num_object_copies":17496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9843,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11412","reported_seq":11541,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:14.155871+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:17:14.155871+0000","last_peered":"2024-09-11T01:17:14.155871+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:17:14.155871+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35762200,"num_objects":8732,"num_object_clones":0,"num_object_copies":17464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8732,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9713,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2558},{"shards":"2","objects":4836}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11416","reported_seq":11486,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:14.098088+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:17:14.098088+0000","last_peered":"2024-09-11T01:17:14.098088+0000","last_clean":"2024-09-11T01:17:14.098088+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:17:14.098088+0000","last_undegraded":"2024-09-11T01:17:14.098088+0000","last_fullsized":"2024-09-11T01:17:14.098088+0000","mapping_epoch":35,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35708928,"num_objects":8718,"num_object_clones":0,"num_object_copies":17436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11416,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11432","reported_seq":11560,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:15.875254+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:17:15.875254+0000","last_peered":"2024-09-11T01:17:15.875254+0000","last_clean":"2024-09-11T01:17:15.875254+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:17:15.875254+0000","last_undegraded":"2024-09-11T01:17:15.875254+0000","last_fullsized":"2024-09-11T01:17:15.875254+0000","mapping_epoch":34,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35131392,"num_objects":8577,"num_object_clones":0,"num_object_copies":17154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8437,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11516","reported_seq":11685,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.841462+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:17:15.841462+0000","last_peered":"2024-09-11T01:17:15.841462+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:17:15.841462+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35680256,"num_objects":8711,"num_object_clones":0,"num_object_copies":17422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7857,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4760},{"shards":"0,2","objects":2551}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11395","reported_seq":11526,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.933507+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:17:15.933507+0000","last_peered":"2024-09-11T01:17:15.933507+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:17:15.933507+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34959360,"num_objects":8535,"num_object_clones":0,"num_object_copies":17070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8252,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4549},{"shards":"0,2","objects":2559}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11574","reported_seq":11704,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:15.890033+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:17:15.890033+0000","last_peered":"2024-09-11T01:17:15.890033+0000","last_clean":"2024-09-11T01:17:15.890033+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:17:15.890033+0000","last_undegraded":"2024-09-11T01:17:15.890033+0000","last_fullsized":"2024-09-11T01:17:15.890033+0000","mapping_epoch":34,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35553280,"num_objects":8680,"num_object_clones":0,"num_object_copies":17360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8680,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8450,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683747+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683747+0000","last_peered":"2024-09-11T01:15:21.683747+0000","last_clean":"2024-09-11T01:15:21.683747+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683747+0000","last_undegraded":"2024-09-11T01:15:21.683747+0000","last_fullsized":"2024-09-11T01:15:21.683747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11282","reported_seq":11408,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:14.159721+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:17:14.159721+0000","last_peered":"2024-09-11T01:17:14.159721+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:17:14.159721+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35598336,"num_objects":8691,"num_object_clones":0,"num_object_copies":17382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9253,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2640},{"shards":"2","objects":4758}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11344","reported_seq":11473,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.927295+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:17:15.927295+0000","last_peered":"2024-09-11T01:17:15.927295+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:17:15.927295+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34914304,"num_objects":8524,"num_object_clones":0,"num_object_copies":17048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7931,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4540},{"shards":"0,2","objects":2578}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11486","reported_seq":11605,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:14.125296+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:17:14.125296+0000","last_peered":"2024-09-11T01:17:14.125296+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:17:14.125296+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35721216,"num_objects":8721,"num_object_clones":0,"num_object_copies":17442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9303,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2643},{"shards":"2","objects":4697}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11231","reported_seq":19212,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:14.171112+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:17:14.171112+0000","last_peered":"2024-09-11T01:17:14.171112+0000","last_clean":"2024-09-11T01:17:14.171112+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:17:14.171112+0000","last_undegraded":"2024-09-11T01:17:14.171112+0000","last_fullsized":"2024-09-11T01:17:14.171112+0000","mapping_epoch":44,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35082240,"num_objects":8565,"num_object_clones":0,"num_object_copies":17130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9016,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11586","reported_seq":11718,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.803153+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:17:15.803153+0000","last_peered":"2024-09-11T01:17:15.803153+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:17:15.803153+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35545088,"num_objects":8678,"num_object_clones":0,"num_object_copies":17356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8678,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8323,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4728},{"shards":"0,2","objects":2497}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11329","reported_seq":16425,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:15.942726+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:17:15.942726+0000","last_peered":"2024-09-11T01:17:15.942726+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:17:15.942726+0000","last_undegraded":"2024-09-11T01:17:15.942726+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35041280,"num_objects":8555,"num_object_clones":0,"num_object_copies":17110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1463,"num_objects_unfound":0,"num_objects_dirty":8555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7984,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2479,"num_bytes_recovered":10141696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7092},{"shards":"0,2","objects":1463}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11514","reported_seq":11649,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.872465+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:17:15.872465+0000","last_peered":"2024-09-11T01:17:15.872465+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:17:15.872465+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35479552,"num_objects":8662,"num_object_clones":0,"num_object_copies":17324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8123,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4623},{"shards":"0,2","objects":2615}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11373","reported_seq":11507,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.914861+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:17:15.914861+0000","last_peered":"2024-09-11T01:17:15.914861+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:17:15.914861+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35140201,"num_objects":8580,"num_object_clones":0,"num_object_copies":17160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8443,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4663},{"shards":"0,2","objects":2526}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":566915713,"num_objects":138409,"num_object_clones":0,"num_object_copies":276818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":1463,"num_objects_unfound":0,"num_objects_dirty":138409,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":143878,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6407,"num_bytes_recovered":26222592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1217069056,"data_stored":1216991332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160728,"ondisk_log_size":160728,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182469,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":821784,"kb_used_data":479172,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93550056,"statfs":{"total":96636764160,"available":95795257344,"internally_reserved":0,"allocated":490672128,"data_stored":490412914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":350809771},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,2,2,4,3,2,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986241,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":717152,"kb_used_data":406480,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93654688,"statfs":{"total":96636764160,"available":95902400512,"internally_reserved":0,"allocated":416235520,"data_stored":415976311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659186,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576460,"kb_used_data":306044,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93795380,"statfs":{"total":96636764160,"available":96046469120,"internally_reserved":0,"allocated":313389056,"data_stored":313066627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":276885167},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,1,1,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":488894464,"data_stored":488890985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":312950784,"data_stored":312884090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":415223808,"data_stored":415216257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:18.691 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:19.495 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:17:19.498 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:17:19.609 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":226,"stamp":"2024-09-11T01:17:18.660838+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":566915732,"num_objects":138411,"num_object_clones":0,"num_object_copies":276822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":1463,"num_objects_unfound":0,"num_objects_dirty":138411,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":143880,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6408,"num_bytes_recovered":26222611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160731,"ondisk_log_size":160731,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2115396,"kb_used_data":1191696,"kb_used_omap":12,"kb_used_meta":923635,"kb_avail":281000124,"statfs":{"total":289910292480,"available":287744126976,"internally_reserved":0,"allocated":1220296704,"data_stored":1219455852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":945802244},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,3,3,3,7,11,8,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18006016,"num_objects":-4396,"num_object_clones":0,"num_object_copies":-8792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-280,"num_objects_unfound":0,"num_objects_dirty":-4396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4396,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":1146880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002865"},"pg_stats":[{"pgid":"2.f","version":"45'11534","reported_seq":11611,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.904626+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:17:15.904626+0000","last_peered":"2024-09-11T01:17:15.904626+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:17:15.904626+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35766272,"num_objects":8732,"num_object_clones":0,"num_object_copies":17464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11534,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4660},{"shards":"0,2","objects":2671}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11604","reported_seq":11706,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:14.203934+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:17:14.203934+0000","last_peered":"2024-09-11T01:17:14.203934+0000","last_clean":"2024-09-11T01:17:14.203934+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:17:14.203934+0000","last_undegraded":"2024-09-11T01:17:14.203934+0000","last_fullsized":"2024-09-11T01:17:14.203934+0000","mapping_epoch":35,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35831808,"num_objects":8748,"num_object_clones":0,"num_object_copies":17496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8748,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9843,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11412","reported_seq":11541,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:14.155871+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:17:14.155871+0000","last_peered":"2024-09-11T01:17:14.155871+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:17:14.155871+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35762200,"num_objects":8732,"num_object_clones":0,"num_object_copies":17464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8732,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9713,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2558},{"shards":"2","objects":4836}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11416","reported_seq":11486,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:14.098088+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:17:14.098088+0000","last_peered":"2024-09-11T01:17:14.098088+0000","last_clean":"2024-09-11T01:17:14.098088+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:17:14.098088+0000","last_undegraded":"2024-09-11T01:17:14.098088+0000","last_fullsized":"2024-09-11T01:17:14.098088+0000","mapping_epoch":35,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35708928,"num_objects":8718,"num_object_clones":0,"num_object_copies":17436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8718,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11416,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11432","reported_seq":11560,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:15.875254+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:17:15.875254+0000","last_peered":"2024-09-11T01:17:15.875254+0000","last_clean":"2024-09-11T01:17:15.875254+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:17:15.875254+0000","last_undegraded":"2024-09-11T01:17:15.875254+0000","last_fullsized":"2024-09-11T01:17:15.875254+0000","mapping_epoch":34,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35131392,"num_objects":8577,"num_object_clones":0,"num_object_copies":17154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8437,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11516","reported_seq":11685,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.841462+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:17:15.841462+0000","last_peered":"2024-09-11T01:17:15.841462+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:17:15.841462+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35680256,"num_objects":8711,"num_object_clones":0,"num_object_copies":17422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7857,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4760},{"shards":"0,2","objects":2551}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11395","reported_seq":11526,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.933507+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:17:15.933507+0000","last_peered":"2024-09-11T01:17:15.933507+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:17:15.933507+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34959360,"num_objects":8535,"num_object_clones":0,"num_object_copies":17070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8252,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4549},{"shards":"0,2","objects":2559}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11574","reported_seq":11704,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:15.890033+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:17:15.890033+0000","last_peered":"2024-09-11T01:17:15.890033+0000","last_clean":"2024-09-11T01:17:15.890033+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:17:15.890033+0000","last_undegraded":"2024-09-11T01:17:15.890033+0000","last_fullsized":"2024-09-11T01:17:15.890033+0000","mapping_epoch":34,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35553280,"num_objects":8680,"num_object_clones":0,"num_object_copies":17360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8680,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8450,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683747+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683747+0000","last_peered":"2024-09-11T01:15:21.683747+0000","last_clean":"2024-09-11T01:15:21.683747+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683747+0000","last_undegraded":"2024-09-11T01:15:21.683747+0000","last_fullsized":"2024-09-11T01:15:21.683747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11282","reported_seq":11408,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:14.159721+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:17:14.159721+0000","last_peered":"2024-09-11T01:17:14.159721+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:17:14.159721+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35598336,"num_objects":8691,"num_object_clones":0,"num_object_copies":17382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9253,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2640},{"shards":"2","objects":4758}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11344","reported_seq":11473,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.927295+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:17:15.927295+0000","last_peered":"2024-09-11T01:17:15.927295+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:17:15.927295+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34914304,"num_objects":8524,"num_object_clones":0,"num_object_copies":17048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7931,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4540},{"shards":"0,2","objects":2578}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11486","reported_seq":11605,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:14.125296+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:17:14.125296+0000","last_peered":"2024-09-11T01:17:14.125296+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:17:14.125296+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35721216,"num_objects":8721,"num_object_clones":0,"num_object_copies":17442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8721,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9303,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2643},{"shards":"2","objects":4697}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":79,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684269+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684269+0000","last_peered":"2024-09-11T01:15:21.684269+0000","last_clean":"2024-09-11T01:15:21.684269+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684269+0000","last_undegraded":"2024-09-11T01:15:21.684269+0000","last_fullsized":"2024-09-11T01:15:21.684269+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11231","reported_seq":19212,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:14.171112+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:17:14.171112+0000","last_peered":"2024-09-11T01:17:14.171112+0000","last_clean":"2024-09-11T01:17:14.171112+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:17:14.171112+0000","last_undegraded":"2024-09-11T01:17:14.171112+0000","last_fullsized":"2024-09-11T01:17:14.171112+0000","mapping_epoch":44,"log_start":"36'1200","ondisk_log_start":"36'1200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35082240,"num_objects":8565,"num_object_clones":0,"num_object_copies":17130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9016,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11586","reported_seq":11718,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.803153+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:17:15.803153+0000","last_peered":"2024-09-11T01:17:15.803153+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:17:15.803153+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35545088,"num_objects":8678,"num_object_clones":0,"num_object_copies":17356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8678,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8323,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4728},{"shards":"0,2","objects":2497}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684588+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684588+0000","last_peered":"2024-09-11T01:15:21.684588+0000","last_clean":"2024-09-11T01:15:21.684588+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684588+0000","last_undegraded":"2024-09-11T01:15:21.684588+0000","last_fullsized":"2024-09-11T01:15:21.684588+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11329","reported_seq":16425,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:15.942726+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:17:15.942726+0000","last_peered":"2024-09-11T01:17:15.942726+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:17:15.942726+0000","last_undegraded":"2024-09-11T01:17:15.942726+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35041280,"num_objects":8555,"num_object_clones":0,"num_object_copies":17110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1463,"num_objects_unfound":0,"num_objects_dirty":8555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7984,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2479,"num_bytes_recovered":10141696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7092},{"shards":"0,2","objects":1463}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11514","reported_seq":11649,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.872465+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:17:15.872465+0000","last_peered":"2024-09-11T01:17:15.872465+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:17:15.872465+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35479552,"num_objects":8662,"num_object_clones":0,"num_object_copies":17324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8123,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4623},{"shards":"0,2","objects":2615}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.691932+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.691932+0000","last_peered":"2024-09-11T01:15:21.691932+0000","last_clean":"2024-09-11T01:15:21.691932+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.691932+0000","last_undegraded":"2024-09-11T01:15:21.691932+0000","last_fullsized":"2024-09-11T01:15:21.691932+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11373","reported_seq":11507,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.914861+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:17:15.914861+0000","last_peered":"2024-09-11T01:17:15.914861+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:17:15.914861+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35140201,"num_objects":8580,"num_object_clones":0,"num_object_copies":17160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8443,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4663},{"shards":"0,2","objects":2526}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":566915713,"num_objects":138409,"num_object_clones":0,"num_object_copies":276818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":1463,"num_objects_unfound":0,"num_objects_dirty":138409,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":143878,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6407,"num_bytes_recovered":26222592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1217069056,"data_stored":1216991332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160728,"ondisk_log_size":160728,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182469,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":821784,"kb_used_data":479172,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93550056,"statfs":{"total":96636764160,"available":95795257344,"internally_reserved":0,"allocated":490672128,"data_stored":490412914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":350809771},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,2,2,4,3,2,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986241,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":717152,"kb_used_data":406480,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93654688,"statfs":{"total":96636764160,"available":95902400512,"internally_reserved":0,"allocated":416235520,"data_stored":415976311,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,2,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659186,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576460,"kb_used_data":306044,"kb_used_omap":4,"kb_used_meta":270395,"kb_avail":93795380,"statfs":{"total":96636764160,"available":96046469120,"internally_reserved":0,"allocated":313389056,"data_stored":313066627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":276885167},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,1,1,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":488894464,"data_stored":488890985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":312950784,"data_stored":312884090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":415223808,"data_stored":415216257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:19.611 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:17:19.611 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'11534", 'reported_seq': 11611, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:15.904626+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:17:15.904626+0000', 'last_peered': '2024-09-11T01:17:15.904626+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:17:15.904626+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35766272, 'num_objects': 8732, 'num_object_clones': 0, 'num_object_copies': 17464, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8732, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11534, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4660}, {'shards': '0,2', 'objects': 2671}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:19.611 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:17:19.611 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'11412", 'reported_seq': 11541, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:14.155871+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:17:14.155871+0000', 'last_peered': '2024-09-11T01:17:14.155871+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:17:14.155871+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35762200, 'num_objects': 8732, 'num_object_clones': 0, 'num_object_copies': 17464, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8732, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9713, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2558}, {'shards': '2', 'objects': 4836}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:19.611 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:17:19.611 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'11516", 'reported_seq': 11685, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:15.841462+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:17:15.841462+0000', 'last_peered': '2024-09-11T01:17:15.841462+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:17:15.841462+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35680256, 'num_objects': 8711, 'num_object_clones': 0, 'num_object_copies': 17422, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8711, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7857, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4760}, {'shards': '0,2', 'objects': 2551}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:19.611 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:17:19.612 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'11395", 'reported_seq': 11526, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:15.933507+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:17:15.933507+0000', 'last_peered': '2024-09-11T01:17:15.933507+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:17:15.933507+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34959360, 'num_objects': 8535, 'num_object_clones': 0, 'num_object_copies': 17070, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8535, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8252, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4549}, {'shards': '0,2', 'objects': 2559}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:19.612 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:17:19.612 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'11282", 'reported_seq': 11408, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:14.159721+0000', 'last_change': '2024-09-11T01:13:10.159069+0000', 'last_active': '2024-09-11T01:17:14.159721+0000', 'last_peered': '2024-09-11T01:17:14.159721+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:17:14.159721+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "36'1200", 'ondisk_log_start': "36'1200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35598336, 'num_objects': 8691, 'num_object_clones': 0, 'num_object_copies': 17382, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8691, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9253, 'num_write_kb': 31844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2640}, {'shards': '2', 'objects': 4758}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:19.612 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:17:19.612 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'11344", 'reported_seq': 11473, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:15.927295+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:17:15.927295+0000', 'last_peered': '2024-09-11T01:17:15.927295+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:17:15.927295+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34914304, 'num_objects': 8524, 'num_object_clones': 0, 'num_object_copies': 17048, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8524, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7931, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4540}, {'shards': '0,2', 'objects': 2578}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:19.612 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:17:19.612 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'11486", 'reported_seq': 11605, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:14.125296+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:17:14.125296+0000', 'last_peered': '2024-09-11T01:17:14.125296+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:17:14.125296+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35721216, 'num_objects': 8721, 'num_object_clones': 0, 'num_object_copies': 17442, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8721, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9303, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2643}, {'shards': '2', 'objects': 4697}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:19.612 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:17:19.612 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'11586", 'reported_seq': 11718, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:15.803153+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:17:15.803153+0000', 'last_peered': '2024-09-11T01:17:15.803153+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:17:15.803153+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35545088, 'num_objects': 8678, 'num_object_clones': 0, 'num_object_copies': 17356, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8678, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8323, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4728}, {'shards': '0,2', 'objects': 2497}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:19.612 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:17:19.613 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'11329", 'reported_seq': 16425, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:17:15.942726+0000', 'last_change': '2024-09-11T01:15:19.913288+0000', 'last_active': '2024-09-11T01:17:15.942726+0000', 'last_peered': '2024-09-11T01:17:15.942726+0000', 'last_clean': '2024-09-11T01:13:07.624565+0000', 'last_became_active': '2024-09-11T01:13:10.199795+0000', 'last_became_peered': '2024-09-11T01:13:10.199795+0000', 'last_unstale': '2024-09-11T01:17:15.942726+0000', 'last_undegraded': '2024-09-11T01:17:15.942726+0000', 'last_fullsized': '2024-09-11T01:13:09.652311+0000', 'mapping_epoch': 42, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3415", 'last_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'objects_scrubbed': 3337, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1975, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35041280, 'num_objects': 8555, 'num_object_clones': 0, 'num_object_copies': 17110, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1463, 'num_objects_unfound': 0, 'num_objects_dirty': 8555, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 7984, 'num_write_kb': 26404, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2479, 'num_bytes_recovered': 10141696, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7092}, {'shards': '0,2', 'objects': 1463}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:19.613 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:17:19.613 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'11514", 'reported_seq': 11649, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:15.872465+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:17:15.872465+0000', 'last_peered': '2024-09-11T01:17:15.872465+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:17:15.872465+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35479552, 'num_objects': 8662, 'num_object_clones': 0, 'num_object_copies': 17324, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8662, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8123, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4623}, {'shards': '0,2', 'objects': 2615}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:19.613 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:17:19.613 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'11373", 'reported_seq': 11507, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:15.914861+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:17:15.914861+0000', 'last_peered': '2024-09-11T01:17:15.914861+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:17:15.914861+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "36'1300", 'ondisk_log_start': "36'1300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10073, 'log_dups_size': 0, 'ondisk_log_size': 10073, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35140201, 'num_objects': 8580, 'num_object_clones': 0, 'num_object_copies': 17160, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8580, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8443, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4663}, {'shards': '0,2', 'objects': 2526}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:19.613 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:17:20.574 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:17:20.574 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39769/276822 objects degraded (14.366%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":377,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":138411,"data_bytes":566915732,"bytes_used":2166165504,"bytes_avail":287744126976,"bytes_total":289910292480,"degraded_objects":39769,"degraded_total":276822,"degraded_ratio":0.14366271466863184,"misplaced_objects":1463,"misplaced_total":276822,"misplaced_ratio":0.0052849845749254028,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":95550,"recovering_keys_per_sec":0,"num_objects_recovered":280,"num_bytes_recovered":1146880,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":366},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:17:20.613 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:17:20.613 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:21.445 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:17:21.446 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:17:21.507 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":227,"stamp":"2024-09-11T01:17:20.661261+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":563360404,"num_objects":137543,"num_object_clones":0,"num_object_copies":275086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":1463,"num_objects_unfound":0,"num_objects_dirty":137543,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":144748,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6408,"num_bytes_recovered":26222611,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160799,"ondisk_log_size":160799,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2101940,"kb_used_data":1186560,"kb_used_omap":12,"kb_used_meta":915315,"kb_avail":281013580,"statfs":{"total":289910292480,"available":287757905920,"internally_reserved":0,"allocated":1215037440,"data_stored":1214196588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":937282564},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,4,3,5,5,5,6,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21561344,"num_objects":-5264,"num_object_clones":0,"num_object_copies":-10528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-280,"num_objects_unfound":0,"num_objects_dirty":-5264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5264,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":280,"num_bytes_recovered":1146880,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002797"},"pg_stats":[{"pgid":"2.f","version":"45'11534","reported_seq":11611,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.904626+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:17:15.904626+0000","last_peered":"2024-09-11T01:17:15.904626+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:17:15.904626+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35766272,"num_objects":8732,"num_object_clones":0,"num_object_copies":17464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11534,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4660},{"shards":"0,2","objects":2671}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11746","reported_seq":11847,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:19.235475+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:17:19.235475+0000","last_peered":"2024-09-11T01:17:19.235475+0000","last_clean":"2024-09-11T01:17:19.235475+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:17:19.235475+0000","last_undegraded":"2024-09-11T01:17:19.235475+0000","last_fullsized":"2024-09-11T01:17:19.235475+0000","mapping_epoch":35,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35250176,"num_objects":8606,"num_object_clones":0,"num_object_copies":17212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8606,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9985,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11529","reported_seq":11657,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:19.208230+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:17:19.208230+0000","last_peered":"2024-09-11T01:17:19.208230+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:17:19.208230+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35282968,"num_objects":8615,"num_object_clones":0,"num_object_copies":17230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8615,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9830,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2441},{"shards":"2","objects":4719}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11571","reported_seq":11641,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:19.186347+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:17:19.186347+0000","last_peered":"2024-09-11T01:17:19.186347+0000","last_clean":"2024-09-11T01:17:19.186347+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:17:19.186347+0000","last_undegraded":"2024-09-11T01:17:19.186347+0000","last_fullsized":"2024-09-11T01:17:19.186347+0000","mapping_epoch":35,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35074048,"num_objects":8563,"num_object_clones":0,"num_object_copies":17126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8563,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11571,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11432","reported_seq":11560,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:15.875254+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:17:15.875254+0000","last_peered":"2024-09-11T01:17:15.875254+0000","last_clean":"2024-09-11T01:17:15.875254+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:17:15.875254+0000","last_undegraded":"2024-09-11T01:17:15.875254+0000","last_fullsized":"2024-09-11T01:17:15.875254+0000","mapping_epoch":34,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35131392,"num_objects":8577,"num_object_clones":0,"num_object_copies":17154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8437,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11516","reported_seq":11685,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.841462+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:17:15.841462+0000","last_peered":"2024-09-11T01:17:15.841462+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:17:15.841462+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35680256,"num_objects":8711,"num_object_clones":0,"num_object_copies":17422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7857,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4760},{"shards":"0,2","objects":2551}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11395","reported_seq":11526,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.933507+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:17:15.933507+0000","last_peered":"2024-09-11T01:17:15.933507+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:17:15.933507+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34959360,"num_objects":8535,"num_object_clones":0,"num_object_copies":17070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8535,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8252,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4549},{"shards":"0,2","objects":2559}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11574","reported_seq":11704,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:15.890033+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:17:15.890033+0000","last_peered":"2024-09-11T01:17:15.890033+0000","last_clean":"2024-09-11T01:17:15.890033+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:17:15.890033+0000","last_undegraded":"2024-09-11T01:17:15.890033+0000","last_fullsized":"2024-09-11T01:17:15.890033+0000","mapping_epoch":34,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35553280,"num_objects":8680,"num_object_clones":0,"num_object_copies":17360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8680,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8450,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683747+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683747+0000","last_peered":"2024-09-11T01:15:21.683747+0000","last_clean":"2024-09-11T01:15:21.683747+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683747+0000","last_undegraded":"2024-09-11T01:15:21.683747+0000","last_fullsized":"2024-09-11T01:15:21.683747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11420","reported_seq":11548,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:19.227153+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:17:19.227153+0000","last_peered":"2024-09-11T01:17:19.227153+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:17:19.227153+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35033088,"num_objects":8553,"num_object_clones":0,"num_object_copies":17106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8553,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9391,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2502},{"shards":"2","objects":4620}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11344","reported_seq":11473,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.927295+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:17:15.927295+0000","last_peered":"2024-09-11T01:17:15.927295+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:17:15.927295+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34914304,"num_objects":8524,"num_object_clones":0,"num_object_copies":17048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7931,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4540},{"shards":"0,2","objects":2578}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11640","reported_seq":11759,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:19.067601+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:17:19.067601+0000","last_peered":"2024-09-11T01:17:19.067601+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:17:19.067601+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35090432,"num_objects":8567,"num_object_clones":0,"num_object_copies":17134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8567,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9457,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2489},{"shards":"2","objects":4543}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684827+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684827+0000","last_peered":"2024-09-11T01:15:21.684827+0000","last_clean":"2024-09-11T01:15:21.684827+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684827+0000","last_undegraded":"2024-09-11T01:15:21.684827+0000","last_fullsized":"2024-09-11T01:15:21.684827+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11393","reported_seq":19374,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:19.216670+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:17:19.216670+0000","last_peered":"2024-09-11T01:17:19.216670+0000","last_clean":"2024-09-11T01:17:19.216670+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:17:19.216670+0000","last_undegraded":"2024-09-11T01:17:19.216670+0000","last_fullsized":"2024-09-11T01:17:19.216670+0000","mapping_epoch":44,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34418688,"num_objects":8403,"num_object_clones":0,"num_object_copies":16806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9178,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11586","reported_seq":11718,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.803153+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:17:15.803153+0000","last_peered":"2024-09-11T01:17:15.803153+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:17:15.803153+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35545088,"num_objects":8678,"num_object_clones":0,"num_object_copies":17356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8678,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8323,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4728},{"shards":"0,2","objects":2497}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684623+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684623+0000","last_peered":"2024-09-11T01:15:21.684623+0000","last_clean":"2024-09-11T01:15:21.684623+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684623+0000","last_undegraded":"2024-09-11T01:15:21.684623+0000","last_fullsized":"2024-09-11T01:15:21.684623+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11329","reported_seq":16425,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:15.942726+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:17:15.942726+0000","last_peered":"2024-09-11T01:17:15.942726+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:17:15.942726+0000","last_undegraded":"2024-09-11T01:17:15.942726+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35041280,"num_objects":8555,"num_object_clones":0,"num_object_copies":17110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1463,"num_objects_unfound":0,"num_objects_dirty":8555,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7984,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2479,"num_bytes_recovered":10141696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7092},{"shards":"0,2","objects":1463}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11514","reported_seq":11649,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.872465+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:17:15.872465+0000","last_peered":"2024-09-11T01:17:15.872465+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:17:15.872465+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35479552,"num_objects":8662,"num_object_clones":0,"num_object_copies":17324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8123,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4623},{"shards":"0,2","objects":2615}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692049+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.692049+0000","last_peered":"2024-09-11T01:15:21.692049+0000","last_clean":"2024-09-11T01:15:21.692049+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.692049+0000","last_undegraded":"2024-09-11T01:15:21.692049+0000","last_fullsized":"2024-09-11T01:15:21.692049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11373","reported_seq":11507,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:15.914861+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:17:15.914861+0000","last_peered":"2024-09-11T01:17:15.914861+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:17:15.914861+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'1300","ondisk_log_start":"36'1300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35140201,"num_objects":8580,"num_object_clones":0,"num_object_copies":17160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8443,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4663},{"shards":"0,2","objects":2526}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":563360385,"num_objects":137541,"num_object_clones":0,"num_object_copies":275082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":1463,"num_objects_unfound":0,"num_objects_dirty":137541,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":144746,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6407,"num_bytes_recovered":26222592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1211572224,"data_stored":1211494500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160796,"ondisk_log_size":160796,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182469,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":821784,"kb_used_data":479172,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93550056,"statfs":{"total":96636764160,"available":95795257344,"internally_reserved":0,"allocated":490672128,"data_stored":490412914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":350809771},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,2,2,4,3,2,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986242,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":700128,"kb_used_data":401936,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93671712,"statfs":{"total":96636764160,"available":95919833088,"internally_reserved":0,"allocated":411582464,"data_stored":411323255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305327786},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,1,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659187,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":580028,"kb_used_data":305452,"kb_used_omap":4,"kb_used_meta":274555,"kb_avail":93791812,"statfs":{"total":96636764160,"available":96042815488,"internally_reserved":0,"allocated":312782848,"data_stored":312460419,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":281145007},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,1,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":488894464,"data_stored":488890985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":312238080,"data_stored":312171386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":410439680,"data_stored":410432129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:24.509 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:25.243 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:17:25.244 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:17:25.298 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":229,"stamp":"2024-09-11T01:17:24.667676+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":554226324,"num_objects":135313,"num_object_clones":0,"num_object_copies":270626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":1322,"num_objects_unfound":0,"num_objects_dirty":135313,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":146978,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6549,"num_bytes_recovered":26800147,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160929,"ondisk_log_size":160929,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2130288,"kb_used_data":1174656,"kb_used_omap":12,"kb_used_meta":955571,"kb_avail":280985232,"statfs":{"total":289910292480,"available":287728877568,"internally_reserved":0,"allocated":1202847744,"data_stored":1202006892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":978504708},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,5,0,6,15,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21798912,"num_objects":-5322,"num_object_clones":0,"num_object_copies":-10644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-284,"num_objects_unfound":0,"num_objects_dirty":-5322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5322,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":1163264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008314"},"pg_stats":[{"pgid":"2.f","version":"45'11680","reported_seq":11758,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:21.020192+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:17:21.020192+0000","last_peered":"2024-09-11T01:17:21.020192+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:17:21.020192+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35168256,"num_objects":8586,"num_object_clones":0,"num_object_copies":17172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11680,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4514},{"shards":"0,2","objects":2525}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11885","reported_seq":11987,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:24.248241+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:17:24.248241+0000","last_peered":"2024-09-11T01:17:24.248241+0000","last_clean":"2024-09-11T01:17:24.248241+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:17:24.248241+0000","last_undegraded":"2024-09-11T01:17:24.248241+0000","last_fullsized":"2024-09-11T01:17:24.248241+0000","mapping_epoch":35,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34680832,"num_objects":8467,"num_object_clones":0,"num_object_copies":16934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10124,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11669","reported_seq":11798,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:24.271966+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:17:24.271966+0000","last_peered":"2024-09-11T01:17:24.271966+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:17:24.271966+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34709528,"num_objects":8475,"num_object_clones":0,"num_object_copies":16950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8475,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9970,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2301},{"shards":"2","objects":4579}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11692","reported_seq":11762,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:24.225142+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:17:24.225142+0000","last_peered":"2024-09-11T01:17:24.225142+0000","last_clean":"2024-09-11T01:17:24.225142+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:17:24.225142+0000","last_undegraded":"2024-09-11T01:17:24.225142+0000","last_fullsized":"2024-09-11T01:17:24.225142+0000","mapping_epoch":35,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34578432,"num_objects":8442,"num_object_clones":0,"num_object_copies":16884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11692,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11565","reported_seq":11693,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:20.945654+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:17:20.945654+0000","last_peered":"2024-09-11T01:17:20.945654+0000","last_clean":"2024-09-11T01:17:20.945654+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:17:20.945654+0000","last_undegraded":"2024-09-11T01:17:20.945654+0000","last_fullsized":"2024-09-11T01:17:20.945654+0000","mapping_epoch":34,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34586624,"num_objects":8444,"num_object_clones":0,"num_object_copies":16888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8570,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11686","reported_seq":11854,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:20.995922+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:17:20.995922+0000","last_peered":"2024-09-11T01:17:20.995922+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:17:20.995922+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34983936,"num_objects":8541,"num_object_clones":0,"num_object_copies":17082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8027,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4590},{"shards":"0,2","objects":2381}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11520","reported_seq":11651,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:20.898369+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:17:20.898369+0000","last_peered":"2024-09-11T01:17:20.898369+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:17:20.898369+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34447360,"num_objects":8410,"num_object_clones":0,"num_object_copies":16820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8377,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4424},{"shards":"0,2","objects":2434}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11710","reported_seq":11840,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:20.876691+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:17:20.876691+0000","last_peered":"2024-09-11T01:17:20.876691+0000","last_clean":"2024-09-11T01:17:20.876691+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:17:20.876691+0000","last_undegraded":"2024-09-11T01:17:20.876691+0000","last_fullsized":"2024-09-11T01:17:20.876691+0000","mapping_epoch":34,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34996224,"num_objects":8544,"num_object_clones":0,"num_object_copies":17088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8586,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683747+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683747+0000","last_peered":"2024-09-11T01:15:21.683747+0000","last_clean":"2024-09-11T01:15:21.683747+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683747+0000","last_undegraded":"2024-09-11T01:15:21.683747+0000","last_fullsized":"2024-09-11T01:15:21.683747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11567","reported_seq":11695,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:24.209347+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:17:24.209347+0000","last_peered":"2024-09-11T01:17:24.209347+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:17:24.209347+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34430976,"num_objects":8406,"num_object_clones":0,"num_object_copies":16812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9538,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2355},{"shards":"2","objects":4473}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11478","reported_seq":11607,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:20.985252+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:17:20.985252+0000","last_peered":"2024-09-11T01:17:20.985252+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:17:20.985252+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34365440,"num_objects":8390,"num_object_clones":0,"num_object_copies":16780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8065,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4406},{"shards":"0,2","objects":2444}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11787","reported_seq":11906,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:24.251148+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:17:24.251148+0000","last_peered":"2024-09-11T01:17:24.251148+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:17:24.251148+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34488320,"num_objects":8420,"num_object_clones":0,"num_object_copies":16840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9604,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2342},{"shards":"2","objects":4396}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684827+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684827+0000","last_peered":"2024-09-11T01:15:21.684827+0000","last_clean":"2024-09-11T01:15:21.684827+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684827+0000","last_undegraded":"2024-09-11T01:15:21.684827+0000","last_fullsized":"2024-09-11T01:15:21.684827+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11544","reported_seq":19523,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:24.212654+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:17:24.212654+0000","last_peered":"2024-09-11T01:17:24.212654+0000","last_clean":"2024-09-11T01:17:24.212654+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:17:24.212654+0000","last_undegraded":"2024-09-11T01:17:24.212654+0000","last_fullsized":"2024-09-11T01:17:24.212654+0000","mapping_epoch":44,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33800192,"num_objects":8252,"num_object_clones":0,"num_object_copies":16504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8252,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9329,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11706","reported_seq":11838,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:20.993784+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:17:20.993784+0000","last_peered":"2024-09-11T01:17:20.993784+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:17:20.993784+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35053568,"num_objects":8558,"num_object_clones":0,"num_object_copies":17116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8443,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4608},{"shards":"0,2","objects":2377}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684623+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684623+0000","last_peered":"2024-09-11T01:15:21.684623+0000","last_clean":"2024-09-11T01:15:21.684623+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684623+0000","last_undegraded":"2024-09-11T01:15:21.684623+0000","last_fullsized":"2024-09-11T01:15:21.684623+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11475","reported_seq":16853,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:20.980657+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:17:20.980657+0000","last_peered":"2024-09-11T01:17:20.980657+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:17:20.980657+0000","last_undegraded":"2024-09-11T01:17:20.980657+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34443264,"num_objects":8409,"num_object_clones":0,"num_object_copies":16818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1322,"num_objects_unfound":0,"num_objects_dirty":8409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8130,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2620,"num_bytes_recovered":10719232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7087},{"shards":"0,2","objects":1322}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11642","reported_seq":11777,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:20.939842+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:17:20.939842+0000","last_peered":"2024-09-11T01:17:20.939842+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:17:20.939842+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34955264,"num_objects":8534,"num_object_clones":0,"num_object_copies":17068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8251,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4495},{"shards":"0,2","objects":2487}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692049+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.692049+0000","last_peered":"2024-09-11T01:15:21.692049+0000","last_clean":"2024-09-11T01:15:21.692049+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.692049+0000","last_undegraded":"2024-09-11T01:15:21.692049+0000","last_fullsized":"2024-09-11T01:15:21.692049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11520","reported_seq":11654,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:21.014325+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:17:21.014325+0000","last_peered":"2024-09-11T01:17:21.014325+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:17:21.014325+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34538089,"num_objects":8433,"num_object_clones":0,"num_object_copies":16866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8590,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4516},{"shards":"0,2","objects":2379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":554226305,"num_objects":135311,"num_object_clones":0,"num_object_copies":270622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":1322,"num_objects_unfound":0,"num_objects_dirty":135311,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":146976,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6548,"num_bytes_recovered":26800128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1199583232,"data_stored":1199505508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160926,"ondisk_log_size":160926,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182470,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":841340,"kb_used_data":472364,"kb_used_omap":4,"kb_used_meta":368955,"kb_avail":93530500,"statfs":{"total":96636764160,"available":95775232000,"internally_reserved":0,"allocated":483700736,"data_stored":483441522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":377810603},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,3,10,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986243,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":699820,"kb_used_data":397468,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93672020,"statfs":{"total":96636764160,"available":95920148480,"internally_reserved":0,"allocated":407007232,"data_stored":406748023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,0,2,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659188,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589128,"kb_used_data":304824,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93782712,"statfs":{"total":96636764160,"available":96033497088,"internally_reserved":0,"allocated":312139776,"data_stored":311817347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":291106479},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,1,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":482127872,"data_stored":482124393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":311611392,"data_stored":311544698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":405843968,"data_stored":405836417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:25.300 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:26.074 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:17:26.077 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:17:26.133 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":229,"stamp":"2024-09-11T01:17:24.667676+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":554226324,"num_objects":135313,"num_object_clones":0,"num_object_copies":270626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":1322,"num_objects_unfound":0,"num_objects_dirty":135313,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":146978,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6549,"num_bytes_recovered":26800147,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160929,"ondisk_log_size":160929,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2130288,"kb_used_data":1174656,"kb_used_omap":12,"kb_used_meta":955571,"kb_avail":280985232,"statfs":{"total":289910292480,"available":287728877568,"internally_reserved":0,"allocated":1202847744,"data_stored":1202006892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":978504708},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,5,0,6,15,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21798912,"num_objects":-5322,"num_object_clones":0,"num_object_copies":-10644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-284,"num_objects_unfound":0,"num_objects_dirty":-5322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5322,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":284,"num_bytes_recovered":1163264,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008314"},"pg_stats":[{"pgid":"2.f","version":"45'11680","reported_seq":11758,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:21.020192+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:17:21.020192+0000","last_peered":"2024-09-11T01:17:21.020192+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:17:21.020192+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35168256,"num_objects":8586,"num_object_clones":0,"num_object_copies":17172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8586,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11680,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4514},{"shards":"0,2","objects":2525}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11885","reported_seq":11987,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:24.248241+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:17:24.248241+0000","last_peered":"2024-09-11T01:17:24.248241+0000","last_clean":"2024-09-11T01:17:24.248241+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:17:24.248241+0000","last_undegraded":"2024-09-11T01:17:24.248241+0000","last_fullsized":"2024-09-11T01:17:24.248241+0000","mapping_epoch":35,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34680832,"num_objects":8467,"num_object_clones":0,"num_object_copies":16934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10124,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11669","reported_seq":11798,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:24.271966+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:17:24.271966+0000","last_peered":"2024-09-11T01:17:24.271966+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:17:24.271966+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34709528,"num_objects":8475,"num_object_clones":0,"num_object_copies":16950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8475,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9970,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2301},{"shards":"2","objects":4579}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11692","reported_seq":11762,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:24.225142+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:17:24.225142+0000","last_peered":"2024-09-11T01:17:24.225142+0000","last_clean":"2024-09-11T01:17:24.225142+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:17:24.225142+0000","last_undegraded":"2024-09-11T01:17:24.225142+0000","last_fullsized":"2024-09-11T01:17:24.225142+0000","mapping_epoch":35,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34578432,"num_objects":8442,"num_object_clones":0,"num_object_copies":16884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11692,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11565","reported_seq":11693,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:20.945654+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:17:20.945654+0000","last_peered":"2024-09-11T01:17:20.945654+0000","last_clean":"2024-09-11T01:17:20.945654+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:17:20.945654+0000","last_undegraded":"2024-09-11T01:17:20.945654+0000","last_fullsized":"2024-09-11T01:17:20.945654+0000","mapping_epoch":34,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34586624,"num_objects":8444,"num_object_clones":0,"num_object_copies":16888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8570,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11686","reported_seq":11854,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:20.995922+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:17:20.995922+0000","last_peered":"2024-09-11T01:17:20.995922+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:17:20.995922+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34983936,"num_objects":8541,"num_object_clones":0,"num_object_copies":17082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8541,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8027,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4590},{"shards":"0,2","objects":2381}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11520","reported_seq":11651,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:20.898369+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:17:20.898369+0000","last_peered":"2024-09-11T01:17:20.898369+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:17:20.898369+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34447360,"num_objects":8410,"num_object_clones":0,"num_object_copies":16820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8410,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8377,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4424},{"shards":"0,2","objects":2434}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11710","reported_seq":11840,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:20.876691+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:17:20.876691+0000","last_peered":"2024-09-11T01:17:20.876691+0000","last_clean":"2024-09-11T01:17:20.876691+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:17:20.876691+0000","last_undegraded":"2024-09-11T01:17:20.876691+0000","last_fullsized":"2024-09-11T01:17:20.876691+0000","mapping_epoch":34,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34996224,"num_objects":8544,"num_object_clones":0,"num_object_copies":17088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8586,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683747+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683747+0000","last_peered":"2024-09-11T01:15:21.683747+0000","last_clean":"2024-09-11T01:15:21.683747+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683747+0000","last_undegraded":"2024-09-11T01:15:21.683747+0000","last_fullsized":"2024-09-11T01:15:21.683747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11567","reported_seq":11695,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:24.209347+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:17:24.209347+0000","last_peered":"2024-09-11T01:17:24.209347+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:17:24.209347+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34430976,"num_objects":8406,"num_object_clones":0,"num_object_copies":16812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9538,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2355},{"shards":"2","objects":4473}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11478","reported_seq":11607,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:20.985252+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:17:20.985252+0000","last_peered":"2024-09-11T01:17:20.985252+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:17:20.985252+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34365440,"num_objects":8390,"num_object_clones":0,"num_object_copies":16780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8065,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4406},{"shards":"0,2","objects":2444}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11787","reported_seq":11906,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:24.251148+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:17:24.251148+0000","last_peered":"2024-09-11T01:17:24.251148+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:17:24.251148+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34488320,"num_objects":8420,"num_object_clones":0,"num_object_copies":16840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9604,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2342},{"shards":"2","objects":4396}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684827+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684827+0000","last_peered":"2024-09-11T01:15:21.684827+0000","last_clean":"2024-09-11T01:15:21.684827+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684827+0000","last_undegraded":"2024-09-11T01:15:21.684827+0000","last_fullsized":"2024-09-11T01:15:21.684827+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11544","reported_seq":19523,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:24.212654+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:17:24.212654+0000","last_peered":"2024-09-11T01:17:24.212654+0000","last_clean":"2024-09-11T01:17:24.212654+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:17:24.212654+0000","last_undegraded":"2024-09-11T01:17:24.212654+0000","last_fullsized":"2024-09-11T01:17:24.212654+0000","mapping_epoch":44,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33800192,"num_objects":8252,"num_object_clones":0,"num_object_copies":16504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8252,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9329,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11706","reported_seq":11838,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:20.993784+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:17:20.993784+0000","last_peered":"2024-09-11T01:17:20.993784+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:17:20.993784+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":35053568,"num_objects":8558,"num_object_clones":0,"num_object_copies":17116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8558,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8443,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4608},{"shards":"0,2","objects":2377}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684623+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684623+0000","last_peered":"2024-09-11T01:15:21.684623+0000","last_clean":"2024-09-11T01:15:21.684623+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684623+0000","last_undegraded":"2024-09-11T01:15:21.684623+0000","last_fullsized":"2024-09-11T01:15:21.684623+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11475","reported_seq":16853,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:20.980657+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:17:20.980657+0000","last_peered":"2024-09-11T01:17:20.980657+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:17:20.980657+0000","last_undegraded":"2024-09-11T01:17:20.980657+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"36'1400","ondisk_log_start":"36'1400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34443264,"num_objects":8409,"num_object_clones":0,"num_object_copies":16818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1322,"num_objects_unfound":0,"num_objects_dirty":8409,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8130,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2620,"num_bytes_recovered":10719232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7087},{"shards":"0,2","objects":1322}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11642","reported_seq":11777,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:20.939842+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:17:20.939842+0000","last_peered":"2024-09-11T01:17:20.939842+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:17:20.939842+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34955264,"num_objects":8534,"num_object_clones":0,"num_object_copies":17068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8251,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4495},{"shards":"0,2","objects":2487}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692049+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.692049+0000","last_peered":"2024-09-11T01:15:21.692049+0000","last_clean":"2024-09-11T01:15:21.692049+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.692049+0000","last_undegraded":"2024-09-11T01:15:21.692049+0000","last_fullsized":"2024-09-11T01:15:21.692049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11520","reported_seq":11654,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:21.014325+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:17:21.014325+0000","last_peered":"2024-09-11T01:17:21.014325+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:17:21.014325+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34538089,"num_objects":8433,"num_object_clones":0,"num_object_copies":16866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8433,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8590,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4516},{"shards":"0,2","objects":2379}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":554226305,"num_objects":135311,"num_object_clones":0,"num_object_copies":270622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":1322,"num_objects_unfound":0,"num_objects_dirty":135311,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":146976,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6548,"num_bytes_recovered":26800128,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1199583232,"data_stored":1199505508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160926,"ondisk_log_size":160926,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182470,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":841340,"kb_used_data":472364,"kb_used_omap":4,"kb_used_meta":368955,"kb_avail":93530500,"statfs":{"total":96636764160,"available":95775232000,"internally_reserved":0,"allocated":483700736,"data_stored":483441522,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":377810603},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,3,10,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986243,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":699820,"kb_used_data":397468,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93672020,"statfs":{"total":96636764160,"available":95920148480,"internally_reserved":0,"allocated":407007232,"data_stored":406748023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,0,2,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659188,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589128,"kb_used_data":304824,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93782712,"statfs":{"total":96636764160,"available":96033497088,"internally_reserved":0,"allocated":312139776,"data_stored":311817347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":291106479},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,1,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":482127872,"data_stored":482124393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":311611392,"data_stored":311544698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":405843968,"data_stored":405836417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:26.134 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:17:26.134 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'11680", 'reported_seq': 11758, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:21.020192+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:17:21.020192+0000', 'last_peered': '2024-09-11T01:17:21.020192+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:17:21.020192+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "36'1600", 'ondisk_log_start': "36'1600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35168256, 'num_objects': 8586, 'num_object_clones': 0, 'num_object_copies': 17172, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8586, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11680, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4514}, {'shards': '0,2', 'objects': 2525}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:26.134 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:17:26.135 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'11669", 'reported_seq': 11798, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:24.271966+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:17:24.271966+0000', 'last_peered': '2024-09-11T01:17:24.271966+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:17:24.271966+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "36'1600", 'ondisk_log_start': "36'1600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34709528, 'num_objects': 8475, 'num_object_clones': 0, 'num_object_copies': 16950, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8475, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 9970, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2301}, {'shards': '2', 'objects': 4579}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:26.135 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:17:26.135 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'11686", 'reported_seq': 11854, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:20.995922+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:17:20.995922+0000', 'last_peered': '2024-09-11T01:17:20.995922+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:17:20.995922+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "36'1600", 'ondisk_log_start': "36'1600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34983936, 'num_objects': 8541, 'num_object_clones': 0, 'num_object_copies': 17082, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8541, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8027, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4590}, {'shards': '0,2', 'objects': 2381}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:26.135 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:17:26.135 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'11520", 'reported_seq': 11651, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:20.898369+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:17:20.898369+0000', 'last_peered': '2024-09-11T01:17:20.898369+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:17:20.898369+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34447360, 'num_objects': 8410, 'num_object_clones': 0, 'num_object_copies': 16820, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8410, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8377, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4424}, {'shards': '0,2', 'objects': 2434}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:26.135 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:17:26.135 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'11567", 'reported_seq': 11695, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:24.209347+0000', 'last_change': '2024-09-11T01:13:10.159069+0000', 'last_active': '2024-09-11T01:17:24.209347+0000', 'last_peered': '2024-09-11T01:17:24.209347+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:17:24.209347+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34430976, 'num_objects': 8406, 'num_object_clones': 0, 'num_object_copies': 16812, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8406, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9538, 'num_write_kb': 31844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2355}, {'shards': '2', 'objects': 4473}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:26.135 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:17:26.135 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'11478", 'reported_seq': 11607, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:20.985252+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:17:20.985252+0000', 'last_peered': '2024-09-11T01:17:20.985252+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:17:20.985252+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34365440, 'num_objects': 8390, 'num_object_clones': 0, 'num_object_copies': 16780, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8390, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8065, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4406}, {'shards': '0,2', 'objects': 2444}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:26.135 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:17:26.136 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'11787", 'reported_seq': 11906, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:24.251148+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:17:24.251148+0000', 'last_peered': '2024-09-11T01:17:24.251148+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:17:24.251148+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34488320, 'num_objects': 8420, 'num_object_clones': 0, 'num_object_copies': 16840, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8420, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9604, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2342}, {'shards': '2', 'objects': 4396}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:26.136 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:17:26.136 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'11706", 'reported_seq': 11838, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:20.993784+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:17:20.993784+0000', 'last_peered': '2024-09-11T01:17:20.993784+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:17:20.993784+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 35053568, 'num_objects': 8558, 'num_object_clones': 0, 'num_object_copies': 17116, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8558, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8443, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4608}, {'shards': '0,2', 'objects': 2377}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:26.136 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:17:26.136 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'11475", 'reported_seq': 16853, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:17:20.980657+0000', 'last_change': '2024-09-11T01:15:19.913288+0000', 'last_active': '2024-09-11T01:17:20.980657+0000', 'last_peered': '2024-09-11T01:17:20.980657+0000', 'last_clean': '2024-09-11T01:13:07.624565+0000', 'last_became_active': '2024-09-11T01:13:10.199795+0000', 'last_became_peered': '2024-09-11T01:13:10.199795+0000', 'last_unstale': '2024-09-11T01:17:20.980657+0000', 'last_undegraded': '2024-09-11T01:17:20.980657+0000', 'last_fullsized': '2024-09-11T01:13:09.652311+0000', 'mapping_epoch': 42, 'log_start': "36'1400", 'ondisk_log_start': "36'1400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3415", 'last_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'objects_scrubbed': 3337, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1975, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34443264, 'num_objects': 8409, 'num_object_clones': 0, 'num_object_copies': 16818, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1322, 'num_objects_unfound': 0, 'num_objects_dirty': 8409, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8130, 'num_write_kb': 26404, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2620, 'num_bytes_recovered': 10719232, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7087}, {'shards': '0,2', 'objects': 1322}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:26.136 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:17:26.136 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'11642", 'reported_seq': 11777, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:20.939842+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:17:20.939842+0000', 'last_peered': '2024-09-11T01:17:20.939842+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:17:20.939842+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "36'1600", 'ondisk_log_start': "36'1600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34955264, 'num_objects': 8534, 'num_object_clones': 0, 'num_object_copies': 17068, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8534, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8251, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4495}, {'shards': '0,2', 'objects': 2487}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:26.136 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:17:26.136 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'11520", 'reported_seq': 11654, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:21.014325+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:17:21.014325+0000', 'last_peered': '2024-09-11T01:17:21.014325+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:17:21.014325+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "36'1500", 'ondisk_log_start': "36'1500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34538089, 'num_objects': 8433, 'num_object_clones': 0, 'num_object_copies': 16866, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8433, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8590, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4516}, {'shards': '0,2', 'objects': 2379}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:26.137 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:17:27.014 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:17:27.014 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39769/267776 objects degraded (14.852%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":383,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":133888,"data_bytes":548389524,"bytes_used":2160402432,"bytes_avail":287749890048,"bytes_total":289910292480,"degraded_objects":39769,"degraded_total":267776,"degraded_ratio":0.14851592375717018,"misplaced_objects":1189,"misplaced_total":267776,"misplaced_ratio":0.0044402784416826,"recovering_objects_per_sec":34,"recovering_bytes_per_sec":142237,"recovering_keys_per_sec":0,"num_objects_recovered":417,"num_bytes_recovered":1708032,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":494},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:17:27.129 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:17:27.129 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:27.900 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:17:27.903 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:17:27.967 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":230,"stamp":"2024-09-11T01:17:26.668153+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":548389524,"num_objects":133888,"num_object_clones":0,"num_object_copies":267776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":1189,"num_objects_unfound":0,"num_objects_dirty":133888,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":148403,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6682,"num_bytes_recovered":27344915,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160954,"ondisk_log_size":160954,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2109768,"kb_used_data":1166616,"kb_used_omap":12,"kb_used_meta":943091,"kb_avail":281005752,"statfs":{"total":289910292480,"available":287749890048,"internally_reserved":0,"allocated":1194614784,"data_stored":1193773932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":965725188},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,8,1,3,11,8,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24309760,"num_objects":-5935,"num_object_clones":0,"num_object_copies":-11870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-417,"num_objects_unfound":0,"num_objects_dirty":-5935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5935,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":417,"num_bytes_recovered":1708032,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008301"},"pg_stats":[{"pgid":"2.f","version":"45'11797","reported_seq":11875,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:26.044119+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:17:26.044119+0000","last_peered":"2024-09-11T01:17:26.044119+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:17:26.044119+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34689024,"num_objects":8469,"num_object_clones":0,"num_object_copies":16938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11797,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4397},{"shards":"0,2","objects":2408}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'11885","reported_seq":11987,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:24.248241+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:17:24.248241+0000","last_peered":"2024-09-11T01:17:24.248241+0000","last_clean":"2024-09-11T01:17:24.248241+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:17:24.248241+0000","last_undegraded":"2024-09-11T01:17:24.248241+0000","last_fullsized":"2024-09-11T01:17:24.248241+0000","mapping_epoch":35,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34680832,"num_objects":8467,"num_object_clones":0,"num_object_copies":16934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10124,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11669","reported_seq":11798,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:24.271966+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:17:24.271966+0000","last_peered":"2024-09-11T01:17:24.271966+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:17:24.271966+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34709528,"num_objects":8475,"num_object_clones":0,"num_object_copies":16950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8475,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9970,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2301},{"shards":"2","objects":4579}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11692","reported_seq":11762,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:24.225142+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:17:24.225142+0000","last_peered":"2024-09-11T01:17:24.225142+0000","last_clean":"2024-09-11T01:17:24.225142+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:17:24.225142+0000","last_undegraded":"2024-09-11T01:17:24.225142+0000","last_fullsized":"2024-09-11T01:17:24.225142+0000","mapping_epoch":35,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34578432,"num_objects":8442,"num_object_clones":0,"num_object_copies":16884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8442,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11692,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11721","reported_seq":11849,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:25.967470+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:17:25.967470+0000","last_peered":"2024-09-11T01:17:25.967470+0000","last_clean":"2024-09-11T01:17:25.967470+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:17:25.967470+0000","last_undegraded":"2024-09-11T01:17:25.967470+0000","last_fullsized":"2024-09-11T01:17:25.967470+0000","mapping_epoch":34,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33947648,"num_objects":8288,"num_object_clones":0,"num_object_copies":16576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8288,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8726,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11834","reported_seq":12005,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:25.911177+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:17:25.911177+0000","last_peered":"2024-09-11T01:17:25.911177+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:17:25.911177+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34377728,"num_objects":8393,"num_object_clones":0,"num_object_copies":16786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8393,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8175,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4442},{"shards":"0,2","objects":2233}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11669","reported_seq":11797,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:26.065736+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:17:26.065736+0000","last_peered":"2024-09-11T01:17:26.065736+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:17:26.065736+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33837056,"num_objects":8261,"num_object_clones":0,"num_object_copies":16522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8526,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4275},{"shards":"0,2","objects":2285}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11867","reported_seq":11996,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:26.044114+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:17:26.044114+0000","last_peered":"2024-09-11T01:17:26.044114+0000","last_clean":"2024-09-11T01:17:26.044114+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:17:26.044114+0000","last_undegraded":"2024-09-11T01:17:26.044114+0000","last_fullsized":"2024-09-11T01:17:26.044114+0000","mapping_epoch":34,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34353152,"num_objects":8387,"num_object_clones":0,"num_object_copies":16774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8743,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683747+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683747+0000","last_peered":"2024-09-11T01:15:21.683747+0000","last_clean":"2024-09-11T01:15:21.683747+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683747+0000","last_undegraded":"2024-09-11T01:15:21.683747+0000","last_fullsized":"2024-09-11T01:15:21.683747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11567","reported_seq":11695,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:24.209347+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:17:24.209347+0000","last_peered":"2024-09-11T01:17:24.209347+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:17:24.209347+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34430976,"num_objects":8406,"num_object_clones":0,"num_object_copies":16812,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8406,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9538,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2355},{"shards":"2","objects":4473}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11620","reported_seq":11749,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:26.064779+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:17:26.064779+0000","last_peered":"2024-09-11T01:17:26.064779+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:17:26.064779+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33783808,"num_objects":8248,"num_object_clones":0,"num_object_copies":16496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8207,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4264},{"shards":"0,2","objects":2302}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11787","reported_seq":11906,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:24.251148+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:17:24.251148+0000","last_peered":"2024-09-11T01:17:24.251148+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:17:24.251148+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34488320,"num_objects":8420,"num_object_clones":0,"num_object_copies":16840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9604,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2342},{"shards":"2","objects":4396}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684827+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684827+0000","last_peered":"2024-09-11T01:15:21.684827+0000","last_clean":"2024-09-11T01:15:21.684827+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684827+0000","last_undegraded":"2024-09-11T01:15:21.684827+0000","last_fullsized":"2024-09-11T01:15:21.684827+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11544","reported_seq":19523,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:24.212654+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:17:24.212654+0000","last_peered":"2024-09-11T01:17:24.212654+0000","last_clean":"2024-09-11T01:17:24.212654+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:17:24.212654+0000","last_undegraded":"2024-09-11T01:17:24.212654+0000","last_fullsized":"2024-09-11T01:17:24.212654+0000","mapping_epoch":44,"log_start":"36'1500","ondisk_log_start":"36'1500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33800192,"num_objects":8252,"num_object_clones":0,"num_object_copies":16504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8252,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9329,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11859","reported_seq":11990,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:26.123584+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:17:26.123584+0000","last_peered":"2024-09-11T01:17:26.123584+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:17:26.123584+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34426880,"num_objects":8405,"num_object_clones":0,"num_object_copies":16810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8596,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4455},{"shards":"0,2","objects":2224}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684623+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684623+0000","last_peered":"2024-09-11T01:15:21.684623+0000","last_clean":"2024-09-11T01:15:21.684623+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684623+0000","last_undegraded":"2024-09-11T01:15:21.684623+0000","last_fullsized":"2024-09-11T01:15:21.684623+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11630","reported_seq":17274,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:26.128849+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:17:26.128849+0000","last_peered":"2024-09-11T01:17:26.128849+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:17:26.128849+0000","last_undegraded":"2024-09-11T01:17:26.128849+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33808384,"num_objects":8254,"num_object_clones":0,"num_object_copies":16508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1189,"num_objects_unfound":0,"num_objects_dirty":8254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8285,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2753,"num_bytes_recovered":11264000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7065},{"shards":"0,2","objects":1189}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11786","reported_seq":11921,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:26.111542+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:17:26.111542+0000","last_peered":"2024-09-11T01:17:26.111542+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:17:26.111542+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34365440,"num_objects":8390,"num_object_clones":0,"num_object_copies":16780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8395,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4351},{"shards":"0,2","objects":2343}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692049+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.692049+0000","last_peered":"2024-09-11T01:15:21.692049+0000","last_clean":"2024-09-11T01:15:21.692049+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.692049+0000","last_undegraded":"2024-09-11T01:15:21.692049+0000","last_fullsized":"2024-09-11T01:15:21.692049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":74,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684358+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684358+0000","last_peered":"2024-09-11T01:15:21.684358+0000","last_clean":"2024-09-11T01:15:21.684358+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684358+0000","last_undegraded":"2024-09-11T01:15:21.684358+0000","last_fullsized":"2024-09-11T01:15:21.684358+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11624","reported_seq":11758,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:26.021428+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:17:26.021428+0000","last_peered":"2024-09-11T01:17:26.021428+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:17:26.021428+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34112105,"num_objects":8329,"num_object_clones":0,"num_object_copies":16658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8694,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4412},{"shards":"0,2","objects":2275}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":548389505,"num_objects":133886,"num_object_clones":0,"num_object_copies":267772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":1189,"num_objects_unfound":0,"num_objects_dirty":133886,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":148401,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6681,"num_bytes_recovered":27344896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1192423424,"data_stored":1192345700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160951,"ondisk_log_size":160951,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182471,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":820820,"kb_used_data":464324,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93551020,"statfs":{"total":96636764160,"available":95796244480,"internally_reserved":0,"allocated":475467776,"data_stored":475208562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":365031083},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,0,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986243,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":699820,"kb_used_data":397468,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93672020,"statfs":{"total":96636764160,"available":95920148480,"internally_reserved":0,"allocated":407007232,"data_stored":406748023,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,0,2,2,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659188,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589128,"kb_used_data":304824,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93782712,"statfs":{"total":96636764160,"available":96033497088,"internally_reserved":0,"allocated":312139776,"data_stored":311817347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":291106479},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,0,1,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":474968064,"data_stored":474964585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":311611392,"data_stored":311544698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":405843968,"data_stored":405836417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:30.968 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:31.943 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:17:31.946 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:17:31.983 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":232,"stamp":"2024-09-11T01:17:30.669132+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":544993940,"num_objects":133059,"num_object_clones":0,"num_object_copies":266118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":1189,"num_objects_unfound":0,"num_objects_dirty":133059,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":149232,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6682,"num_bytes_recovered":27344915,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160783,"ondisk_log_size":160783,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2113080,"kb_used_data":1161608,"kb_used_omap":12,"kb_used_meta":951411,"kb_avail":281002440,"statfs":{"total":289910292480,"available":287746498560,"internally_reserved":0,"allocated":1189486592,"data_stored":1188645740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":974244868},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,3,5,7,6,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21921792,"num_objects":-5352,"num_object_clones":0,"num_object_copies":-10704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-274,"num_objects_unfound":0,"num_objects_dirty":-5352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5352,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":274,"num_bytes_recovered":1122304,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008294"},"pg_stats":[{"pgid":"2.f","version":"45'11797","reported_seq":11875,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:26.044119+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:17:26.044119+0000","last_peered":"2024-09-11T01:17:26.044119+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:17:26.044119+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34689024,"num_objects":8469,"num_object_clones":0,"num_object_copies":16938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8469,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11797,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4397},{"shards":"0,2","objects":2408}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12055","reported_seq":12157,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:29.300908+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:17:29.300908+0000","last_peered":"2024-09-11T01:17:29.300908+0000","last_clean":"2024-09-11T01:17:29.300908+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:17:29.300908+0000","last_undegraded":"2024-09-11T01:17:29.300908+0000","last_fullsized":"2024-09-11T01:17:29.300908+0000","mapping_epoch":35,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33984512,"num_objects":8297,"num_object_clones":0,"num_object_copies":16594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10294,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11787","reported_seq":11916,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:29.259744+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:17:29.259744+0000","last_peered":"2024-09-11T01:17:29.259744+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:17:29.259744+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34226200,"num_objects":8357,"num_object_clones":0,"num_object_copies":16714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8357,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10088,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2183},{"shards":"2","objects":4461}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11819","reported_seq":11889,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:29.267268+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:17:29.267268+0000","last_peered":"2024-09-11T01:17:29.267268+0000","last_clean":"2024-09-11T01:17:29.267268+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:17:29.267268+0000","last_undegraded":"2024-09-11T01:17:29.267268+0000","last_fullsized":"2024-09-11T01:17:29.267268+0000","mapping_epoch":35,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34058240,"num_objects":8315,"num_object_clones":0,"num_object_copies":16630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8315,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11819,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11721","reported_seq":11849,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:25.967470+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:17:25.967470+0000","last_peered":"2024-09-11T01:17:25.967470+0000","last_clean":"2024-09-11T01:17:25.967470+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:17:25.967470+0000","last_undegraded":"2024-09-11T01:17:25.967470+0000","last_fullsized":"2024-09-11T01:17:25.967470+0000","mapping_epoch":34,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33947648,"num_objects":8288,"num_object_clones":0,"num_object_copies":16576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8288,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8726,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11834","reported_seq":12005,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:25.911177+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:17:25.911177+0000","last_peered":"2024-09-11T01:17:25.911177+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:17:25.911177+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34377728,"num_objects":8393,"num_object_clones":0,"num_object_copies":16786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8393,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8175,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4442},{"shards":"0,2","objects":2233}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11669","reported_seq":11797,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:26.065736+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:17:26.065736+0000","last_peered":"2024-09-11T01:17:26.065736+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:17:26.065736+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33837056,"num_objects":8261,"num_object_clones":0,"num_object_copies":16522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8526,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4275},{"shards":"0,2","objects":2285}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'11867","reported_seq":11996,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:26.044114+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:17:26.044114+0000","last_peered":"2024-09-11T01:17:26.044114+0000","last_clean":"2024-09-11T01:17:26.044114+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:17:26.044114+0000","last_undegraded":"2024-09-11T01:17:26.044114+0000","last_fullsized":"2024-09-11T01:17:26.044114+0000","mapping_epoch":34,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34353152,"num_objects":8387,"num_object_clones":0,"num_object_copies":16774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8387,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8743,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683747+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683747+0000","last_peered":"2024-09-11T01:15:21.683747+0000","last_clean":"2024-09-11T01:15:21.683747+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683747+0000","last_undegraded":"2024-09-11T01:15:21.683747+0000","last_fullsized":"2024-09-11T01:15:21.683747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11718","reported_seq":11846,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:29.298827+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:17:29.298827+0000","last_peered":"2024-09-11T01:17:29.298827+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:17:29.298827+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33812480,"num_objects":8255,"num_object_clones":0,"num_object_copies":16510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9689,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2204},{"shards":"2","objects":4322}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11620","reported_seq":11749,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:26.064779+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:17:26.064779+0000","last_peered":"2024-09-11T01:17:26.064779+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:17:26.064779+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33783808,"num_objects":8248,"num_object_clones":0,"num_object_copies":16496,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8248,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8207,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4264},{"shards":"0,2","objects":2302}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11912","reported_seq":12027,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:29.285628+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:17:29.285628+0000","last_peered":"2024-09-11T01:17:29.285628+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:17:29.285628+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33976320,"num_objects":8295,"num_object_clones":0,"num_object_copies":16590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9729,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2217},{"shards":"2","objects":4271}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684827+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684827+0000","last_peered":"2024-09-11T01:15:21.684827+0000","last_clean":"2024-09-11T01:15:21.684827+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684827+0000","last_undegraded":"2024-09-11T01:15:21.684827+0000","last_fullsized":"2024-09-11T01:15:21.684827+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11682","reported_seq":19663,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:29.304738+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:17:29.304738+0000","last_peered":"2024-09-11T01:17:29.304738+0000","last_clean":"2024-09-11T01:17:29.304738+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:17:29.304738+0000","last_undegraded":"2024-09-11T01:17:29.304738+0000","last_fullsized":"2024-09-11T01:17:29.304738+0000","mapping_epoch":44,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33234944,"num_objects":8114,"num_object_clones":0,"num_object_copies":16228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9467,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11859","reported_seq":11990,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:26.123584+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:17:26.123584+0000","last_peered":"2024-09-11T01:17:26.123584+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:17:26.123584+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34426880,"num_objects":8405,"num_object_clones":0,"num_object_copies":16810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8596,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4455},{"shards":"0,2","objects":2224}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684623+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684623+0000","last_peered":"2024-09-11T01:15:21.684623+0000","last_clean":"2024-09-11T01:15:21.684623+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684623+0000","last_undegraded":"2024-09-11T01:15:21.684623+0000","last_fullsized":"2024-09-11T01:15:21.684623+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11630","reported_seq":17274,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:26.128849+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:17:26.128849+0000","last_peered":"2024-09-11T01:17:26.128849+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:17:26.128849+0000","last_undegraded":"2024-09-11T01:17:26.128849+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33808384,"num_objects":8254,"num_object_clones":0,"num_object_copies":16508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1189,"num_objects_unfound":0,"num_objects_dirty":8254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8285,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2753,"num_bytes_recovered":11264000,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7065},{"shards":"0,2","objects":1189}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11786","reported_seq":11921,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:26.111542+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:17:26.111542+0000","last_peered":"2024-09-11T01:17:26.111542+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:17:26.111542+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34365440,"num_objects":8390,"num_object_clones":0,"num_object_copies":16780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8395,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4351},{"shards":"0,2","objects":2343}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692049+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.692049+0000","last_peered":"2024-09-11T01:15:21.692049+0000","last_clean":"2024-09-11T01:15:21.692049+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.692049+0000","last_undegraded":"2024-09-11T01:15:21.692049+0000","last_fullsized":"2024-09-11T01:15:21.692049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684417+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684417+0000","last_peered":"2024-09-11T01:15:21.684417+0000","last_clean":"2024-09-11T01:15:21.684417+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684417+0000","last_undegraded":"2024-09-11T01:15:21.684417+0000","last_fullsized":"2024-09-11T01:15:21.684417+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11624","reported_seq":11758,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:26.021428+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:17:26.021428+0000","last_peered":"2024-09-11T01:17:26.021428+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:17:26.021428+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34112105,"num_objects":8329,"num_object_clones":0,"num_object_copies":16658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8694,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4412},{"shards":"0,2","objects":2275}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":544993921,"num_objects":133057,"num_object_clones":0,"num_object_copies":266114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":1189,"num_objects_unfound":0,"num_objects_dirty":133057,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":149230,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6681,"num_bytes_recovered":27344896,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187278848,"data_stored":1187201124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160780,"ondisk_log_size":160780,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182471,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":820820,"kb_used_data":464324,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93551020,"statfs":{"total":96636764160,"available":95796244480,"internally_reserved":0,"allocated":475467776,"data_stored":475208562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":365031083},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,0,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986244,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":699704,"kb_used_data":393192,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93672136,"statfs":{"total":96636764160,"available":95920267264,"internally_reserved":0,"allocated":402628608,"data_stored":402369399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,0,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659189,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592556,"kb_used_data":304092,"kb_used_omap":4,"kb_used_meta":288443,"kb_avail":93779284,"statfs":{"total":96636764160,"available":96029986816,"internally_reserved":0,"allocated":311390208,"data_stored":311067779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":295366319},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,1,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":474968064,"data_stored":474964585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":310943744,"data_stored":310877050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":401367040,"data_stored":401359489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:31.985 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:32.801 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:17:32.803 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:17:32.841 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":233,"stamp":"2024-09-11T01:17:32.669616+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":539177620,"num_objects":131639,"num_object_clones":0,"num_object_copies":263278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":1039,"num_objects_unfound":0,"num_objects_dirty":131639,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":150652,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6832,"num_bytes_recovered":27959315,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160803,"ondisk_log_size":160803,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2111768,"kb_used_data":1156136,"kb_used_omap":12,"kb_used_meta":955571,"kb_avail":281003752,"statfs":{"total":289910292480,"available":287747842048,"internally_reserved":0,"allocated":1183883264,"data_stored":1183042412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":978504708},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,3,10,6,4,11],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24182784,"num_objects":-5904,"num_object_clones":0,"num_object_copies":-11808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-424,"num_objects_unfound":0,"num_objects_dirty":-5904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5904,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":424,"num_bytes_recovered":1736704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008354"},"pg_stats":[{"pgid":"2.f","version":"45'11932","reported_seq":12010,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:31.000332+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:17:31.000332+0000","last_peered":"2024-09-11T01:17:31.000332+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:17:31.000332+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34136064,"num_objects":8334,"num_object_clones":0,"num_object_copies":16668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11932,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4262},{"shards":"0,2","objects":2273}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12055","reported_seq":12157,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:29.300908+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:17:29.300908+0000","last_peered":"2024-09-11T01:17:29.300908+0000","last_clean":"2024-09-11T01:17:29.300908+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:17:29.300908+0000","last_undegraded":"2024-09-11T01:17:29.300908+0000","last_fullsized":"2024-09-11T01:17:29.300908+0000","mapping_epoch":35,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33984512,"num_objects":8297,"num_object_clones":0,"num_object_copies":16594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10294,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11787","reported_seq":11916,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:29.259744+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:17:29.259744+0000","last_peered":"2024-09-11T01:17:29.259744+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:17:29.259744+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34226200,"num_objects":8357,"num_object_clones":0,"num_object_copies":16714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8357,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10088,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2183},{"shards":"2","objects":4461}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11819","reported_seq":11889,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:29.267268+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:17:29.267268+0000","last_peered":"2024-09-11T01:17:29.267268+0000","last_clean":"2024-09-11T01:17:29.267268+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:17:29.267268+0000","last_undegraded":"2024-09-11T01:17:29.267268+0000","last_fullsized":"2024-09-11T01:17:29.267268+0000","mapping_epoch":35,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34058240,"num_objects":8315,"num_object_clones":0,"num_object_copies":16630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8315,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11819,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11852","reported_seq":11980,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:31.186635+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:17:31.186635+0000","last_peered":"2024-09-11T01:17:31.186635+0000","last_clean":"2024-09-11T01:17:31.186635+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:17:31.186635+0000","last_undegraded":"2024-09-11T01:17:31.186635+0000","last_fullsized":"2024-09-11T01:17:31.186635+0000","mapping_epoch":34,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33411072,"num_objects":8157,"num_object_clones":0,"num_object_copies":16314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8857,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11990","reported_seq":12161,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:31.135868+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:17:31.135868+0000","last_peered":"2024-09-11T01:17:31.135868+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:17:31.135868+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33738752,"num_objects":8237,"num_object_clones":0,"num_object_copies":16474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8331,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4286},{"shards":"0,2","objects":2077}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11817","reported_seq":11948,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:31.081870+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:17:31.081870+0000","last_peered":"2024-09-11T01:17:31.081870+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:17:31.081870+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33230848,"num_objects":8113,"num_object_clones":0,"num_object_copies":16226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8674,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4127},{"shards":"0,2","objects":2137}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12013","reported_seq":12143,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:31.087597+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:17:31.087597+0000","last_peered":"2024-09-11T01:17:31.087597+0000","last_clean":"2024-09-11T01:17:31.087597+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:17:31.087597+0000","last_undegraded":"2024-09-11T01:17:31.087597+0000","last_fullsized":"2024-09-11T01:17:31.087597+0000","mapping_epoch":34,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33755136,"num_objects":8241,"num_object_clones":0,"num_object_copies":16482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8889,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683747+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683747+0000","last_peered":"2024-09-11T01:15:21.683747+0000","last_clean":"2024-09-11T01:15:21.683747+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683747+0000","last_undegraded":"2024-09-11T01:15:21.683747+0000","last_fullsized":"2024-09-11T01:15:21.683747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11718","reported_seq":11846,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:29.298827+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:17:29.298827+0000","last_peered":"2024-09-11T01:17:29.298827+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:17:29.298827+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33812480,"num_objects":8255,"num_object_clones":0,"num_object_copies":16510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9689,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2204},{"shards":"2","objects":4322}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11766","reported_seq":11895,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:31.163990+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:17:31.163990+0000","last_peered":"2024-09-11T01:17:31.163990+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:17:31.163990+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33185792,"num_objects":8102,"num_object_clones":0,"num_object_copies":16204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8353,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4118},{"shards":"0,2","objects":2156}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11912","reported_seq":12027,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:29.285628+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:17:29.285628+0000","last_peered":"2024-09-11T01:17:29.285628+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:17:29.285628+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33976320,"num_objects":8295,"num_object_clones":0,"num_object_copies":16590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9729,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2217},{"shards":"2","objects":4271}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684827+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684827+0000","last_peered":"2024-09-11T01:15:21.684827+0000","last_clean":"2024-09-11T01:15:21.684827+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684827+0000","last_undegraded":"2024-09-11T01:15:21.684827+0000","last_fullsized":"2024-09-11T01:15:21.684827+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11682","reported_seq":19663,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:29.304738+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:17:29.304738+0000","last_peered":"2024-09-11T01:17:29.304738+0000","last_clean":"2024-09-11T01:17:29.304738+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:17:29.304738+0000","last_undegraded":"2024-09-11T01:17:29.304738+0000","last_fullsized":"2024-09-11T01:17:29.304738+0000","mapping_epoch":44,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33234944,"num_objects":8114,"num_object_clones":0,"num_object_copies":16228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9467,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11988","reported_seq":12120,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:31.127831+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:17:31.127831+0000","last_peered":"2024-09-11T01:17:31.127831+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:17:31.127831+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33898496,"num_objects":8276,"num_object_clones":0,"num_object_copies":16552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8276,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8725,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4326},{"shards":"0,2","objects":2095}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684623+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684623+0000","last_peered":"2024-09-11T01:15:21.684623+0000","last_clean":"2024-09-11T01:15:21.684623+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684623+0000","last_undegraded":"2024-09-11T01:15:21.684623+0000","last_fullsized":"2024-09-11T01:15:21.684623+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11779","reported_seq":17723,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:31.166848+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:17:31.166848+0000","last_peered":"2024-09-11T01:17:31.166848+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:17:31.166848+0000","last_undegraded":"2024-09-11T01:17:31.166848+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33198080,"num_objects":8105,"num_object_clones":0,"num_object_copies":16210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1039,"num_objects_unfound":0,"num_objects_dirty":8105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8434,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2903,"num_bytes_recovered":11878400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7066},{"shards":"0,2","objects":1039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11920","reported_seq":12055,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:31.100348+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:17:31.100348+0000","last_peered":"2024-09-11T01:17:31.100348+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:17:31.100348+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33816576,"num_objects":8256,"num_object_clones":0,"num_object_copies":16512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8529,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4217},{"shards":"0,2","objects":2209}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692049+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.692049+0000","last_peered":"2024-09-11T01:15:21.692049+0000","last_clean":"2024-09-11T01:15:21.692049+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.692049+0000","last_undegraded":"2024-09-11T01:15:21.692049+0000","last_fullsized":"2024-09-11T01:15:21.692049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684417+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684417+0000","last_peered":"2024-09-11T01:15:21.684417+0000","last_clean":"2024-09-11T01:15:21.684417+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684417+0000","last_undegraded":"2024-09-11T01:15:21.684417+0000","last_fullsized":"2024-09-11T01:15:21.684417+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11770","reported_seq":11904,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:31.101377+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:17:31.101377+0000","last_peered":"2024-09-11T01:17:31.101377+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:17:31.101377+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33514089,"num_objects":8183,"num_object_clones":0,"num_object_copies":16366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8840,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4266},{"shards":"0,2","objects":2129}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":539177601,"num_objects":131637,"num_object_clones":0,"num_object_copies":263274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":1039,"num_objects_unfound":0,"num_objects_dirty":131637,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":150650,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6831,"num_bytes_recovered":27959296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1180377088,"data_stored":1180299364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160800,"ondisk_log_size":160800,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182472,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":819508,"kb_used_data":458852,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93552332,"statfs":{"total":96636764160,"available":95797587968,"internally_reserved":0,"allocated":469864448,"data_stored":469605234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":369290923},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,5,5,1,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986244,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":699704,"kb_used_data":393192,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93672136,"statfs":{"total":96636764160,"available":95920267264,"internally_reserved":0,"allocated":402628608,"data_stored":402369399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,0,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659189,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592556,"kb_used_data":304092,"kb_used_omap":4,"kb_used_meta":288443,"kb_avail":93779284,"statfs":{"total":96636764160,"available":96029986816,"internally_reserved":0,"allocated":311390208,"data_stored":311067779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":295366319},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,1,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":468066304,"data_stored":468062825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":310943744,"data_stored":310877050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":401367040,"data_stored":401359489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:32.843 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:17:32.843 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'11932", 'reported_seq': 12010, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:31.000332+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:17:31.000332+0000', 'last_peered': '2024-09-11T01:17:31.000332+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:17:31.000332+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34136064, 'num_objects': 8334, 'num_object_clones': 0, 'num_object_copies': 16668, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8334, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11932, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4262}, {'shards': '0,2', 'objects': 2273}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:32.843 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:17:32.843 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'11787", 'reported_seq': 11916, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:29.259744+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:17:29.259744+0000', 'last_peered': '2024-09-11T01:17:29.259744+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:17:29.259744+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 34226200, 'num_objects': 8357, 'num_object_clones': 0, 'num_object_copies': 16714, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8357, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10088, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2183}, {'shards': '2', 'objects': 4461}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:32.843 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:17:32.843 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'11990", 'reported_seq': 12161, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:31.135868+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:17:31.135868+0000', 'last_peered': '2024-09-11T01:17:31.135868+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:17:31.135868+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33738752, 'num_objects': 8237, 'num_object_clones': 0, 'num_object_copies': 16474, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8237, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8331, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4286}, {'shards': '0,2', 'objects': 2077}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:32.844 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:17:32.844 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'11817", 'reported_seq': 11948, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:31.081870+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:17:31.081870+0000', 'last_peered': '2024-09-11T01:17:31.081870+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:17:31.081870+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33230848, 'num_objects': 8113, 'num_object_clones': 0, 'num_object_copies': 16226, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8113, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8674, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4127}, {'shards': '0,2', 'objects': 2137}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:32.844 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:17:32.844 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'11718", 'reported_seq': 11846, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:29.298827+0000', 'last_change': '2024-09-11T01:13:10.159069+0000', 'last_active': '2024-09-11T01:17:29.298827+0000', 'last_peered': '2024-09-11T01:17:29.298827+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:17:29.298827+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33812480, 'num_objects': 8255, 'num_object_clones': 0, 'num_object_copies': 16510, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8255, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9689, 'num_write_kb': 31844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2204}, {'shards': '2', 'objects': 4322}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:32.844 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:17:32.844 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'11766", 'reported_seq': 11895, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:31.163990+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:17:31.163990+0000', 'last_peered': '2024-09-11T01:17:31.163990+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:17:31.163990+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33185792, 'num_objects': 8102, 'num_object_clones': 0, 'num_object_copies': 16204, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8102, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8353, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4118}, {'shards': '0,2', 'objects': 2156}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:32.844 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:17:32.844 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'11912", 'reported_seq': 12027, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:29.285628+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:17:29.285628+0000', 'last_peered': '2024-09-11T01:17:29.285628+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:17:29.285628+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33976320, 'num_objects': 8295, 'num_object_clones': 0, 'num_object_copies': 16590, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8295, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9729, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2217}, {'shards': '2', 'objects': 4271}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:32.844 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:17:32.844 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'11988", 'reported_seq': 12120, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:31.127831+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:17:31.127831+0000', 'last_peered': '2024-09-11T01:17:31.127831+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:17:31.127831+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33898496, 'num_objects': 8276, 'num_object_clones': 0, 'num_object_copies': 16552, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8276, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8725, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4326}, {'shards': '0,2', 'objects': 2095}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:32.845 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:17:32.845 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'11779", 'reported_seq': 17723, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:17:31.166848+0000', 'last_change': '2024-09-11T01:15:19.913288+0000', 'last_active': '2024-09-11T01:17:31.166848+0000', 'last_peered': '2024-09-11T01:17:31.166848+0000', 'last_clean': '2024-09-11T01:13:07.624565+0000', 'last_became_active': '2024-09-11T01:13:10.199795+0000', 'last_became_peered': '2024-09-11T01:13:10.199795+0000', 'last_unstale': '2024-09-11T01:17:31.166848+0000', 'last_undegraded': '2024-09-11T01:17:31.166848+0000', 'last_fullsized': '2024-09-11T01:13:09.652311+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3415", 'last_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'objects_scrubbed': 3337, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1975, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33198080, 'num_objects': 8105, 'num_object_clones': 0, 'num_object_copies': 16210, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 1039, 'num_objects_unfound': 0, 'num_objects_dirty': 8105, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8434, 'num_write_kb': 26404, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2903, 'num_bytes_recovered': 11878400, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7066}, {'shards': '0,2', 'objects': 1039}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:32.845 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:17:32.845 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'11920", 'reported_seq': 12055, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:31.100348+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:17:31.100348+0000', 'last_peered': '2024-09-11T01:17:31.100348+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:17:31.100348+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33816576, 'num_objects': 8256, 'num_object_clones': 0, 'num_object_copies': 16512, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8256, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8529, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4217}, {'shards': '0,2', 'objects': 2209}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:32.845 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:17:32.845 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'11770", 'reported_seq': 11904, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:31.101377+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:17:31.101377+0000', 'last_peered': '2024-09-11T01:17:31.101377+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:17:31.101377+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "36'1700", 'ondisk_log_start': "36'1700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10070, 'log_dups_size': 0, 'ondisk_log_size': 10070, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33514089, 'num_objects': 8183, 'num_object_clones': 0, 'num_object_copies': 16366, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8183, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8840, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4266}, {'shards': '0,2', 'objects': 2129}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:32.845 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:17:33.690 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:17:33.690 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39769/263278 objects degraded (15.105%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":390,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":131639,"data_bytes":539177620,"bytes_used":2162450432,"bytes_avail":287747842048,"bytes_total":289910292480,"degraded_objects":39769,"degraded_total":263278,"degraded_ratio":0.15105325929245891,"misplaced_objects":1039,"misplaced_total":263278,"misplaced_ratio":0.0039463988635586719,"recovering_objects_per_sec":35,"recovering_bytes_per_sec":144624,"recovering_keys_per_sec":0,"num_objects_recovered":424,"num_bytes_recovered":1736704,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":491},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:17:33.727 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:17:33.727 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:34.554 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:17:34.555 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:17:34.604 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":233,"stamp":"2024-09-11T01:17:32.669616+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":539177620,"num_objects":131639,"num_object_clones":0,"num_object_copies":263278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":1039,"num_objects_unfound":0,"num_objects_dirty":131639,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":150652,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6832,"num_bytes_recovered":27959315,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160803,"ondisk_log_size":160803,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2111768,"kb_used_data":1156136,"kb_used_omap":12,"kb_used_meta":955571,"kb_avail":281003752,"statfs":{"total":289910292480,"available":287747842048,"internally_reserved":0,"allocated":1183883264,"data_stored":1183042412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":978504708},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,3,10,6,4,11],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24182784,"num_objects":-5904,"num_object_clones":0,"num_object_copies":-11808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-424,"num_objects_unfound":0,"num_objects_dirty":-5904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5904,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":424,"num_bytes_recovered":1736704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008354"},"pg_stats":[{"pgid":"2.f","version":"45'11932","reported_seq":12010,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:31.000332+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:17:31.000332+0000","last_peered":"2024-09-11T01:17:31.000332+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:17:31.000332+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34136064,"num_objects":8334,"num_object_clones":0,"num_object_copies":16668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8334,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11932,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4262},{"shards":"0,2","objects":2273}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12055","reported_seq":12157,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:29.300908+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:17:29.300908+0000","last_peered":"2024-09-11T01:17:29.300908+0000","last_clean":"2024-09-11T01:17:29.300908+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:17:29.300908+0000","last_undegraded":"2024-09-11T01:17:29.300908+0000","last_fullsized":"2024-09-11T01:17:29.300908+0000","mapping_epoch":35,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33984512,"num_objects":8297,"num_object_clones":0,"num_object_copies":16594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8297,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10294,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11787","reported_seq":11916,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:29.259744+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:17:29.259744+0000","last_peered":"2024-09-11T01:17:29.259744+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:17:29.259744+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34226200,"num_objects":8357,"num_object_clones":0,"num_object_copies":16714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8357,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10088,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2183},{"shards":"2","objects":4461}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11819","reported_seq":11889,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:29.267268+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:17:29.267268+0000","last_peered":"2024-09-11T01:17:29.267268+0000","last_clean":"2024-09-11T01:17:29.267268+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:17:29.267268+0000","last_undegraded":"2024-09-11T01:17:29.267268+0000","last_fullsized":"2024-09-11T01:17:29.267268+0000","mapping_epoch":35,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34058240,"num_objects":8315,"num_object_clones":0,"num_object_copies":16630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8315,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11819,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11852","reported_seq":11980,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:31.186635+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:17:31.186635+0000","last_peered":"2024-09-11T01:17:31.186635+0000","last_clean":"2024-09-11T01:17:31.186635+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:17:31.186635+0000","last_undegraded":"2024-09-11T01:17:31.186635+0000","last_fullsized":"2024-09-11T01:17:31.186635+0000","mapping_epoch":34,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33411072,"num_objects":8157,"num_object_clones":0,"num_object_copies":16314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8157,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8857,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'11990","reported_seq":12161,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:31.135868+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:17:31.135868+0000","last_peered":"2024-09-11T01:17:31.135868+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:17:31.135868+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33738752,"num_objects":8237,"num_object_clones":0,"num_object_copies":16474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8237,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8331,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4286},{"shards":"0,2","objects":2077}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11817","reported_seq":11948,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:31.081870+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:17:31.081870+0000","last_peered":"2024-09-11T01:17:31.081870+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:17:31.081870+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33230848,"num_objects":8113,"num_object_clones":0,"num_object_copies":16226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8674,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4127},{"shards":"0,2","objects":2137}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12013","reported_seq":12143,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:31.087597+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:17:31.087597+0000","last_peered":"2024-09-11T01:17:31.087597+0000","last_clean":"2024-09-11T01:17:31.087597+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:17:31.087597+0000","last_undegraded":"2024-09-11T01:17:31.087597+0000","last_fullsized":"2024-09-11T01:17:31.087597+0000","mapping_epoch":34,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33755136,"num_objects":8241,"num_object_clones":0,"num_object_copies":16482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8889,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683747+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683747+0000","last_peered":"2024-09-11T01:15:21.683747+0000","last_clean":"2024-09-11T01:15:21.683747+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683747+0000","last_undegraded":"2024-09-11T01:15:21.683747+0000","last_fullsized":"2024-09-11T01:15:21.683747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11718","reported_seq":11846,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:29.298827+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:17:29.298827+0000","last_peered":"2024-09-11T01:17:29.298827+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:17:29.298827+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33812480,"num_objects":8255,"num_object_clones":0,"num_object_copies":16510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9689,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2204},{"shards":"2","objects":4322}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11766","reported_seq":11895,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:31.163990+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:17:31.163990+0000","last_peered":"2024-09-11T01:17:31.163990+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:17:31.163990+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33185792,"num_objects":8102,"num_object_clones":0,"num_object_copies":16204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8102,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8353,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4118},{"shards":"0,2","objects":2156}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'11912","reported_seq":12027,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:29.285628+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:17:29.285628+0000","last_peered":"2024-09-11T01:17:29.285628+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:17:29.285628+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33976320,"num_objects":8295,"num_object_clones":0,"num_object_copies":16590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8295,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9729,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2217},{"shards":"2","objects":4271}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684827+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684827+0000","last_peered":"2024-09-11T01:15:21.684827+0000","last_clean":"2024-09-11T01:15:21.684827+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684827+0000","last_undegraded":"2024-09-11T01:15:21.684827+0000","last_fullsized":"2024-09-11T01:15:21.684827+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11682","reported_seq":19663,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:29.304738+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:17:29.304738+0000","last_peered":"2024-09-11T01:17:29.304738+0000","last_clean":"2024-09-11T01:17:29.304738+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:17:29.304738+0000","last_undegraded":"2024-09-11T01:17:29.304738+0000","last_fullsized":"2024-09-11T01:17:29.304738+0000","mapping_epoch":44,"log_start":"36'1600","ondisk_log_start":"36'1600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33234944,"num_objects":8114,"num_object_clones":0,"num_object_copies":16228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8114,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9467,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'11988","reported_seq":12120,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:31.127831+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:17:31.127831+0000","last_peered":"2024-09-11T01:17:31.127831+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:17:31.127831+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33898496,"num_objects":8276,"num_object_clones":0,"num_object_copies":16552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8276,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8725,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4326},{"shards":"0,2","objects":2095}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684623+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684623+0000","last_peered":"2024-09-11T01:15:21.684623+0000","last_clean":"2024-09-11T01:15:21.684623+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684623+0000","last_undegraded":"2024-09-11T01:15:21.684623+0000","last_fullsized":"2024-09-11T01:15:21.684623+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11779","reported_seq":17723,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:31.166848+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:17:31.166848+0000","last_peered":"2024-09-11T01:17:31.166848+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:17:31.166848+0000","last_undegraded":"2024-09-11T01:17:31.166848+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33198080,"num_objects":8105,"num_object_clones":0,"num_object_copies":16210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":1039,"num_objects_unfound":0,"num_objects_dirty":8105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8434,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2903,"num_bytes_recovered":11878400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7066},{"shards":"0,2","objects":1039}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'11920","reported_seq":12055,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:31.100348+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:17:31.100348+0000","last_peered":"2024-09-11T01:17:31.100348+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:17:31.100348+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33816576,"num_objects":8256,"num_object_clones":0,"num_object_copies":16512,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8256,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8529,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4217},{"shards":"0,2","objects":2209}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692049+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.692049+0000","last_peered":"2024-09-11T01:15:21.692049+0000","last_clean":"2024-09-11T01:15:21.692049+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.692049+0000","last_undegraded":"2024-09-11T01:15:21.692049+0000","last_fullsized":"2024-09-11T01:15:21.692049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684417+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684417+0000","last_peered":"2024-09-11T01:15:21.684417+0000","last_clean":"2024-09-11T01:15:21.684417+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684417+0000","last_undegraded":"2024-09-11T01:15:21.684417+0000","last_fullsized":"2024-09-11T01:15:21.684417+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11770","reported_seq":11904,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:31.101377+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:17:31.101377+0000","last_peered":"2024-09-11T01:17:31.101377+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:17:31.101377+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'1700","ondisk_log_start":"36'1700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33514089,"num_objects":8183,"num_object_clones":0,"num_object_copies":16366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8840,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4266},{"shards":"0,2","objects":2129}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":539177601,"num_objects":131637,"num_object_clones":0,"num_object_copies":263274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":1039,"num_objects_unfound":0,"num_objects_dirty":131637,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":150650,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6831,"num_bytes_recovered":27959296,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1180377088,"data_stored":1180299364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160800,"ondisk_log_size":160800,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182472,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":819508,"kb_used_data":458852,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93552332,"statfs":{"total":96636764160,"available":95797587968,"internally_reserved":0,"allocated":469864448,"data_stored":469605234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":369290923},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,5,5,1,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986244,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":699704,"kb_used_data":393192,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93672136,"statfs":{"total":96636764160,"available":95920267264,"internally_reserved":0,"allocated":402628608,"data_stored":402369399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,3,0,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659189,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592556,"kb_used_data":304092,"kb_used_omap":4,"kb_used_meta":288443,"kb_avail":93779284,"statfs":{"total":96636764160,"available":96029986816,"internally_reserved":0,"allocated":311390208,"data_stored":311067779,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":295366319},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,2,1,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":468066304,"data_stored":468062825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":310943744,"data_stored":310877050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":401367040,"data_stored":401359489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:37.606 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:38.386 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:17:38.387 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:17:38.436 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":235,"stamp":"2024-09-11T01:17:36.670612+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":530317972,"num_objects":129476,"num_object_clones":0,"num_object_copies":258952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":868,"num_objects_unfound":0,"num_objects_dirty":129476,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":152815,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7003,"num_bytes_recovered":28659731,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160766,"ondisk_log_size":160766,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2087580,"kb_used_data":1144424,"kb_used_omap":12,"kb_used_meta":943091,"kb_avail":281027940,"statfs":{"total":289910292480,"available":287772610560,"internally_reserved":0,"allocated":1171890176,"data_stored":1171049324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":965725188},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,6,5,8,13,5,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23908352,"num_objects":-5837,"num_object_clones":0,"num_object_copies":-11674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-454,"num_objects_unfound":0,"num_objects_dirty":-5837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5837,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":454,"num_bytes_recovered":1859584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002936"},"pg_stats":[{"pgid":"2.f","version":"45'12072","reported_seq":12145,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:36.144721+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:17:36.144721+0000","last_peered":"2024-09-11T01:17:36.144721+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:17:36.144721+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33562624,"num_objects":8194,"num_object_clones":0,"num_object_copies":16388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12072,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4122},{"shards":"0,2","objects":2133}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12192","reported_seq":12294,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:34.306321+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:17:34.306321+0000","last_peered":"2024-09-11T01:17:34.306321+0000","last_clean":"2024-09-11T01:17:34.306321+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:17:34.306321+0000","last_undegraded":"2024-09-11T01:17:34.306321+0000","last_fullsized":"2024-09-11T01:17:34.306321+0000","mapping_epoch":35,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33423360,"num_objects":8160,"num_object_clones":0,"num_object_copies":16320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10431,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11910","reported_seq":12039,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:34.325031+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:17:34.325031+0000","last_peered":"2024-09-11T01:17:34.325031+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:17:34.325031+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33722392,"num_objects":8234,"num_object_clones":0,"num_object_copies":16468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8234,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10211,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2060},{"shards":"2","objects":4338}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11953","reported_seq":12023,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:34.250230+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:17:34.250230+0000","last_peered":"2024-09-11T01:17:34.250230+0000","last_clean":"2024-09-11T01:17:34.250230+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:17:34.250230+0000","last_undegraded":"2024-09-11T01:17:34.250230+0000","last_fullsized":"2024-09-11T01:17:34.250230+0000","mapping_epoch":35,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33509376,"num_objects":8181,"num_object_clones":0,"num_object_copies":16362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11953,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11977","reported_seq":12105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:36.103728+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:17:36.103728+0000","last_peered":"2024-09-11T01:17:36.103728+0000","last_clean":"2024-09-11T01:17:36.103728+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:17:36.103728+0000","last_undegraded":"2024-09-11T01:17:36.103728+0000","last_fullsized":"2024-09-11T01:17:36.103728+0000","mapping_epoch":34,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32899072,"num_objects":8032,"num_object_clones":0,"num_object_copies":16064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8982,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12121","reported_seq":12292,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:36.154084+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:17:36.154084+0000","last_peered":"2024-09-11T01:17:36.154084+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:17:36.154084+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33202176,"num_objects":8106,"num_object_clones":0,"num_object_copies":16212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8462,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4155},{"shards":"0,2","objects":1946}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11943","reported_seq":12073,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:36.217509+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:17:36.217509+0000","last_peered":"2024-09-11T01:17:36.217509+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:17:36.217509+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32714752,"num_objects":7987,"num_object_clones":0,"num_object_copies":15974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7987,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8800,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4001},{"shards":"0,2","objects":2011}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12151","reported_seq":12281,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:36.060292+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:17:36.060292+0000","last_peered":"2024-09-11T01:17:36.060292+0000","last_clean":"2024-09-11T01:17:36.060292+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:17:36.060292+0000","last_undegraded":"2024-09-11T01:17:36.060292+0000","last_fullsized":"2024-09-11T01:17:36.060292+0000","mapping_epoch":34,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33189888,"num_objects":8103,"num_object_clones":0,"num_object_copies":16206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9027,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683747+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683747+0000","last_peered":"2024-09-11T01:15:21.683747+0000","last_clean":"2024-09-11T01:15:21.683747+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683747+0000","last_undegraded":"2024-09-11T01:15:21.683747+0000","last_fullsized":"2024-09-11T01:15:21.683747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11854","reported_seq":11982,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:34.284136+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:17:34.284136+0000","last_peered":"2024-09-11T01:17:34.284136+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:17:34.284136+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33255424,"num_objects":8119,"num_object_clones":0,"num_object_copies":16238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9825,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2068},{"shards":"2","objects":4186}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11923","reported_seq":12051,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:36.214494+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:17:36.214494+0000","last_peered":"2024-09-11T01:17:36.214494+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:17:36.214494+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32542720,"num_objects":7945,"num_object_clones":0,"num_object_copies":15890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8510,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3961},{"shards":"0,2","objects":1999}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12069","reported_seq":12187,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:34.320381+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:17:34.320381+0000","last_peered":"2024-09-11T01:17:34.320381+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:17:34.320381+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33333248,"num_objects":8138,"num_object_clones":0,"num_object_copies":16276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9886,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2060},{"shards":"2","objects":4114}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684827+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684827+0000","last_peered":"2024-09-11T01:15:21.684827+0000","last_clean":"2024-09-11T01:15:21.684827+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684827+0000","last_undegraded":"2024-09-11T01:15:21.684827+0000","last_fullsized":"2024-09-11T01:15:21.684827+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11813","reported_seq":19794,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:34.340417+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:17:34.340417+0000","last_peered":"2024-09-11T01:17:34.340417+0000","last_clean":"2024-09-11T01:17:34.340417+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:17:34.340417+0000","last_undegraded":"2024-09-11T01:17:34.340417+0000","last_fullsized":"2024-09-11T01:17:34.340417+0000","mapping_epoch":44,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32698368,"num_objects":7983,"num_object_clones":0,"num_object_copies":15966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9598,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12129","reported_seq":12261,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:36.143159+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:17:36.143159+0000","last_peered":"2024-09-11T01:17:36.143159+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:17:36.143159+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33320960,"num_objects":8135,"num_object_clones":0,"num_object_copies":16270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8866,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4185},{"shards":"0,2","objects":1954}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684623+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684623+0000","last_peered":"2024-09-11T01:15:21.684623+0000","last_clean":"2024-09-11T01:15:21.684623+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684623+0000","last_undegraded":"2024-09-11T01:15:21.684623+0000","last_fullsized":"2024-09-11T01:15:21.684623+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11902","reported_seq":18188,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:36.205071+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:17:36.205071+0000","last_peered":"2024-09-11T01:17:36.205071+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:17:36.205071+0000","last_undegraded":"2024-09-11T01:17:36.205071+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32694272,"num_objects":7982,"num_object_clones":0,"num_object_copies":15964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":868,"num_objects_unfound":0,"num_objects_dirty":7982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8557,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3074,"num_bytes_recovered":12578816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7114},{"shards":"0,2","objects":868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12061","reported_seq":12196,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:36.044982+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:17:36.044982+0000","last_peered":"2024-09-11T01:17:36.044982+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:17:36.044982+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33239040,"num_objects":8115,"num_object_clones":0,"num_object_copies":16230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8670,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4076},{"shards":"0,2","objects":2068}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692049+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.692049+0000","last_peered":"2024-09-11T01:15:21.692049+0000","last_clean":"2024-09-11T01:15:21.692049+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.692049+0000","last_undegraded":"2024-09-11T01:15:21.692049+0000","last_fullsized":"2024-09-11T01:15:21.692049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684417+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684417+0000","last_peered":"2024-09-11T01:15:21.684417+0000","last_clean":"2024-09-11T01:15:21.684417+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684417+0000","last_undegraded":"2024-09-11T01:15:21.684417+0000","last_fullsized":"2024-09-11T01:15:21.684417+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11893","reported_seq":12027,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:36.141732+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:17:36.141732+0000","last_peered":"2024-09-11T01:17:36.141732+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:17:36.141732+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33010281,"num_objects":8060,"num_object_clones":0,"num_object_copies":16120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8060,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8963,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4143},{"shards":"0,2","objects":2006}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":530317953,"num_objects":129474,"num_object_clones":0,"num_object_copies":258948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":868,"num_objects_unfound":0,"num_objects_dirty":129474,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":152813,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7002,"num_bytes_recovered":28659712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1168822272,"data_stored":1168744548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160763,"ondisk_log_size":160763,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182473,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":804536,"kb_used_data":452200,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93567304,"statfs":{"total":96636764160,"available":95812919296,"internally_reserved":0,"allocated":463052800,"data_stored":462793586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":360771243},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,4,1,8,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986245,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":699268,"kb_used_data":388596,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93672572,"statfs":{"total":96636764160,"available":95920713728,"internally_reserved":0,"allocated":397922304,"data_stored":397663095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,2,0,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659190,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":583776,"kb_used_data":303628,"kb_used_omap":4,"kb_used_meta":280123,"kb_avail":93788064,"statfs":{"total":96636764160,"available":96038977536,"internally_reserved":0,"allocated":310915072,"data_stored":310592643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":286846639},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,1,5,5,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":461524992,"data_stored":461521513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":310464512,"data_stored":310397818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":396832768,"data_stored":396825217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:38.438 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:39.239 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:17:39.239 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:17:39.270 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":236,"stamp":"2024-09-11T01:17:38.671086+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":530317972,"num_objects":129476,"num_object_clones":0,"num_object_copies":258952,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":868,"num_objects_unfound":0,"num_objects_dirty":129476,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":152815,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7003,"num_bytes_recovered":28659731,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160766,"ondisk_log_size":160766,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2087580,"kb_used_data":1144424,"kb_used_omap":12,"kb_used_meta":943091,"kb_avail":281027940,"statfs":{"total":289910292480,"available":287772610560,"internally_reserved":0,"allocated":1171890176,"data_stored":1171049324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":965725188},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,6,5,8,13,5,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18071552,"num_objects":-4412,"num_object_clones":0,"num_object_copies":-8824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-321,"num_objects_unfound":0,"num_objects_dirty":-4412,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4412,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":321,"num_bytes_recovered":1314816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002933"},"pg_stats":[{"pgid":"2.f","version":"45'12072","reported_seq":12145,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:36.144721+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:17:36.144721+0000","last_peered":"2024-09-11T01:17:36.144721+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:17:36.144721+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33562624,"num_objects":8194,"num_object_clones":0,"num_object_copies":16388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12072,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4122},{"shards":"0,2","objects":2133}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12192","reported_seq":12294,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:34.306321+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:17:34.306321+0000","last_peered":"2024-09-11T01:17:34.306321+0000","last_clean":"2024-09-11T01:17:34.306321+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:17:34.306321+0000","last_undegraded":"2024-09-11T01:17:34.306321+0000","last_fullsized":"2024-09-11T01:17:34.306321+0000","mapping_epoch":35,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33423360,"num_objects":8160,"num_object_clones":0,"num_object_copies":16320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10431,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'11910","reported_seq":12039,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:34.325031+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:17:34.325031+0000","last_peered":"2024-09-11T01:17:34.325031+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:17:34.325031+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33722392,"num_objects":8234,"num_object_clones":0,"num_object_copies":16468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8234,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10211,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2060},{"shards":"2","objects":4338}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'11953","reported_seq":12023,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:34.250230+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:17:34.250230+0000","last_peered":"2024-09-11T01:17:34.250230+0000","last_clean":"2024-09-11T01:17:34.250230+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:17:34.250230+0000","last_undegraded":"2024-09-11T01:17:34.250230+0000","last_fullsized":"2024-09-11T01:17:34.250230+0000","mapping_epoch":35,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33509376,"num_objects":8181,"num_object_clones":0,"num_object_copies":16362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8181,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11953,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11977","reported_seq":12105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:36.103728+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:17:36.103728+0000","last_peered":"2024-09-11T01:17:36.103728+0000","last_clean":"2024-09-11T01:17:36.103728+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:17:36.103728+0000","last_undegraded":"2024-09-11T01:17:36.103728+0000","last_fullsized":"2024-09-11T01:17:36.103728+0000","mapping_epoch":34,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32899072,"num_objects":8032,"num_object_clones":0,"num_object_copies":16064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8982,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12121","reported_seq":12292,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:36.154084+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:17:36.154084+0000","last_peered":"2024-09-11T01:17:36.154084+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:17:36.154084+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33202176,"num_objects":8106,"num_object_clones":0,"num_object_copies":16212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8462,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4155},{"shards":"0,2","objects":1946}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11943","reported_seq":12073,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:36.217509+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:17:36.217509+0000","last_peered":"2024-09-11T01:17:36.217509+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:17:36.217509+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32714752,"num_objects":7987,"num_object_clones":0,"num_object_copies":15974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7987,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8800,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4001},{"shards":"0,2","objects":2011}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12151","reported_seq":12281,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:36.060292+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:17:36.060292+0000","last_peered":"2024-09-11T01:17:36.060292+0000","last_clean":"2024-09-11T01:17:36.060292+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:17:36.060292+0000","last_undegraded":"2024-09-11T01:17:36.060292+0000","last_fullsized":"2024-09-11T01:17:36.060292+0000","mapping_epoch":34,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33189888,"num_objects":8103,"num_object_clones":0,"num_object_copies":16206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9027,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683747+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683747+0000","last_peered":"2024-09-11T01:15:21.683747+0000","last_clean":"2024-09-11T01:15:21.683747+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683747+0000","last_undegraded":"2024-09-11T01:15:21.683747+0000","last_fullsized":"2024-09-11T01:15:21.683747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11854","reported_seq":11982,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:34.284136+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:17:34.284136+0000","last_peered":"2024-09-11T01:17:34.284136+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:17:34.284136+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33255424,"num_objects":8119,"num_object_clones":0,"num_object_copies":16238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9825,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2068},{"shards":"2","objects":4186}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11923","reported_seq":12051,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:36.214494+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:17:36.214494+0000","last_peered":"2024-09-11T01:17:36.214494+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:17:36.214494+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32542720,"num_objects":7945,"num_object_clones":0,"num_object_copies":15890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8510,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3961},{"shards":"0,2","objects":1999}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12069","reported_seq":12187,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:34.320381+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:17:34.320381+0000","last_peered":"2024-09-11T01:17:34.320381+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:17:34.320381+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33333248,"num_objects":8138,"num_object_clones":0,"num_object_copies":16276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9886,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":2060},{"shards":"2","objects":4114}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684827+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684827+0000","last_peered":"2024-09-11T01:15:21.684827+0000","last_clean":"2024-09-11T01:15:21.684827+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684827+0000","last_undegraded":"2024-09-11T01:15:21.684827+0000","last_fullsized":"2024-09-11T01:15:21.684827+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11813","reported_seq":19794,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:34.340417+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:17:34.340417+0000","last_peered":"2024-09-11T01:17:34.340417+0000","last_clean":"2024-09-11T01:17:34.340417+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:17:34.340417+0000","last_undegraded":"2024-09-11T01:17:34.340417+0000","last_fullsized":"2024-09-11T01:17:34.340417+0000","mapping_epoch":44,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32698368,"num_objects":7983,"num_object_clones":0,"num_object_copies":15966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9598,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12129","reported_seq":12261,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:36.143159+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:17:36.143159+0000","last_peered":"2024-09-11T01:17:36.143159+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:17:36.143159+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33320960,"num_objects":8135,"num_object_clones":0,"num_object_copies":16270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8866,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4185},{"shards":"0,2","objects":1954}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684623+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684623+0000","last_peered":"2024-09-11T01:15:21.684623+0000","last_clean":"2024-09-11T01:15:21.684623+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684623+0000","last_undegraded":"2024-09-11T01:15:21.684623+0000","last_fullsized":"2024-09-11T01:15:21.684623+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11902","reported_seq":18188,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:36.205071+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:17:36.205071+0000","last_peered":"2024-09-11T01:17:36.205071+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:17:36.205071+0000","last_undegraded":"2024-09-11T01:17:36.205071+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32694272,"num_objects":7982,"num_object_clones":0,"num_object_copies":15964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":868,"num_objects_unfound":0,"num_objects_dirty":7982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8557,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3074,"num_bytes_recovered":12578816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7114},{"shards":"0,2","objects":868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12061","reported_seq":12196,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:36.044982+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:17:36.044982+0000","last_peered":"2024-09-11T01:17:36.044982+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:17:36.044982+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33239040,"num_objects":8115,"num_object_clones":0,"num_object_copies":16230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8670,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4076},{"shards":"0,2","objects":2068}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692049+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.692049+0000","last_peered":"2024-09-11T01:15:21.692049+0000","last_clean":"2024-09-11T01:15:21.692049+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.692049+0000","last_undegraded":"2024-09-11T01:15:21.692049+0000","last_fullsized":"2024-09-11T01:15:21.692049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684417+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684417+0000","last_peered":"2024-09-11T01:15:21.684417+0000","last_clean":"2024-09-11T01:15:21.684417+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684417+0000","last_undegraded":"2024-09-11T01:15:21.684417+0000","last_fullsized":"2024-09-11T01:15:21.684417+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11893","reported_seq":12027,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:36.141732+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:17:36.141732+0000","last_peered":"2024-09-11T01:17:36.141732+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:17:36.141732+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33010281,"num_objects":8060,"num_object_clones":0,"num_object_copies":16120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8060,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8963,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4143},{"shards":"0,2","objects":2006}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":530317953,"num_objects":129474,"num_object_clones":0,"num_object_copies":258948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":868,"num_objects_unfound":0,"num_objects_dirty":129474,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":152813,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7002,"num_bytes_recovered":28659712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1168822272,"data_stored":1168744548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160763,"ondisk_log_size":160763,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182473,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":804536,"kb_used_data":452200,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93567304,"statfs":{"total":96636764160,"available":95812919296,"internally_reserved":0,"allocated":463052800,"data_stored":462793586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":360771243},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,4,1,8,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986245,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":699268,"kb_used_data":388596,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93672572,"statfs":{"total":96636764160,"available":95920713728,"internally_reserved":0,"allocated":397922304,"data_stored":397663095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,2,0,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659190,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":583776,"kb_used_data":303628,"kb_used_omap":4,"kb_used_meta":280123,"kb_avail":93788064,"statfs":{"total":96636764160,"available":96038977536,"internally_reserved":0,"allocated":310915072,"data_stored":310592643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":286846639},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,1,5,5,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":461524992,"data_stored":461521513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":310464512,"data_stored":310397818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":396832768,"data_stored":396825217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:39.271 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:17:39.271 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'12072", 'reported_seq': 12145, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:36.144721+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:17:36.144721+0000', 'last_peered': '2024-09-11T01:17:36.144721+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:17:36.144721+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33562624, 'num_objects': 8194, 'num_object_clones': 0, 'num_object_copies': 16388, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8194, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12072, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4122}, {'shards': '0,2', 'objects': 2133}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:39.271 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:17:39.271 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'11910", 'reported_seq': 12039, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:34.325031+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:17:34.325031+0000', 'last_peered': '2024-09-11T01:17:34.325031+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:17:34.325031+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33722392, 'num_objects': 8234, 'num_object_clones': 0, 'num_object_copies': 16468, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8234, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10211, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2060}, {'shards': '2', 'objects': 4338}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:39.272 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:17:39.272 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'12121", 'reported_seq': 12292, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:36.154084+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:17:36.154084+0000', 'last_peered': '2024-09-11T01:17:36.154084+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:17:36.154084+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "36'2100", 'ondisk_log_start': "36'2100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33202176, 'num_objects': 8106, 'num_object_clones': 0, 'num_object_copies': 16212, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8106, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8462, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4155}, {'shards': '0,2', 'objects': 1946}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:39.272 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:17:39.272 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'11943", 'reported_seq': 12073, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:36.217509+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:17:36.217509+0000', 'last_peered': '2024-09-11T01:17:36.217509+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:17:36.217509+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32714752, 'num_objects': 7987, 'num_object_clones': 0, 'num_object_copies': 15974, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7987, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8800, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4001}, {'shards': '0,2', 'objects': 2011}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:39.272 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:17:39.272 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'11854", 'reported_seq': 11982, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:34.284136+0000', 'last_change': '2024-09-11T01:13:10.159069+0000', 'last_active': '2024-09-11T01:17:34.284136+0000', 'last_peered': '2024-09-11T01:17:34.284136+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:17:34.284136+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33255424, 'num_objects': 8119, 'num_object_clones': 0, 'num_object_copies': 16238, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8119, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9825, 'num_write_kb': 31844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2068}, {'shards': '2', 'objects': 4186}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:39.272 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:17:39.272 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'11923", 'reported_seq': 12051, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:36.214494+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:17:36.214494+0000', 'last_peered': '2024-09-11T01:17:36.214494+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:17:36.214494+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32542720, 'num_objects': 7945, 'num_object_clones': 0, 'num_object_copies': 15890, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7945, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8510, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3961}, {'shards': '0,2', 'objects': 1999}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:39.272 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:17:39.272 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'12069", 'reported_seq': 12187, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:34.320381+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:17:34.320381+0000', 'last_peered': '2024-09-11T01:17:34.320381+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:17:34.320381+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10069, 'log_dups_size': 0, 'ondisk_log_size': 10069, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33333248, 'num_objects': 8138, 'num_object_clones': 0, 'num_object_copies': 16276, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8138, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9886, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 2060}, {'shards': '2', 'objects': 4114}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:39.273 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:17:39.273 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'12129", 'reported_seq': 12261, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:36.143159+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:17:36.143159+0000', 'last_peered': '2024-09-11T01:17:36.143159+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:17:36.143159+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "36'2100", 'ondisk_log_start': "36'2100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33320960, 'num_objects': 8135, 'num_object_clones': 0, 'num_object_copies': 16270, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8135, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8866, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4185}, {'shards': '0,2', 'objects': 1954}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:39.273 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:17:39.273 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'11902", 'reported_seq': 18188, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:17:36.205071+0000', 'last_change': '2024-09-11T01:15:19.913288+0000', 'last_active': '2024-09-11T01:17:36.205071+0000', 'last_peered': '2024-09-11T01:17:36.205071+0000', 'last_clean': '2024-09-11T01:13:07.624565+0000', 'last_became_active': '2024-09-11T01:13:10.199795+0000', 'last_became_peered': '2024-09-11T01:13:10.199795+0000', 'last_unstale': '2024-09-11T01:17:36.205071+0000', 'last_undegraded': '2024-09-11T01:17:36.205071+0000', 'last_fullsized': '2024-09-11T01:13:09.652311+0000', 'mapping_epoch': 42, 'log_start': "36'1900", 'ondisk_log_start': "36'1900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3415", 'last_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'objects_scrubbed': 3337, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1975, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32694272, 'num_objects': 7982, 'num_object_clones': 0, 'num_object_copies': 15964, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 868, 'num_objects_unfound': 0, 'num_objects_dirty': 7982, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8557, 'num_write_kb': 26404, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3074, 'num_bytes_recovered': 12578816, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7114}, {'shards': '0,2', 'objects': 868}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:39.273 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:17:39.273 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'12061", 'reported_seq': 12196, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:36.044982+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:17:36.044982+0000', 'last_peered': '2024-09-11T01:17:36.044982+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:17:36.044982+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33239040, 'num_objects': 8115, 'num_object_clones': 0, 'num_object_copies': 16230, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8115, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8670, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4076}, {'shards': '0,2', 'objects': 2068}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:39.273 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:17:39.273 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'11893", 'reported_seq': 12027, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:36.141732+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:17:36.141732+0000', 'last_peered': '2024-09-11T01:17:36.141732+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:17:36.141732+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "36'1800", 'ondisk_log_start': "36'1800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 33010281, 'num_objects': 8060, 'num_object_clones': 0, 'num_object_copies': 16120, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8060, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8963, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4143}, {'shards': '0,2', 'objects': 2006}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:39.274 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:17:40.260 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:17:40.260 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39769/258952 objects degraded (15.358%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":396,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":129476,"data_bytes":530317972,"bytes_used":2137681920,"bytes_avail":287772610560,"bytes_total":289910292480,"degraded_objects":39769,"degraded_total":258952,"degraded_ratio":0.15357672464394945,"misplaced_objects":868,"misplaced_total":258952,"misplaced_ratio":0.0033519725663443425,"recovering_objects_per_sec":26,"recovering_bytes_per_sec":109541,"recovering_keys_per_sec":0,"num_objects_recovered":321,"num_bytes_recovered":1314816,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":367},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:17:40.319 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:17:40.319 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:41.198 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:17:41.200 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:17:41.265 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":237,"stamp":"2024-09-11T01:17:40.671428+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":526955156,"num_objects":128655,"num_object_clones":0,"num_object_copies":257310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":868,"num_objects_unfound":0,"num_objects_dirty":128655,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":153636,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7003,"num_bytes_recovered":28659731,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160787,"ondisk_log_size":160787,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2078656,"kb_used_data":1139664,"kb_used_omap":12,"kb_used_meta":938931,"kb_avail":281036864,"statfs":{"total":289910292480,"available":287781748736,"internally_reserved":0,"allocated":1167015936,"data_stored":1166175084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":961465348},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,3,6,10,15,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21434368,"num_objects":-5233,"num_object_clones":0,"num_object_copies":-10466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-321,"num_objects_unfound":0,"num_objects_dirty":-5233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5233,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":321,"num_bytes_recovered":1314816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002750"},"pg_stats":[{"pgid":"2.f","version":"45'12072","reported_seq":12145,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:36.144721+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:17:36.144721+0000","last_peered":"2024-09-11T01:17:36.144721+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:17:36.144721+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33562624,"num_objects":8194,"num_object_clones":0,"num_object_copies":16388,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8194,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12072,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4122},{"shards":"0,2","objects":2133}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12333","reported_seq":12435,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:39.293930+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:17:39.293930+0000","last_peered":"2024-09-11T01:17:39.293930+0000","last_clean":"2024-09-11T01:17:39.293930+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:17:39.293930+0000","last_undegraded":"2024-09-11T01:17:39.293930+0000","last_fullsized":"2024-09-11T01:17:39.293930+0000","mapping_epoch":35,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32845824,"num_objects":8019,"num_object_clones":0,"num_object_copies":16038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10572,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12068","reported_seq":12197,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:38.965836+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:17:38.965836+0000","last_peered":"2024-09-11T01:17:38.965836+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:17:38.965836+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33075224,"num_objects":8076,"num_object_clones":0,"num_object_copies":16152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8076,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10369,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1902},{"shards":"2","objects":4180}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12064","reported_seq":12134,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:39.282985+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:17:39.282985+0000","last_peered":"2024-09-11T01:17:39.282985+0000","last_clean":"2024-09-11T01:17:39.282985+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:17:39.282985+0000","last_undegraded":"2024-09-11T01:17:39.282985+0000","last_fullsized":"2024-09-11T01:17:39.282985+0000","mapping_epoch":35,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33054720,"num_objects":8070,"num_object_clones":0,"num_object_copies":16140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8070,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12064,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'11977","reported_seq":12105,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:36.103728+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:17:36.103728+0000","last_peered":"2024-09-11T01:17:36.103728+0000","last_clean":"2024-09-11T01:17:36.103728+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:17:36.103728+0000","last_undegraded":"2024-09-11T01:17:36.103728+0000","last_fullsized":"2024-09-11T01:17:36.103728+0000","mapping_epoch":34,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32899072,"num_objects":8032,"num_object_clones":0,"num_object_copies":16064,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8032,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8982,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12121","reported_seq":12292,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:36.154084+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:17:36.154084+0000","last_peered":"2024-09-11T01:17:36.154084+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:17:36.154084+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33202176,"num_objects":8106,"num_object_clones":0,"num_object_copies":16212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8462,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4155},{"shards":"0,2","objects":1946}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'11943","reported_seq":12073,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:36.217509+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:17:36.217509+0000","last_peered":"2024-09-11T01:17:36.217509+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:17:36.217509+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32714752,"num_objects":7987,"num_object_clones":0,"num_object_copies":15974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7987,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8800,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4001},{"shards":"0,2","objects":2011}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12151","reported_seq":12281,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:36.060292+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:17:36.060292+0000","last_peered":"2024-09-11T01:17:36.060292+0000","last_clean":"2024-09-11T01:17:36.060292+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:17:36.060292+0000","last_undegraded":"2024-09-11T01:17:36.060292+0000","last_fullsized":"2024-09-11T01:17:36.060292+0000","mapping_epoch":34,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33189888,"num_objects":8103,"num_object_clones":0,"num_object_copies":16206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8103,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9027,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683747+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683747+0000","last_peered":"2024-09-11T01:15:21.683747+0000","last_clean":"2024-09-11T01:15:21.683747+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683747+0000","last_undegraded":"2024-09-11T01:15:21.683747+0000","last_fullsized":"2024-09-11T01:15:21.683747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'11999","reported_seq":12126,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:39.328142+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:17:39.328142+0000","last_peered":"2024-09-11T01:17:39.328142+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:17:39.328142+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32661504,"num_objects":7974,"num_object_clones":0,"num_object_copies":15948,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7974,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9970,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1923},{"shards":"2","objects":4041}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'11923","reported_seq":12051,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:36.214494+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:17:36.214494+0000","last_peered":"2024-09-11T01:17:36.214494+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:17:36.214494+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32542720,"num_objects":7945,"num_object_clones":0,"num_object_copies":15890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8510,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3961},{"shards":"0,2","objects":1999}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12205","reported_seq":12324,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:39.203584+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:17:39.203584+0000","last_peered":"2024-09-11T01:17:39.203584+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:17:39.203584+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32776192,"num_objects":8002,"num_object_clones":0,"num_object_copies":16004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10022,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1924},{"shards":"2","objects":3978}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684827+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684827+0000","last_peered":"2024-09-11T01:15:21.684827+0000","last_clean":"2024-09-11T01:15:21.684827+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684827+0000","last_undegraded":"2024-09-11T01:15:21.684827+0000","last_fullsized":"2024-09-11T01:15:21.684827+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'11943","reported_seq":19924,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:39.258471+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:17:39.258471+0000","last_peered":"2024-09-11T01:17:39.258471+0000","last_clean":"2024-09-11T01:17:39.258471+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:17:39.258471+0000","last_undegraded":"2024-09-11T01:17:39.258471+0000","last_fullsized":"2024-09-11T01:17:39.258471+0000","mapping_epoch":44,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32165888,"num_objects":7853,"num_object_clones":0,"num_object_copies":15706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9728,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12129","reported_seq":12261,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:36.143159+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:17:36.143159+0000","last_peered":"2024-09-11T01:17:36.143159+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:17:36.143159+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33320960,"num_objects":8135,"num_object_clones":0,"num_object_copies":16270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8866,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4185},{"shards":"0,2","objects":1954}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684623+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684623+0000","last_peered":"2024-09-11T01:15:21.684623+0000","last_clean":"2024-09-11T01:15:21.684623+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684623+0000","last_undegraded":"2024-09-11T01:15:21.684623+0000","last_fullsized":"2024-09-11T01:15:21.684623+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'11902","reported_seq":18188,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:36.205071+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:17:36.205071+0000","last_peered":"2024-09-11T01:17:36.205071+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:17:36.205071+0000","last_undegraded":"2024-09-11T01:17:36.205071+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"36'1900","ondisk_log_start":"36'1900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32694272,"num_objects":7982,"num_object_clones":0,"num_object_copies":15964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":868,"num_objects_unfound":0,"num_objects_dirty":7982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8557,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3074,"num_bytes_recovered":12578816,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7114},{"shards":"0,2","objects":868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12061","reported_seq":12196,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:36.044982+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:17:36.044982+0000","last_peered":"2024-09-11T01:17:36.044982+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:17:36.044982+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33239040,"num_objects":8115,"num_object_clones":0,"num_object_copies":16230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8670,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4076},{"shards":"0,2","objects":2068}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692049+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.692049+0000","last_peered":"2024-09-11T01:15:21.692049+0000","last_clean":"2024-09-11T01:15:21.692049+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.692049+0000","last_undegraded":"2024-09-11T01:15:21.692049+0000","last_fullsized":"2024-09-11T01:15:21.692049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684417+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684417+0000","last_peered":"2024-09-11T01:15:21.684417+0000","last_clean":"2024-09-11T01:15:21.684417+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684417+0000","last_undegraded":"2024-09-11T01:15:21.684417+0000","last_fullsized":"2024-09-11T01:15:21.684417+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'11893","reported_seq":12027,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:36.141732+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:17:36.141732+0000","last_peered":"2024-09-11T01:17:36.141732+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:17:36.141732+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'1800","ondisk_log_start":"36'1800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":33010281,"num_objects":8060,"num_object_clones":0,"num_object_copies":16120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8060,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8963,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4143},{"shards":"0,2","objects":2006}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":526955137,"num_objects":128653,"num_object_clones":0,"num_object_copies":257306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":868,"num_objects_unfound":0,"num_objects_dirty":128653,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":153634,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7002,"num_bytes_recovered":28659712,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1164189696,"data_stored":1164111972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160784,"ondisk_log_size":160784,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182473,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":804536,"kb_used_data":452200,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93567304,"statfs":{"total":96636764160,"available":95812919296,"internally_reserved":0,"allocated":463052800,"data_stored":462793586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":360771243},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,4,1,8,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986246,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":682316,"kb_used_data":384124,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93689524,"statfs":{"total":96636764160,"available":95938072576,"internally_reserved":0,"allocated":393342976,"data_stored":393083767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305327786},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,4,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659191,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591804,"kb_used_data":303340,"kb_used_omap":4,"kb_used_meta":288443,"kb_avail":93780036,"statfs":{"total":96636764160,"available":96030756864,"internally_reserved":0,"allocated":310620160,"data_stored":310297731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":295366319},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,1,5,5,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":461524992,"data_stored":461521513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":310259712,"data_stored":310193018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":392404992,"data_stored":392397441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:44.268 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:45.076 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:17:45.077 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:17:45.140 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":239,"stamp":"2024-09-11T01:17:44.672388+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":517800596,"num_objects":126420,"num_object_clones":0,"num_object_copies":252840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":671,"num_objects_unfound":0,"num_objects_dirty":126420,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":155871,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7200,"num_bytes_recovered":29466643,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160822,"ondisk_log_size":160822,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2076184,"kb_used_data":1128872,"kb_used_omap":12,"kb_used_meta":947251,"kb_avail":281039336,"statfs":{"total":289910292480,"available":287784280064,"internally_reserved":0,"allocated":1155964928,"data_stored":1155124076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":969985028},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,4,5,12,9,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21377024,"num_objects":-5219,"num_object_clones":0,"num_object_copies":-10438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-368,"num_objects_unfound":0,"num_objects_dirty":-5219,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5219,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":368,"num_bytes_recovered":1507328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002772"},"pg_stats":[{"pgid":"2.f","version":"45'12240","reported_seq":12316,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:41.266132+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:17:41.266132+0000","last_peered":"2024-09-11T01:17:41.266132+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:17:41.266132+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32874496,"num_objects":8026,"num_object_clones":0,"num_object_copies":16052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12240,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3954},{"shards":"0,2","objects":1965}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12465","reported_seq":12567,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:44.311403+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:17:44.311403+0000","last_peered":"2024-09-11T01:17:44.311403+0000","last_clean":"2024-09-11T01:17:44.311403+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:17:44.311403+0000","last_undegraded":"2024-09-11T01:17:44.311403+0000","last_fullsized":"2024-09-11T01:17:44.311403+0000","mapping_epoch":35,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32305152,"num_objects":7887,"num_object_clones":0,"num_object_copies":15774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10704,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12218","reported_seq":12347,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:44.265525+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:17:44.265525+0000","last_peered":"2024-09-11T01:17:44.265525+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:17:44.265525+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32460824,"num_objects":7926,"num_object_clones":0,"num_object_copies":15852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7926,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10519,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1752},{"shards":"2","objects":4030}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12224","reported_seq":12294,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:44.217408+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:17:44.217408+0000","last_peered":"2024-09-11T01:17:44.217408+0000","last_clean":"2024-09-11T01:17:44.217408+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:17:44.217408+0000","last_undegraded":"2024-09-11T01:17:44.217408+0000","last_fullsized":"2024-09-11T01:17:44.217408+0000","mapping_epoch":35,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32399360,"num_objects":7910,"num_object_clones":0,"num_object_copies":15820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12224,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12085","reported_seq":12213,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:41.152963+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:17:41.152963+0000","last_peered":"2024-09-11T01:17:41.152963+0000","last_clean":"2024-09-11T01:17:41.152963+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:17:41.152963+0000","last_undegraded":"2024-09-11T01:17:41.152963+0000","last_fullsized":"2024-09-11T01:17:41.152963+0000","mapping_epoch":34,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32456704,"num_objects":7924,"num_object_clones":0,"num_object_copies":15848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9090,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12263","reported_seq":12432,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:41.218477+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:17:41.218477+0000","last_peered":"2024-09-11T01:17:41.218477+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:17:41.218477+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32620544,"num_objects":7964,"num_object_clones":0,"num_object_copies":15928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8604,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4013},{"shards":"0,2","objects":1804}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12061","reported_seq":12192,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:41.125267+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:17:41.125267+0000","last_peered":"2024-09-11T01:17:41.125267+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:17:41.125267+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32231424,"num_objects":7869,"num_object_clones":0,"num_object_copies":15738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8918,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3883},{"shards":"0,2","objects":1893}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12288","reported_seq":12418,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:41.195026+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:17:41.195026+0000","last_peered":"2024-09-11T01:17:41.195026+0000","last_clean":"2024-09-11T01:17:41.195026+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:17:41.195026+0000","last_undegraded":"2024-09-11T01:17:41.195026+0000","last_fullsized":"2024-09-11T01:17:41.195026+0000","mapping_epoch":34,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32628736,"num_objects":7966,"num_object_clones":0,"num_object_copies":15932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9164,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683747+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683747+0000","last_peered":"2024-09-11T01:15:21.683747+0000","last_clean":"2024-09-11T01:15:21.683747+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683747+0000","last_undegraded":"2024-09-11T01:15:21.683747+0000","last_fullsized":"2024-09-11T01:15:21.683747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12159","reported_seq":12287,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:44.311644+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:17:44.311644+0000","last_peered":"2024-09-11T01:17:44.311644+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:17:44.311644+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32006144,"num_objects":7814,"num_object_clones":0,"num_object_copies":15628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7814,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10130,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1763},{"shards":"2","objects":3881}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12054","reported_seq":12181,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:41.236826+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:17:41.236826+0000","last_peered":"2024-09-11T01:17:41.236826+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:17:41.236826+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32006144,"num_objects":7814,"num_object_clones":0,"num_object_copies":15628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7814,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8641,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3830},{"shards":"0,2","objects":1868}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12343","reported_seq":12461,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:44.256190+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:17:44.256190+0000","last_peered":"2024-09-11T01:17:44.256190+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:17:44.256190+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32210944,"num_objects":7864,"num_object_clones":0,"num_object_copies":15728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10160,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1786},{"shards":"2","objects":3840}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684827+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684827+0000","last_peered":"2024-09-11T01:15:21.684827+0000","last_clean":"2024-09-11T01:15:21.684827+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684827+0000","last_undegraded":"2024-09-11T01:15:21.684827+0000","last_fullsized":"2024-09-11T01:15:21.684827+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12088","reported_seq":20069,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:44.271345+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:17:44.271345+0000","last_peered":"2024-09-11T01:17:44.271345+0000","last_clean":"2024-09-11T01:17:44.271345+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:17:44.271345+0000","last_undegraded":"2024-09-11T01:17:44.271345+0000","last_fullsized":"2024-09-11T01:17:44.271345+0000","mapping_epoch":44,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31571968,"num_objects":7708,"num_object_clones":0,"num_object_copies":15416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9873,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12280","reported_seq":12412,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:41.118566+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:17:41.118566+0000","last_peered":"2024-09-11T01:17:41.118566+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:17:41.118566+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32702464,"num_objects":7984,"num_object_clones":0,"num_object_copies":15968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9017,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4034},{"shards":"0,2","objects":1803}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684623+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684623+0000","last_peered":"2024-09-11T01:15:21.684623+0000","last_clean":"2024-09-11T01:15:21.684623+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684623+0000","last_undegraded":"2024-09-11T01:15:21.684623+0000","last_fullsized":"2024-09-11T01:15:21.684623+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12023","reported_seq":18703,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:41.260045+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:17:41.260045+0000","last_peered":"2024-09-11T01:17:41.260045+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:17:41.260045+0000","last_undegraded":"2024-09-11T01:17:41.260045+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32198656,"num_objects":7861,"num_object_clones":0,"num_object_copies":15722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":671,"num_objects_unfound":0,"num_objects_dirty":7861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8678,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3271,"num_bytes_recovered":13385728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7190},{"shards":"0,2","objects":671}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12203","reported_seq":12336,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:41.176073+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:17:41.176073+0000","last_peered":"2024-09-11T01:17:41.176073+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:17:41.176073+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32657408,"num_objects":7973,"num_object_clones":0,"num_object_copies":15946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8812,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3934},{"shards":"0,2","objects":1926}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692049+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.692049+0000","last_peered":"2024-09-11T01:15:21.692049+0000","last_clean":"2024-09-11T01:15:21.692049+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.692049+0000","last_undegraded":"2024-09-11T01:15:21.692049+0000","last_fullsized":"2024-09-11T01:15:21.692049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684417+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684417+0000","last_peered":"2024-09-11T01:15:21.684417+0000","last_clean":"2024-09-11T01:15:21.684417+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684417+0000","last_undegraded":"2024-09-11T01:15:21.684417+0000","last_fullsized":"2024-09-11T01:15:21.684417+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12025","reported_seq":12159,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:41.245616+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:17:41.245616+0000","last_peered":"2024-09-11T01:17:41.245616+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:17:41.245616+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32469609,"num_objects":7928,"num_object_clones":0,"num_object_copies":15856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9095,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4011},{"shards":"0,2","objects":1874}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":517800577,"num_objects":126418,"num_object_clones":0,"num_object_copies":252836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":671,"num_objects_unfound":0,"num_objects_dirty":126418,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":155869,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7199,"num_bytes_recovered":29466624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1152897024,"data_stored":1152819300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160819,"ondisk_log_size":160819,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182474,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":802320,"kb_used_data":445824,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93569520,"statfs":{"total":96636764160,"available":95815188480,"internally_reserved":0,"allocated":456523776,"data_stored":456264562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":365031083},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,4,5,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986247,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":682060,"kb_used_data":379708,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93689780,"statfs":{"total":96636764160,"available":95938334720,"internally_reserved":0,"allocated":388820992,"data_stored":388561783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659191,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591804,"kb_used_data":303340,"kb_used_omap":4,"kb_used_meta":288443,"kb_avail":93780036,"statfs":{"total":96636764160,"available":96030756864,"internally_reserved":0,"allocated":310620160,"data_stored":310297731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":295366319},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,1,5,5,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":454897664,"data_stored":454894185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":310259712,"data_stored":310193018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":387739648,"data_stored":387732097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:45.141 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:45.885 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:17:45.886 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:17:45.953 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":239,"stamp":"2024-09-11T01:17:44.672388+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":517800596,"num_objects":126420,"num_object_clones":0,"num_object_copies":252840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":671,"num_objects_unfound":0,"num_objects_dirty":126420,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":155871,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7200,"num_bytes_recovered":29466643,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160822,"ondisk_log_size":160822,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2076184,"kb_used_data":1128872,"kb_used_omap":12,"kb_used_meta":947251,"kb_avail":281039336,"statfs":{"total":289910292480,"available":287784280064,"internally_reserved":0,"allocated":1155964928,"data_stored":1155124076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":969985028},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,4,5,12,9,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21377024,"num_objects":-5219,"num_object_clones":0,"num_object_copies":-10438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-368,"num_objects_unfound":0,"num_objects_dirty":-5219,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5219,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":368,"num_bytes_recovered":1507328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002772"},"pg_stats":[{"pgid":"2.f","version":"45'12240","reported_seq":12316,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:41.266132+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:17:41.266132+0000","last_peered":"2024-09-11T01:17:41.266132+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:17:41.266132+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32874496,"num_objects":8026,"num_object_clones":0,"num_object_copies":16052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8026,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12240,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3954},{"shards":"0,2","objects":1965}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12465","reported_seq":12567,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:44.311403+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:17:44.311403+0000","last_peered":"2024-09-11T01:17:44.311403+0000","last_clean":"2024-09-11T01:17:44.311403+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:17:44.311403+0000","last_undegraded":"2024-09-11T01:17:44.311403+0000","last_fullsized":"2024-09-11T01:17:44.311403+0000","mapping_epoch":35,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32305152,"num_objects":7887,"num_object_clones":0,"num_object_copies":15774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10704,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12218","reported_seq":12347,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:44.265525+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:17:44.265525+0000","last_peered":"2024-09-11T01:17:44.265525+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:17:44.265525+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32460824,"num_objects":7926,"num_object_clones":0,"num_object_copies":15852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7926,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10519,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1752},{"shards":"2","objects":4030}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12224","reported_seq":12294,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:44.217408+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:17:44.217408+0000","last_peered":"2024-09-11T01:17:44.217408+0000","last_clean":"2024-09-11T01:17:44.217408+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:17:44.217408+0000","last_undegraded":"2024-09-11T01:17:44.217408+0000","last_fullsized":"2024-09-11T01:17:44.217408+0000","mapping_epoch":35,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32399360,"num_objects":7910,"num_object_clones":0,"num_object_copies":15820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12224,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12085","reported_seq":12213,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:41.152963+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:17:41.152963+0000","last_peered":"2024-09-11T01:17:41.152963+0000","last_clean":"2024-09-11T01:17:41.152963+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:17:41.152963+0000","last_undegraded":"2024-09-11T01:17:41.152963+0000","last_fullsized":"2024-09-11T01:17:41.152963+0000","mapping_epoch":34,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32456704,"num_objects":7924,"num_object_clones":0,"num_object_copies":15848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9090,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12263","reported_seq":12432,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:41.218477+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:17:41.218477+0000","last_peered":"2024-09-11T01:17:41.218477+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:17:41.218477+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32620544,"num_objects":7964,"num_object_clones":0,"num_object_copies":15928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8604,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4013},{"shards":"0,2","objects":1804}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12061","reported_seq":12192,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:41.125267+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:17:41.125267+0000","last_peered":"2024-09-11T01:17:41.125267+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:17:41.125267+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32231424,"num_objects":7869,"num_object_clones":0,"num_object_copies":15738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8918,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3883},{"shards":"0,2","objects":1893}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12288","reported_seq":12418,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:41.195026+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:17:41.195026+0000","last_peered":"2024-09-11T01:17:41.195026+0000","last_clean":"2024-09-11T01:17:41.195026+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:17:41.195026+0000","last_undegraded":"2024-09-11T01:17:41.195026+0000","last_fullsized":"2024-09-11T01:17:41.195026+0000","mapping_epoch":34,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32628736,"num_objects":7966,"num_object_clones":0,"num_object_copies":15932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9164,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683747+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683747+0000","last_peered":"2024-09-11T01:15:21.683747+0000","last_clean":"2024-09-11T01:15:21.683747+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683747+0000","last_undegraded":"2024-09-11T01:15:21.683747+0000","last_fullsized":"2024-09-11T01:15:21.683747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12159","reported_seq":12287,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:44.311644+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:17:44.311644+0000","last_peered":"2024-09-11T01:17:44.311644+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:17:44.311644+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32006144,"num_objects":7814,"num_object_clones":0,"num_object_copies":15628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7814,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10130,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1763},{"shards":"2","objects":3881}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12054","reported_seq":12181,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:41.236826+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:17:41.236826+0000","last_peered":"2024-09-11T01:17:41.236826+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:17:41.236826+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32006144,"num_objects":7814,"num_object_clones":0,"num_object_copies":15628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7814,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8641,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3830},{"shards":"0,2","objects":1868}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12343","reported_seq":12461,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:44.256190+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:17:44.256190+0000","last_peered":"2024-09-11T01:17:44.256190+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:17:44.256190+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32210944,"num_objects":7864,"num_object_clones":0,"num_object_copies":15728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10160,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1786},{"shards":"2","objects":3840}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684827+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684827+0000","last_peered":"2024-09-11T01:15:21.684827+0000","last_clean":"2024-09-11T01:15:21.684827+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684827+0000","last_undegraded":"2024-09-11T01:15:21.684827+0000","last_fullsized":"2024-09-11T01:15:21.684827+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12088","reported_seq":20069,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:44.271345+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:17:44.271345+0000","last_peered":"2024-09-11T01:17:44.271345+0000","last_clean":"2024-09-11T01:17:44.271345+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:17:44.271345+0000","last_undegraded":"2024-09-11T01:17:44.271345+0000","last_fullsized":"2024-09-11T01:17:44.271345+0000","mapping_epoch":44,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31571968,"num_objects":7708,"num_object_clones":0,"num_object_copies":15416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9873,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12280","reported_seq":12412,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:41.118566+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:17:41.118566+0000","last_peered":"2024-09-11T01:17:41.118566+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:17:41.118566+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32702464,"num_objects":7984,"num_object_clones":0,"num_object_copies":15968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9017,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4034},{"shards":"0,2","objects":1803}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684623+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684623+0000","last_peered":"2024-09-11T01:15:21.684623+0000","last_clean":"2024-09-11T01:15:21.684623+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684623+0000","last_undegraded":"2024-09-11T01:15:21.684623+0000","last_fullsized":"2024-09-11T01:15:21.684623+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12023","reported_seq":18703,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:41.260045+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:17:41.260045+0000","last_peered":"2024-09-11T01:17:41.260045+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:17:41.260045+0000","last_undegraded":"2024-09-11T01:17:41.260045+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32198656,"num_objects":7861,"num_object_clones":0,"num_object_copies":15722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":671,"num_objects_unfound":0,"num_objects_dirty":7861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8678,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3271,"num_bytes_recovered":13385728,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7190},{"shards":"0,2","objects":671}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12203","reported_seq":12336,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:41.176073+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:17:41.176073+0000","last_peered":"2024-09-11T01:17:41.176073+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:17:41.176073+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32657408,"num_objects":7973,"num_object_clones":0,"num_object_copies":15946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8812,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3934},{"shards":"0,2","objects":1926}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692049+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.692049+0000","last_peered":"2024-09-11T01:15:21.692049+0000","last_clean":"2024-09-11T01:15:21.692049+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.692049+0000","last_undegraded":"2024-09-11T01:15:21.692049+0000","last_fullsized":"2024-09-11T01:15:21.692049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684417+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684417+0000","last_peered":"2024-09-11T01:15:21.684417+0000","last_clean":"2024-09-11T01:15:21.684417+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684417+0000","last_undegraded":"2024-09-11T01:15:21.684417+0000","last_fullsized":"2024-09-11T01:15:21.684417+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12025","reported_seq":12159,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:41.245616+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:17:41.245616+0000","last_peered":"2024-09-11T01:17:41.245616+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:17:41.245616+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32469609,"num_objects":7928,"num_object_clones":0,"num_object_copies":15856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7928,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9095,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":4011},{"shards":"0,2","objects":1874}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":517800577,"num_objects":126418,"num_object_clones":0,"num_object_copies":252836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":671,"num_objects_unfound":0,"num_objects_dirty":126418,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":155869,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7199,"num_bytes_recovered":29466624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1152897024,"data_stored":1152819300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160819,"ondisk_log_size":160819,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182474,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":802320,"kb_used_data":445824,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93569520,"statfs":{"total":96636764160,"available":95815188480,"internally_reserved":0,"allocated":456523776,"data_stored":456264562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":365031083},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,4,5,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986247,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":682060,"kb_used_data":379708,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93689780,"statfs":{"total":96636764160,"available":95938334720,"internally_reserved":0,"allocated":388820992,"data_stored":388561783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659191,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591804,"kb_used_data":303340,"kb_used_omap":4,"kb_used_meta":288443,"kb_avail":93780036,"statfs":{"total":96636764160,"available":96030756864,"internally_reserved":0,"allocated":310620160,"data_stored":310297731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":295366319},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,1,5,5,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":454897664,"data_stored":454894185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":310259712,"data_stored":310193018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":387739648,"data_stored":387732097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:45.954 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:17:45.954 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'12240", 'reported_seq': 12316, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:41.266132+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:17:41.266132+0000', 'last_peered': '2024-09-11T01:17:41.266132+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:17:41.266132+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32874496, 'num_objects': 8026, 'num_object_clones': 0, 'num_object_copies': 16052, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 8026, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12240, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3954}, {'shards': '0,2', 'objects': 1965}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:45.954 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:17:45.954 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'12218", 'reported_seq': 12347, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:44.265525+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:17:44.265525+0000', 'last_peered': '2024-09-11T01:17:44.265525+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:17:44.265525+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32460824, 'num_objects': 7926, 'num_object_clones': 0, 'num_object_copies': 15852, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7926, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10519, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1752}, {'shards': '2', 'objects': 4030}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:45.955 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:17:45.955 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'12263", 'reported_seq': 12432, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:41.218477+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:17:41.218477+0000', 'last_peered': '2024-09-11T01:17:41.218477+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:17:41.218477+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32620544, 'num_objects': 7964, 'num_object_clones': 0, 'num_object_copies': 15928, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7964, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8604, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4013}, {'shards': '0,2', 'objects': 1804}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:45.955 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:17:45.955 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'12061", 'reported_seq': 12192, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:41.125267+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:17:41.125267+0000', 'last_peered': '2024-09-11T01:17:41.125267+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:17:41.125267+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32231424, 'num_objects': 7869, 'num_object_clones': 0, 'num_object_copies': 15738, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7869, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8918, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3883}, {'shards': '0,2', 'objects': 1893}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:45.955 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:17:45.955 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'12159", 'reported_seq': 12287, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:44.311644+0000', 'last_change': '2024-09-11T01:13:10.159069+0000', 'last_active': '2024-09-11T01:17:44.311644+0000', 'last_peered': '2024-09-11T01:17:44.311644+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:17:44.311644+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "36'2100", 'ondisk_log_start': "36'2100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32006144, 'num_objects': 7814, 'num_object_clones': 0, 'num_object_copies': 15628, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7814, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10130, 'num_write_kb': 31844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1763}, {'shards': '2', 'objects': 3881}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:45.955 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:17:45.955 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'12054", 'reported_seq': 12181, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:41.236826+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:17:41.236826+0000', 'last_peered': '2024-09-11T01:17:41.236826+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:17:41.236826+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32006144, 'num_objects': 7814, 'num_object_clones': 0, 'num_object_copies': 15628, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7814, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8641, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3830}, {'shards': '0,2', 'objects': 1868}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:45.955 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:17:45.955 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'12343", 'reported_seq': 12461, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:44.256190+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:17:44.256190+0000', 'last_peered': '2024-09-11T01:17:44.256190+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:17:44.256190+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32210944, 'num_objects': 7864, 'num_object_clones': 0, 'num_object_copies': 15728, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7864, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10160, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1786}, {'shards': '2', 'objects': 3840}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:45.956 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:17:45.956 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'12280", 'reported_seq': 12412, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:41.118566+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:17:41.118566+0000', 'last_peered': '2024-09-11T01:17:41.118566+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:17:41.118566+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32702464, 'num_objects': 7984, 'num_object_clones': 0, 'num_object_copies': 15968, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7984, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9017, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4034}, {'shards': '0,2', 'objects': 1803}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:45.956 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:17:45.956 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'12023", 'reported_seq': 18703, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:17:41.260045+0000', 'last_change': '2024-09-11T01:15:19.913288+0000', 'last_active': '2024-09-11T01:17:41.260045+0000', 'last_peered': '2024-09-11T01:17:41.260045+0000', 'last_clean': '2024-09-11T01:13:07.624565+0000', 'last_became_active': '2024-09-11T01:13:10.199795+0000', 'last_became_peered': '2024-09-11T01:13:10.199795+0000', 'last_unstale': '2024-09-11T01:17:41.260045+0000', 'last_undegraded': '2024-09-11T01:17:41.260045+0000', 'last_fullsized': '2024-09-11T01:13:09.652311+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3415", 'last_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'objects_scrubbed': 3337, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1975, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32198656, 'num_objects': 7861, 'num_object_clones': 0, 'num_object_copies': 15722, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 671, 'num_objects_unfound': 0, 'num_objects_dirty': 7861, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8678, 'num_write_kb': 26404, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3271, 'num_bytes_recovered': 13385728, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7190}, {'shards': '0,2', 'objects': 671}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:45.956 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:17:45.956 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'12203", 'reported_seq': 12336, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:41.176073+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:17:41.176073+0000', 'last_peered': '2024-09-11T01:17:41.176073+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:17:41.176073+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "36'2200", 'ondisk_log_start': "36'2200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32657408, 'num_objects': 7973, 'num_object_clones': 0, 'num_object_copies': 15946, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7973, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8812, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3934}, {'shards': '0,2', 'objects': 1926}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:45.956 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:17:45.956 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'12025", 'reported_seq': 12159, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:41.245616+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:17:41.245616+0000', 'last_peered': '2024-09-11T01:17:41.245616+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:17:41.245616+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "36'2000", 'ondisk_log_start': "36'2000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32469609, 'num_objects': 7928, 'num_object_clones': 0, 'num_object_copies': 15856, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7928, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9095, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 4011}, {'shards': '0,2', 'objects': 1874}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:45.957 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:17:46.924 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:17:46.924 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39769/250040 objects degraded (15.905%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":403,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":125020,"data_bytes":512066196,"bytes_used":2134573056,"bytes_avail":287775719424,"bytes_total":289910292480,"degraded_objects":39769,"degraded_total":250040,"degraded_ratio":0.15905055191169412,"misplaced_objects":483,"misplaced_total":250040,"misplaced_ratio":0.0019316909294512879,"recovering_objects_per_sec":46,"recovering_bytes_per_sec":189738,"recovering_keys_per_sec":0,"num_objects_recovered":556,"num_bytes_recovered":2277376,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":483},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:17:47.003 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:17:47.003 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:47.821 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:17:47.822 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:17:47.903 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":240,"stamp":"2024-09-11T01:17:46.672846+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":512066196,"num_objects":125020,"num_object_clones":0,"num_object_copies":250040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":483,"num_objects_unfound":0,"num_objects_dirty":125020,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":157271,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7388,"num_bytes_recovered":30236691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160722,"ondisk_log_size":160722,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2084544,"kb_used_data":1120592,"kb_used_omap":12,"kb_used_meta":963891,"kb_avail":281030976,"statfs":{"total":289910292480,"available":287775719424,"internally_reserved":0,"allocated":1147486208,"data_stored":1146645356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":987024388},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,8,9,6,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23760896,"num_objects":-5801,"num_object_clones":0,"num_object_copies":-11602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-556,"num_objects_unfound":0,"num_objects_dirty":-5801,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5801,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":556,"num_bytes_recovered":2277376,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002722"},"pg_stats":[{"pgid":"2.f","version":"45'12409","reported_seq":12487,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:46.257811+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:17:46.257811+0000","last_peered":"2024-09-11T01:17:46.257811+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:17:46.257811+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32182272,"num_objects":7857,"num_object_clones":0,"num_object_copies":15714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12409,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3785},{"shards":"0,2","objects":1796}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12465","reported_seq":12567,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:44.311403+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:17:44.311403+0000","last_peered":"2024-09-11T01:17:44.311403+0000","last_clean":"2024-09-11T01:17:44.311403+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:17:44.311403+0000","last_undegraded":"2024-09-11T01:17:44.311403+0000","last_fullsized":"2024-09-11T01:17:44.311403+0000","mapping_epoch":35,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32305152,"num_objects":7887,"num_object_clones":0,"num_object_copies":15774,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7887,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10704,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12218","reported_seq":12347,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:44.265525+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:17:44.265525+0000","last_peered":"2024-09-11T01:17:44.265525+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:17:44.265525+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32460824,"num_objects":7926,"num_object_clones":0,"num_object_copies":15852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7926,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10519,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1752},{"shards":"2","objects":4030}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12224","reported_seq":12294,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:44.217408+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:17:44.217408+0000","last_peered":"2024-09-11T01:17:44.217408+0000","last_clean":"2024-09-11T01:17:44.217408+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:17:44.217408+0000","last_undegraded":"2024-09-11T01:17:44.217408+0000","last_fullsized":"2024-09-11T01:17:44.217408+0000","mapping_epoch":35,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32399360,"num_objects":7910,"num_object_clones":0,"num_object_copies":15820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12224,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12224","reported_seq":12352,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:46.311405+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:17:46.311405+0000","last_peered":"2024-09-11T01:17:46.311405+0000","last_clean":"2024-09-11T01:17:46.311405+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:17:46.311405+0000","last_undegraded":"2024-09-11T01:17:46.311405+0000","last_fullsized":"2024-09-11T01:17:46.311405+0000","mapping_epoch":34,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31887360,"num_objects":7785,"num_object_clones":0,"num_object_copies":15570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9229,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12404","reported_seq":12574,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:46.266666+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:17:46.266666+0000","last_peered":"2024-09-11T01:17:46.266666+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:17:46.266666+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32043008,"num_objects":7823,"num_object_clones":0,"num_object_copies":15646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8745,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3872},{"shards":"0,2","objects":1663}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12198","reported_seq":12329,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:46.219245+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:17:46.219245+0000","last_peered":"2024-09-11T01:17:46.219245+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:17:46.219245+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31670272,"num_objects":7732,"num_object_clones":0,"num_object_copies":15464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9055,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3746},{"shards":"0,2","objects":1756}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12409","reported_seq":12539,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:46.277258+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:17:46.277258+0000","last_peered":"2024-09-11T01:17:46.277258+0000","last_clean":"2024-09-11T01:17:46.277258+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:17:46.277258+0000","last_undegraded":"2024-09-11T01:17:46.277258+0000","last_fullsized":"2024-09-11T01:17:46.277258+0000","mapping_epoch":34,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32133120,"num_objects":7845,"num_object_clones":0,"num_object_copies":15690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9285,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683747+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683747+0000","last_peered":"2024-09-11T01:15:21.683747+0000","last_clean":"2024-09-11T01:15:21.683747+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683747+0000","last_undegraded":"2024-09-11T01:15:21.683747+0000","last_fullsized":"2024-09-11T01:15:21.683747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12159","reported_seq":12287,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:44.311644+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:17:44.311644+0000","last_peered":"2024-09-11T01:17:44.311644+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:17:44.311644+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32006144,"num_objects":7814,"num_object_clones":0,"num_object_copies":15628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7814,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10130,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1763},{"shards":"2","objects":3881}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12180","reported_seq":12309,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:46.282862+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:17:46.282862+0000","last_peered":"2024-09-11T01:17:46.282862+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:17:46.282862+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31490048,"num_objects":7688,"num_object_clones":0,"num_object_copies":15376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8767,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3704},{"shards":"0,2","objects":1742}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12343","reported_seq":12461,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:44.256190+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:17:44.256190+0000","last_peered":"2024-09-11T01:17:44.256190+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:17:44.256190+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32210944,"num_objects":7864,"num_object_clones":0,"num_object_copies":15728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7864,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10160,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1786},{"shards":"2","objects":3840}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684827+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684827+0000","last_peered":"2024-09-11T01:15:21.684827+0000","last_clean":"2024-09-11T01:15:21.684827+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684827+0000","last_undegraded":"2024-09-11T01:15:21.684827+0000","last_fullsized":"2024-09-11T01:15:21.684827+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12088","reported_seq":20069,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:44.271345+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:17:44.271345+0000","last_peered":"2024-09-11T01:17:44.271345+0000","last_clean":"2024-09-11T01:17:44.271345+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:17:44.271345+0000","last_undegraded":"2024-09-11T01:17:44.271345+0000","last_fullsized":"2024-09-11T01:17:44.271345+0000","mapping_epoch":44,"log_start":"36'2000","ondisk_log_start":"36'2000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31571968,"num_objects":7708,"num_object_clones":0,"num_object_copies":15416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7708,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9873,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12415","reported_seq":12547,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:46.242018+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:17:46.242018+0000","last_peered":"2024-09-11T01:17:46.242018+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:17:46.242018+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32149504,"num_objects":7849,"num_object_clones":0,"num_object_copies":15698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7849,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9152,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3899},{"shards":"0,2","objects":1668}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684623+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684623+0000","last_peered":"2024-09-11T01:15:21.684623+0000","last_clean":"2024-09-11T01:15:21.684623+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684623+0000","last_undegraded":"2024-09-11T01:15:21.684623+0000","last_fullsized":"2024-09-11T01:15:21.684623+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12168","reported_seq":19224,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:46.243077+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:17:46.243077+0000","last_peered":"2024-09-11T01:17:46.243077+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:17:46.243077+0000","last_undegraded":"2024-09-11T01:17:46.243077+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31604736,"num_objects":7716,"num_object_clones":0,"num_object_copies":15432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":483,"num_objects_unfound":0,"num_objects_dirty":7716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8823,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3459,"num_bytes_recovered":14155776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7233},{"shards":"0,2","objects":483}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12352","reported_seq":12487,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:46.223281+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:17:46.223281+0000","last_peered":"2024-09-11T01:17:46.223281+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:17:46.223281+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32047104,"num_objects":7824,"num_object_clones":0,"num_object_copies":15648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8961,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3785},{"shards":"0,2","objects":1777}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692049+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.692049+0000","last_peered":"2024-09-11T01:15:21.692049+0000","last_clean":"2024-09-11T01:15:21.692049+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.692049+0000","last_undegraded":"2024-09-11T01:15:21.692049+0000","last_fullsized":"2024-09-11T01:15:21.692049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684417+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684417+0000","last_peered":"2024-09-11T01:15:21.684417+0000","last_clean":"2024-09-11T01:15:21.684417+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684417+0000","last_undegraded":"2024-09-11T01:15:21.684417+0000","last_fullsized":"2024-09-11T01:15:21.684417+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12163","reported_seq":12296,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:46.291281+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:17:46.291281+0000","last_peered":"2024-09-11T01:17:46.291281+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:17:46.291281+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31904361,"num_objects":7790,"num_object_clones":0,"num_object_copies":15580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7790,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9233,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3873},{"shards":"0,2","objects":1736}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":512066177,"num_objects":125018,"num_object_clones":0,"num_object_copies":250036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":483,"num_objects_unfound":0,"num_objects_dirty":125018,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":157269,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7387,"num_bytes_recovered":30236672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1145577472,"data_stored":1145499748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160719,"ondisk_log_size":160719,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182475,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":802720,"kb_used_data":437904,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93569120,"statfs":{"total":96636764160,"available":95814778880,"internally_reserved":0,"allocated":448413696,"data_stored":448154482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":373550763},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,4,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986247,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":682060,"kb_used_data":379708,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93689780,"statfs":{"total":96636764160,"available":95938334720,"internally_reserved":0,"allocated":388820992,"data_stored":388561783,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659192,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":599764,"kb_used_data":302980,"kb_used_omap":4,"kb_used_meta":296763,"kb_avail":93772076,"statfs":{"total":96636764160,"available":96022605824,"internally_reserved":0,"allocated":310251520,"data_stored":309929091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":303885999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,4,2,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":447975424,"data_stored":447971945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":309862400,"data_stored":309795706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":387739648,"data_stored":387732097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:50.905 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:51.825 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:17:51.826 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:17:51.868 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":242,"stamp":"2024-09-11T01:17:50.673773+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":508514964,"num_objects":124153,"num_object_clones":0,"num_object_copies":248306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":483,"num_objects_unfound":0,"num_objects_dirty":124153,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":158138,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7388,"num_bytes_recovered":30236691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160689,"ondisk_log_size":160689,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2071484,"kb_used_data":1115852,"kb_used_omap":12,"kb_used_meta":955571,"kb_avail":281044036,"statfs":{"total":289910292480,"available":287789092864,"internally_reserved":0,"allocated":1142632448,"data_stored":1141791596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":978504708},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,2,8,8,9,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21803008,"num_objects":-5323,"num_object_clones":0,"num_object_copies":-10646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-385,"num_objects_unfound":0,"num_objects_dirty":-5323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5323,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":385,"num_bytes_recovered":1576960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002687"},"pg_stats":[{"pgid":"2.f","version":"45'12409","reported_seq":12487,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:46.257811+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:17:46.257811+0000","last_peered":"2024-09-11T01:17:46.257811+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:17:46.257811+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32182272,"num_objects":7857,"num_object_clones":0,"num_object_copies":15714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12409,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3785},{"shards":"0,2","objects":1796}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12613","reported_seq":12715,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:49.345616+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:17:49.345616+0000","last_peered":"2024-09-11T01:17:49.345616+0000","last_clean":"2024-09-11T01:17:49.345616+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:17:49.345616+0000","last_undegraded":"2024-09-11T01:17:49.345616+0000","last_fullsized":"2024-09-11T01:17:49.345616+0000","mapping_epoch":35,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31698944,"num_objects":7739,"num_object_clones":0,"num_object_copies":15478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10852,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12381","reported_seq":12510,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:49.378296+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:17:49.378296+0000","last_peered":"2024-09-11T01:17:49.378296+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:17:49.378296+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31793176,"num_objects":7763,"num_object_clones":0,"num_object_copies":15526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7763,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10682,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1589},{"shards":"2","objects":3867}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12364","reported_seq":12434,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:49.335050+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:17:49.335050+0000","last_peered":"2024-09-11T01:17:49.335050+0000","last_clean":"2024-09-11T01:17:49.335050+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:17:49.335050+0000","last_undegraded":"2024-09-11T01:17:49.335050+0000","last_fullsized":"2024-09-11T01:17:49.335050+0000","mapping_epoch":35,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31825920,"num_objects":7770,"num_object_clones":0,"num_object_copies":15540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12364,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12224","reported_seq":12352,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:46.311405+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:17:46.311405+0000","last_peered":"2024-09-11T01:17:46.311405+0000","last_clean":"2024-09-11T01:17:46.311405+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:17:46.311405+0000","last_undegraded":"2024-09-11T01:17:46.311405+0000","last_fullsized":"2024-09-11T01:17:46.311405+0000","mapping_epoch":34,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31887360,"num_objects":7785,"num_object_clones":0,"num_object_copies":15570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9229,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12404","reported_seq":12574,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:46.266666+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:17:46.266666+0000","last_peered":"2024-09-11T01:17:46.266666+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:17:46.266666+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32043008,"num_objects":7823,"num_object_clones":0,"num_object_copies":15646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8745,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3872},{"shards":"0,2","objects":1663}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12198","reported_seq":12329,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:46.219245+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:17:46.219245+0000","last_peered":"2024-09-11T01:17:46.219245+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:17:46.219245+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31670272,"num_objects":7732,"num_object_clones":0,"num_object_copies":15464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9055,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3746},{"shards":"0,2","objects":1756}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12409","reported_seq":12539,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:46.277258+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:17:46.277258+0000","last_peered":"2024-09-11T01:17:46.277258+0000","last_clean":"2024-09-11T01:17:46.277258+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:17:46.277258+0000","last_undegraded":"2024-09-11T01:17:46.277258+0000","last_fullsized":"2024-09-11T01:17:46.277258+0000","mapping_epoch":34,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32133120,"num_objects":7845,"num_object_clones":0,"num_object_copies":15690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9285,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683747+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683747+0000","last_peered":"2024-09-11T01:15:21.683747+0000","last_clean":"2024-09-11T01:15:21.683747+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683747+0000","last_undegraded":"2024-09-11T01:15:21.683747+0000","last_fullsized":"2024-09-11T01:15:21.683747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12301","reported_seq":12427,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:49.361358+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:17:49.361358+0000","last_peered":"2024-09-11T01:17:49.361358+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:17:49.361358+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31424512,"num_objects":7672,"num_object_clones":0,"num_object_copies":15344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10272,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1621},{"shards":"2","objects":3739}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12180","reported_seq":12309,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:46.282862+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:17:46.282862+0000","last_peered":"2024-09-11T01:17:46.282862+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:17:46.282862+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31490048,"num_objects":7688,"num_object_clones":0,"num_object_copies":15376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8767,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3704},{"shards":"0,2","objects":1742}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12480","reported_seq":12599,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:49.348628+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:17:49.348628+0000","last_peered":"2024-09-11T01:17:49.348628+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:17:49.348628+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31649792,"num_objects":7727,"num_object_clones":0,"num_object_copies":15454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10297,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1649},{"shards":"2","objects":3703}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684827+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684827+0000","last_peered":"2024-09-11T01:15:21.684827+0000","last_clean":"2024-09-11T01:15:21.684827+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684827+0000","last_undegraded":"2024-09-11T01:15:21.684827+0000","last_fullsized":"2024-09-11T01:15:21.684827+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12225","reported_seq":20206,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:49.358809+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:17:49.358809+0000","last_peered":"2024-09-11T01:17:49.358809+0000","last_clean":"2024-09-11T01:17:49.358809+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:17:49.358809+0000","last_undegraded":"2024-09-11T01:17:49.358809+0000","last_fullsized":"2024-09-11T01:17:49.358809+0000","mapping_epoch":44,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31010816,"num_objects":7571,"num_object_clones":0,"num_object_copies":15142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10010,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12415","reported_seq":12547,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:46.242018+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:17:46.242018+0000","last_peered":"2024-09-11T01:17:46.242018+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:17:46.242018+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32149504,"num_objects":7849,"num_object_clones":0,"num_object_copies":15698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7849,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9152,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3899},{"shards":"0,2","objects":1668}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684623+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684623+0000","last_peered":"2024-09-11T01:15:21.684623+0000","last_clean":"2024-09-11T01:15:21.684623+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684623+0000","last_undegraded":"2024-09-11T01:15:21.684623+0000","last_fullsized":"2024-09-11T01:15:21.684623+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12168","reported_seq":19224,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:46.243077+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:17:46.243077+0000","last_peered":"2024-09-11T01:17:46.243077+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:17:46.243077+0000","last_undegraded":"2024-09-11T01:17:46.243077+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31604736,"num_objects":7716,"num_object_clones":0,"num_object_copies":15432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":483,"num_objects_unfound":0,"num_objects_dirty":7716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8823,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3459,"num_bytes_recovered":14155776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7233},{"shards":"0,2","objects":483}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12352","reported_seq":12487,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:46.223281+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:17:46.223281+0000","last_peered":"2024-09-11T01:17:46.223281+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:17:46.223281+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32047104,"num_objects":7824,"num_object_clones":0,"num_object_copies":15648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8961,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3785},{"shards":"0,2","objects":1777}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692049+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.692049+0000","last_peered":"2024-09-11T01:15:21.692049+0000","last_clean":"2024-09-11T01:15:21.692049+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.692049+0000","last_undegraded":"2024-09-11T01:15:21.692049+0000","last_fullsized":"2024-09-11T01:15:21.692049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684417+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684417+0000","last_peered":"2024-09-11T01:15:21.684417+0000","last_clean":"2024-09-11T01:15:21.684417+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684417+0000","last_undegraded":"2024-09-11T01:15:21.684417+0000","last_fullsized":"2024-09-11T01:15:21.684417+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12163","reported_seq":12296,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:46.291281+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:17:46.291281+0000","last_peered":"2024-09-11T01:17:46.291281+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:17:46.291281+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31904361,"num_objects":7790,"num_object_clones":0,"num_object_copies":15580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7790,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9233,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3873},{"shards":"0,2","objects":1736}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":508514945,"num_objects":124151,"num_object_clones":0,"num_object_copies":248302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":483,"num_objects_unfound":0,"num_objects_dirty":124151,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":158136,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7387,"num_bytes_recovered":30236672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1140731904,"data_stored":1140654180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160686,"ondisk_log_size":160686,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182475,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":802720,"kb_used_data":437904,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93569120,"statfs":{"total":96636764160,"available":95814778880,"internally_reserved":0,"allocated":448413696,"data_stored":448154482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":373550763},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,4,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986248,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":681732,"kb_used_data":375220,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93690108,"statfs":{"total":96636764160,"available":95938670592,"internally_reserved":0,"allocated":384225280,"data_stored":383966071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659193,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":587032,"kb_used_data":302728,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93784808,"statfs":{"total":96636764160,"available":96035643392,"internally_reserved":0,"allocated":309993472,"data_stored":309671043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":291106479},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,2,4,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":447975424,"data_stored":447971945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":309604352,"data_stored":309537658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":383152128,"data_stored":383144577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:51.869 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:52.648 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:17:52.650 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:17:52.738 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":242,"stamp":"2024-09-11T01:17:50.673773+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":508514964,"num_objects":124153,"num_object_clones":0,"num_object_copies":248306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":483,"num_objects_unfound":0,"num_objects_dirty":124153,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":158138,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7388,"num_bytes_recovered":30236691,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160689,"ondisk_log_size":160689,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2071484,"kb_used_data":1115852,"kb_used_omap":12,"kb_used_meta":955571,"kb_avail":281044036,"statfs":{"total":289910292480,"available":287789092864,"internally_reserved":0,"allocated":1142632448,"data_stored":1141791596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":978504708},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,2,8,8,9,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21803008,"num_objects":-5323,"num_object_clones":0,"num_object_copies":-10646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-385,"num_objects_unfound":0,"num_objects_dirty":-5323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5323,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":385,"num_bytes_recovered":1576960,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002687"},"pg_stats":[{"pgid":"2.f","version":"45'12409","reported_seq":12487,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:46.257811+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:17:46.257811+0000","last_peered":"2024-09-11T01:17:46.257811+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:17:46.257811+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32182272,"num_objects":7857,"num_object_clones":0,"num_object_copies":15714,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7857,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12409,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3785},{"shards":"0,2","objects":1796}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12613","reported_seq":12715,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:49.345616+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:17:49.345616+0000","last_peered":"2024-09-11T01:17:49.345616+0000","last_clean":"2024-09-11T01:17:49.345616+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:17:49.345616+0000","last_undegraded":"2024-09-11T01:17:49.345616+0000","last_fullsized":"2024-09-11T01:17:49.345616+0000","mapping_epoch":35,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31698944,"num_objects":7739,"num_object_clones":0,"num_object_copies":15478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10852,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12381","reported_seq":12510,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:49.378296+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:17:49.378296+0000","last_peered":"2024-09-11T01:17:49.378296+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:17:49.378296+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31793176,"num_objects":7763,"num_object_clones":0,"num_object_copies":15526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7763,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10682,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1589},{"shards":"2","objects":3867}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12364","reported_seq":12434,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:49.335050+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:17:49.335050+0000","last_peered":"2024-09-11T01:17:49.335050+0000","last_clean":"2024-09-11T01:17:49.335050+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:17:49.335050+0000","last_undegraded":"2024-09-11T01:17:49.335050+0000","last_fullsized":"2024-09-11T01:17:49.335050+0000","mapping_epoch":35,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31825920,"num_objects":7770,"num_object_clones":0,"num_object_copies":15540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12364,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12224","reported_seq":12352,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:46.311405+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:17:46.311405+0000","last_peered":"2024-09-11T01:17:46.311405+0000","last_clean":"2024-09-11T01:17:46.311405+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:17:46.311405+0000","last_undegraded":"2024-09-11T01:17:46.311405+0000","last_fullsized":"2024-09-11T01:17:46.311405+0000","mapping_epoch":34,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31887360,"num_objects":7785,"num_object_clones":0,"num_object_copies":15570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9229,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12404","reported_seq":12574,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:46.266666+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:17:46.266666+0000","last_peered":"2024-09-11T01:17:46.266666+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:17:46.266666+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32043008,"num_objects":7823,"num_object_clones":0,"num_object_copies":15646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8745,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3872},{"shards":"0,2","objects":1663}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12198","reported_seq":12329,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:46.219245+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:17:46.219245+0000","last_peered":"2024-09-11T01:17:46.219245+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:17:46.219245+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31670272,"num_objects":7732,"num_object_clones":0,"num_object_copies":15464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7732,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9055,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3746},{"shards":"0,2","objects":1756}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12409","reported_seq":12539,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:46.277258+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:17:46.277258+0000","last_peered":"2024-09-11T01:17:46.277258+0000","last_clean":"2024-09-11T01:17:46.277258+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:17:46.277258+0000","last_undegraded":"2024-09-11T01:17:46.277258+0000","last_fullsized":"2024-09-11T01:17:46.277258+0000","mapping_epoch":34,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32133120,"num_objects":7845,"num_object_clones":0,"num_object_copies":15690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9285,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683747+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683747+0000","last_peered":"2024-09-11T01:15:21.683747+0000","last_clean":"2024-09-11T01:15:21.683747+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683747+0000","last_undegraded":"2024-09-11T01:15:21.683747+0000","last_fullsized":"2024-09-11T01:15:21.683747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12301","reported_seq":12427,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:49.361358+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:17:49.361358+0000","last_peered":"2024-09-11T01:17:49.361358+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:17:49.361358+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31424512,"num_objects":7672,"num_object_clones":0,"num_object_copies":15344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10272,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1621},{"shards":"2","objects":3739}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12180","reported_seq":12309,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:46.282862+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:17:46.282862+0000","last_peered":"2024-09-11T01:17:46.282862+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:17:46.282862+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31490048,"num_objects":7688,"num_object_clones":0,"num_object_copies":15376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8767,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3704},{"shards":"0,2","objects":1742}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12480","reported_seq":12599,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:49.348628+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:17:49.348628+0000","last_peered":"2024-09-11T01:17:49.348628+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:17:49.348628+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31649792,"num_objects":7727,"num_object_clones":0,"num_object_copies":15454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10297,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1649},{"shards":"2","objects":3703}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684827+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684827+0000","last_peered":"2024-09-11T01:15:21.684827+0000","last_clean":"2024-09-11T01:15:21.684827+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684827+0000","last_undegraded":"2024-09-11T01:15:21.684827+0000","last_fullsized":"2024-09-11T01:15:21.684827+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12225","reported_seq":20206,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:49.358809+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:17:49.358809+0000","last_peered":"2024-09-11T01:17:49.358809+0000","last_clean":"2024-09-11T01:17:49.358809+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:17:49.358809+0000","last_undegraded":"2024-09-11T01:17:49.358809+0000","last_fullsized":"2024-09-11T01:17:49.358809+0000","mapping_epoch":44,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31010816,"num_objects":7571,"num_object_clones":0,"num_object_copies":15142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10010,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12415","reported_seq":12547,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:46.242018+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:17:46.242018+0000","last_peered":"2024-09-11T01:17:46.242018+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:17:46.242018+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32149504,"num_objects":7849,"num_object_clones":0,"num_object_copies":15698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7849,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9152,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3899},{"shards":"0,2","objects":1668}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684623+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684623+0000","last_peered":"2024-09-11T01:15:21.684623+0000","last_clean":"2024-09-11T01:15:21.684623+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684623+0000","last_undegraded":"2024-09-11T01:15:21.684623+0000","last_fullsized":"2024-09-11T01:15:21.684623+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12168","reported_seq":19224,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:46.243077+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:17:46.243077+0000","last_peered":"2024-09-11T01:17:46.243077+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:17:46.243077+0000","last_undegraded":"2024-09-11T01:17:46.243077+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31604736,"num_objects":7716,"num_object_clones":0,"num_object_copies":15432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":483,"num_objects_unfound":0,"num_objects_dirty":7716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8823,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3459,"num_bytes_recovered":14155776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7233},{"shards":"0,2","objects":483}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12352","reported_seq":12487,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:46.223281+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:17:46.223281+0000","last_peered":"2024-09-11T01:17:46.223281+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:17:46.223281+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":32047104,"num_objects":7824,"num_object_clones":0,"num_object_copies":15648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8961,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3785},{"shards":"0,2","objects":1777}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692049+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.692049+0000","last_peered":"2024-09-11T01:15:21.692049+0000","last_clean":"2024-09-11T01:15:21.692049+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.692049+0000","last_undegraded":"2024-09-11T01:15:21.692049+0000","last_fullsized":"2024-09-11T01:15:21.692049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684417+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684417+0000","last_peered":"2024-09-11T01:15:21.684417+0000","last_clean":"2024-09-11T01:15:21.684417+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684417+0000","last_undegraded":"2024-09-11T01:15:21.684417+0000","last_fullsized":"2024-09-11T01:15:21.684417+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12163","reported_seq":12296,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:46.291281+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:17:46.291281+0000","last_peered":"2024-09-11T01:17:46.291281+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:17:46.291281+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'2100","ondisk_log_start":"36'2100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31904361,"num_objects":7790,"num_object_clones":0,"num_object_copies":15580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7790,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9233,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3873},{"shards":"0,2","objects":1736}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":508514945,"num_objects":124151,"num_object_clones":0,"num_object_copies":248302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":483,"num_objects_unfound":0,"num_objects_dirty":124151,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":158136,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7387,"num_bytes_recovered":30236672,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1140731904,"data_stored":1140654180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160686,"ondisk_log_size":160686,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182475,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":802720,"kb_used_data":437904,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93569120,"statfs":{"total":96636764160,"available":95814778880,"internally_reserved":0,"allocated":448413696,"data_stored":448154482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":373550763},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,4,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986248,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":681732,"kb_used_data":375220,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93690108,"statfs":{"total":96636764160,"available":95938670592,"internally_reserved":0,"allocated":384225280,"data_stored":383966071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659193,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":587032,"kb_used_data":302728,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93784808,"statfs":{"total":96636764160,"available":96035643392,"internally_reserved":0,"allocated":309993472,"data_stored":309671043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":291106479},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,2,4,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":447975424,"data_stored":447971945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":309604352,"data_stored":309537658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":383152128,"data_stored":383144577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:52.739 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:17:52.739 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'12409", 'reported_seq': 12487, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:46.257811+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:17:46.257811+0000', 'last_peered': '2024-09-11T01:17:46.257811+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:17:46.257811+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32182272, 'num_objects': 7857, 'num_object_clones': 0, 'num_object_copies': 15714, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7857, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12409, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3785}, {'shards': '0,2', 'objects': 1796}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:52.739 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:17:52.739 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'12381", 'reported_seq': 12510, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:49.378296+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:17:49.378296+0000', 'last_peered': '2024-09-11T01:17:49.378296+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:17:49.378296+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31793176, 'num_objects': 7763, 'num_object_clones': 0, 'num_object_copies': 15526, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7763, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10682, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1589}, {'shards': '2', 'objects': 3867}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:52.740 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:17:52.740 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'12404", 'reported_seq': 12574, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:46.266666+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:17:46.266666+0000', 'last_peered': '2024-09-11T01:17:46.266666+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:17:46.266666+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32043008, 'num_objects': 7823, 'num_object_clones': 0, 'num_object_copies': 15646, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7823, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8745, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3872}, {'shards': '0,2', 'objects': 1663}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:52.740 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:17:52.740 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'12198", 'reported_seq': 12329, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:46.219245+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:17:46.219245+0000', 'last_peered': '2024-09-11T01:17:46.219245+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:17:46.219245+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "36'2100", 'ondisk_log_start': "36'2100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31670272, 'num_objects': 7732, 'num_object_clones': 0, 'num_object_copies': 15464, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7732, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9055, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3746}, {'shards': '0,2', 'objects': 1756}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:52.740 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:17:52.740 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'12301", 'reported_seq': 12427, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:49.361358+0000', 'last_change': '2024-09-11T01:13:10.159069+0000', 'last_active': '2024-09-11T01:17:49.361358+0000', 'last_peered': '2024-09-11T01:17:49.361358+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:17:49.361358+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31424512, 'num_objects': 7672, 'num_object_clones': 0, 'num_object_copies': 15344, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7672, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10272, 'num_write_kb': 31844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1621}, {'shards': '2', 'objects': 3739}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:52.740 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:17:52.740 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'12180", 'reported_seq': 12309, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:46.282862+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:17:46.282862+0000', 'last_peered': '2024-09-11T01:17:46.282862+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:17:46.282862+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "36'2100", 'ondisk_log_start': "36'2100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31490048, 'num_objects': 7688, 'num_object_clones': 0, 'num_object_copies': 15376, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7688, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8767, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3704}, {'shards': '0,2', 'objects': 1742}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:52.740 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:17:52.740 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'12480", 'reported_seq': 12599, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:49.348628+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:17:49.348628+0000', 'last_peered': '2024-09-11T01:17:49.348628+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:17:49.348628+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31649792, 'num_objects': 7727, 'num_object_clones': 0, 'num_object_copies': 15454, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7727, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10297, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1649}, {'shards': '2', 'objects': 3703}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:52.740 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:17:52.741 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'12415", 'reported_seq': 12547, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:46.242018+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:17:46.242018+0000', 'last_peered': '2024-09-11T01:17:46.242018+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:17:46.242018+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32149504, 'num_objects': 7849, 'num_object_clones': 0, 'num_object_copies': 15698, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7849, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9152, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3899}, {'shards': '0,2', 'objects': 1668}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:52.741 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:17:52.741 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'12168", 'reported_seq': 19224, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:17:46.243077+0000', 'last_change': '2024-09-11T01:15:19.913288+0000', 'last_active': '2024-09-11T01:17:46.243077+0000', 'last_peered': '2024-09-11T01:17:46.243077+0000', 'last_clean': '2024-09-11T01:13:07.624565+0000', 'last_became_active': '2024-09-11T01:13:10.199795+0000', 'last_became_peered': '2024-09-11T01:13:10.199795+0000', 'last_unstale': '2024-09-11T01:17:46.243077+0000', 'last_undegraded': '2024-09-11T01:17:46.243077+0000', 'last_fullsized': '2024-09-11T01:13:09.652311+0000', 'mapping_epoch': 42, 'log_start': "36'2100", 'ondisk_log_start': "36'2100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3415", 'last_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'objects_scrubbed': 3337, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1975, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31604736, 'num_objects': 7716, 'num_object_clones': 0, 'num_object_copies': 15432, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 483, 'num_objects_unfound': 0, 'num_objects_dirty': 7716, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8823, 'num_write_kb': 26404, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3459, 'num_bytes_recovered': 14155776, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7233}, {'shards': '0,2', 'objects': 483}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:52.741 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:17:52.741 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'12352", 'reported_seq': 12487, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:46.223281+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:17:46.223281+0000', 'last_peered': '2024-09-11T01:17:46.223281+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:17:46.223281+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "36'2300", 'ondisk_log_start': "36'2300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 32047104, 'num_objects': 7824, 'num_object_clones': 0, 'num_object_copies': 15648, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7824, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 8961, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3785}, {'shards': '0,2', 'objects': 1777}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:52.741 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:17:52.741 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'12163", 'reported_seq': 12296, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:46.291281+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:17:46.291281+0000', 'last_peered': '2024-09-11T01:17:46.291281+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:17:46.291281+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "36'2100", 'ondisk_log_start': "36'2100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31904361, 'num_objects': 7790, 'num_object_clones': 0, 'num_object_copies': 15580, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7790, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9233, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3873}, {'shards': '0,2', 'objects': 1736}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:52.741 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:17:53.730 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:17:53.730 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39769/245574 objects degraded (16.194%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":410,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":45,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":11},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":122787,"data_bytes":502919828,"bytes_used":2101645312,"bytes_avail":287808647168,"bytes_total":289910292480,"degraded_objects":39769,"degraded_total":245574,"degraded_ratio":0.16194303957259318,"misplaced_objects":278,"misplaced_total":245574,"misplaced_ratio":0.0011320416656486435,"recovering_objects_per_sec":49,"recovering_bytes_per_sec":201308,"recovering_keys_per_sec":0,"num_objects_recovered":590,"num_bytes_recovered":2416640,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":488},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:17:53.805 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:17:53.805 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:54.580 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:17:54.583 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:17:54.661 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2024-09-11T01:17:52.676071+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":502919828,"num_objects":122787,"num_object_clones":0,"num_object_copies":245574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":278,"num_objects_unfound":0,"num_objects_dirty":122787,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":159504,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7593,"num_bytes_recovered":31076371,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160755,"ondisk_log_size":160755,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2052388,"kb_used_data":1109228,"kb_used_omap":12,"kb_used_meta":943091,"kb_avail":281063132,"statfs":{"total":289910292480,"available":287808647168,"internally_reserved":0,"allocated":1135849472,"data_stored":1135008620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":965725188},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,1,1,6,7,6,10],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24035328,"num_objects":-5868,"num_object_clones":0,"num_object_copies":-11736,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-590,"num_objects_unfound":0,"num_objects_dirty":-5868,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5868,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":590,"num_bytes_recovered":2416640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004643"},"pg_stats":[{"pgid":"2.f","version":"45'12549","reported_seq":12627,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:51.301088+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:17:51.301088+0000","last_peered":"2024-09-11T01:17:51.301088+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:17:51.301088+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31608832,"num_objects":7717,"num_object_clones":0,"num_object_copies":15434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12549,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3645},{"shards":"0,2","objects":1656}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12613","reported_seq":12715,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:49.345616+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:17:49.345616+0000","last_peered":"2024-09-11T01:17:49.345616+0000","last_clean":"2024-09-11T01:17:49.345616+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:17:49.345616+0000","last_undegraded":"2024-09-11T01:17:49.345616+0000","last_fullsized":"2024-09-11T01:17:49.345616+0000","mapping_epoch":35,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31698944,"num_objects":7739,"num_object_clones":0,"num_object_copies":15478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10852,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12381","reported_seq":12510,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:49.378296+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:17:49.378296+0000","last_peered":"2024-09-11T01:17:49.378296+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:17:49.378296+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31793176,"num_objects":7763,"num_object_clones":0,"num_object_copies":15526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7763,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10682,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1589},{"shards":"2","objects":3867}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12364","reported_seq":12434,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:49.335050+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:17:49.335050+0000","last_peered":"2024-09-11T01:17:49.335050+0000","last_clean":"2024-09-11T01:17:49.335050+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:17:49.335050+0000","last_undegraded":"2024-09-11T01:17:49.335050+0000","last_fullsized":"2024-09-11T01:17:49.335050+0000","mapping_epoch":35,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31825920,"num_objects":7770,"num_object_clones":0,"num_object_copies":15540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12364,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12373","reported_seq":12501,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:51.292796+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:17:51.292796+0000","last_peered":"2024-09-11T01:17:51.292796+0000","last_clean":"2024-09-11T01:17:51.292796+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:17:51.292796+0000","last_undegraded":"2024-09-11T01:17:51.292796+0000","last_fullsized":"2024-09-11T01:17:51.292796+0000","mapping_epoch":34,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31277056,"num_objects":7636,"num_object_clones":0,"num_object_copies":15272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9378,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12540","reported_seq":12711,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:51.320626+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:17:51.320626+0000","last_peered":"2024-09-11T01:17:51.320626+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:17:51.320626+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31485952,"num_objects":7687,"num_object_clones":0,"num_object_copies":15374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7687,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8881,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3736},{"shards":"0,2","objects":1527}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12327","reported_seq":12458,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:51.328125+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:17:51.328125+0000","last_peered":"2024-09-11T01:17:51.328125+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:17:51.328125+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31141888,"num_objects":7603,"num_object_clones":0,"num_object_copies":15206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7603,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9184,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3617},{"shards":"0,2","objects":1627}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12537","reported_seq":12667,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:51.290902+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:17:51.290902+0000","last_peered":"2024-09-11T01:17:51.290902+0000","last_clean":"2024-09-11T01:17:51.290902+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:17:51.290902+0000","last_undegraded":"2024-09-11T01:17:51.290902+0000","last_fullsized":"2024-09-11T01:17:51.290902+0000","mapping_epoch":34,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31608832,"num_objects":7717,"num_object_clones":0,"num_object_copies":15434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7717,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9413,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683747+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683747+0000","last_peered":"2024-09-11T01:15:21.683747+0000","last_clean":"2024-09-11T01:15:21.683747+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683747+0000","last_undegraded":"2024-09-11T01:15:21.683747+0000","last_fullsized":"2024-09-11T01:15:21.683747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12301","reported_seq":12427,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:49.361358+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:17:49.361358+0000","last_peered":"2024-09-11T01:17:49.361358+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:17:49.361358+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31424512,"num_objects":7672,"num_object_clones":0,"num_object_copies":15344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10272,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1621},{"shards":"2","objects":3739}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12307","reported_seq":12436,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:51.273132+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:17:51.273132+0000","last_peered":"2024-09-11T01:17:51.273132+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:17:51.273132+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30969856,"num_objects":7561,"num_object_clones":0,"num_object_copies":15122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7561,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8894,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3577},{"shards":"0,2","objects":1615}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12480","reported_seq":12599,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:49.348628+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:17:49.348628+0000","last_peered":"2024-09-11T01:17:49.348628+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:17:49.348628+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31649792,"num_objects":7727,"num_object_clones":0,"num_object_copies":15454,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7727,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10297,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1649},{"shards":"2","objects":3703}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684827+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684827+0000","last_peered":"2024-09-11T01:15:21.684827+0000","last_clean":"2024-09-11T01:15:21.684827+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684827+0000","last_undegraded":"2024-09-11T01:15:21.684827+0000","last_fullsized":"2024-09-11T01:15:21.684827+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12225","reported_seq":20206,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:49.358809+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:17:49.358809+0000","last_peered":"2024-09-11T01:17:49.358809+0000","last_clean":"2024-09-11T01:17:49.358809+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:17:49.358809+0000","last_undegraded":"2024-09-11T01:17:49.358809+0000","last_fullsized":"2024-09-11T01:17:49.358809+0000","mapping_epoch":44,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31010816,"num_objects":7571,"num_object_clones":0,"num_object_copies":15142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7571,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10010,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12553","reported_seq":12685,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:51.227986+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:17:51.227986+0000","last_peered":"2024-09-11T01:17:51.227986+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:17:51.227986+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31584256,"num_objects":7711,"num_object_clones":0,"num_object_copies":15422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9290,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3761},{"shards":"0,2","objects":1530}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684623+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684623+0000","last_peered":"2024-09-11T01:15:21.684623+0000","last_clean":"2024-09-11T01:15:21.684623+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684623+0000","last_undegraded":"2024-09-11T01:15:21.684623+0000","last_fullsized":"2024-09-11T01:15:21.684623+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12296","reported_seq":19762,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:51.321830+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:17:51.321830+0000","last_peered":"2024-09-11T01:17:51.321830+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:17:51.321830+0000","last_undegraded":"2024-09-11T01:17:51.321830+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"36'2200","ondisk_log_start":"36'2200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31080448,"num_objects":7588,"num_object_clones":0,"num_object_copies":15176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":278,"num_objects_unfound":0,"num_objects_dirty":7588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8951,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3664,"num_bytes_recovered":14995456,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7310},{"shards":"0,2","objects":278}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12495","reported_seq":12630,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:51.240598+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:17:51.240598+0000","last_peered":"2024-09-11T01:17:51.240598+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:17:51.240598+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31461376,"num_objects":7681,"num_object_clones":0,"num_object_copies":15362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9104,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3642},{"shards":"0,2","objects":1634}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692049+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.692049+0000","last_peered":"2024-09-11T01:15:21.692049+0000","last_clean":"2024-09-11T01:15:21.692049+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.692049+0000","last_undegraded":"2024-09-11T01:15:21.692049+0000","last_fullsized":"2024-09-11T01:15:21.692049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684417+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684417+0000","last_peered":"2024-09-11T01:15:21.684417+0000","last_clean":"2024-09-11T01:15:21.684417+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684417+0000","last_undegraded":"2024-09-11T01:15:21.684417+0000","last_fullsized":"2024-09-11T01:15:21.684417+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12311","reported_seq":12445,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:51.274447+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:17:51.274447+0000","last_peered":"2024-09-11T01:17:51.274447+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:17:51.274447+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31298153,"num_objects":7642,"num_object_clones":0,"num_object_copies":15284,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7642,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9381,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3725},{"shards":"0,2","objects":1588}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":502919809,"num_objects":122785,"num_object_clones":0,"num_object_copies":245570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":278,"num_objects_unfound":0,"num_objects_dirty":122785,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":159502,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7592,"num_bytes_recovered":31076352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1133903872,"data_stored":1133826148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160752,"ondisk_log_size":160752,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182476,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":783624,"kb_used_data":431280,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93588216,"statfs":{"total":96636764160,"available":95834333184,"internally_reserved":0,"allocated":441630720,"data_stored":441371506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":360771243},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,4,1,2,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986248,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":681732,"kb_used_data":375220,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93690108,"statfs":{"total":96636764160,"available":95938670592,"internally_reserved":0,"allocated":384225280,"data_stored":383966071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,3],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659193,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":587032,"kb_used_data":302728,"kb_used_omap":4,"kb_used_meta":284283,"kb_avail":93784808,"statfs":{"total":96636764160,"available":96035643392,"internally_reserved":0,"allocated":309993472,"data_stored":309671043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":291106479},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,2,4,1,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":441147392,"data_stored":441143913,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":309604352,"data_stored":309537658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":383152128,"data_stored":383144577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:57.663 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:58.335 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:17:58.335 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:17:58.365 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":246,"stamp":"2024-09-11T01:17:57.912199+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":493703828,"num_objects":120537,"num_object_clones":0,"num_object_copies":241074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":51,"num_objects_unfound":0,"num_objects_dirty":120537,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":161754,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7820,"num_bytes_recovered":32006163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160905,"ondisk_log_size":160905,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2079772,"kb_used_data":1097764,"kb_used_omap":12,"kb_used_meta":981939,"kb_avail":281035748,"statfs":{"total":289910292480,"available":287780605952,"internally_reserved":0,"allocated":1124110336,"data_stored":1123269484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1005505540},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,6,5,9,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18362368,"num_objects":-4483,"num_object_clones":0,"num_object_copies":-8966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-432,"num_objects_unfound":0,"num_objects_dirty":-4483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4483,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":432,"num_bytes_recovered":1769472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.239353"},"pg_stats":[{"pgid":"2.f","version":"45'12686","reported_seq":12764,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:56.308208+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:17:56.308208+0000","last_peered":"2024-09-11T01:17:56.308208+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:17:56.308208+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31047680,"num_objects":7580,"num_object_clones":0,"num_object_copies":15160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12686,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3508},{"shards":"0,2","objects":1519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12748","reported_seq":12850,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:54.234465+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:17:54.234465+0000","last_peered":"2024-09-11T01:17:54.234465+0000","last_clean":"2024-09-11T01:17:54.234465+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:17:54.234465+0000","last_undegraded":"2024-09-11T01:17:54.234465+0000","last_fullsized":"2024-09-11T01:17:54.234465+0000","mapping_epoch":35,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31145984,"num_objects":7604,"num_object_clones":0,"num_object_copies":15208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10987,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12533","reported_seq":12662,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:54.283457+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:17:54.283457+0000","last_peered":"2024-09-11T01:17:54.283457+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:17:54.283457+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31170584,"num_objects":7611,"num_object_clones":0,"num_object_copies":15222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7611,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10834,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1437},{"shards":"2","objects":3715}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12500","reported_seq":12570,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:54.361116+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:17:54.361116+0000","last_peered":"2024-09-11T01:17:54.361116+0000","last_clean":"2024-09-11T01:17:54.361116+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:17:54.361116+0000","last_undegraded":"2024-09-11T01:17:54.361116+0000","last_fullsized":"2024-09-11T01:17:54.361116+0000","mapping_epoch":35,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31268864,"num_objects":7634,"num_object_clones":0,"num_object_copies":15268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12500,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12516","reported_seq":12644,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:56.297716+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:17:56.297716+0000","last_peered":"2024-09-11T01:17:56.297716+0000","last_clean":"2024-09-11T01:17:56.297716+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:17:56.297716+0000","last_undegraded":"2024-09-11T01:17:56.297716+0000","last_fullsized":"2024-09-11T01:17:56.297716+0000","mapping_epoch":34,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30691328,"num_objects":7493,"num_object_clones":0,"num_object_copies":14986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9521,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12689","reported_seq":12859,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:56.356631+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:17:56.356631+0000","last_peered":"2024-09-11T01:17:56.356631+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:17:56.356631+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30875648,"num_objects":7538,"num_object_clones":0,"num_object_copies":15076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9030,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3587},{"shards":"0,2","objects":1378}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12450","reported_seq":12581,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:56.288938+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:17:56.288938+0000","last_peered":"2024-09-11T01:17:56.288938+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:17:56.288938+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30638080,"num_objects":7480,"num_object_clones":0,"num_object_copies":14960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7480,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9307,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3494},{"shards":"0,2","objects":1504}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12675","reported_seq":12805,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:56.314154+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:17:56.314154+0000","last_peered":"2024-09-11T01:17:56.314154+0000","last_clean":"2024-09-11T01:17:56.314154+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:17:56.314154+0000","last_undegraded":"2024-09-11T01:17:56.314154+0000","last_fullsized":"2024-09-11T01:17:56.314154+0000","mapping_epoch":34,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31043584,"num_objects":7579,"num_object_clones":0,"num_object_copies":15158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9551,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683747+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683747+0000","last_peered":"2024-09-11T01:15:21.683747+0000","last_clean":"2024-09-11T01:15:21.683747+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683747+0000","last_undegraded":"2024-09-11T01:15:21.683747+0000","last_fullsized":"2024-09-11T01:15:21.683747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12452","reported_seq":12580,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:54.307347+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:17:54.307347+0000","last_peered":"2024-09-11T01:17:54.307347+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:17:54.307347+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30806016,"num_objects":7521,"num_object_clones":0,"num_object_copies":15042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10423,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1470},{"shards":"2","objects":3588}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12445","reported_seq":12572,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:56.361368+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:17:56.361368+0000","last_peered":"2024-09-11T01:17:56.361368+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:17:56.361368+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30404608,"num_objects":7423,"num_object_clones":0,"num_object_copies":14846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9032,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3439},{"shards":"0,2","objects":1477}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12613","reported_seq":12732,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:54.308030+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:17:54.308030+0000","last_peered":"2024-09-11T01:17:54.308030+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:17:54.308030+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31105024,"num_objects":7594,"num_object_clones":0,"num_object_copies":15188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7594,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10430,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1516},{"shards":"2","objects":3570}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684827+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684827+0000","last_peered":"2024-09-11T01:15:21.684827+0000","last_clean":"2024-09-11T01:15:21.684827+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684827+0000","last_undegraded":"2024-09-11T01:15:21.684827+0000","last_fullsized":"2024-09-11T01:15:21.684827+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12357","reported_seq":20338,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:54.385777+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:17:54.385777+0000","last_peered":"2024-09-11T01:17:54.385777+0000","last_clean":"2024-09-11T01:17:54.385777+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:17:54.385777+0000","last_undegraded":"2024-09-11T01:17:54.385777+0000","last_fullsized":"2024-09-11T01:17:54.385777+0000","mapping_epoch":44,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30470144,"num_objects":7439,"num_object_clones":0,"num_object_copies":14878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7439,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10142,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12687","reported_seq":12819,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:56.218357+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:17:56.218357+0000","last_peered":"2024-09-11T01:17:56.218357+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:17:56.218357+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31035392,"num_objects":7577,"num_object_clones":0,"num_object_copies":15154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9424,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3627},{"shards":"0,2","objects":1396}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684623+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684623+0000","last_peered":"2024-09-11T01:15:21.684623+0000","last_clean":"2024-09-11T01:15:21.684623+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684623+0000","last_undegraded":"2024-09-11T01:15:21.684623+0000","last_fullsized":"2024-09-11T01:15:21.684623+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12452","reported_seq":20372,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:56.362937+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:17:56.362937+0000","last_peered":"2024-09-11T01:17:56.362937+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:17:56.362937+0000","last_undegraded":"2024-09-11T01:17:56.362937+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30441472,"num_objects":7432,"num_object_clones":0,"num_object_copies":14864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":51,"num_objects_unfound":0,"num_objects_dirty":7432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9107,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3891,"num_bytes_recovered":15925248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7381},{"shards":"0,2","objects":51}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12657","reported_seq":12792,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:56.330190+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:17:56.330190+0000","last_peered":"2024-09-11T01:17:56.330190+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:17:56.330190+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30797824,"num_objects":7519,"num_object_clones":0,"num_object_copies":15038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9266,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3480},{"shards":"0,2","objects":1472}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692049+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.692049+0000","last_peered":"2024-09-11T01:15:21.692049+0000","last_clean":"2024-09-11T01:15:21.692049+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.692049+0000","last_undegraded":"2024-09-11T01:15:21.692049+0000","last_fullsized":"2024-09-11T01:15:21.692049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684417+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684417+0000","last_peered":"2024-09-11T01:15:21.684417+0000","last_clean":"2024-09-11T01:15:21.684417+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684417+0000","last_undegraded":"2024-09-11T01:15:21.684417+0000","last_fullsized":"2024-09-11T01:15:21.684417+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12442","reported_seq":12576,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:56.343721+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:17:56.343721+0000","last_peered":"2024-09-11T01:17:56.343721+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:17:56.343721+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30761577,"num_objects":7511,"num_object_clones":0,"num_object_copies":15022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9512,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3594},{"shards":"0,2","objects":1457}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":493703809,"num_objects":120535,"num_object_clones":0,"num_object_copies":241070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":51,"num_objects_unfound":0,"num_objects_dirty":120535,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":161752,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7819,"num_bytes_recovered":32006144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1122193408,"data_stored":1122115684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160902,"ondisk_log_size":160902,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182477,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":785348,"kb_used_data":424684,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93586492,"statfs":{"total":96636764160,"available":95832567808,"internally_reserved":0,"allocated":434876416,"data_stored":434617202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":369290923},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,5,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986249,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":681352,"kb_used_data":370680,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93690488,"statfs":{"total":96636764160,"available":95939059712,"internally_reserved":0,"allocated":379576320,"data_stored":379317111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,1,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659194,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":613072,"kb_used_data":302400,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93758768,"statfs":{"total":96636764160,"available":96008978432,"internally_reserved":0,"allocated":309657600,"data_stored":309335171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":318107311},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":434331648,"data_stored":434328169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":309297152,"data_stored":309230458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":378564608,"data_stored":378557057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:58.367 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:17:58.692 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:17:58.692 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:17:58.708 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":247,"stamp":"2024-09-11T01:17:58.681412+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":493703828,"num_objects":120537,"num_object_clones":0,"num_object_copies":241074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":51,"num_objects_unfound":0,"num_objects_dirty":120537,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":161754,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7820,"num_bytes_recovered":32006163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160905,"ondisk_log_size":160905,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2079772,"kb_used_data":1097764,"kb_used_omap":12,"kb_used_meta":981939,"kb_avail":281035748,"statfs":{"total":289910292480,"available":287780605952,"internally_reserved":0,"allocated":1124110336,"data_stored":1123269484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1005505540},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,6,5,9,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18362368,"num_objects":-4483,"num_object_clones":0,"num_object_copies":-8966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-432,"num_objects_unfound":0,"num_objects_dirty":-4483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4483,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":432,"num_bytes_recovered":1769472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.008088"},"pg_stats":[{"pgid":"2.f","version":"45'12686","reported_seq":12764,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:56.308208+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:17:56.308208+0000","last_peered":"2024-09-11T01:17:56.308208+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:17:56.308208+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31047680,"num_objects":7580,"num_object_clones":0,"num_object_copies":15160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12686,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3508},{"shards":"0,2","objects":1519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12748","reported_seq":12850,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:54.234465+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:17:54.234465+0000","last_peered":"2024-09-11T01:17:54.234465+0000","last_clean":"2024-09-11T01:17:54.234465+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:17:54.234465+0000","last_undegraded":"2024-09-11T01:17:54.234465+0000","last_fullsized":"2024-09-11T01:17:54.234465+0000","mapping_epoch":35,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31145984,"num_objects":7604,"num_object_clones":0,"num_object_copies":15208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10987,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12533","reported_seq":12662,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:54.283457+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:17:54.283457+0000","last_peered":"2024-09-11T01:17:54.283457+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:17:54.283457+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31170584,"num_objects":7611,"num_object_clones":0,"num_object_copies":15222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7611,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10834,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1437},{"shards":"2","objects":3715}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12500","reported_seq":12570,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:54.361116+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:17:54.361116+0000","last_peered":"2024-09-11T01:17:54.361116+0000","last_clean":"2024-09-11T01:17:54.361116+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:17:54.361116+0000","last_undegraded":"2024-09-11T01:17:54.361116+0000","last_fullsized":"2024-09-11T01:17:54.361116+0000","mapping_epoch":35,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31268864,"num_objects":7634,"num_object_clones":0,"num_object_copies":15268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12500,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12516","reported_seq":12644,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:56.297716+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:17:56.297716+0000","last_peered":"2024-09-11T01:17:56.297716+0000","last_clean":"2024-09-11T01:17:56.297716+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:17:56.297716+0000","last_undegraded":"2024-09-11T01:17:56.297716+0000","last_fullsized":"2024-09-11T01:17:56.297716+0000","mapping_epoch":34,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30691328,"num_objects":7493,"num_object_clones":0,"num_object_copies":14986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9521,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12689","reported_seq":12859,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:56.356631+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:17:56.356631+0000","last_peered":"2024-09-11T01:17:56.356631+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:17:56.356631+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30875648,"num_objects":7538,"num_object_clones":0,"num_object_copies":15076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9030,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3587},{"shards":"0,2","objects":1378}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12450","reported_seq":12581,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:56.288938+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:17:56.288938+0000","last_peered":"2024-09-11T01:17:56.288938+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:17:56.288938+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30638080,"num_objects":7480,"num_object_clones":0,"num_object_copies":14960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7480,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9307,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3494},{"shards":"0,2","objects":1504}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12675","reported_seq":12805,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:56.314154+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:17:56.314154+0000","last_peered":"2024-09-11T01:17:56.314154+0000","last_clean":"2024-09-11T01:17:56.314154+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:17:56.314154+0000","last_undegraded":"2024-09-11T01:17:56.314154+0000","last_fullsized":"2024-09-11T01:17:56.314154+0000","mapping_epoch":34,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31043584,"num_objects":7579,"num_object_clones":0,"num_object_copies":15158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9551,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683747+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683747+0000","last_peered":"2024-09-11T01:15:21.683747+0000","last_clean":"2024-09-11T01:15:21.683747+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683747+0000","last_undegraded":"2024-09-11T01:15:21.683747+0000","last_fullsized":"2024-09-11T01:15:21.683747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12452","reported_seq":12580,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:54.307347+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:17:54.307347+0000","last_peered":"2024-09-11T01:17:54.307347+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:17:54.307347+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30806016,"num_objects":7521,"num_object_clones":0,"num_object_copies":15042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10423,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1470},{"shards":"2","objects":3588}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12445","reported_seq":12572,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:56.361368+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:17:56.361368+0000","last_peered":"2024-09-11T01:17:56.361368+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:17:56.361368+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30404608,"num_objects":7423,"num_object_clones":0,"num_object_copies":14846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9032,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3439},{"shards":"0,2","objects":1477}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12613","reported_seq":12732,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:54.308030+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:17:54.308030+0000","last_peered":"2024-09-11T01:17:54.308030+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:17:54.308030+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31105024,"num_objects":7594,"num_object_clones":0,"num_object_copies":15188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7594,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10430,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1516},{"shards":"2","objects":3570}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684827+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684827+0000","last_peered":"2024-09-11T01:15:21.684827+0000","last_clean":"2024-09-11T01:15:21.684827+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684827+0000","last_undegraded":"2024-09-11T01:15:21.684827+0000","last_fullsized":"2024-09-11T01:15:21.684827+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12357","reported_seq":20338,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:54.385777+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:17:54.385777+0000","last_peered":"2024-09-11T01:17:54.385777+0000","last_clean":"2024-09-11T01:17:54.385777+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:17:54.385777+0000","last_undegraded":"2024-09-11T01:17:54.385777+0000","last_fullsized":"2024-09-11T01:17:54.385777+0000","mapping_epoch":44,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30470144,"num_objects":7439,"num_object_clones":0,"num_object_copies":14878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7439,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10142,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12687","reported_seq":12819,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:56.218357+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:17:56.218357+0000","last_peered":"2024-09-11T01:17:56.218357+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:17:56.218357+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31035392,"num_objects":7577,"num_object_clones":0,"num_object_copies":15154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9424,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3627},{"shards":"0,2","objects":1396}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684623+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684623+0000","last_peered":"2024-09-11T01:15:21.684623+0000","last_clean":"2024-09-11T01:15:21.684623+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684623+0000","last_undegraded":"2024-09-11T01:15:21.684623+0000","last_fullsized":"2024-09-11T01:15:21.684623+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12452","reported_seq":20372,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:56.362937+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:17:56.362937+0000","last_peered":"2024-09-11T01:17:56.362937+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:17:56.362937+0000","last_undegraded":"2024-09-11T01:17:56.362937+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30441472,"num_objects":7432,"num_object_clones":0,"num_object_copies":14864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":51,"num_objects_unfound":0,"num_objects_dirty":7432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9107,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3891,"num_bytes_recovered":15925248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7381},{"shards":"0,2","objects":51}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12657","reported_seq":12792,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:56.330190+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:17:56.330190+0000","last_peered":"2024-09-11T01:17:56.330190+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:17:56.330190+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30797824,"num_objects":7519,"num_object_clones":0,"num_object_copies":15038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9266,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3480},{"shards":"0,2","objects":1472}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692049+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.692049+0000","last_peered":"2024-09-11T01:15:21.692049+0000","last_clean":"2024-09-11T01:15:21.692049+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.692049+0000","last_undegraded":"2024-09-11T01:15:21.692049+0000","last_fullsized":"2024-09-11T01:15:21.692049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684417+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684417+0000","last_peered":"2024-09-11T01:15:21.684417+0000","last_clean":"2024-09-11T01:15:21.684417+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684417+0000","last_undegraded":"2024-09-11T01:15:21.684417+0000","last_fullsized":"2024-09-11T01:15:21.684417+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12442","reported_seq":12576,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:56.343721+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:17:56.343721+0000","last_peered":"2024-09-11T01:17:56.343721+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:17:56.343721+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30761577,"num_objects":7511,"num_object_clones":0,"num_object_copies":15022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9512,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3594},{"shards":"0,2","objects":1457}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":493703809,"num_objects":120535,"num_object_clones":0,"num_object_copies":241070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":51,"num_objects_unfound":0,"num_objects_dirty":120535,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":161752,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7819,"num_bytes_recovered":32006144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1122193408,"data_stored":1122115684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160902,"ondisk_log_size":160902,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182477,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":785348,"kb_used_data":424684,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93586492,"statfs":{"total":96636764160,"available":95832567808,"internally_reserved":0,"allocated":434876416,"data_stored":434617202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":369290923},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,5,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986249,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":681352,"kb_used_data":370680,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93690488,"statfs":{"total":96636764160,"available":95939059712,"internally_reserved":0,"allocated":379576320,"data_stored":379317111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,1,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659194,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":613072,"kb_used_data":302400,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93758768,"statfs":{"total":96636764160,"available":96008978432,"internally_reserved":0,"allocated":309657600,"data_stored":309335171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":318107311},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":434331648,"data_stored":434328169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":309297152,"data_stored":309230458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":378564608,"data_stored":378557057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:17:58.710 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:17:58.710 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "45'12686", 'reported_seq': 12764, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:56.308208+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:17:56.308208+0000', 'last_peered': '2024-09-11T01:17:56.308208+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:17:56.308208+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31047680, 'num_objects': 7580, 'num_object_clones': 0, 'num_object_copies': 15160, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7580, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12686, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3508}, {'shards': '0,2', 'objects': 1519}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:58.710 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:17:58.710 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "45'12533", 'reported_seq': 12662, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:54.283457+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:17:54.283457+0000', 'last_peered': '2024-09-11T01:17:54.283457+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:17:54.283457+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "36'2500", 'ondisk_log_start': "36'2500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31170584, 'num_objects': 7611, 'num_object_clones': 0, 'num_object_copies': 15222, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7611, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 10834, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1437}, {'shards': '2', 'objects': 3715}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:58.710 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:17:58.710 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "45'12689", 'reported_seq': 12859, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:56.356631+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:17:56.356631+0000', 'last_peered': '2024-09-11T01:17:56.356631+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:17:56.356631+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30875648, 'num_objects': 7538, 'num_object_clones': 0, 'num_object_copies': 15076, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7538, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9030, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3587}, {'shards': '0,2', 'objects': 1378}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:58.711 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:17:58.711 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "45'12450", 'reported_seq': 12581, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:56.288938+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:17:56.288938+0000', 'last_peered': '2024-09-11T01:17:56.288938+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:17:56.288938+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30638080, 'num_objects': 7480, 'num_object_clones': 0, 'num_object_copies': 14960, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7480, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9307, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3494}, {'shards': '0,2', 'objects': 1504}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:58.711 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:17:58.711 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "45'12452", 'reported_seq': 12580, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:54.307347+0000', 'last_change': '2024-09-11T01:13:10.159069+0000', 'last_active': '2024-09-11T01:17:54.307347+0000', 'last_peered': '2024-09-11T01:17:54.307347+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:17:54.307347+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30806016, 'num_objects': 7521, 'num_object_clones': 0, 'num_object_copies': 15042, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3935, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7521, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10423, 'num_write_kb': 31844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1470}, {'shards': '2', 'objects': 3588}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:58.711 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:17:58.711 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "45'12445", 'reported_seq': 12572, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:56.361368+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:17:56.361368+0000', 'last_peered': '2024-09-11T01:17:56.361368+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:17:56.361368+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30404608, 'num_objects': 7423, 'num_object_clones': 0, 'num_object_copies': 14846, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7423, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9032, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3439}, {'shards': '0,2', 'objects': 1477}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:58.711 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:17:58.711 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "45'12613", 'reported_seq': 12732, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:54.308030+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:17:54.308030+0000', 'last_peered': '2024-09-11T01:17:54.308030+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:17:54.308030+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31105024, 'num_objects': 7594, 'num_object_clones': 0, 'num_object_copies': 15188, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7594, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10430, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1516}, {'shards': '2', 'objects': 3570}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:17:58.711 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:17:58.711 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "45'12687", 'reported_seq': 12819, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:56.218357+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:17:56.218357+0000', 'last_peered': '2024-09-11T01:17:56.218357+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:17:56.218357+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 31035392, 'num_objects': 7577, 'num_object_clones': 0, 'num_object_copies': 15154, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7577, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9424, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3627}, {'shards': '0,2', 'objects': 1396}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:58.711 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.6 is not active+clean 2024-09-11T01:17:58.712 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.6', 'version': "45'12452", 'reported_seq': 20372, 'reported_epoch': 45, 'state': 'active+recovering+undersized+remapped', 'last_fresh': '2024-09-11T01:17:56.362937+0000', 'last_change': '2024-09-11T01:15:19.913288+0000', 'last_active': '2024-09-11T01:17:56.362937+0000', 'last_peered': '2024-09-11T01:17:56.362937+0000', 'last_clean': '2024-09-11T01:13:07.624565+0000', 'last_became_active': '2024-09-11T01:13:10.199795+0000', 'last_became_peered': '2024-09-11T01:13:10.199795+0000', 'last_unstale': '2024-09-11T01:17:56.362937+0000', 'last_undegraded': '2024-09-11T01:17:56.362937+0000', 'last_fullsized': '2024-09-11T01:13:09.652311+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3415", 'last_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:58.109581+0000', 'objects_scrubbed': 3337, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1975, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30441472, 'num_objects': 7432, 'num_object_clones': 0, 'num_object_copies': 14864, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 51, 'num_objects_unfound': 0, 'num_objects_dirty': 7432, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9107, 'num_write_kb': 26404, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3891, 'num_bytes_recovered': 15925248, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 7381}, {'shards': '0,2', 'objects': 51}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:58.712 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:17:58.712 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "45'12657", 'reported_seq': 12792, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:56.330190+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:17:56.330190+0000', 'last_peered': '2024-09-11T01:17:56.330190+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:17:56.330190+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30797824, 'num_objects': 7519, 'num_object_clones': 0, 'num_object_copies': 15038, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7519, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9266, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3480}, {'shards': '0,2', 'objects': 1472}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:58.712 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:17:58.712 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "45'12442", 'reported_seq': 12576, 'reported_epoch': 45, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:17:56.343721+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:17:56.343721+0000', 'last_peered': '2024-09-11T01:17:56.343721+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:17:56.343721+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "36'2400", 'ondisk_log_start': "36'2400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30761577, 'num_objects': 7511, 'num_object_clones': 0, 'num_object_copies': 15022, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7511, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9512, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3594}, {'shards': '0,2', 'objects': 1457}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:17:58.712 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:17:59.248 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:17:59.248 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 39769/241074 objects degraded (16.497%), 10 pgs degraded, 11 pgs undersized","count":21},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":415,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":13},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":10},{"state_name":"active+recovering+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":120537,"data_bytes":493703828,"bytes_used":2129686528,"bytes_avail":287780605952,"bytes_total":289910292480,"degraded_objects":39769,"degraded_total":241074,"degraded_ratio":0.16496594406696696,"misplaced_objects":51,"misplaced_total":241074,"misplaced_ratio":0.00021155329898703303,"recovering_objects_per_sec":43,"recovering_bytes_per_sec":176804,"recovering_keys_per_sec":0,"num_objects_recovered":432,"num_bytes_recovered":1769472,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":447},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:17:59.288 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:17:59.289 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:00.072 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:18:00.073 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:18:00.147 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":248,"stamp":"2024-09-11T01:17:58.925878+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":493703828,"num_objects":120537,"num_object_clones":0,"num_object_copies":241074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":51,"num_objects_unfound":0,"num_objects_dirty":120537,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":161754,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7820,"num_bytes_recovered":32006163,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160905,"ondisk_log_size":160905,"up":48,"acting":37,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2079772,"kb_used_data":1097764,"kb_used_omap":12,"kb_used_meta":981939,"kb_avail":281035748,"statfs":{"total":289910292480,"available":287780605952,"internally_reserved":0,"allocated":1124110336,"data_stored":1123269484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1005505540},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,6,5,9,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14811136,"num_objects":-3616,"num_object_clones":0,"num_object_copies":-7232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":-432,"num_objects_unfound":0,"num_objects_dirty":-3616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3616,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":432,"num_bytes_recovered":1769472,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.252104"},"pg_stats":[{"pgid":"2.f","version":"45'12686","reported_seq":12764,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:56.308208+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:17:56.308208+0000","last_peered":"2024-09-11T01:17:56.308208+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:17:56.308208+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31047680,"num_objects":7580,"num_object_clones":0,"num_object_copies":15160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12686,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3508},{"shards":"0,2","objects":1519}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"45'12748","reported_seq":12850,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:54.234465+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:17:54.234465+0000","last_peered":"2024-09-11T01:17:54.234465+0000","last_clean":"2024-09-11T01:17:54.234465+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:17:54.234465+0000","last_undegraded":"2024-09-11T01:17:54.234465+0000","last_fullsized":"2024-09-11T01:17:54.234465+0000","mapping_epoch":35,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31145984,"num_objects":7604,"num_object_clones":0,"num_object_copies":15208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10987,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"45'12533","reported_seq":12662,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:54.283457+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:17:54.283457+0000","last_peered":"2024-09-11T01:17:54.283457+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:17:54.283457+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31170584,"num_objects":7611,"num_object_clones":0,"num_object_copies":15222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7611,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10834,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1437},{"shards":"2","objects":3715}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"45'12500","reported_seq":12570,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:54.361116+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:17:54.361116+0000","last_peered":"2024-09-11T01:17:54.361116+0000","last_clean":"2024-09-11T01:17:54.361116+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:17:54.361116+0000","last_undegraded":"2024-09-11T01:17:54.361116+0000","last_fullsized":"2024-09-11T01:17:54.361116+0000","mapping_epoch":35,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31268864,"num_objects":7634,"num_object_clones":0,"num_object_copies":15268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7634,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12500,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"45'12516","reported_seq":12644,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:56.297716+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:17:56.297716+0000","last_peered":"2024-09-11T01:17:56.297716+0000","last_clean":"2024-09-11T01:17:56.297716+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:17:56.297716+0000","last_undegraded":"2024-09-11T01:17:56.297716+0000","last_fullsized":"2024-09-11T01:17:56.297716+0000","mapping_epoch":34,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30691328,"num_objects":7493,"num_object_clones":0,"num_object_copies":14986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9521,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"45'12689","reported_seq":12859,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:56.356631+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:17:56.356631+0000","last_peered":"2024-09-11T01:17:56.356631+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:17:56.356631+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30875648,"num_objects":7538,"num_object_clones":0,"num_object_copies":15076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9030,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3587},{"shards":"0,2","objects":1378}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"45'12450","reported_seq":12581,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:56.288938+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:17:56.288938+0000","last_peered":"2024-09-11T01:17:56.288938+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:17:56.288938+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30638080,"num_objects":7480,"num_object_clones":0,"num_object_copies":14960,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7480,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9307,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3494},{"shards":"0,2","objects":1504}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"45'12675","reported_seq":12805,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:56.314154+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:17:56.314154+0000","last_peered":"2024-09-11T01:17:56.314154+0000","last_clean":"2024-09-11T01:17:56.314154+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:17:56.314154+0000","last_undegraded":"2024-09-11T01:17:56.314154+0000","last_fullsized":"2024-09-11T01:17:56.314154+0000","mapping_epoch":34,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31043584,"num_objects":7579,"num_object_clones":0,"num_object_copies":15158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9551,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.683747+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:15:21.683747+0000","last_peered":"2024-09-11T01:15:21.683747+0000","last_clean":"2024-09-11T01:15:21.683747+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:15:21.683747+0000","last_undegraded":"2024-09-11T01:15:21.683747+0000","last_fullsized":"2024-09-11T01:15:21.683747+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"45'12452","reported_seq":12580,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:54.307347+0000","last_change":"2024-09-11T01:13:10.159069+0000","last_active":"2024-09-11T01:17:54.307347+0000","last_peered":"2024-09-11T01:17:54.307347+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:17:54.307347+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30806016,"num_objects":7521,"num_object_clones":0,"num_object_copies":15042,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3935,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7521,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10423,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1470},{"shards":"2","objects":3588}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:15:21.692373+0000","last_peered":"2024-09-11T01:15:21.692373+0000","last_clean":"2024-09-11T01:15:21.692373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:15:21.692373+0000","last_undegraded":"2024-09-11T01:15:21.692373+0000","last_fullsized":"2024-09-11T01:15:21.692373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"45'12445","reported_seq":12572,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:56.361368+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:17:56.361368+0000","last_peered":"2024-09-11T01:17:56.361368+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:17:56.361368+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30404608,"num_objects":7423,"num_object_clones":0,"num_object_copies":14846,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7423,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9032,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3439},{"shards":"0,2","objects":1477}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":101,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.686780+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:15:21.686780+0000","last_peered":"2024-09-11T01:15:21.686780+0000","last_clean":"2024-09-11T01:15:21.686780+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:15:21.686780+0000","last_undegraded":"2024-09-11T01:15:21.686780+0000","last_fullsized":"2024-09-11T01:15:21.686780+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"45'12613","reported_seq":12732,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:54.308030+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:17:54.308030+0000","last_peered":"2024-09-11T01:17:54.308030+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:17:54.308030+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31105024,"num_objects":7594,"num_object_clones":0,"num_object_copies":15188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7594,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10430,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1516},{"shards":"2","objects":3570}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":80,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684827+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:15:21.684827+0000","last_peered":"2024-09-11T01:15:21.684827+0000","last_clean":"2024-09-11T01:15:21.684827+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:15:21.684827+0000","last_undegraded":"2024-09-11T01:15:21.684827+0000","last_fullsized":"2024-09-11T01:15:21.684827+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"45'12357","reported_seq":20338,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:17:54.385777+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:17:54.385777+0000","last_peered":"2024-09-11T01:17:54.385777+0000","last_clean":"2024-09-11T01:17:54.385777+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:17:54.385777+0000","last_undegraded":"2024-09-11T01:17:54.385777+0000","last_fullsized":"2024-09-11T01:17:54.385777+0000","mapping_epoch":44,"log_start":"36'2300","ondisk_log_start":"36'2300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30470144,"num_objects":7439,"num_object_clones":0,"num_object_copies":14878,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7439,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10142,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"45'12687","reported_seq":12819,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:56.218357+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:17:56.218357+0000","last_peered":"2024-09-11T01:17:56.218357+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:17:56.218357+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":31035392,"num_objects":7577,"num_object_clones":0,"num_object_copies":15154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7577,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9424,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3627},{"shards":"0,2","objects":1396}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684623+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:15:21.684623+0000","last_peered":"2024-09-11T01:15:21.684623+0000","last_clean":"2024-09-11T01:15:21.684623+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:15:21.684623+0000","last_undegraded":"2024-09-11T01:15:21.684623+0000","last_fullsized":"2024-09-11T01:15:21.684623+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"45'12452","reported_seq":20372,"reported_epoch":45,"state":"active+recovering+undersized+remapped","last_fresh":"2024-09-11T01:17:56.362937+0000","last_change":"2024-09-11T01:15:19.913288+0000","last_active":"2024-09-11T01:17:56.362937+0000","last_peered":"2024-09-11T01:17:56.362937+0000","last_clean":"2024-09-11T01:13:07.624565+0000","last_became_active":"2024-09-11T01:13:10.199795+0000","last_became_peered":"2024-09-11T01:13:10.199795+0000","last_unstale":"2024-09-11T01:17:56.362937+0000","last_undegraded":"2024-09-11T01:17:56.362937+0000","last_fullsized":"2024-09-11T01:13:09.652311+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30441472,"num_objects":7432,"num_object_clones":0,"num_object_copies":14864,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":51,"num_objects_unfound":0,"num_objects_dirty":7432,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9107,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3891,"num_bytes_recovered":15925248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":7381},{"shards":"0,2","objects":51}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.694401+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:15:21.694401+0000","last_peered":"2024-09-11T01:15:21.694401+0000","last_clean":"2024-09-11T01:15:21.694401+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:15:21.694401+0000","last_undegraded":"2024-09-11T01:15:21.694401+0000","last_fullsized":"2024-09-11T01:15:21.694401+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"45'12657","reported_seq":12792,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:56.330190+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:17:56.330190+0000","last_peered":"2024-09-11T01:17:56.330190+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:17:56.330190+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30797824,"num_objects":7519,"num_object_clones":0,"num_object_copies":15038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9266,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3480},{"shards":"0,2","objects":1472}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":70,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.692049+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:15:21.692049+0000","last_peered":"2024-09-11T01:15:21.692049+0000","last_clean":"2024-09-11T01:15:21.692049+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:15:21.692049+0000","last_undegraded":"2024-09-11T01:15:21.692049+0000","last_fullsized":"2024-09-11T01:15:21.692049+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":75,"reported_epoch":45,"state":"active+clean","last_fresh":"2024-09-11T01:15:21.684417+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:15:21.684417+0000","last_peered":"2024-09-11T01:15:21.684417+0000","last_clean":"2024-09-11T01:15:21.684417+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:15:21.684417+0000","last_undegraded":"2024-09-11T01:15:21.684417+0000","last_fullsized":"2024-09-11T01:15:21.684417+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"45'12442","reported_seq":12576,"reported_epoch":45,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:56.343721+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:17:56.343721+0000","last_peered":"2024-09-11T01:17:56.343721+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:17:56.343721+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30761577,"num_objects":7511,"num_object_clones":0,"num_object_copies":15022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7511,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9512,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3594},{"shards":"0,2","objects":1457}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":493703809,"num_objects":120535,"num_object_clones":0,"num_object_copies":241070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39769,"num_objects_misplaced":51,"num_objects_unfound":0,"num_objects_dirty":120535,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":161752,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7819,"num_bytes_recovered":32006144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1122193408,"data_stored":1122115684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160902,"ondisk_log_size":160902,"up":32,"acting":21,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182477,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":785348,"kb_used_data":424684,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93586492,"statfs":{"total":96636764160,"available":95832567808,"internally_reserved":0,"allocated":434876416,"data_stored":434617202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":369290923},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,5,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986249,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":681352,"kb_used_data":370680,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93690488,"statfs":{"total":96636764160,"available":95939059712,"internally_reserved":0,"allocated":379576320,"data_stored":379317111,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,1,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659194,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":613072,"kb_used_data":302400,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93758768,"statfs":{"total":96636764160,"available":96008978432,"internally_reserved":0,"allocated":309657600,"data_stored":309335171,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":318107311},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":434331648,"data_stored":434328169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":309297152,"data_stored":309230458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":378564608,"data_stored":378557057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:03.149 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:04.126 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:18:04.127 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:18:04.190 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":250,"stamp":"2024-09-11T01:18:02.682238+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":485896852,"num_objects":118631,"num_object_clones":0,"num_object_copies":237262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38378,"num_objects_misplaced":1307,"num_objects_unfound":0,"num_objects_dirty":118631,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":163660,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7955,"num_bytes_recovered":32550931,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161011,"ondisk_log_size":161011,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2048880,"kb_used_data":1087680,"kb_used_omap":12,"kb_used_meta":961139,"kb_avail":281066640,"statfs":{"total":289910292480,"available":287812239360,"internally_reserved":0,"allocated":1113784320,"data_stored":1112924324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":984206340},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,4,5,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-13586432,"num_objects":-3317,"num_object_clones":0,"num_object_copies":-6634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1391,"num_objects_misplaced":1029,"num_objects_unfound":0,"num_objects_dirty":-3317,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3317,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":362,"num_bytes_recovered":1474560,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001807"},"pg_stats":[{"pgid":"2.f","version":"47'12810","reported_seq":12892,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:01.376878+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:18:01.376878+0000","last_peered":"2024-09-11T01:18:01.376878+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:18:01.376878+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30539776,"num_objects":7456,"num_object_clones":0,"num_object_copies":14912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12810,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3384},{"shards":"0,2","objects":1395}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'12879","reported_seq":12985,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:59.248708+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:17:59.248708+0000","last_peered":"2024-09-11T01:17:59.248708+0000","last_clean":"2024-09-11T01:17:59.248708+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:17:59.248708+0000","last_undegraded":"2024-09-11T01:17:59.248708+0000","last_fullsized":"2024-09-11T01:17:59.248708+0000","mapping_epoch":35,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30609408,"num_objects":7473,"num_object_clones":0,"num_object_copies":14946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11118,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'12654","reported_seq":12787,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:59.318765+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:17:59.318765+0000","last_peered":"2024-09-11T01:17:59.318765+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:17:59.318765+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30674968,"num_objects":7490,"num_object_clones":0,"num_object_copies":14980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7490,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10955,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1316},{"shards":"2","objects":3594}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'12615","reported_seq":12689,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:59.393868+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:17:59.393868+0000","last_peered":"2024-09-11T01:17:59.393868+0000","last_clean":"2024-09-11T01:17:59.393868+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:17:59.393868+0000","last_undegraded":"2024-09-11T01:17:59.393868+0000","last_fullsized":"2024-09-11T01:17:59.393868+0000","mapping_epoch":35,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30797824,"num_objects":7519,"num_object_clones":0,"num_object_copies":15038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12615,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'12648","reported_seq":12780,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:01.269420+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:18:01.269420+0000","last_peered":"2024-09-11T01:18:01.269420+0000","last_clean":"2024-09-11T01:18:01.269420+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:18:01.269420+0000","last_undegraded":"2024-09-11T01:18:01.269420+0000","last_fullsized":"2024-09-11T01:18:01.269420+0000","mapping_epoch":34,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30150656,"num_objects":7361,"num_object_clones":0,"num_object_copies":14722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9653,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'12799","reported_seq":12973,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:01.371950+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:18:01.371950+0000","last_peered":"2024-09-11T01:18:01.371950+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:18:01.371950+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30425088,"num_objects":7428,"num_object_clones":0,"num_object_copies":14856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9140,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3477},{"shards":"0,2","objects":1268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'12586","reported_seq":12716,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:01.356307+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:18:01.356307+0000","last_peered":"2024-09-11T01:18:01.356307+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:18:01.356307+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30081024,"num_objects":7344,"num_object_clones":0,"num_object_copies":14688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9443,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3358},{"shards":"0,2","objects":1368}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'12796","reported_seq":12930,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:01.354474+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:18:01.354474+0000","last_peered":"2024-09-11T01:18:01.354474+0000","last_clean":"2024-09-11T01:18:01.354474+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:18:01.354474+0000","last_undegraded":"2024-09-11T01:18:01.354474+0000","last_fullsized":"2024-09-11T01:18:01.354474+0000","mapping_epoch":34,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30547968,"num_objects":7458,"num_object_clones":0,"num_object_copies":14916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9672,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934359+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934359+0000","last_peered":"2024-09-11T01:17:58.934359+0000","last_clean":"2024-09-11T01:17:58.934359+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934359+0000","last_undegraded":"2024-09-11T01:17:58.934359+0000","last_fullsized":"2024-09-11T01:17:58.934359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'12557","reported_seq":12856,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:59.391686+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:17:59.391686+0000","last_peered":"2024-09-11T01:17:59.391686+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:17:59.391686+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30375936,"num_objects":7416,"num_object_clones":0,"num_object_copies":14832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2544,"num_objects_misplaced":1307,"num_objects_unfound":0,"num_objects_dirty":7416,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10528,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":84,"num_bytes_recovered":335872,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1307},{"shards":"2","objects":3565}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938294+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938294+0000","last_peered":"2024-09-11T01:17:58.938294+0000","last_clean":"2024-09-11T01:17:58.938294+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938294+0000","last_undegraded":"2024-09-11T01:17:58.938294+0000","last_fullsized":"2024-09-11T01:17:58.938294+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'12575","reported_seq":12708,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:01.327804+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:18:01.327804+0000","last_peered":"2024-09-11T01:18:01.327804+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:18:01.327804+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29872128,"num_objects":7293,"num_object_clones":0,"num_object_copies":14586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9162,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3309},{"shards":"0,2","objects":1347}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928034+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928034+0000","last_peered":"2024-09-11T01:17:58.928034+0000","last_clean":"2024-09-11T01:17:58.928034+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928034+0000","last_undegraded":"2024-09-11T01:17:58.928034+0000","last_fullsized":"2024-09-11T01:17:58.928034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'12732","reported_seq":12853,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:17:59.366872+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:17:59.366872+0000","last_peered":"2024-09-11T01:17:59.366872+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:17:59.366872+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30617600,"num_objects":7475,"num_object_clones":0,"num_object_copies":14950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10549,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1397},{"shards":"2","objects":3451}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935080+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935080+0000","last_peered":"2024-09-11T01:17:58.935080+0000","last_clean":"2024-09-11T01:17:58.935080+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935080+0000","last_undegraded":"2024-09-11T01:17:58.935080+0000","last_fullsized":"2024-09-11T01:17:58.935080+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'12474","reported_seq":20459,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:59.342188+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:17:59.342188+0000","last_peered":"2024-09-11T01:17:59.342188+0000","last_clean":"2024-09-11T01:17:59.342188+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:17:59.342188+0000","last_undegraded":"2024-09-11T01:17:59.342188+0000","last_fullsized":"2024-09-11T01:17:59.342188+0000","mapping_epoch":44,"log_start":"36'2400","ondisk_log_start":"36'2400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29990912,"num_objects":7322,"num_object_clones":0,"num_object_copies":14644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10259,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'12791","reported_seq":12927,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:01.331232+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:18:01.331232+0000","last_peered":"2024-09-11T01:18:01.331232+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:18:01.331232+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30609408,"num_objects":7473,"num_object_clones":0,"num_object_copies":14946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9528,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3523},{"shards":"0,2","objects":1292}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938540+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938540+0000","last_peered":"2024-09-11T01:17:58.938540+0000","last_clean":"2024-09-11T01:17:58.938540+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938540+0000","last_undegraded":"2024-09-11T01:17:58.938540+0000","last_fullsized":"2024-09-11T01:17:58.938540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'12534","reported_seq":20570,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:00.086742+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:18:00.086742+0000","last_peered":"2024-09-11T01:18:00.086742+0000","last_clean":"2024-09-11T01:18:00.086742+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:18:00.086742+0000","last_undegraded":"2024-09-11T01:18:00.086742+0000","last_fullsized":"2024-09-11T01:18:00.086742+0000","mapping_epoch":46,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30105600,"num_objects":7350,"num_object_clones":0,"num_object_copies":14700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9189,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941377+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941377+0000","last_peered":"2024-09-11T01:17:58.941377+0000","last_clean":"2024-09-11T01:17:58.941377+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941377+0000","last_undegraded":"2024-09-11T01:17:58.941377+0000","last_fullsized":"2024-09-11T01:17:58.941377+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'12780","reported_seq":12918,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:01.365079+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:18:01.365079+0000","last_peered":"2024-09-11T01:18:01.365079+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:18:01.365079+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30294016,"num_objects":7396,"num_object_clones":0,"num_object_copies":14792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9389,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3357},{"shards":"0,2","objects":1349}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971314+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971314+0000","last_peered":"2024-09-11T01:17:58.971314+0000","last_clean":"2024-09-11T01:17:58.971314+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971314+0000","last_undegraded":"2024-09-11T01:17:58.971314+0000","last_fullsized":"2024-09-11T01:17:58.971314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'12578","reported_seq":12716,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:01.346455+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:18:01.346455+0000","last_peered":"2024-09-11T01:18:01.346455+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:18:01.346455+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30204521,"num_objects":7375,"num_object_clones":0,"num_object_copies":14750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9648,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3458},{"shards":"0,2","objects":1321}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":485896833,"num_objects":118629,"num_object_clones":0,"num_object_copies":237258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38378,"num_objects_misplaced":1307,"num_objects_unfound":0,"num_objects_dirty":118629,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":163658,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7954,"num_bytes_recovered":32550912,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1111879680,"data_stored":1111808914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161008,"ondisk_log_size":161008,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182478,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":783732,"kb_used_data":418916,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93588108,"statfs":{"total":96636764160,"available":95834222592,"internally_reserved":0,"allocated":428969984,"data_stored":428700325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":373550763},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,3,4,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986250,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":664672,"kb_used_data":366480,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93707168,"statfs":{"total":96636764160,"available":95956140032,"internally_reserved":0,"allocated":375275520,"data_stored":375011099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":305327786},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659195,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":600476,"kb_used_data":302284,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93771364,"statfs":{"total":96636764160,"available":96021876736,"internally_reserved":0,"allocated":309538816,"data_stored":309212900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":305327791},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":428261376,"data_stored":428257897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":309022720,"data_stored":308962984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":374595584,"data_stored":374588033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:04.191 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:05.010 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:18:05.011 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:18:05.071 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":251,"stamp":"2024-09-11T01:18:04.682708+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":482439828,"num_objects":117787,"num_object_clones":0,"num_object_copies":235574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38378,"num_objects_misplaced":1217,"num_objects_unfound":0,"num_objects_dirty":117787,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":164504,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8045,"num_bytes_recovered":32919571,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161055,"ondisk_log_size":161055,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2048948,"kb_used_data":1083584,"kb_used_omap":12,"kb_used_meta":965299,"kb_avail":281066572,"statfs":{"total":289910292480,"available":287812169728,"internally_reserved":0,"allocated":1109590016,"data_stored":1108724787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":988466180},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,2,7,7,9],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11264000,"num_objects":-2750,"num_object_clones":0,"num_object_copies":-5500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1391,"num_objects_misplaced":1166,"num_objects_unfound":0,"num_objects_dirty":-2750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2750,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":913408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001805"},"pg_stats":[{"pgid":"2.f","version":"47'12810","reported_seq":12892,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:01.376878+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:18:01.376878+0000","last_peered":"2024-09-11T01:18:01.376878+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:18:01.376878+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30539776,"num_objects":7456,"num_object_clones":0,"num_object_copies":14912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12810,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3384},{"shards":"0,2","objects":1395}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13012","reported_seq":13118,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:04.293132+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:18:04.293132+0000","last_peered":"2024-09-11T01:18:04.293132+0000","last_clean":"2024-09-11T01:18:04.293132+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:18:04.293132+0000","last_undegraded":"2024-09-11T01:18:04.293132+0000","last_fullsized":"2024-09-11T01:18:04.293132+0000","mapping_epoch":35,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30064640,"num_objects":7340,"num_object_clones":0,"num_object_copies":14680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11251,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'12779","reported_seq":12911,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:04.318254+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:18:04.318254+0000","last_peered":"2024-09-11T01:18:04.318254+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:18:04.318254+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30162968,"num_objects":7365,"num_object_clones":0,"num_object_copies":14730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7365,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11080,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1191},{"shards":"2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'12767","reported_seq":12841,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:04.375655+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:18:04.375655+0000","last_peered":"2024-09-11T01:18:04.375655+0000","last_clean":"2024-09-11T01:18:04.375655+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:18:04.375655+0000","last_undegraded":"2024-09-11T01:18:04.375655+0000","last_fullsized":"2024-09-11T01:18:04.375655+0000","mapping_epoch":35,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30175232,"num_objects":7367,"num_object_clones":0,"num_object_copies":14734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12767,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'12648","reported_seq":12780,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:01.269420+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:18:01.269420+0000","last_peered":"2024-09-11T01:18:01.269420+0000","last_clean":"2024-09-11T01:18:01.269420+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:18:01.269420+0000","last_undegraded":"2024-09-11T01:18:01.269420+0000","last_fullsized":"2024-09-11T01:18:01.269420+0000","mapping_epoch":34,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30150656,"num_objects":7361,"num_object_clones":0,"num_object_copies":14722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7361,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9653,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'12799","reported_seq":12973,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:01.371950+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:18:01.371950+0000","last_peered":"2024-09-11T01:18:01.371950+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:18:01.371950+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30425088,"num_objects":7428,"num_object_clones":0,"num_object_copies":14856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9140,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3477},{"shards":"0,2","objects":1268}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'12586","reported_seq":12716,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:01.356307+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:18:01.356307+0000","last_peered":"2024-09-11T01:18:01.356307+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:18:01.356307+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30081024,"num_objects":7344,"num_object_clones":0,"num_object_copies":14688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9443,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3358},{"shards":"0,2","objects":1368}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'12796","reported_seq":12930,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:01.354474+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:18:01.354474+0000","last_peered":"2024-09-11T01:18:01.354474+0000","last_clean":"2024-09-11T01:18:01.354474+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:18:01.354474+0000","last_undegraded":"2024-09-11T01:18:01.354474+0000","last_fullsized":"2024-09-11T01:18:01.354474+0000","mapping_epoch":34,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30547968,"num_objects":7458,"num_object_clones":0,"num_object_copies":14916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7458,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9672,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934359+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934359+0000","last_peered":"2024-09-11T01:17:58.934359+0000","last_clean":"2024-09-11T01:17:58.934359+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934359+0000","last_undegraded":"2024-09-11T01:17:58.934359+0000","last_fullsized":"2024-09-11T01:17:58.934359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'12699","reported_seq":13179,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:04.385952+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:18:04.385952+0000","last_peered":"2024-09-11T01:18:04.385952+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:18:04.385952+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29794304,"num_objects":7274,"num_object_clones":0,"num_object_copies":14548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2544,"num_objects_misplaced":1217,"num_objects_unfound":0,"num_objects_dirty":7274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10670,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":704512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1217},{"shards":"2","objects":3513}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938294+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938294+0000","last_peered":"2024-09-11T01:17:58.938294+0000","last_clean":"2024-09-11T01:17:58.938294+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938294+0000","last_undegraded":"2024-09-11T01:17:58.938294+0000","last_fullsized":"2024-09-11T01:17:58.938294+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'12575","reported_seq":12708,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:01.327804+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:18:01.327804+0000","last_peered":"2024-09-11T01:18:01.327804+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:18:01.327804+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29872128,"num_objects":7293,"num_object_clones":0,"num_object_copies":14586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7293,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9162,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3309},{"shards":"0,2","objects":1347}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928034+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928034+0000","last_peered":"2024-09-11T01:17:58.928034+0000","last_clean":"2024-09-11T01:17:58.928034+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928034+0000","last_undegraded":"2024-09-11T01:17:58.928034+0000","last_fullsized":"2024-09-11T01:17:58.928034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'12885","reported_seq":13008,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:04.386595+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:18:04.386595+0000","last_peered":"2024-09-11T01:18:04.386595+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:18:04.386595+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29990912,"num_objects":7322,"num_object_clones":0,"num_object_copies":14644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10702,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1244},{"shards":"2","objects":3298}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935080+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935080+0000","last_peered":"2024-09-11T01:17:58.935080+0000","last_clean":"2024-09-11T01:17:58.935080+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935080+0000","last_undegraded":"2024-09-11T01:17:58.935080+0000","last_fullsized":"2024-09-11T01:17:58.935080+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'12613","reported_seq":20598,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:04.391163+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:18:04.391163+0000","last_peered":"2024-09-11T01:18:04.391163+0000","last_clean":"2024-09-11T01:18:04.391163+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:18:04.391163+0000","last_undegraded":"2024-09-11T01:18:04.391163+0000","last_fullsized":"2024-09-11T01:18:04.391163+0000","mapping_epoch":44,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29421568,"num_objects":7183,"num_object_clones":0,"num_object_copies":14366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10398,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'12791","reported_seq":12927,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:01.331232+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:18:01.331232+0000","last_peered":"2024-09-11T01:18:01.331232+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:18:01.331232+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30609408,"num_objects":7473,"num_object_clones":0,"num_object_copies":14946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9528,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3523},{"shards":"0,2","objects":1292}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938540+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938540+0000","last_peered":"2024-09-11T01:17:58.938540+0000","last_clean":"2024-09-11T01:17:58.938540+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938540+0000","last_undegraded":"2024-09-11T01:17:58.938540+0000","last_fullsized":"2024-09-11T01:17:58.938540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'12534","reported_seq":20570,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:00.086742+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:18:00.086742+0000","last_peered":"2024-09-11T01:18:00.086742+0000","last_clean":"2024-09-11T01:18:00.086742+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:18:00.086742+0000","last_undegraded":"2024-09-11T01:18:00.086742+0000","last_fullsized":"2024-09-11T01:18:00.086742+0000","mapping_epoch":46,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30105600,"num_objects":7350,"num_object_clones":0,"num_object_copies":14700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9189,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941377+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941377+0000","last_peered":"2024-09-11T01:17:58.941377+0000","last_clean":"2024-09-11T01:17:58.941377+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941377+0000","last_undegraded":"2024-09-11T01:17:58.941377+0000","last_fullsized":"2024-09-11T01:17:58.941377+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'12780","reported_seq":12918,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:01.365079+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:18:01.365079+0000","last_peered":"2024-09-11T01:18:01.365079+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:18:01.365079+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30294016,"num_objects":7396,"num_object_clones":0,"num_object_copies":14792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9389,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3357},{"shards":"0,2","objects":1349}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971314+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971314+0000","last_peered":"2024-09-11T01:17:58.971314+0000","last_clean":"2024-09-11T01:17:58.971314+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971314+0000","last_undegraded":"2024-09-11T01:17:58.971314+0000","last_fullsized":"2024-09-11T01:17:58.971314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'12578","reported_seq":12716,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:01.346455+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:18:01.346455+0000","last_peered":"2024-09-11T01:18:01.346455+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:18:01.346455+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'2500","ondisk_log_start":"36'2500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30204521,"num_objects":7375,"num_object_clones":0,"num_object_copies":14750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9648,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3458},{"shards":"0,2","objects":1321}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":482439809,"num_objects":117785,"num_object_clones":0,"num_object_copies":235570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38378,"num_objects_misplaced":1217,"num_objects_unfound":0,"num_objects_dirty":117785,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":164502,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8044,"num_bytes_recovered":32919552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1107283968,"data_stored":1107213202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161052,"ondisk_log_size":161052,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182478,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":783732,"kb_used_data":418916,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93588108,"statfs":{"total":96636764160,"available":95834222592,"internally_reserved":0,"allocated":428969984,"data_stored":428700325,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":373550763},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,3,4,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986251,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":664740,"kb_used_data":362384,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93707100,"statfs":{"total":96636764160,"available":95956070400,"internally_reserved":0,"allocated":371081216,"data_stored":370811562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659195,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":600476,"kb_used_data":302284,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93771364,"statfs":{"total":96636764160,"available":96021876736,"internally_reserved":0,"allocated":309538816,"data_stored":309212900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":305327791},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":428261376,"data_stored":428257897,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":309022720,"data_stored":308962984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":369999872,"data_stored":369992321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:05.072 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:18:05.073 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'12810", 'reported_seq': 12892, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:01.376878+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:18:01.376878+0000', 'last_peered': '2024-09-11T01:18:01.376878+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:18:01.376878+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "36'2800", 'ondisk_log_start': "36'2800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30539776, 'num_objects': 7456, 'num_object_clones': 0, 'num_object_copies': 14912, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7456, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12810, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3384}, {'shards': '0,2', 'objects': 1395}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:05.073 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:18:05.073 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'12779", 'reported_seq': 12911, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:04.318254+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:18:04.318254+0000', 'last_peered': '2024-09-11T01:18:04.318254+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:18:04.318254+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "36'2700", 'ondisk_log_start': "36'2700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30162968, 'num_objects': 7365, 'num_object_clones': 0, 'num_object_copies': 14730, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7365, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11080, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1191}, {'shards': '2', 'objects': 3469}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:05.073 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:18:05.073 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'12799", 'reported_seq': 12973, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:01.371950+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:18:01.371950+0000', 'last_peered': '2024-09-11T01:18:01.371950+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:18:01.371950+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "36'2700", 'ondisk_log_start': "36'2700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30425088, 'num_objects': 7428, 'num_object_clones': 0, 'num_object_copies': 14856, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7428, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9140, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3477}, {'shards': '0,2', 'objects': 1268}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:05.073 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:18:05.073 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'12586", 'reported_seq': 12716, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:01.356307+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:18:01.356307+0000', 'last_peered': '2024-09-11T01:18:01.356307+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:18:01.356307+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "36'2500", 'ondisk_log_start': "36'2500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30081024, 'num_objects': 7344, 'num_object_clones': 0, 'num_object_copies': 14688, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7344, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9443, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3358}, {'shards': '0,2', 'objects': 1368}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:05.073 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:18:05.073 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'12699", 'reported_seq': 13179, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:04.385952+0000', 'last_change': '2024-09-11T01:17:57.355893+0000', 'last_active': '2024-09-11T01:18:04.385952+0000', 'last_peered': '2024-09-11T01:18:04.385952+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:18:04.385952+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "36'2600", 'ondisk_log_start': "36'2600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29794304, 'num_objects': 7274, 'num_object_clones': 0, 'num_object_copies': 14548, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2544, 'num_objects_misplaced': 1217, 'num_objects_unfound': 0, 'num_objects_dirty': 7274, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10670, 'num_write_kb': 31844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 174, 'num_bytes_recovered': 704512, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1217}, {'shards': '2', 'objects': 3513}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:05.074 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:18:05.074 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'12575", 'reported_seq': 12708, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:01.327804+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:18:01.327804+0000', 'last_peered': '2024-09-11T01:18:01.327804+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:18:01.327804+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "36'2500", 'ondisk_log_start': "36'2500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29872128, 'num_objects': 7293, 'num_object_clones': 0, 'num_object_copies': 14586, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7293, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9162, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3309}, {'shards': '0,2', 'objects': 1347}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:05.074 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:18:05.074 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'12885", 'reported_seq': 13008, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:04.386595+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:18:04.386595+0000', 'last_peered': '2024-09-11T01:18:04.386595+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:18:04.386595+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "36'2800", 'ondisk_log_start': "36'2800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29990912, 'num_objects': 7322, 'num_object_clones': 0, 'num_object_copies': 14644, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7322, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10702, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1244}, {'shards': '2', 'objects': 3298}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:05.074 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:18:05.074 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'12791", 'reported_seq': 12927, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:01.331232+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:18:01.331232+0000', 'last_peered': '2024-09-11T01:18:01.331232+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:18:01.331232+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "36'2700", 'ondisk_log_start': "36'2700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30609408, 'num_objects': 7473, 'num_object_clones': 0, 'num_object_copies': 14946, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7473, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9528, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3523}, {'shards': '0,2', 'objects': 1292}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:05.074 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:18:05.074 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'12780", 'reported_seq': 12918, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:01.365079+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:18:01.365079+0000', 'last_peered': '2024-09-11T01:18:01.365079+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:18:01.365079+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "36'2700", 'ondisk_log_start': "36'2700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30294016, 'num_objects': 7396, 'num_object_clones': 0, 'num_object_copies': 14792, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7396, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9389, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3357}, {'shards': '0,2', 'objects': 1349}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:05.074 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:18:05.074 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'12578", 'reported_seq': 12716, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:01.346455+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:18:01.346455+0000', 'last_peered': '2024-09-11T01:18:01.346455+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:18:01.346455+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "36'2500", 'ondisk_log_start': "36'2500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30204521, 'num_objects': 7375, 'num_object_clones': 0, 'num_object_copies': 14750, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7375, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9648, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3458}, {'shards': '0,2', 'objects': 1321}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:05.075 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:18:06.183 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:18:06.184 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38378/235574 objects degraded (16.291%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":422,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":117787,"data_bytes":482439828,"bytes_used":2098122752,"bytes_avail":287812169728,"bytes_total":289910292480,"degraded_objects":38378,"degraded_total":235574,"degraded_ratio":0.16291271532512078,"misplaced_objects":1217,"misplaced_total":235574,"misplaced_ratio":0.0051661049181998013,"recovering_objects_per_sec":28,"recovering_bytes_per_sec":114150,"recovering_keys_per_sec":0,"num_objects_recovered":225,"num_bytes_recovered":913408,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":343},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:18:06.248 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:18:06.248 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:07.049 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:18:07.049 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:18:07.108 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":252,"stamp":"2024-09-11T01:18:06.683149+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":476725908,"num_objects":116392,"num_object_clones":0,"num_object_copies":232784,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38378,"num_objects_misplaced":1217,"num_objects_unfound":0,"num_objects_dirty":116392,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":165899,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8045,"num_bytes_recovered":32919571,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160750,"ondisk_log_size":160750,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2041004,"kb_used_data":1075640,"kb_used_omap":12,"kb_used_meta":965299,"kb_avail":281074516,"statfs":{"total":289910292480,"available":287820304384,"internally_reserved":0,"allocated":1101455360,"data_stored":1100590131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":988466180},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,8,5,9,8,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-16977920,"num_objects":-4145,"num_object_clones":0,"num_object_copies":-8290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1391,"num_objects_misplaced":1166,"num_objects_unfound":0,"num_objects_dirty":-4145,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4145,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":225,"num_bytes_recovered":913408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.770950"},"pg_stats":[{"pgid":"2.f","version":"47'12954","reported_seq":13036,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:06.392497+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:18:06.392497+0000","last_peered":"2024-09-11T01:18:06.392497+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:18:06.392497+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29949952,"num_objects":7312,"num_object_clones":0,"num_object_copies":14624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12954,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3240},{"shards":"0,2","objects":1251}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13012","reported_seq":13118,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:04.293132+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:18:04.293132+0000","last_peered":"2024-09-11T01:18:04.293132+0000","last_clean":"2024-09-11T01:18:04.293132+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:18:04.293132+0000","last_undegraded":"2024-09-11T01:18:04.293132+0000","last_fullsized":"2024-09-11T01:18:04.293132+0000","mapping_epoch":35,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30064640,"num_objects":7340,"num_object_clones":0,"num_object_copies":14680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11251,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'12779","reported_seq":12911,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:04.318254+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:18:04.318254+0000","last_peered":"2024-09-11T01:18:04.318254+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:18:04.318254+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30162968,"num_objects":7365,"num_object_clones":0,"num_object_copies":14730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7365,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11080,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1191},{"shards":"2","objects":3469}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'12767","reported_seq":12841,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:04.375655+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:18:04.375655+0000","last_peered":"2024-09-11T01:18:04.375655+0000","last_clean":"2024-09-11T01:18:04.375655+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:18:04.375655+0000","last_undegraded":"2024-09-11T01:18:04.375655+0000","last_fullsized":"2024-09-11T01:18:04.375655+0000","mapping_epoch":35,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30175232,"num_objects":7367,"num_object_clones":0,"num_object_copies":14734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12767,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'12793","reported_seq":12925,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:06.364552+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:18:06.364552+0000","last_peered":"2024-09-11T01:18:06.364552+0000","last_clean":"2024-09-11T01:18:06.364552+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:18:06.364552+0000","last_undegraded":"2024-09-11T01:18:06.364552+0000","last_fullsized":"2024-09-11T01:18:06.364552+0000","mapping_epoch":34,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29556736,"num_objects":7216,"num_object_clones":0,"num_object_copies":14432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9798,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'12939","reported_seq":13114,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:06.318450+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:18:06.318450+0000","last_peered":"2024-09-11T01:18:06.318450+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:18:06.318450+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29851648,"num_objects":7288,"num_object_clones":0,"num_object_copies":14576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7288,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9280,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3337},{"shards":"0,2","objects":1128}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'12719","reported_seq":12854,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:06.351068+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:18:06.351068+0000","last_peered":"2024-09-11T01:18:06.351068+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:18:06.351068+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29536256,"num_objects":7211,"num_object_clones":0,"num_object_copies":14422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7211,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9576,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3225},{"shards":"0,2","objects":1235}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'12925","reported_seq":13059,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:06.304904+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:18:06.304904+0000","last_peered":"2024-09-11T01:18:06.304904+0000","last_clean":"2024-09-11T01:18:06.304904+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:18:06.304904+0000","last_undegraded":"2024-09-11T01:18:06.304904+0000","last_fullsized":"2024-09-11T01:18:06.304904+0000","mapping_epoch":34,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30019584,"num_objects":7329,"num_object_clones":0,"num_object_copies":14658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9801,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934359+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934359+0000","last_peered":"2024-09-11T01:17:58.934359+0000","last_clean":"2024-09-11T01:17:58.934359+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934359+0000","last_undegraded":"2024-09-11T01:17:58.934359+0000","last_fullsized":"2024-09-11T01:17:58.934359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'12699","reported_seq":13179,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:04.385952+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:18:04.385952+0000","last_peered":"2024-09-11T01:18:04.385952+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:18:04.385952+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29794304,"num_objects":7274,"num_object_clones":0,"num_object_copies":14548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2544,"num_objects_misplaced":1217,"num_objects_unfound":0,"num_objects_dirty":7274,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10670,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":704512,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1217},{"shards":"2","objects":3513}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938294+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938294+0000","last_peered":"2024-09-11T01:17:58.938294+0000","last_clean":"2024-09-11T01:17:58.938294+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938294+0000","last_undegraded":"2024-09-11T01:17:58.938294+0000","last_fullsized":"2024-09-11T01:17:58.938294+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'12703","reported_seq":12833,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:06.320450+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:18:06.320450+0000","last_peered":"2024-09-11T01:18:06.320450+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:18:06.320450+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29347840,"num_objects":7165,"num_object_clones":0,"num_object_copies":14330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9290,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3181},{"shards":"0,2","objects":1219}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928034+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928034+0000","last_peered":"2024-09-11T01:17:58.928034+0000","last_clean":"2024-09-11T01:17:58.928034+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928034+0000","last_undegraded":"2024-09-11T01:17:58.928034+0000","last_fullsized":"2024-09-11T01:17:58.928034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'12885","reported_seq":13008,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:04.386595+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:18:04.386595+0000","last_peered":"2024-09-11T01:18:04.386595+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:18:04.386595+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29990912,"num_objects":7322,"num_object_clones":0,"num_object_copies":14644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10702,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1244},{"shards":"2","objects":3298}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935080+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935080+0000","last_peered":"2024-09-11T01:17:58.935080+0000","last_clean":"2024-09-11T01:17:58.935080+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935080+0000","last_undegraded":"2024-09-11T01:17:58.935080+0000","last_fullsized":"2024-09-11T01:17:58.935080+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'12613","reported_seq":20598,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:04.391163+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:18:04.391163+0000","last_peered":"2024-09-11T01:18:04.391163+0000","last_clean":"2024-09-11T01:18:04.391163+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:18:04.391163+0000","last_undegraded":"2024-09-11T01:18:04.391163+0000","last_fullsized":"2024-09-11T01:18:04.391163+0000","mapping_epoch":44,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29421568,"num_objects":7183,"num_object_clones":0,"num_object_copies":14366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10398,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'12906","reported_seq":13042,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:06.277743+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:18:06.277743+0000","last_peered":"2024-09-11T01:18:06.277743+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:18:06.277743+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30138368,"num_objects":7358,"num_object_clones":0,"num_object_copies":14716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9643,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3408},{"shards":"0,2","objects":1177}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938540+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938540+0000","last_peered":"2024-09-11T01:17:58.938540+0000","last_clean":"2024-09-11T01:17:58.938540+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938540+0000","last_undegraded":"2024-09-11T01:17:58.938540+0000","last_fullsized":"2024-09-11T01:17:58.938540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'12674","reported_seq":20708,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:05.191962+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:18:05.191962+0000","last_peered":"2024-09-11T01:18:05.191962+0000","last_clean":"2024-09-11T01:18:05.191962+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:18:05.191962+0000","last_undegraded":"2024-09-11T01:18:05.191962+0000","last_fullsized":"2024-09-11T01:18:05.191962+0000","mapping_epoch":46,"log_start":"36'2600","ondisk_log_start":"36'2600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29532160,"num_objects":7210,"num_object_clones":0,"num_object_copies":14420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9329,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941377+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941377+0000","last_peered":"2024-09-11T01:17:58.941377+0000","last_clean":"2024-09-11T01:17:58.941377+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941377+0000","last_undegraded":"2024-09-11T01:17:58.941377+0000","last_fullsized":"2024-09-11T01:17:58.941377+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'12948","reported_seq":13082,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:06.405925+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:18:06.405925+0000","last_peered":"2024-09-11T01:18:06.405925+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:18:06.405925+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29605888,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9557,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3189},{"shards":"0,2","objects":1181}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971314+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971314+0000","last_peered":"2024-09-11T01:17:58.971314+0000","last_clean":"2024-09-11T01:17:58.971314+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971314+0000","last_undegraded":"2024-09-11T01:17:58.971314+0000","last_fullsized":"2024-09-11T01:17:58.971314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'12731","reported_seq":12869,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:06.340760+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:18:06.340760+0000","last_peered":"2024-09-11T01:18:06.340760+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:18:06.340760+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29577833,"num_objects":7222,"num_object_clones":0,"num_object_copies":14444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9801,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3305},{"shards":"0,2","objects":1168}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":476725889,"num_objects":116390,"num_object_clones":0,"num_object_copies":232780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38378,"num_objects_misplaced":1217,"num_objects_unfound":0,"num_objects_dirty":116390,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":165897,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8044,"num_bytes_recovered":32919552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1099018240,"data_stored":1098947474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160747,"ondisk_log_size":160747,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182479,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":768824,"kb_used_data":412328,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93603016,"statfs":{"total":96636764160,"available":95849488384,"internally_reserved":0,"allocated":422223872,"data_stored":421954213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":365031083},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,1,4,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986251,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":664740,"kb_used_data":362384,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93707100,"statfs":{"total":96636764160,"available":95956070400,"internally_reserved":0,"allocated":371081216,"data_stored":370811562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659196,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":607440,"kb_used_data":300928,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93764400,"statfs":{"total":96636764160,"available":96014745600,"internally_reserved":0,"allocated":308150272,"data_stored":307824356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":313847471},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,1,3,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":421363712,"data_stored":421360233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":307654656,"data_stored":307594920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":369999872,"data_stored":369992321,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:10.110 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:10.774 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:18:10.775 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:18:10.831 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":254,"stamp":"2024-09-11T01:18:10.683969+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":472724116,"num_objects":115415,"num_object_clones":0,"num_object_copies":230830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38381,"num_objects_misplaced":1109,"num_objects_unfound":0,"num_objects_dirty":115415,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":166876,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8150,"num_bytes_recovered":33349651,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160627,"ondisk_log_size":160627,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2043752,"kb_used_data":1070072,"kb_used_omap":12,"kb_used_meta":973619,"kb_avail":281071768,"statfs":{"total":289910292480,"available":287817490432,"internally_reserved":0,"allocated":1095753728,"data_stored":1094888499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":996985860},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,7,3,7,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20979712,"num_objects":-5122,"num_object_clones":0,"num_object_copies":-10244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1388,"num_objects_misplaced":1058,"num_objects_unfound":0,"num_objects_dirty":-5122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5122,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":330,"num_bytes_recovered":1343488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.758090"},"pg_stats":[{"pgid":"2.f","version":"47'12954","reported_seq":13036,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:06.392497+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:18:06.392497+0000","last_peered":"2024-09-11T01:18:06.392497+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:18:06.392497+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29949952,"num_objects":7312,"num_object_clones":0,"num_object_copies":14624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12954,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3240},{"shards":"0,2","objects":1251}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13144","reported_seq":13250,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:09.352728+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:18:09.352728+0000","last_peered":"2024-09-11T01:18:09.352728+0000","last_clean":"2024-09-11T01:18:09.352728+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:18:09.352728+0000","last_undegraded":"2024-09-11T01:18:09.352728+0000","last_fullsized":"2024-09-11T01:18:09.352728+0000","mapping_epoch":35,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29523968,"num_objects":7208,"num_object_clones":0,"num_object_copies":14416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11383,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'12900","reported_seq":13033,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:09.288968+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:18:09.288968+0000","last_peered":"2024-09-11T01:18:09.288968+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:18:09.288968+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29667352,"num_objects":7244,"num_object_clones":0,"num_object_copies":14488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7244,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11201,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1070},{"shards":"2","objects":3348}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'12925","reported_seq":12999,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:09.263913+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:18:09.263913+0000","last_peered":"2024-09-11T01:18:09.263913+0000","last_clean":"2024-09-11T01:18:09.263913+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:18:09.263913+0000","last_undegraded":"2024-09-11T01:18:09.263913+0000","last_fullsized":"2024-09-11T01:18:09.263913+0000","mapping_epoch":35,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29528064,"num_objects":7209,"num_object_clones":0,"num_object_copies":14418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12925,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'12793","reported_seq":12925,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:06.364552+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:18:06.364552+0000","last_peered":"2024-09-11T01:18:06.364552+0000","last_clean":"2024-09-11T01:18:06.364552+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:18:06.364552+0000","last_undegraded":"2024-09-11T01:18:06.364552+0000","last_fullsized":"2024-09-11T01:18:06.364552+0000","mapping_epoch":34,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29556736,"num_objects":7216,"num_object_clones":0,"num_object_copies":14432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9798,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'12939","reported_seq":13114,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:06.318450+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:18:06.318450+0000","last_peered":"2024-09-11T01:18:06.318450+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:18:06.318450+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29851648,"num_objects":7288,"num_object_clones":0,"num_object_copies":14576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7288,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9280,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3337},{"shards":"0,2","objects":1128}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'12719","reported_seq":12854,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:06.351068+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:18:06.351068+0000","last_peered":"2024-09-11T01:18:06.351068+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:18:06.351068+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29536256,"num_objects":7211,"num_object_clones":0,"num_object_copies":14422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7211,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9576,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3225},{"shards":"0,2","objects":1235}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'12925","reported_seq":13059,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:06.304904+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:18:06.304904+0000","last_peered":"2024-09-11T01:18:06.304904+0000","last_clean":"2024-09-11T01:18:06.304904+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:18:06.304904+0000","last_undegraded":"2024-09-11T01:18:06.304904+0000","last_fullsized":"2024-09-11T01:18:06.304904+0000","mapping_epoch":34,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30019584,"num_objects":7329,"num_object_clones":0,"num_object_copies":14658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9801,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934359+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934359+0000","last_peered":"2024-09-11T01:17:58.934359+0000","last_clean":"2024-09-11T01:17:58.934359+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934359+0000","last_undegraded":"2024-09-11T01:17:58.934359+0000","last_fullsized":"2024-09-11T01:17:58.934359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'12812","reported_seq":13503,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:09.411383+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:18:09.411383+0000","last_peered":"2024-09-11T01:18:09.411383+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:18:09.411383+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29331456,"num_objects":7161,"num_object_clones":0,"num_object_copies":14322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2547,"num_objects_misplaced":1109,"num_objects_unfound":0,"num_objects_dirty":7161,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10783,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":279,"num_bytes_recovered":1134592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1109},{"shards":"2","objects":3505}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938294+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938294+0000","last_peered":"2024-09-11T01:17:58.938294+0000","last_clean":"2024-09-11T01:17:58.938294+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938294+0000","last_undegraded":"2024-09-11T01:17:58.938294+0000","last_fullsized":"2024-09-11T01:17:58.938294+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'12703","reported_seq":12833,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:06.320450+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:18:06.320450+0000","last_peered":"2024-09-11T01:18:06.320450+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:18:06.320450+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29347840,"num_objects":7165,"num_object_clones":0,"num_object_copies":14330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9290,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3181},{"shards":"0,2","objects":1219}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928034+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928034+0000","last_peered":"2024-09-11T01:17:58.928034+0000","last_clean":"2024-09-11T01:17:58.928034+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928034+0000","last_undegraded":"2024-09-11T01:17:58.928034+0000","last_fullsized":"2024-09-11T01:17:58.928034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13046","reported_seq":13169,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:09.370340+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:18:09.370340+0000","last_peered":"2024-09-11T01:18:09.370340+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:18:09.370340+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29331456,"num_objects":7161,"num_object_clones":0,"num_object_copies":14322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7161,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10863,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1083},{"shards":"2","objects":3137}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935080+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935080+0000","last_peered":"2024-09-11T01:17:58.935080+0000","last_clean":"2024-09-11T01:17:58.935080+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935080+0000","last_undegraded":"2024-09-11T01:17:58.935080+0000","last_fullsized":"2024-09-11T01:17:58.935080+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'12763","reported_seq":20746,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:09.408589+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:18:09.408589+0000","last_peered":"2024-09-11T01:18:09.408589+0000","last_clean":"2024-09-11T01:18:09.408589+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:18:09.408589+0000","last_undegraded":"2024-09-11T01:18:09.408589+0000","last_fullsized":"2024-09-11T01:18:09.408589+0000","mapping_epoch":44,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28807168,"num_objects":7033,"num_object_clones":0,"num_object_copies":14066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10548,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'12906","reported_seq":13042,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:06.277743+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:18:06.277743+0000","last_peered":"2024-09-11T01:18:06.277743+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:18:06.277743+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30138368,"num_objects":7358,"num_object_clones":0,"num_object_copies":14716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9643,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3408},{"shards":"0,2","objects":1177}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938540+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938540+0000","last_peered":"2024-09-11T01:17:58.938540+0000","last_clean":"2024-09-11T01:17:58.938540+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938540+0000","last_undegraded":"2024-09-11T01:17:58.938540+0000","last_fullsized":"2024-09-11T01:17:58.938540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'12816","reported_seq":20852,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:10.231190+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:18:10.231190+0000","last_peered":"2024-09-11T01:18:10.231190+0000","last_clean":"2024-09-11T01:18:10.231190+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:18:10.231190+0000","last_undegraded":"2024-09-11T01:18:10.231190+0000","last_fullsized":"2024-09-11T01:18:10.231190+0000","mapping_epoch":46,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28950528,"num_objects":7068,"num_object_clones":0,"num_object_copies":14136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9471,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941377+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941377+0000","last_peered":"2024-09-11T01:17:58.941377+0000","last_clean":"2024-09-11T01:17:58.941377+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941377+0000","last_undegraded":"2024-09-11T01:17:58.941377+0000","last_fullsized":"2024-09-11T01:17:58.941377+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'12948","reported_seq":13082,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:06.405925+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:18:06.405925+0000","last_peered":"2024-09-11T01:18:06.405925+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:18:06.405925+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29605888,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9557,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3189},{"shards":"0,2","objects":1181}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971314+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971314+0000","last_peered":"2024-09-11T01:17:58.971314+0000","last_clean":"2024-09-11T01:17:58.971314+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971314+0000","last_undegraded":"2024-09-11T01:17:58.971314+0000","last_fullsized":"2024-09-11T01:17:58.971314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'12731","reported_seq":12869,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:06.340760+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:18:06.340760+0000","last_peered":"2024-09-11T01:18:06.340760+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:18:06.340760+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29577833,"num_objects":7222,"num_object_clones":0,"num_object_copies":14444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9801,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3305},{"shards":"0,2","objects":1168}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":472724097,"num_objects":115413,"num_object_clones":0,"num_object_copies":230826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38381,"num_objects_misplaced":1109,"num_objects_unfound":0,"num_objects_dirty":115413,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":166874,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8149,"num_bytes_recovered":33349632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1093283840,"data_stored":1093213074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160624,"ondisk_log_size":160624,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182479,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":768824,"kb_used_data":412328,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93603016,"statfs":{"total":96636764160,"available":95849488384,"internally_reserved":0,"allocated":422223872,"data_stored":421954213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":365031083},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,1,4,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986252,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":664456,"kb_used_data":357944,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93707384,"statfs":{"total":96636764160,"available":95956361216,"internally_reserved":0,"allocated":366534656,"data_stored":366265002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659197,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610472,"kb_used_data":299800,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93761368,"statfs":{"total":96636764160,"available":96011640832,"internally_reserved":0,"allocated":306995200,"data_stored":306669284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":318107311},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,2,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":421363712,"data_stored":421360233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":306491392,"data_stored":306431656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":365428736,"data_stored":365421185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:10.833 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:11.569 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:18:11.577 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:18:11.633 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":254,"stamp":"2024-09-11T01:18:10.683969+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":472724116,"num_objects":115415,"num_object_clones":0,"num_object_copies":230830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38381,"num_objects_misplaced":1109,"num_objects_unfound":0,"num_objects_dirty":115415,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":166876,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8150,"num_bytes_recovered":33349651,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160627,"ondisk_log_size":160627,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2043752,"kb_used_data":1070072,"kb_used_omap":12,"kb_used_meta":973619,"kb_avail":281071768,"statfs":{"total":289910292480,"available":287817490432,"internally_reserved":0,"allocated":1095753728,"data_stored":1094888499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":996985860},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,7,3,7,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20979712,"num_objects":-5122,"num_object_clones":0,"num_object_copies":-10244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1388,"num_objects_misplaced":1058,"num_objects_unfound":0,"num_objects_dirty":-5122,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5122,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":330,"num_bytes_recovered":1343488,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.758090"},"pg_stats":[{"pgid":"2.f","version":"47'12954","reported_seq":13036,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:06.392497+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:18:06.392497+0000","last_peered":"2024-09-11T01:18:06.392497+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:18:06.392497+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29949952,"num_objects":7312,"num_object_clones":0,"num_object_copies":14624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12954,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3240},{"shards":"0,2","objects":1251}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13144","reported_seq":13250,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:09.352728+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:18:09.352728+0000","last_peered":"2024-09-11T01:18:09.352728+0000","last_clean":"2024-09-11T01:18:09.352728+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:18:09.352728+0000","last_undegraded":"2024-09-11T01:18:09.352728+0000","last_fullsized":"2024-09-11T01:18:09.352728+0000","mapping_epoch":35,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29523968,"num_objects":7208,"num_object_clones":0,"num_object_copies":14416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11383,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'12900","reported_seq":13033,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:09.288968+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:18:09.288968+0000","last_peered":"2024-09-11T01:18:09.288968+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:18:09.288968+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29667352,"num_objects":7244,"num_object_clones":0,"num_object_copies":14488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7244,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11201,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1070},{"shards":"2","objects":3348}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'12925","reported_seq":12999,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:09.263913+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:18:09.263913+0000","last_peered":"2024-09-11T01:18:09.263913+0000","last_clean":"2024-09-11T01:18:09.263913+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:18:09.263913+0000","last_undegraded":"2024-09-11T01:18:09.263913+0000","last_fullsized":"2024-09-11T01:18:09.263913+0000","mapping_epoch":35,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29528064,"num_objects":7209,"num_object_clones":0,"num_object_copies":14418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12925,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'12793","reported_seq":12925,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:06.364552+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:18:06.364552+0000","last_peered":"2024-09-11T01:18:06.364552+0000","last_clean":"2024-09-11T01:18:06.364552+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:18:06.364552+0000","last_undegraded":"2024-09-11T01:18:06.364552+0000","last_fullsized":"2024-09-11T01:18:06.364552+0000","mapping_epoch":34,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29556736,"num_objects":7216,"num_object_clones":0,"num_object_copies":14432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7216,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9798,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'12939","reported_seq":13114,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:06.318450+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:18:06.318450+0000","last_peered":"2024-09-11T01:18:06.318450+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:18:06.318450+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29851648,"num_objects":7288,"num_object_clones":0,"num_object_copies":14576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7288,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9280,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3337},{"shards":"0,2","objects":1128}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'12719","reported_seq":12854,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:06.351068+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:18:06.351068+0000","last_peered":"2024-09-11T01:18:06.351068+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:18:06.351068+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29536256,"num_objects":7211,"num_object_clones":0,"num_object_copies":14422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7211,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9576,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3225},{"shards":"0,2","objects":1235}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'12925","reported_seq":13059,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:06.304904+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:18:06.304904+0000","last_peered":"2024-09-11T01:18:06.304904+0000","last_clean":"2024-09-11T01:18:06.304904+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:18:06.304904+0000","last_undegraded":"2024-09-11T01:18:06.304904+0000","last_fullsized":"2024-09-11T01:18:06.304904+0000","mapping_epoch":34,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30019584,"num_objects":7329,"num_object_clones":0,"num_object_copies":14658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7329,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9801,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934359+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934359+0000","last_peered":"2024-09-11T01:17:58.934359+0000","last_clean":"2024-09-11T01:17:58.934359+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934359+0000","last_undegraded":"2024-09-11T01:17:58.934359+0000","last_fullsized":"2024-09-11T01:17:58.934359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'12812","reported_seq":13503,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:09.411383+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:18:09.411383+0000","last_peered":"2024-09-11T01:18:09.411383+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:18:09.411383+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29331456,"num_objects":7161,"num_object_clones":0,"num_object_copies":14322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2547,"num_objects_misplaced":1109,"num_objects_unfound":0,"num_objects_dirty":7161,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10783,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":279,"num_bytes_recovered":1134592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1109},{"shards":"2","objects":3505}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938294+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938294+0000","last_peered":"2024-09-11T01:17:58.938294+0000","last_clean":"2024-09-11T01:17:58.938294+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938294+0000","last_undegraded":"2024-09-11T01:17:58.938294+0000","last_fullsized":"2024-09-11T01:17:58.938294+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'12703","reported_seq":12833,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:06.320450+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:18:06.320450+0000","last_peered":"2024-09-11T01:18:06.320450+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:18:06.320450+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29347840,"num_objects":7165,"num_object_clones":0,"num_object_copies":14330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7165,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9290,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3181},{"shards":"0,2","objects":1219}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":105,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928034+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928034+0000","last_peered":"2024-09-11T01:17:58.928034+0000","last_clean":"2024-09-11T01:17:58.928034+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928034+0000","last_undegraded":"2024-09-11T01:17:58.928034+0000","last_fullsized":"2024-09-11T01:17:58.928034+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13046","reported_seq":13169,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:09.370340+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:18:09.370340+0000","last_peered":"2024-09-11T01:18:09.370340+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:18:09.370340+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29331456,"num_objects":7161,"num_object_clones":0,"num_object_copies":14322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7161,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10863,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1083},{"shards":"2","objects":3137}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935080+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935080+0000","last_peered":"2024-09-11T01:17:58.935080+0000","last_clean":"2024-09-11T01:17:58.935080+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935080+0000","last_undegraded":"2024-09-11T01:17:58.935080+0000","last_fullsized":"2024-09-11T01:17:58.935080+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'12763","reported_seq":20746,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:09.408589+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:18:09.408589+0000","last_peered":"2024-09-11T01:18:09.408589+0000","last_clean":"2024-09-11T01:18:09.408589+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:18:09.408589+0000","last_undegraded":"2024-09-11T01:18:09.408589+0000","last_fullsized":"2024-09-11T01:18:09.408589+0000","mapping_epoch":44,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28807168,"num_objects":7033,"num_object_clones":0,"num_object_copies":14066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10548,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'12906","reported_seq":13042,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:06.277743+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:18:06.277743+0000","last_peered":"2024-09-11T01:18:06.277743+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:18:06.277743+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":30138368,"num_objects":7358,"num_object_clones":0,"num_object_copies":14716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9643,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3408},{"shards":"0,2","objects":1177}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938540+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938540+0000","last_peered":"2024-09-11T01:17:58.938540+0000","last_clean":"2024-09-11T01:17:58.938540+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938540+0000","last_undegraded":"2024-09-11T01:17:58.938540+0000","last_fullsized":"2024-09-11T01:17:58.938540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'12816","reported_seq":20852,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:10.231190+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:18:10.231190+0000","last_peered":"2024-09-11T01:18:10.231190+0000","last_clean":"2024-09-11T01:18:10.231190+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:18:10.231190+0000","last_undegraded":"2024-09-11T01:18:10.231190+0000","last_fullsized":"2024-09-11T01:18:10.231190+0000","mapping_epoch":46,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28950528,"num_objects":7068,"num_object_clones":0,"num_object_copies":14136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9471,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941377+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941377+0000","last_peered":"2024-09-11T01:17:58.941377+0000","last_clean":"2024-09-11T01:17:58.941377+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941377+0000","last_undegraded":"2024-09-11T01:17:58.941377+0000","last_fullsized":"2024-09-11T01:17:58.941377+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'12948","reported_seq":13082,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:06.405925+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:18:06.405925+0000","last_peered":"2024-09-11T01:18:06.405925+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:18:06.405925+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29605888,"num_objects":7228,"num_object_clones":0,"num_object_copies":14456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9557,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3189},{"shards":"0,2","objects":1181}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971314+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971314+0000","last_peered":"2024-09-11T01:17:58.971314+0000","last_clean":"2024-09-11T01:17:58.971314+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971314+0000","last_undegraded":"2024-09-11T01:17:58.971314+0000","last_fullsized":"2024-09-11T01:17:58.971314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'12731","reported_seq":12869,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:06.340760+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:18:06.340760+0000","last_peered":"2024-09-11T01:18:06.340760+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:18:06.340760+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29577833,"num_objects":7222,"num_object_clones":0,"num_object_copies":14444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9801,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3305},{"shards":"0,2","objects":1168}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":472724097,"num_objects":115413,"num_object_clones":0,"num_object_copies":230826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38381,"num_objects_misplaced":1109,"num_objects_unfound":0,"num_objects_dirty":115413,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":166874,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8149,"num_bytes_recovered":33349632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1093283840,"data_stored":1093213074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160624,"ondisk_log_size":160624,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182479,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":768824,"kb_used_data":412328,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93603016,"statfs":{"total":96636764160,"available":95849488384,"internally_reserved":0,"allocated":422223872,"data_stored":421954213,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":365031083},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,1,4,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986252,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":664456,"kb_used_data":357944,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93707384,"statfs":{"total":96636764160,"available":95956361216,"internally_reserved":0,"allocated":366534656,"data_stored":366265002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659197,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610472,"kb_used_data":299800,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93761368,"statfs":{"total":96636764160,"available":96011640832,"internally_reserved":0,"allocated":306995200,"data_stored":306669284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":318107311},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,2,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":421363712,"data_stored":421360233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":306491392,"data_stored":306431656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":365428736,"data_stored":365421185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:11.634 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:18:11.634 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'12954", 'reported_seq': 13036, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:06.392497+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:18:06.392497+0000', 'last_peered': '2024-09-11T01:18:06.392497+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:18:06.392497+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29949952, 'num_objects': 7312, 'num_object_clones': 0, 'num_object_copies': 14624, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7312, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12954, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3240}, {'shards': '0,2', 'objects': 1251}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:11.634 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:18:11.634 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'12900", 'reported_seq': 13033, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:09.288968+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:18:09.288968+0000', 'last_peered': '2024-09-11T01:18:09.288968+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:18:09.288968+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "36'2800", 'ondisk_log_start': "36'2800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29667352, 'num_objects': 7244, 'num_object_clones': 0, 'num_object_copies': 14488, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7244, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11201, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1070}, {'shards': '2', 'objects': 3348}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:11.634 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:18:11.634 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'12939", 'reported_seq': 13114, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:06.318450+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:18:06.318450+0000', 'last_peered': '2024-09-11T01:18:06.318450+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:18:06.318450+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29851648, 'num_objects': 7288, 'num_object_clones': 0, 'num_object_copies': 14576, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7288, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9280, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3337}, {'shards': '0,2', 'objects': 1128}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:11.635 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:18:11.635 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'12719", 'reported_seq': 12854, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:06.351068+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:18:06.351068+0000', 'last_peered': '2024-09-11T01:18:06.351068+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:18:06.351068+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "36'2700", 'ondisk_log_start': "36'2700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29536256, 'num_objects': 7211, 'num_object_clones': 0, 'num_object_copies': 14422, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7211, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9576, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3225}, {'shards': '0,2', 'objects': 1235}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:11.635 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:18:11.635 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'12812", 'reported_seq': 13503, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:09.411383+0000', 'last_change': '2024-09-11T01:17:57.355893+0000', 'last_active': '2024-09-11T01:18:09.411383+0000', 'last_peered': '2024-09-11T01:18:09.411383+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:18:09.411383+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "36'2800", 'ondisk_log_start': "36'2800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29331456, 'num_objects': 7161, 'num_object_clones': 0, 'num_object_copies': 14322, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2547, 'num_objects_misplaced': 1109, 'num_objects_unfound': 0, 'num_objects_dirty': 7161, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10783, 'num_write_kb': 31844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 279, 'num_bytes_recovered': 1134592, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1109}, {'shards': '2', 'objects': 3505}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:11.635 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:18:11.635 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'12703", 'reported_seq': 12833, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:06.320450+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:18:06.320450+0000', 'last_peered': '2024-09-11T01:18:06.320450+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:18:06.320450+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "36'2700", 'ondisk_log_start': "36'2700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29347840, 'num_objects': 7165, 'num_object_clones': 0, 'num_object_copies': 14330, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7165, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9290, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3181}, {'shards': '0,2', 'objects': 1219}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:11.635 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:18:11.635 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'13046", 'reported_seq': 13169, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:09.370340+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:18:09.370340+0000', 'last_peered': '2024-09-11T01:18:09.370340+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:18:09.370340+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29331456, 'num_objects': 7161, 'num_object_clones': 0, 'num_object_copies': 14322, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7161, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10863, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 1083}, {'shards': '2', 'objects': 3137}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:11.635 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:18:11.635 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'12906", 'reported_seq': 13042, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:06.277743+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:18:06.277743+0000', 'last_peered': '2024-09-11T01:18:06.277743+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:18:06.277743+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 30138368, 'num_objects': 7358, 'num_object_clones': 0, 'num_object_copies': 14716, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7358, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9643, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3408}, {'shards': '0,2', 'objects': 1177}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:11.636 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:18:11.636 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'12948", 'reported_seq': 13082, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:06.405925+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:18:06.405925+0000', 'last_peered': '2024-09-11T01:18:06.405925+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:18:06.405925+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29605888, 'num_objects': 7228, 'num_object_clones': 0, 'num_object_copies': 14456, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7228, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9557, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3189}, {'shards': '0,2', 'objects': 1181}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:11.636 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:18:11.636 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'12731", 'reported_seq': 12869, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:06.340760+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:18:06.340760+0000', 'last_peered': '2024-09-11T01:18:06.340760+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:18:06.340760+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "36'2700", 'ondisk_log_start': "36'2700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29577833, 'num_objects': 7222, 'num_object_clones': 0, 'num_object_copies': 14444, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7222, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9801, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3305}, {'shards': '0,2', 'objects': 1168}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:11.636 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:18:12.625 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:18:12.625 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38381/230830 objects degraded (16.627%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":429,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":115415,"data_bytes":472724116,"bytes_used":2092802048,"bytes_avail":287817490432,"bytes_total":289910292480,"degraded_objects":38381,"degraded_total":230830,"degraded_ratio":0.16627388121128103,"misplaced_objects":1109,"misplaced_total":230830,"misplaced_ratio":0.0048044015076029976,"recovering_objects_per_sec":28,"recovering_bytes_per_sec":114260,"recovering_keys_per_sec":0,"num_objects_recovered":330,"num_bytes_recovered":1343488,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":435},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:18:12.709 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:18:12.709 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:13.456 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:18:13.458 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:18:13.523 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":255,"stamp":"2024-09-11T01:18:12.684469+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":467493524,"num_objects":114138,"num_object_clones":0,"num_object_copies":228276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38381,"num_objects_misplaced":1109,"num_objects_unfound":0,"num_objects_dirty":114138,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":168153,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8150,"num_bytes_recovered":33349651,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160904,"ondisk_log_size":160904,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2059284,"kb_used_data":1063396,"kb_used_omap":12,"kb_used_meta":995827,"kb_avail":281056236,"statfs":{"total":289910292480,"available":287801585664,"internally_reserved":0,"allocated":1088917504,"data_stored":1088052275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1019726852},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,0,5,2,11,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22974464,"num_objects":-5609,"num_object_clones":0,"num_object_copies":-11218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3,"num_objects_misplaced":-198,"num_objects_unfound":0,"num_objects_dirty":-5609,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5609,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":195,"num_bytes_recovered":798720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002709"},"pg_stats":[{"pgid":"2.f","version":"47'13097","reported_seq":13179,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:11.407542+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:18:11.407542+0000","last_peered":"2024-09-11T01:18:11.407542+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:18:11.407542+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29364224,"num_objects":7169,"num_object_clones":0,"num_object_copies":14338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13097,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3097},{"shards":"0,2","objects":1108}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13144","reported_seq":13250,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:09.352728+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:18:09.352728+0000","last_peered":"2024-09-11T01:18:09.352728+0000","last_clean":"2024-09-11T01:18:09.352728+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:18:09.352728+0000","last_undegraded":"2024-09-11T01:18:09.352728+0000","last_fullsized":"2024-09-11T01:18:09.352728+0000","mapping_epoch":35,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29523968,"num_objects":7208,"num_object_clones":0,"num_object_copies":14416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11383,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'12900","reported_seq":13033,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:09.288968+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:18:09.288968+0000","last_peered":"2024-09-11T01:18:09.288968+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:18:09.288968+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29667352,"num_objects":7244,"num_object_clones":0,"num_object_copies":14488,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7244,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11201,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1070},{"shards":"2","objects":3348}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'12925","reported_seq":12999,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:09.263913+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:18:09.263913+0000","last_peered":"2024-09-11T01:18:09.263913+0000","last_clean":"2024-09-11T01:18:09.263913+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:18:09.263913+0000","last_undegraded":"2024-09-11T01:18:09.263913+0000","last_fullsized":"2024-09-11T01:18:09.263913+0000","mapping_epoch":35,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29528064,"num_objects":7209,"num_object_clones":0,"num_object_copies":14418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12925,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'12924","reported_seq":13055,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:11.388587+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:18:11.388587+0000","last_peered":"2024-09-11T01:18:11.388587+0000","last_clean":"2024-09-11T01:18:11.388587+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:18:11.388587+0000","last_undegraded":"2024-09-11T01:18:11.388587+0000","last_fullsized":"2024-09-11T01:18:11.388587+0000","mapping_epoch":34,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29020160,"num_objects":7085,"num_object_clones":0,"num_object_copies":14170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9929,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13083","reported_seq":13258,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:11.452401+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:18:11.452401+0000","last_peered":"2024-09-11T01:18:11.452401+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:18:11.452401+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29261824,"num_objects":7144,"num_object_clones":0,"num_object_copies":14288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7144,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9424,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3193},{"shards":"0,2","objects":984}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'12874","reported_seq":13009,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:11.428598+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:18:11.428598+0000","last_peered":"2024-09-11T01:18:11.428598+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:18:11.428598+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28901376,"num_objects":7056,"num_object_clones":0,"num_object_copies":14112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7056,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9731,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3070},{"shards":"0,2","objects":1080}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13077","reported_seq":13211,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:11.379563+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:18:11.379563+0000","last_peered":"2024-09-11T01:18:11.379563+0000","last_clean":"2024-09-11T01:18:11.379563+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:18:11.379563+0000","last_undegraded":"2024-09-11T01:18:11.379563+0000","last_fullsized":"2024-09-11T01:18:11.379563+0000","mapping_epoch":34,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29396992,"num_objects":7177,"num_object_clones":0,"num_object_copies":14354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9953,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934359+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934359+0000","last_peered":"2024-09-11T01:17:58.934359+0000","last_clean":"2024-09-11T01:17:58.934359+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934359+0000","last_undegraded":"2024-09-11T01:17:58.934359+0000","last_fullsized":"2024-09-11T01:17:58.934359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'12812","reported_seq":13503,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:09.411383+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:18:09.411383+0000","last_peered":"2024-09-11T01:18:09.411383+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:18:09.411383+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29331456,"num_objects":7161,"num_object_clones":0,"num_object_copies":14322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2547,"num_objects_misplaced":1109,"num_objects_unfound":0,"num_objects_dirty":7161,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10783,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":279,"num_bytes_recovered":1134592,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1109},{"shards":"2","objects":3505}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938294+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938294+0000","last_peered":"2024-09-11T01:17:58.938294+0000","last_clean":"2024-09-11T01:17:58.938294+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938294+0000","last_undegraded":"2024-09-11T01:17:58.938294+0000","last_fullsized":"2024-09-11T01:17:58.938294+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'12834","reported_seq":12966,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:11.451445+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:18:11.451445+0000","last_peered":"2024-09-11T01:18:11.451445+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:18:11.451445+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28811264,"num_objects":7034,"num_object_clones":0,"num_object_copies":14068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7034,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9421,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3050},{"shards":"0,2","objects":1088}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13046","reported_seq":13169,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:09.370340+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:18:09.370340+0000","last_peered":"2024-09-11T01:18:09.370340+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:18:09.370340+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29331456,"num_objects":7161,"num_object_clones":0,"num_object_copies":14322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7161,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10863,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":1083},{"shards":"2","objects":3137}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935080+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935080+0000","last_peered":"2024-09-11T01:17:58.935080+0000","last_clean":"2024-09-11T01:17:58.935080+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935080+0000","last_undegraded":"2024-09-11T01:17:58.935080+0000","last_fullsized":"2024-09-11T01:17:58.935080+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'12763","reported_seq":20746,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:09.408589+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:18:09.408589+0000","last_peered":"2024-09-11T01:18:09.408589+0000","last_clean":"2024-09-11T01:18:09.408589+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:18:09.408589+0000","last_undegraded":"2024-09-11T01:18:09.408589+0000","last_fullsized":"2024-09-11T01:18:09.408589+0000","mapping_epoch":44,"log_start":"36'2700","ondisk_log_start":"36'2700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28807168,"num_objects":7033,"num_object_clones":0,"num_object_copies":14066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10548,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13042","reported_seq":13178,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:11.464619+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:18:11.464619+0000","last_peered":"2024-09-11T01:18:11.464619+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:18:11.464619+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29581312,"num_objects":7222,"num_object_clones":0,"num_object_copies":14444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9779,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3272},{"shards":"0,2","objects":1041}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938540+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938540+0000","last_peered":"2024-09-11T01:17:58.938540+0000","last_clean":"2024-09-11T01:17:58.938540+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938540+0000","last_undegraded":"2024-09-11T01:17:58.938540+0000","last_fullsized":"2024-09-11T01:17:58.938540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'12816","reported_seq":20852,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:10.231190+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:18:10.231190+0000","last_peered":"2024-09-11T01:18:10.231190+0000","last_clean":"2024-09-11T01:18:10.231190+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:18:10.231190+0000","last_undegraded":"2024-09-11T01:18:10.231190+0000","last_fullsized":"2024-09-11T01:18:10.231190+0000","mapping_epoch":46,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28950528,"num_objects":7068,"num_object_clones":0,"num_object_copies":14136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7068,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9471,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941377+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941377+0000","last_peered":"2024-09-11T01:17:58.941377+0000","last_clean":"2024-09-11T01:17:58.941377+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941377+0000","last_undegraded":"2024-09-11T01:17:58.941377+0000","last_fullsized":"2024-09-11T01:17:58.941377+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13075","reported_seq":13213,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:11.465238+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:18:11.465238+0000","last_peered":"2024-09-11T01:18:11.465238+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:18:11.465238+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29085696,"num_objects":7101,"num_object_clones":0,"num_object_copies":14202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7101,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9684,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3062},{"shards":"0,2","objects":1054}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971314+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971314+0000","last_peered":"2024-09-11T01:17:58.971314+0000","last_clean":"2024-09-11T01:17:58.971314+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971314+0000","last_undegraded":"2024-09-11T01:17:58.971314+0000","last_fullsized":"2024-09-11T01:17:58.971314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'12889","reported_seq":13024,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:11.439236+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:18:11.439236+0000","last_peered":"2024-09-11T01:18:11.439236+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:18:11.439236+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28930665,"num_objects":7064,"num_object_clones":0,"num_object_copies":14128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7064,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9959,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3147},{"shards":"0,2","objects":1010}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":467493505,"num_objects":114136,"num_object_clones":0,"num_object_copies":228272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38381,"num_objects_misplaced":1109,"num_objects_unfound":0,"num_objects_dirty":114136,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":168151,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8149,"num_bytes_recovered":33349632,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1086304256,"data_stored":1086233490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160901,"ondisk_log_size":160901,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182480,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":784356,"kb_used_data":405652,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93587484,"statfs":{"total":96636764160,"available":95833583616,"internally_reserved":0,"allocated":415387648,"data_stored":415117989,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":387772075},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,3,0,8,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986252,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":664456,"kb_used_data":357944,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93707384,"statfs":{"total":96636764160,"available":95956361216,"internally_reserved":0,"allocated":366534656,"data_stored":366265002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659197,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610472,"kb_used_data":299800,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93761368,"statfs":{"total":96636764160,"available":96011640832,"internally_reserved":0,"allocated":306995200,"data_stored":306669284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":318107311},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,2,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":414384128,"data_stored":414380649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":306491392,"data_stored":306431656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":365428736,"data_stored":365421185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:16.525 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:17.249 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:18:17.249 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:18:17.327 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":257,"stamp":"2024-09-11T01:18:16.686605+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":458412692,"num_objects":111921,"num_object_clones":0,"num_object_copies":223842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38428,"num_objects_misplaced":972,"num_objects_unfound":0,"num_objects_dirty":111921,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":170370,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8240,"num_bytes_recovered":33718291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160821,"ondisk_log_size":160821,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2047216,"kb_used_data":1051328,"kb_used_omap":12,"kb_used_meta":995827,"kb_avail":281068304,"statfs":{"total":289910292480,"available":287813943296,"internally_reserved":0,"allocated":1076559872,"data_stored":1075694643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1019726852},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,6,10,8,5,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24027136,"num_objects":-5866,"num_object_clones":0,"num_object_copies":-11732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":50,"num_objects_misplaced":-245,"num_objects_unfound":0,"num_objects_dirty":-5866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5866,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":195,"num_bytes_recovered":798720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003897"},"pg_stats":[{"pgid":"2.f","version":"47'13235","reported_seq":13317,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:16.418510+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:18:16.418510+0000","last_peered":"2024-09-11T01:18:16.418510+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:18:16.418510+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28798976,"num_objects":7031,"num_object_clones":0,"num_object_copies":14062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13235,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2959},{"shards":"0,2","objects":970}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13291","reported_seq":13397,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:14.413798+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:18:14.413798+0000","last_peered":"2024-09-11T01:18:14.413798+0000","last_clean":"2024-09-11T01:18:14.413798+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:18:14.413798+0000","last_undegraded":"2024-09-11T01:18:14.413798+0000","last_fullsized":"2024-09-11T01:18:14.413798+0000","mapping_epoch":35,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28921856,"num_objects":7061,"num_object_clones":0,"num_object_copies":14122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7061,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11530,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13018","reported_seq":13151,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:14.386686+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:18:14.386686+0000","last_peered":"2024-09-11T01:18:14.386686+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:18:14.386686+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29184024,"num_objects":7126,"num_object_clones":0,"num_object_copies":14252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7126,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11319,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":952},{"shards":"2","objects":3230}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'13065","reported_seq":13137,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:14.409663+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:18:14.409663+0000","last_peered":"2024-09-11T01:18:14.409663+0000","last_clean":"2024-09-11T01:18:14.409663+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:18:14.409663+0000","last_undegraded":"2024-09-11T01:18:14.409663+0000","last_fullsized":"2024-09-11T01:18:14.409663+0000","mapping_epoch":35,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28954624,"num_objects":7069,"num_object_clones":0,"num_object_copies":14138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13065,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13049","reported_seq":13181,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:16.456062+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:18:16.456062+0000","last_peered":"2024-09-11T01:18:16.456062+0000","last_clean":"2024-09-11T01:18:16.456062+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:18:16.456062+0000","last_undegraded":"2024-09-11T01:18:16.456062+0000","last_fullsized":"2024-09-11T01:18:16.456062+0000","mapping_epoch":34,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28508160,"num_objects":6960,"num_object_clones":0,"num_object_copies":13920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13227","reported_seq":13400,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:16.477832+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:18:16.477832+0000","last_peered":"2024-09-11T01:18:16.477832+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:18:16.477832+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28672000,"num_objects":7000,"num_object_clones":0,"num_object_copies":14000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3049},{"shards":"0,2","objects":840}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'13010","reported_seq":13145,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:16.342959+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:18:16.342959+0000","last_peered":"2024-09-11T01:18:16.342959+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:18:16.342959+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28344320,"num_objects":6920,"num_object_clones":0,"num_object_copies":13840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9867,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2934},{"shards":"0,2","objects":944}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13240","reported_seq":13374,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:16.449817+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:18:16.449817+0000","last_peered":"2024-09-11T01:18:16.449817+0000","last_clean":"2024-09-11T01:18:16.449817+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:18:16.449817+0000","last_undegraded":"2024-09-11T01:18:16.449817+0000","last_fullsized":"2024-09-11T01:18:16.449817+0000","mapping_epoch":34,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28729344,"num_objects":7014,"num_object_clones":0,"num_object_copies":14028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10116,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934359+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934359+0000","last_peered":"2024-09-11T01:17:58.934359+0000","last_clean":"2024-09-11T01:17:58.934359+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934359+0000","last_undegraded":"2024-09-11T01:17:58.934359+0000","last_fullsized":"2024-09-11T01:17:58.934359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'12950","reported_seq":13821,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:14.390395+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:18:14.390395+0000","last_peered":"2024-09-11T01:18:14.390395+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:18:14.390395+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28766208,"num_objects":7023,"num_object_clones":0,"num_object_copies":14046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2594,"num_objects_misplaced":972,"num_objects_unfound":0,"num_objects_dirty":7023,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10921,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":369,"num_bytes_recovered":1503232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":972},{"shards":"2","objects":3457}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938294+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938294+0000","last_peered":"2024-09-11T01:17:58.938294+0000","last_clean":"2024-09-11T01:17:58.938294+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938294+0000","last_undegraded":"2024-09-11T01:17:58.938294+0000","last_fullsized":"2024-09-11T01:17:58.938294+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'12995","reported_seq":13125,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:16.374476+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:18:16.374476+0000","last_peered":"2024-09-11T01:18:16.374476+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:18:16.374476+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28151808,"num_objects":6873,"num_object_clones":0,"num_object_copies":13746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9582,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2889},{"shards":"0,2","objects":927}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13168","reported_seq":13291,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:14.429140+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:18:14.429140+0000","last_peered":"2024-09-11T01:18:14.429140+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:18:14.429140+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28831744,"num_objects":7039,"num_object_clones":0,"num_object_copies":14078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10985,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":961},{"shards":"2","objects":3015}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935080+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935080+0000","last_peered":"2024-09-11T01:17:58.935080+0000","last_clean":"2024-09-11T01:17:58.935080+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935080+0000","last_undegraded":"2024-09-11T01:17:58.935080+0000","last_fullsized":"2024-09-11T01:17:58.935080+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'12892","reported_seq":20878,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:14.343284+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:18:14.343284+0000","last_peered":"2024-09-11T01:18:14.343284+0000","last_clean":"2024-09-11T01:18:14.343284+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:18:14.343284+0000","last_undegraded":"2024-09-11T01:18:14.343284+0000","last_fullsized":"2024-09-11T01:18:14.343284+0000","mapping_epoch":44,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28278784,"num_objects":6904,"num_object_clones":0,"num_object_copies":13808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10677,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13178","reported_seq":13314,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:16.361108+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:18:16.361108+0000","last_peered":"2024-09-11T01:18:16.361108+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:18:16.361108+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29024256,"num_objects":7086,"num_object_clones":0,"num_object_copies":14172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9915,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3136},{"shards":"0,2","objects":905}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938540+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938540+0000","last_peered":"2024-09-11T01:17:58.938540+0000","last_clean":"2024-09-11T01:17:58.938540+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938540+0000","last_undegraded":"2024-09-11T01:17:58.938540+0000","last_fullsized":"2024-09-11T01:17:58.938540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'12954","reported_seq":20989,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:15.326562+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:18:15.326562+0000","last_peered":"2024-09-11T01:18:15.326562+0000","last_clean":"2024-09-11T01:18:15.326562+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:18:15.326562+0000","last_undegraded":"2024-09-11T01:18:15.326562+0000","last_fullsized":"2024-09-11T01:18:15.326562+0000","mapping_epoch":46,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28385280,"num_objects":6930,"num_object_clones":0,"num_object_copies":13860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9609,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941377+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941377+0000","last_peered":"2024-09-11T01:17:58.941377+0000","last_clean":"2024-09-11T01:17:58.941377+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941377+0000","last_undegraded":"2024-09-11T01:17:58.941377+0000","last_fullsized":"2024-09-11T01:17:58.941377+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13215","reported_seq":13354,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:16.414044+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:18:16.414044+0000","last_peered":"2024-09-11T01:18:16.414044+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:18:16.414044+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28512256,"num_objects":6961,"num_object_clones":0,"num_object_copies":13922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9824,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2922},{"shards":"0,2","objects":914}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971314+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971314+0000","last_peered":"2024-09-11T01:17:58.971314+0000","last_clean":"2024-09-11T01:17:58.971314+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971314+0000","last_undegraded":"2024-09-11T01:17:58.971314+0000","last_fullsized":"2024-09-11T01:17:58.971314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13031","reported_seq":13168,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:16.466681+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:18:16.466681+0000","last_peered":"2024-09-11T01:18:16.466681+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:18:16.466681+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28349033,"num_objects":6922,"num_object_clones":0,"num_object_copies":13844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6922,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3005},{"shards":"0,2","objects":868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":458412673,"num_objects":111919,"num_object_clones":0,"num_object_copies":223838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38428,"num_objects_misplaced":972,"num_objects_unfound":0,"num_objects_dirty":111919,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":170368,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8239,"num_bytes_recovered":33718272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1073905664,"data_stored":1073834898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160818,"ondisk_log_size":160818,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182481,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":781760,"kb_used_data":398896,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93590080,"statfs":{"total":96636764160,"available":95836241920,"internally_reserved":0,"allocated":408469504,"data_stored":408199845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":392031915},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,3,5,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986253,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":664368,"kb_used_data":353696,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93707472,"statfs":{"total":96636764160,"available":95956451328,"internally_reserved":0,"allocated":362184704,"data_stored":361915050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,5,1,0,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659198,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":601088,"kb_used_data":298736,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93770752,"statfs":{"total":96636764160,"available":96021250048,"internally_reserved":0,"allocated":305905664,"data_stored":305579748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":309587631},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,2,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":407437312,"data_stored":407433833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":305410048,"data_stored":305350312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":361058304,"data_stored":361050753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:17.329 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:18.224 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:18:18.225 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:18:18.263 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":257,"stamp":"2024-09-11T01:18:16.686605+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":458412692,"num_objects":111921,"num_object_clones":0,"num_object_copies":223842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38428,"num_objects_misplaced":972,"num_objects_unfound":0,"num_objects_dirty":111921,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":170370,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8240,"num_bytes_recovered":33718291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160821,"ondisk_log_size":160821,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2047216,"kb_used_data":1051328,"kb_used_omap":12,"kb_used_meta":995827,"kb_avail":281068304,"statfs":{"total":289910292480,"available":287813943296,"internally_reserved":0,"allocated":1076559872,"data_stored":1075694643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1019726852},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,6,10,8,5,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24027136,"num_objects":-5866,"num_object_clones":0,"num_object_copies":-11732,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":50,"num_objects_misplaced":-245,"num_objects_unfound":0,"num_objects_dirty":-5866,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5866,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":195,"num_bytes_recovered":798720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003897"},"pg_stats":[{"pgid":"2.f","version":"47'13235","reported_seq":13317,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:16.418510+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:18:16.418510+0000","last_peered":"2024-09-11T01:18:16.418510+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:18:16.418510+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28798976,"num_objects":7031,"num_object_clones":0,"num_object_copies":14062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13235,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2959},{"shards":"0,2","objects":970}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13291","reported_seq":13397,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:14.413798+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:18:14.413798+0000","last_peered":"2024-09-11T01:18:14.413798+0000","last_clean":"2024-09-11T01:18:14.413798+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:18:14.413798+0000","last_undegraded":"2024-09-11T01:18:14.413798+0000","last_fullsized":"2024-09-11T01:18:14.413798+0000","mapping_epoch":35,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28921856,"num_objects":7061,"num_object_clones":0,"num_object_copies":14122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7061,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11530,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13018","reported_seq":13151,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:14.386686+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:18:14.386686+0000","last_peered":"2024-09-11T01:18:14.386686+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:18:14.386686+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29184024,"num_objects":7126,"num_object_clones":0,"num_object_copies":14252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7126,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11319,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":952},{"shards":"2","objects":3230}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'13065","reported_seq":13137,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:14.409663+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:18:14.409663+0000","last_peered":"2024-09-11T01:18:14.409663+0000","last_clean":"2024-09-11T01:18:14.409663+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:18:14.409663+0000","last_undegraded":"2024-09-11T01:18:14.409663+0000","last_fullsized":"2024-09-11T01:18:14.409663+0000","mapping_epoch":35,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28954624,"num_objects":7069,"num_object_clones":0,"num_object_copies":14138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13065,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13049","reported_seq":13181,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:16.456062+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:18:16.456062+0000","last_peered":"2024-09-11T01:18:16.456062+0000","last_clean":"2024-09-11T01:18:16.456062+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:18:16.456062+0000","last_undegraded":"2024-09-11T01:18:16.456062+0000","last_fullsized":"2024-09-11T01:18:16.456062+0000","mapping_epoch":34,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28508160,"num_objects":6960,"num_object_clones":0,"num_object_copies":13920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13227","reported_seq":13400,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:16.477832+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:18:16.477832+0000","last_peered":"2024-09-11T01:18:16.477832+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:18:16.477832+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28672000,"num_objects":7000,"num_object_clones":0,"num_object_copies":14000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3049},{"shards":"0,2","objects":840}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'13010","reported_seq":13145,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:16.342959+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:18:16.342959+0000","last_peered":"2024-09-11T01:18:16.342959+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:18:16.342959+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28344320,"num_objects":6920,"num_object_clones":0,"num_object_copies":13840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9867,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2934},{"shards":"0,2","objects":944}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13240","reported_seq":13374,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:16.449817+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:18:16.449817+0000","last_peered":"2024-09-11T01:18:16.449817+0000","last_clean":"2024-09-11T01:18:16.449817+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:18:16.449817+0000","last_undegraded":"2024-09-11T01:18:16.449817+0000","last_fullsized":"2024-09-11T01:18:16.449817+0000","mapping_epoch":34,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28729344,"num_objects":7014,"num_object_clones":0,"num_object_copies":14028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10116,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934359+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934359+0000","last_peered":"2024-09-11T01:17:58.934359+0000","last_clean":"2024-09-11T01:17:58.934359+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934359+0000","last_undegraded":"2024-09-11T01:17:58.934359+0000","last_fullsized":"2024-09-11T01:17:58.934359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'12950","reported_seq":13821,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:14.390395+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:18:14.390395+0000","last_peered":"2024-09-11T01:18:14.390395+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:18:14.390395+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28766208,"num_objects":7023,"num_object_clones":0,"num_object_copies":14046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2594,"num_objects_misplaced":972,"num_objects_unfound":0,"num_objects_dirty":7023,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10921,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":369,"num_bytes_recovered":1503232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":972},{"shards":"2","objects":3457}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938294+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938294+0000","last_peered":"2024-09-11T01:17:58.938294+0000","last_clean":"2024-09-11T01:17:58.938294+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938294+0000","last_undegraded":"2024-09-11T01:17:58.938294+0000","last_fullsized":"2024-09-11T01:17:58.938294+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'12995","reported_seq":13125,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:16.374476+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:18:16.374476+0000","last_peered":"2024-09-11T01:18:16.374476+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:18:16.374476+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28151808,"num_objects":6873,"num_object_clones":0,"num_object_copies":13746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9582,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2889},{"shards":"0,2","objects":927}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13168","reported_seq":13291,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:14.429140+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:18:14.429140+0000","last_peered":"2024-09-11T01:18:14.429140+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:18:14.429140+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28831744,"num_objects":7039,"num_object_clones":0,"num_object_copies":14078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10985,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":961},{"shards":"2","objects":3015}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935080+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935080+0000","last_peered":"2024-09-11T01:17:58.935080+0000","last_clean":"2024-09-11T01:17:58.935080+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935080+0000","last_undegraded":"2024-09-11T01:17:58.935080+0000","last_fullsized":"2024-09-11T01:17:58.935080+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'12892","reported_seq":20878,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:14.343284+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:18:14.343284+0000","last_peered":"2024-09-11T01:18:14.343284+0000","last_clean":"2024-09-11T01:18:14.343284+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:18:14.343284+0000","last_undegraded":"2024-09-11T01:18:14.343284+0000","last_fullsized":"2024-09-11T01:18:14.343284+0000","mapping_epoch":44,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28278784,"num_objects":6904,"num_object_clones":0,"num_object_copies":13808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10677,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13178","reported_seq":13314,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:16.361108+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:18:16.361108+0000","last_peered":"2024-09-11T01:18:16.361108+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:18:16.361108+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29024256,"num_objects":7086,"num_object_clones":0,"num_object_copies":14172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9915,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3136},{"shards":"0,2","objects":905}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938540+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938540+0000","last_peered":"2024-09-11T01:17:58.938540+0000","last_clean":"2024-09-11T01:17:58.938540+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938540+0000","last_undegraded":"2024-09-11T01:17:58.938540+0000","last_fullsized":"2024-09-11T01:17:58.938540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'12954","reported_seq":20989,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:15.326562+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:18:15.326562+0000","last_peered":"2024-09-11T01:18:15.326562+0000","last_clean":"2024-09-11T01:18:15.326562+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:18:15.326562+0000","last_undegraded":"2024-09-11T01:18:15.326562+0000","last_fullsized":"2024-09-11T01:18:15.326562+0000","mapping_epoch":46,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28385280,"num_objects":6930,"num_object_clones":0,"num_object_copies":13860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9609,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941377+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941377+0000","last_peered":"2024-09-11T01:17:58.941377+0000","last_clean":"2024-09-11T01:17:58.941377+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941377+0000","last_undegraded":"2024-09-11T01:17:58.941377+0000","last_fullsized":"2024-09-11T01:17:58.941377+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13215","reported_seq":13354,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:16.414044+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:18:16.414044+0000","last_peered":"2024-09-11T01:18:16.414044+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:18:16.414044+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28512256,"num_objects":6961,"num_object_clones":0,"num_object_copies":13922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9824,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2922},{"shards":"0,2","objects":914}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971314+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971314+0000","last_peered":"2024-09-11T01:17:58.971314+0000","last_clean":"2024-09-11T01:17:58.971314+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971314+0000","last_undegraded":"2024-09-11T01:17:58.971314+0000","last_fullsized":"2024-09-11T01:17:58.971314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13031","reported_seq":13168,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:16.466681+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:18:16.466681+0000","last_peered":"2024-09-11T01:18:16.466681+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:18:16.466681+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28349033,"num_objects":6922,"num_object_clones":0,"num_object_copies":13844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6922,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3005},{"shards":"0,2","objects":868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":458412673,"num_objects":111919,"num_object_clones":0,"num_object_copies":223838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38428,"num_objects_misplaced":972,"num_objects_unfound":0,"num_objects_dirty":111919,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":170368,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8239,"num_bytes_recovered":33718272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1073905664,"data_stored":1073834898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160818,"ondisk_log_size":160818,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182481,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":781760,"kb_used_data":398896,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93590080,"statfs":{"total":96636764160,"available":95836241920,"internally_reserved":0,"allocated":408469504,"data_stored":408199845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":392031915},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,3,5,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986253,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":664368,"kb_used_data":353696,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93707472,"statfs":{"total":96636764160,"available":95956451328,"internally_reserved":0,"allocated":362184704,"data_stored":361915050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,5,1,0,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659198,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":601088,"kb_used_data":298736,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93770752,"statfs":{"total":96636764160,"available":96021250048,"internally_reserved":0,"allocated":305905664,"data_stored":305579748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":309587631},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,2,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":407437312,"data_stored":407433833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":305410048,"data_stored":305350312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":361058304,"data_stored":361050753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:18.265 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:18:18.265 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'13235", 'reported_seq': 13317, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:16.418510+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:18:16.418510+0000', 'last_peered': '2024-09-11T01:18:16.418510+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:18:16.418510+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "36'3200", 'ondisk_log_start': "36'3200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28798976, 'num_objects': 7031, 'num_object_clones': 0, 'num_object_copies': 14062, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7031, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13235, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2959}, {'shards': '0,2', 'objects': 970}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:18.265 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:18:18.265 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'13018", 'reported_seq': 13151, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:14.386686+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:18:14.386686+0000', 'last_peered': '2024-09-11T01:18:14.386686+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:18:14.386686+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29184024, 'num_objects': 7126, 'num_object_clones': 0, 'num_object_copies': 14252, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7126, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11319, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 952}, {'shards': '2', 'objects': 3230}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:18.265 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:18:18.265 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'13227", 'reported_seq': 13400, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:16.477832+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:18:16.477832+0000', 'last_peered': '2024-09-11T01:18:16.477832+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:18:16.477832+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "36'3200", 'ondisk_log_start': "36'3200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28672000, 'num_objects': 7000, 'num_object_clones': 0, 'num_object_copies': 14000, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7000, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9568, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3049}, {'shards': '0,2', 'objects': 840}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:18.265 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:18:18.265 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'13010", 'reported_seq': 13145, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:16.342959+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:18:16.342959+0000', 'last_peered': '2024-09-11T01:18:16.342959+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:18:16.342959+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28344320, 'num_objects': 6920, 'num_object_clones': 0, 'num_object_copies': 13840, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6920, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9867, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2934}, {'shards': '0,2', 'objects': 944}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:18.265 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:18:18.266 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'12950", 'reported_seq': 13821, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:14.390395+0000', 'last_change': '2024-09-11T01:17:57.355893+0000', 'last_active': '2024-09-11T01:18:14.390395+0000', 'last_peered': '2024-09-11T01:18:14.390395+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:18:14.390395+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28766208, 'num_objects': 7023, 'num_object_clones': 0, 'num_object_copies': 14046, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2594, 'num_objects_misplaced': 972, 'num_objects_unfound': 0, 'num_objects_dirty': 7023, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10921, 'num_write_kb': 31844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 369, 'num_bytes_recovered': 1503232, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 972}, {'shards': '2', 'objects': 3457}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:18.266 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:18:18.266 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'12995", 'reported_seq': 13125, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:16.374476+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:18:16.374476+0000', 'last_peered': '2024-09-11T01:18:16.374476+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:18:16.374476+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "36'2900", 'ondisk_log_start': "36'2900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28151808, 'num_objects': 6873, 'num_object_clones': 0, 'num_object_copies': 13746, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6873, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9582, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2889}, {'shards': '0,2', 'objects': 927}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:18.266 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:18:18.266 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'13168", 'reported_seq': 13291, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:14.429140+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:18:14.429140+0000', 'last_peered': '2024-09-11T01:18:14.429140+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:18:14.429140+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "36'3100", 'ondisk_log_start': "36'3100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28831744, 'num_objects': 7039, 'num_object_clones': 0, 'num_object_copies': 14078, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7039, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10985, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 961}, {'shards': '2', 'objects': 3015}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:18.266 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:18:18.266 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'13178", 'reported_seq': 13314, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:16.361108+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:18:16.361108+0000', 'last_peered': '2024-09-11T01:18:16.361108+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:18:16.361108+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "36'3100", 'ondisk_log_start': "36'3100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 29024256, 'num_objects': 7086, 'num_object_clones': 0, 'num_object_copies': 14172, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 7086, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9915, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3136}, {'shards': '0,2', 'objects': 905}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:18.266 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:18:18.266 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'13215", 'reported_seq': 13354, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:16.414044+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:18:16.414044+0000', 'last_peered': '2024-09-11T01:18:16.414044+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:18:16.414044+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "36'3200", 'ondisk_log_start': "36'3200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28512256, 'num_objects': 6961, 'num_object_clones': 0, 'num_object_copies': 13922, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6961, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9824, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2922}, {'shards': '0,2', 'objects': 914}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:18.266 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:18:18.266 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'13031", 'reported_seq': 13168, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:16.466681+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:18:16.466681+0000', 'last_peered': '2024-09-11T01:18:16.466681+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:18:16.466681+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "36'3000", 'ondisk_log_start': "36'3000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28349033, 'num_objects': 6922, 'num_object_clones': 0, 'num_object_copies': 13844, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6922, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10101, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 3005}, {'shards': '0,2', 'objects': 868}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:18.267 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:18:19.197 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:18:19.197 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38428/223842 objects degraded (17.167%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":435,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":111921,"data_bytes":458412692,"bytes_used":2096349184,"bytes_avail":287813943296,"bytes_total":289910292480,"degraded_objects":38428,"degraded_total":223842,"degraded_ratio":0.17167466337863313,"misplaced_objects":972,"misplaced_total":223842,"misplaced_ratio":0.0043423486209022439,"recovering_objects_per_sec":16,"recovering_bytes_per_sec":66538,"recovering_keys_per_sec":0,"num_objects_recovered":195,"num_bytes_recovered":798720,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":372},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:18:19.244 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:18:19.244 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:20.074 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:18:20.076 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:18:20.144 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":258,"stamp":"2024-09-11T01:18:18.687105+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":458412692,"num_objects":111921,"num_object_clones":0,"num_object_copies":223842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38428,"num_objects_misplaced":972,"num_objects_unfound":0,"num_objects_dirty":111921,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":170370,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8240,"num_bytes_recovered":33718291,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160821,"ondisk_log_size":160821,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2047216,"kb_used_data":1051328,"kb_used_omap":12,"kb_used_meta":995827,"kb_avail":281068304,"statfs":{"total":289910292480,"available":287813943296,"internally_reserved":0,"allocated":1076559872,"data_stored":1075694643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1019726852},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,6,10,8,5,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18313216,"num_objects":-4471,"num_object_clones":0,"num_object_copies":-8942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":50,"num_objects_misplaced":-245,"num_objects_unfound":0,"num_objects_dirty":-4471,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4471,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":195,"num_bytes_recovered":798720,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003955"},"pg_stats":[{"pgid":"2.f","version":"47'13235","reported_seq":13317,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:16.418510+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:18:16.418510+0000","last_peered":"2024-09-11T01:18:16.418510+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:18:16.418510+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28798976,"num_objects":7031,"num_object_clones":0,"num_object_copies":14062,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7031,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13235,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2959},{"shards":"0,2","objects":970}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13291","reported_seq":13397,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:14.413798+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:18:14.413798+0000","last_peered":"2024-09-11T01:18:14.413798+0000","last_clean":"2024-09-11T01:18:14.413798+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:18:14.413798+0000","last_undegraded":"2024-09-11T01:18:14.413798+0000","last_fullsized":"2024-09-11T01:18:14.413798+0000","mapping_epoch":35,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28921856,"num_objects":7061,"num_object_clones":0,"num_object_copies":14122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7061,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11530,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13018","reported_seq":13151,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:14.386686+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:18:14.386686+0000","last_peered":"2024-09-11T01:18:14.386686+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:18:14.386686+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29184024,"num_objects":7126,"num_object_clones":0,"num_object_copies":14252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7126,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11319,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":952},{"shards":"2","objects":3230}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'13065","reported_seq":13137,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:14.409663+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:18:14.409663+0000","last_peered":"2024-09-11T01:18:14.409663+0000","last_clean":"2024-09-11T01:18:14.409663+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:18:14.409663+0000","last_undegraded":"2024-09-11T01:18:14.409663+0000","last_fullsized":"2024-09-11T01:18:14.409663+0000","mapping_epoch":35,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28954624,"num_objects":7069,"num_object_clones":0,"num_object_copies":14138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13065,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13049","reported_seq":13181,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:16.456062+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:18:16.456062+0000","last_peered":"2024-09-11T01:18:16.456062+0000","last_clean":"2024-09-11T01:18:16.456062+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:18:16.456062+0000","last_undegraded":"2024-09-11T01:18:16.456062+0000","last_fullsized":"2024-09-11T01:18:16.456062+0000","mapping_epoch":34,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28508160,"num_objects":6960,"num_object_clones":0,"num_object_copies":13920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10054,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13227","reported_seq":13400,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:16.477832+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:18:16.477832+0000","last_peered":"2024-09-11T01:18:16.477832+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:18:16.477832+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28672000,"num_objects":7000,"num_object_clones":0,"num_object_copies":14000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3049},{"shards":"0,2","objects":840}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'13010","reported_seq":13145,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:16.342959+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:18:16.342959+0000","last_peered":"2024-09-11T01:18:16.342959+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:18:16.342959+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28344320,"num_objects":6920,"num_object_clones":0,"num_object_copies":13840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9867,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2934},{"shards":"0,2","objects":944}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13240","reported_seq":13374,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:16.449817+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:18:16.449817+0000","last_peered":"2024-09-11T01:18:16.449817+0000","last_clean":"2024-09-11T01:18:16.449817+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:18:16.449817+0000","last_undegraded":"2024-09-11T01:18:16.449817+0000","last_fullsized":"2024-09-11T01:18:16.449817+0000","mapping_epoch":34,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28729344,"num_objects":7014,"num_object_clones":0,"num_object_copies":14028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7014,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10116,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934359+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934359+0000","last_peered":"2024-09-11T01:17:58.934359+0000","last_clean":"2024-09-11T01:17:58.934359+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934359+0000","last_undegraded":"2024-09-11T01:17:58.934359+0000","last_fullsized":"2024-09-11T01:17:58.934359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'12950","reported_seq":13821,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:14.390395+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:18:14.390395+0000","last_peered":"2024-09-11T01:18:14.390395+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:18:14.390395+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28766208,"num_objects":7023,"num_object_clones":0,"num_object_copies":14046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2594,"num_objects_misplaced":972,"num_objects_unfound":0,"num_objects_dirty":7023,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10921,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":369,"num_bytes_recovered":1503232,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":972},{"shards":"2","objects":3457}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938294+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938294+0000","last_peered":"2024-09-11T01:17:58.938294+0000","last_clean":"2024-09-11T01:17:58.938294+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938294+0000","last_undegraded":"2024-09-11T01:17:58.938294+0000","last_fullsized":"2024-09-11T01:17:58.938294+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'12995","reported_seq":13125,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:16.374476+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:18:16.374476+0000","last_peered":"2024-09-11T01:18:16.374476+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:18:16.374476+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28151808,"num_objects":6873,"num_object_clones":0,"num_object_copies":13746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6873,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9582,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2889},{"shards":"0,2","objects":927}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13168","reported_seq":13291,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:14.429140+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:18:14.429140+0000","last_peered":"2024-09-11T01:18:14.429140+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:18:14.429140+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28831744,"num_objects":7039,"num_object_clones":0,"num_object_copies":14078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10985,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":961},{"shards":"2","objects":3015}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935080+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935080+0000","last_peered":"2024-09-11T01:17:58.935080+0000","last_clean":"2024-09-11T01:17:58.935080+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935080+0000","last_undegraded":"2024-09-11T01:17:58.935080+0000","last_fullsized":"2024-09-11T01:17:58.935080+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'12892","reported_seq":20878,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:14.343284+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:18:14.343284+0000","last_peered":"2024-09-11T01:18:14.343284+0000","last_clean":"2024-09-11T01:18:14.343284+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:18:14.343284+0000","last_undegraded":"2024-09-11T01:18:14.343284+0000","last_fullsized":"2024-09-11T01:18:14.343284+0000","mapping_epoch":44,"log_start":"36'2800","ondisk_log_start":"36'2800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28278784,"num_objects":6904,"num_object_clones":0,"num_object_copies":13808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10677,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13178","reported_seq":13314,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:16.361108+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:18:16.361108+0000","last_peered":"2024-09-11T01:18:16.361108+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:18:16.361108+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29024256,"num_objects":7086,"num_object_clones":0,"num_object_copies":14172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9915,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3136},{"shards":"0,2","objects":905}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938540+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938540+0000","last_peered":"2024-09-11T01:17:58.938540+0000","last_clean":"2024-09-11T01:17:58.938540+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938540+0000","last_undegraded":"2024-09-11T01:17:58.938540+0000","last_fullsized":"2024-09-11T01:17:58.938540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'12954","reported_seq":20989,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:15.326562+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:18:15.326562+0000","last_peered":"2024-09-11T01:18:15.326562+0000","last_clean":"2024-09-11T01:18:15.326562+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:18:15.326562+0000","last_undegraded":"2024-09-11T01:18:15.326562+0000","last_fullsized":"2024-09-11T01:18:15.326562+0000","mapping_epoch":46,"log_start":"36'2900","ondisk_log_start":"36'2900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28385280,"num_objects":6930,"num_object_clones":0,"num_object_copies":13860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6930,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9609,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941377+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941377+0000","last_peered":"2024-09-11T01:17:58.941377+0000","last_clean":"2024-09-11T01:17:58.941377+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941377+0000","last_undegraded":"2024-09-11T01:17:58.941377+0000","last_fullsized":"2024-09-11T01:17:58.941377+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13215","reported_seq":13354,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:16.414044+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:18:16.414044+0000","last_peered":"2024-09-11T01:18:16.414044+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:18:16.414044+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28512256,"num_objects":6961,"num_object_clones":0,"num_object_copies":13922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9824,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2922},{"shards":"0,2","objects":914}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971314+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971314+0000","last_peered":"2024-09-11T01:17:58.971314+0000","last_clean":"2024-09-11T01:17:58.971314+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971314+0000","last_undegraded":"2024-09-11T01:17:58.971314+0000","last_fullsized":"2024-09-11T01:17:58.971314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13031","reported_seq":13168,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:16.466681+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:18:16.466681+0000","last_peered":"2024-09-11T01:18:16.466681+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:18:16.466681+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28349033,"num_objects":6922,"num_object_clones":0,"num_object_copies":13844,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6922,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10101,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":3005},{"shards":"0,2","objects":868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":458412673,"num_objects":111919,"num_object_clones":0,"num_object_copies":223838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38428,"num_objects_misplaced":972,"num_objects_unfound":0,"num_objects_dirty":111919,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":170368,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8239,"num_bytes_recovered":33718272,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1073905664,"data_stored":1073834898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160818,"ondisk_log_size":160818,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182481,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":781760,"kb_used_data":398896,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93590080,"statfs":{"total":96636764160,"available":95836241920,"internally_reserved":0,"allocated":408469504,"data_stored":408199845,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":392031915},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,3,5,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986253,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":664368,"kb_used_data":353696,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93707472,"statfs":{"total":96636764160,"available":95956451328,"internally_reserved":0,"allocated":362184704,"data_stored":361915050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,5,1,0,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659198,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":601088,"kb_used_data":298736,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93770752,"statfs":{"total":96636764160,"available":96021250048,"internally_reserved":0,"allocated":305905664,"data_stored":305579748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":309587631},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,2,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":407437312,"data_stored":407433833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":305410048,"data_stored":305350312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":361058304,"data_stored":361050753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:23.146 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:23.971 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:18:23.971 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:18:24.033 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":260,"stamp":"2024-09-11T01:18:22.688407+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":449532564,"num_objects":109753,"num_object_clones":0,"num_object_copies":219506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38447,"num_objects_misplaced":839,"num_objects_unfound":0,"num_objects_dirty":109753,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":172538,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8354,"num_bytes_recovered":34185235,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160789,"ondisk_log_size":160789,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2035224,"kb_used_data":1039336,"kb_used_omap":12,"kb_used_meta":995827,"kb_avail":281080296,"statfs":{"total":289910292480,"available":287826223104,"internally_reserved":0,"allocated":1064280064,"data_stored":1063414835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1019726852},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,0,1,6,6,8,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23191552,"num_objects":-5662,"num_object_clones":0,"num_object_copies":-11324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":66,"num_objects_misplaced":-270,"num_objects_unfound":0,"num_objects_dirty":-5662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5662,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":835584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004437"},"pg_stats":[{"pgid":"2.f","version":"47'13385","reported_seq":13466,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:21.504437+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:18:21.504437+0000","last_peered":"2024-09-11T01:18:21.504437+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:18:21.504437+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28184576,"num_objects":6881,"num_object_clones":0,"num_object_copies":13762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13385,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2809},{"shards":"0,2","objects":820}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13407","reported_seq":13513,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:19.356617+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:18:19.356617+0000","last_peered":"2024-09-11T01:18:19.356617+0000","last_clean":"2024-09-11T01:18:19.356617+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:18:19.356617+0000","last_undegraded":"2024-09-11T01:18:19.356617+0000","last_fullsized":"2024-09-11T01:18:19.356617+0000","mapping_epoch":35,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28446720,"num_objects":6945,"num_object_clones":0,"num_object_copies":13890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6945,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11646,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13147","reported_seq":13280,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:19.266212+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:18:19.266212+0000","last_peered":"2024-09-11T01:18:19.266212+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:18:19.266212+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28655640,"num_objects":6997,"num_object_clones":0,"num_object_copies":13994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6997,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11448,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":823},{"shards":"2","objects":3101}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'13208","reported_seq":13282,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:19.352580+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:18:19.352580+0000","last_peered":"2024-09-11T01:18:19.352580+0000","last_clean":"2024-09-11T01:18:19.352580+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:18:19.352580+0000","last_undegraded":"2024-09-11T01:18:19.352580+0000","last_fullsized":"2024-09-11T01:18:19.352580+0000","mapping_epoch":35,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28368896,"num_objects":6926,"num_object_clones":0,"num_object_copies":13852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13208,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13181","reported_seq":13313,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:21.410946+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:18:21.410946+0000","last_peered":"2024-09-11T01:18:21.410946+0000","last_clean":"2024-09-11T01:18:21.410946+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:18:21.410946+0000","last_undegraded":"2024-09-11T01:18:21.410946+0000","last_fullsized":"2024-09-11T01:18:21.410946+0000","mapping_epoch":34,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27967488,"num_objects":6828,"num_object_clones":0,"num_object_copies":13656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10186,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13362","reported_seq":13537,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:21.493210+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:18:21.493210+0000","last_peered":"2024-09-11T01:18:21.493210+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:18:21.493210+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28119040,"num_objects":6865,"num_object_clones":0,"num_object_copies":13730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9703,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2914},{"shards":"0,2","objects":705}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'13154","reported_seq":13288,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:21.480981+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:18:21.480981+0000","last_peered":"2024-09-11T01:18:21.480981+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:18:21.480981+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27754496,"num_objects":6776,"num_object_clones":0,"num_object_copies":13552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10011,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2790},{"shards":"0,2","objects":800}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13373","reported_seq":13507,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:21.410847+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:18:21.410847+0000","last_peered":"2024-09-11T01:18:21.410847+0000","last_clean":"2024-09-11T01:18:21.410847+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:18:21.410847+0000","last_undegraded":"2024-09-11T01:18:21.410847+0000","last_fullsized":"2024-09-11T01:18:21.410847+0000","mapping_epoch":34,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28184576,"num_objects":6881,"num_object_clones":0,"num_object_copies":13762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10249,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934359+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934359+0000","last_peered":"2024-09-11T01:17:58.934359+0000","last_clean":"2024-09-11T01:17:58.934359+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934359+0000","last_undegraded":"2024-09-11T01:17:58.934359+0000","last_fullsized":"2024-09-11T01:17:58.934359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13075","reported_seq":14174,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:19.395063+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:18:19.395063+0000","last_peered":"2024-09-11T01:18:19.395063+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:18:19.395063+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28254208,"num_objects":6898,"num_object_clones":0,"num_object_copies":13796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2613,"num_objects_misplaced":839,"num_objects_unfound":0,"num_objects_dirty":6898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11046,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":483,"num_bytes_recovered":1970176,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":839},{"shards":"2","objects":3446}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938294+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938294+0000","last_peered":"2024-09-11T01:17:58.938294+0000","last_clean":"2024-09-11T01:17:58.938294+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938294+0000","last_undegraded":"2024-09-11T01:17:58.938294+0000","last_fullsized":"2024-09-11T01:17:58.938294+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13123","reported_seq":13255,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:21.494649+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:18:21.494649+0000","last_peered":"2024-09-11T01:18:21.494649+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:18:21.494649+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27627520,"num_objects":6745,"num_object_clones":0,"num_object_copies":13490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9710,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2761},{"shards":"0,2","objects":799}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13302","reported_seq":13425,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:19.396363+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:18:19.396363+0000","last_peered":"2024-09-11T01:18:19.396363+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:18:19.396363+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28282880,"num_objects":6905,"num_object_clones":0,"num_object_copies":13810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11119,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":827},{"shards":"2","objects":2881}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935080+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935080+0000","last_peered":"2024-09-11T01:17:58.935080+0000","last_clean":"2024-09-11T01:17:58.935080+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935080+0000","last_undegraded":"2024-09-11T01:17:58.935080+0000","last_fullsized":"2024-09-11T01:17:58.935080+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13021","reported_seq":21007,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:19.341554+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:18:19.341554+0000","last_peered":"2024-09-11T01:18:19.341554+0000","last_clean":"2024-09-11T01:18:19.341554+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:18:19.341554+0000","last_undegraded":"2024-09-11T01:18:19.341554+0000","last_fullsized":"2024-09-11T01:18:19.341554+0000","mapping_epoch":44,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27750400,"num_objects":6775,"num_object_clones":0,"num_object_copies":13550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10806,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13340","reported_seq":13476,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:21.448881+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:18:21.448881+0000","last_peered":"2024-09-11T01:18:21.448881+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:18:21.448881+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28360704,"num_objects":6924,"num_object_clones":0,"num_object_copies":13848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10077,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2974},{"shards":"0,2","objects":743}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938540+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938540+0000","last_peered":"2024-09-11T01:17:58.938540+0000","last_clean":"2024-09-11T01:17:58.938540+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938540+0000","last_undegraded":"2024-09-11T01:17:58.938540+0000","last_fullsized":"2024-09-11T01:17:58.938540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13082","reported_seq":21114,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:20.354117+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:18:20.354117+0000","last_peered":"2024-09-11T01:18:20.354117+0000","last_clean":"2024-09-11T01:18:20.354117+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:18:20.354117+0000","last_undegraded":"2024-09-11T01:18:20.354117+0000","last_fullsized":"2024-09-11T01:18:20.354117+0000","mapping_epoch":46,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27860992,"num_objects":6802,"num_object_clones":0,"num_object_copies":13604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9737,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941377+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941377+0000","last_peered":"2024-09-11T01:17:58.941377+0000","last_clean":"2024-09-11T01:17:58.941377+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941377+0000","last_undegraded":"2024-09-11T01:17:58.941377+0000","last_fullsized":"2024-09-11T01:17:58.941377+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13367","reported_seq":13505,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:21.303448+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:18:21.303448+0000","last_peered":"2024-09-11T01:18:21.303448+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:18:21.303448+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27889664,"num_objects":6809,"num_object_clones":0,"num_object_copies":13618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6809,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9976,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2770},{"shards":"0,2","objects":762}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971314+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971314+0000","last_peered":"2024-09-11T01:17:58.971314+0000","last_clean":"2024-09-11T01:17:58.971314+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971314+0000","last_undegraded":"2024-09-11T01:17:58.971314+0000","last_fullsized":"2024-09-11T01:17:58.971314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13159","reported_seq":13296,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:21.519878+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:18:21.519878+0000","last_peered":"2024-09-11T01:18:21.519878+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:18:21.519878+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27824745,"num_objects":6794,"num_object_clones":0,"num_object_copies":13588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6794,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10229,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2877},{"shards":"0,2","objects":740}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":449532545,"num_objects":109751,"num_object_clones":0,"num_object_copies":219502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38447,"num_objects_misplaced":839,"num_objects_unfound":0,"num_objects_dirty":109751,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":172536,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8353,"num_bytes_recovered":34185216,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1061736448,"data_stored":1061665682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160786,"ondisk_log_size":160786,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182482,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":766824,"kb_used_data":392280,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93605016,"statfs":{"total":96636764160,"available":95851536384,"internally_reserved":0,"allocated":401694720,"data_stored":401425061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":383512235},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,2,1,2,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986254,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":664192,"kb_used_data":349360,"kb_used_omap":4,"kb_used_meta":314811,"kb_avail":93707648,"statfs":{"total":96636764160,"available":95956631552,"internally_reserved":0,"allocated":357744640,"data_stored":357474986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":322367146},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659199,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":604208,"kb_used_data":297696,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93767632,"statfs":{"total":96636764160,"available":96018055168,"internally_reserved":0,"allocated":304840704,"data_stored":304514788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":313847471},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,4,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":400601088,"data_stored":400597609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":304328704,"data_stored":304268968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":356806656,"data_stored":356799105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:24.034 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:24.957 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:18:24.960 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:18:25.042 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":261,"stamp":"2024-09-11T01:18:24.688859+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":446087828,"num_objects":108912,"num_object_clones":0,"num_object_copies":217824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38469,"num_objects_misplaced":709,"num_objects_unfound":0,"num_objects_dirty":108912,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":173379,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8462,"num_bytes_recovered":34627603,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160830,"ondisk_log_size":160830,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2018144,"kb_used_data":1034736,"kb_used_omap":12,"kb_used_meta":983347,"kb_avail":281097376,"statfs":{"total":289910292480,"available":287843713024,"internally_reserved":0,"allocated":1059569664,"data_stored":1058704435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1006947332},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[5,0,1,4,7,9,11,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21405696,"num_objects":-5226,"num_object_clones":0,"num_object_copies":-10452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":88,"num_objects_misplaced":-400,"num_objects_unfound":0,"num_objects_dirty":-5226,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5226,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":312,"num_bytes_recovered":1277952,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004390"},"pg_stats":[{"pgid":"2.f","version":"47'13385","reported_seq":13466,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:21.504437+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:18:21.504437+0000","last_peered":"2024-09-11T01:18:21.504437+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:18:21.504437+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28184576,"num_objects":6881,"num_object_clones":0,"num_object_copies":13762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13385,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2809},{"shards":"0,2","objects":820}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13548","reported_seq":13654,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:24.365012+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:18:24.365012+0000","last_peered":"2024-09-11T01:18:24.365012+0000","last_clean":"2024-09-11T01:18:24.365012+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:18:24.365012+0000","last_undegraded":"2024-09-11T01:18:24.365012+0000","last_fullsized":"2024-09-11T01:18:24.365012+0000","mapping_epoch":35,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27869184,"num_objects":6804,"num_object_clones":0,"num_object_copies":13608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11787,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13301","reported_seq":13433,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:24.441775+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:18:24.441775+0000","last_peered":"2024-09-11T01:18:24.441775+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:18:24.441775+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28024856,"num_objects":6843,"num_object_clones":0,"num_object_copies":13686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6843,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11602,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":669},{"shards":"2","objects":2947}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'13347","reported_seq":13421,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:24.368325+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:18:24.368325+0000","last_peered":"2024-09-11T01:18:24.368325+0000","last_clean":"2024-09-11T01:18:24.368325+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:18:24.368325+0000","last_undegraded":"2024-09-11T01:18:24.368325+0000","last_fullsized":"2024-09-11T01:18:24.368325+0000","mapping_epoch":35,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27799552,"num_objects":6787,"num_object_clones":0,"num_object_copies":13574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6787,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13347,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13181","reported_seq":13313,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:21.410946+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:18:21.410946+0000","last_peered":"2024-09-11T01:18:21.410946+0000","last_clean":"2024-09-11T01:18:21.410946+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:18:21.410946+0000","last_undegraded":"2024-09-11T01:18:21.410946+0000","last_fullsized":"2024-09-11T01:18:21.410946+0000","mapping_epoch":34,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27967488,"num_objects":6828,"num_object_clones":0,"num_object_copies":13656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10186,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13362","reported_seq":13537,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:21.493210+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:18:21.493210+0000","last_peered":"2024-09-11T01:18:21.493210+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:18:21.493210+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28119040,"num_objects":6865,"num_object_clones":0,"num_object_copies":13730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6865,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9703,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2914},{"shards":"0,2","objects":705}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'13154","reported_seq":13288,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:21.480981+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:18:21.480981+0000","last_peered":"2024-09-11T01:18:21.480981+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:18:21.480981+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27754496,"num_objects":6776,"num_object_clones":0,"num_object_copies":13552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6776,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10011,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2790},{"shards":"0,2","objects":800}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13373","reported_seq":13507,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:21.410847+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:18:21.410847+0000","last_peered":"2024-09-11T01:18:21.410847+0000","last_clean":"2024-09-11T01:18:21.410847+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:18:21.410847+0000","last_undegraded":"2024-09-11T01:18:21.410847+0000","last_fullsized":"2024-09-11T01:18:21.410847+0000","mapping_epoch":34,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28184576,"num_objects":6881,"num_object_clones":0,"num_object_copies":13762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6881,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10249,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934359+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934359+0000","last_peered":"2024-09-11T01:17:58.934359+0000","last_clean":"2024-09-11T01:17:58.934359+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934359+0000","last_undegraded":"2024-09-11T01:17:58.934359+0000","last_fullsized":"2024-09-11T01:17:58.934359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13207","reported_seq":14522,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:24.409919+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:18:24.409919+0000","last_peered":"2024-09-11T01:18:24.409919+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:18:24.409919+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27713536,"num_objects":6766,"num_object_clones":0,"num_object_copies":13532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2635,"num_objects_misplaced":709,"num_objects_unfound":0,"num_objects_dirty":6766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11178,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":591,"num_bytes_recovered":2412544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":709},{"shards":"2","objects":3422}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13123","reported_seq":13255,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:21.494649+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:18:21.494649+0000","last_peered":"2024-09-11T01:18:21.494649+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:18:21.494649+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27627520,"num_objects":6745,"num_object_clones":0,"num_object_copies":13490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9710,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2761},{"shards":"0,2","objects":799}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13439","reported_seq":13562,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:24.460464+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:18:24.460464+0000","last_peered":"2024-09-11T01:18:24.460464+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:18:24.460464+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27721728,"num_objects":6768,"num_object_clones":0,"num_object_copies":13536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11256,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":690},{"shards":"2","objects":2744}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935080+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935080+0000","last_peered":"2024-09-11T01:17:58.935080+0000","last_clean":"2024-09-11T01:17:58.935080+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935080+0000","last_undegraded":"2024-09-11T01:17:58.935080+0000","last_fullsized":"2024-09-11T01:17:58.935080+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13159","reported_seq":21145,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:24.386637+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:18:24.386637+0000","last_peered":"2024-09-11T01:18:24.386637+0000","last_clean":"2024-09-11T01:18:24.386637+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:18:24.386637+0000","last_undegraded":"2024-09-11T01:18:24.386637+0000","last_fullsized":"2024-09-11T01:18:24.386637+0000","mapping_epoch":44,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27185152,"num_objects":6637,"num_object_clones":0,"num_object_copies":13274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10944,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13340","reported_seq":13476,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:21.448881+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:18:21.448881+0000","last_peered":"2024-09-11T01:18:21.448881+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:18:21.448881+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28360704,"num_objects":6924,"num_object_clones":0,"num_object_copies":13848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10077,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2974},{"shards":"0,2","objects":743}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938540+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938540+0000","last_peered":"2024-09-11T01:17:58.938540+0000","last_clean":"2024-09-11T01:17:58.938540+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938540+0000","last_undegraded":"2024-09-11T01:17:58.938540+0000","last_fullsized":"2024-09-11T01:17:58.938540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13082","reported_seq":21114,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:20.354117+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:18:20.354117+0000","last_peered":"2024-09-11T01:18:20.354117+0000","last_clean":"2024-09-11T01:18:20.354117+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:18:20.354117+0000","last_undegraded":"2024-09-11T01:18:20.354117+0000","last_fullsized":"2024-09-11T01:18:20.354117+0000","mapping_epoch":46,"log_start":"36'3000","ondisk_log_start":"36'3000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27860992,"num_objects":6802,"num_object_clones":0,"num_object_copies":13604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6802,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9737,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13367","reported_seq":13505,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:21.303448+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:18:21.303448+0000","last_peered":"2024-09-11T01:18:21.303448+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:18:21.303448+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27889664,"num_objects":6809,"num_object_clones":0,"num_object_copies":13618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6809,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9976,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2770},{"shards":"0,2","objects":762}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971314+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971314+0000","last_peered":"2024-09-11T01:17:58.971314+0000","last_clean":"2024-09-11T01:17:58.971314+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971314+0000","last_undegraded":"2024-09-11T01:17:58.971314+0000","last_fullsized":"2024-09-11T01:17:58.971314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13159","reported_seq":13296,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:21.519878+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:18:21.519878+0000","last_peered":"2024-09-11T01:18:21.519878+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:18:21.519878+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27824745,"num_objects":6794,"num_object_clones":0,"num_object_copies":13588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6794,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10229,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2877},{"shards":"0,2","objects":740}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":446087809,"num_objects":108910,"num_object_clones":0,"num_object_copies":217820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38469,"num_objects_misplaced":709,"num_objects_unfound":0,"num_objects_dirty":108910,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":173377,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8461,"num_bytes_recovered":34627584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1057026048,"data_stored":1056955282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160827,"ondisk_log_size":160827,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182482,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":766824,"kb_used_data":392280,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93605016,"statfs":{"total":96636764160,"available":95851536384,"internally_reserved":0,"allocated":401694720,"data_stored":401425061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":383512235},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,0,2,1,2,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986255,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":647112,"kb_used_data":344760,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93724728,"statfs":{"total":96636764160,"available":95974121472,"internally_reserved":0,"allocated":353034240,"data_stored":352764586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659199,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":604208,"kb_used_data":297696,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93767632,"statfs":{"total":96636764160,"available":96018055168,"internally_reserved":0,"allocated":304840704,"data_stored":304514788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":313847471},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,4,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":400601088,"data_stored":400597609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":304328704,"data_stored":304268968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":352096256,"data_stored":352088705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:25.043 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:18:25.043 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'13385", 'reported_seq': 13466, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:21.504437+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:18:21.504437+0000', 'last_peered': '2024-09-11T01:18:21.504437+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:18:21.504437+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "36'3300", 'ondisk_log_start': "36'3300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28184576, 'num_objects': 6881, 'num_object_clones': 0, 'num_object_copies': 13762, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6881, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13385, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2809}, {'shards': '0,2', 'objects': 820}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:25.043 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:18:25.043 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'13301", 'reported_seq': 13433, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:24.441775+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:18:24.441775+0000', 'last_peered': '2024-09-11T01:18:24.441775+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:18:24.441775+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "36'3300", 'ondisk_log_start': "36'3300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28024856, 'num_objects': 6843, 'num_object_clones': 0, 'num_object_copies': 13686, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6843, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11602, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 669}, {'shards': '2', 'objects': 2947}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:25.043 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:18:25.044 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'13362", 'reported_seq': 13537, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:21.493210+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:18:21.493210+0000', 'last_peered': '2024-09-11T01:18:21.493210+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:18:21.493210+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "36'3300", 'ondisk_log_start': "36'3300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28119040, 'num_objects': 6865, 'num_object_clones': 0, 'num_object_copies': 13730, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6865, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9703, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2914}, {'shards': '0,2', 'objects': 705}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:25.044 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:18:25.044 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'13154", 'reported_seq': 13288, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:21.480981+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:18:21.480981+0000', 'last_peered': '2024-09-11T01:18:21.480981+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:18:21.480981+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "36'3100", 'ondisk_log_start': "36'3100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10054, 'log_dups_size': 0, 'ondisk_log_size': 10054, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27754496, 'num_objects': 6776, 'num_object_clones': 0, 'num_object_copies': 13552, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6776, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10011, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2790}, {'shards': '0,2', 'objects': 800}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:25.044 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:18:25.044 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'13207", 'reported_seq': 14522, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:24.409919+0000', 'last_change': '2024-09-11T01:17:57.355893+0000', 'last_active': '2024-09-11T01:18:24.409919+0000', 'last_peered': '2024-09-11T01:18:24.409919+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:18:24.409919+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "36'3200", 'ondisk_log_start': "36'3200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27713536, 'num_objects': 6766, 'num_object_clones': 0, 'num_object_copies': 13532, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2635, 'num_objects_misplaced': 709, 'num_objects_unfound': 0, 'num_objects_dirty': 6766, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11178, 'num_write_kb': 31844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 591, 'num_bytes_recovered': 2412544, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 709}, {'shards': '2', 'objects': 3422}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:25.044 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:18:25.044 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'13123", 'reported_seq': 13255, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:21.494649+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:18:21.494649+0000', 'last_peered': '2024-09-11T01:18:21.494649+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:18:21.494649+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "36'3100", 'ondisk_log_start': "36'3100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27627520, 'num_objects': 6745, 'num_object_clones': 0, 'num_object_copies': 13490, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6745, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9710, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2761}, {'shards': '0,2', 'objects': 799}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:25.044 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:18:25.044 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'13439", 'reported_seq': 13562, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:24.460464+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:18:24.460464+0000', 'last_peered': '2024-09-11T01:18:24.460464+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:18:24.460464+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "36'3400", 'ondisk_log_start': "36'3400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27721728, 'num_objects': 6768, 'num_object_clones': 0, 'num_object_copies': 13536, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6768, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11256, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 690}, {'shards': '2', 'objects': 2744}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:25.044 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:18:25.045 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'13340", 'reported_seq': 13476, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:21.448881+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:18:21.448881+0000', 'last_peered': '2024-09-11T01:18:21.448881+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:18:21.448881+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "36'3300", 'ondisk_log_start': "36'3300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 28360704, 'num_objects': 6924, 'num_object_clones': 0, 'num_object_copies': 13848, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6924, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10077, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2974}, {'shards': '0,2', 'objects': 743}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:25.045 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:18:25.045 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'13367", 'reported_seq': 13505, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:21.303448+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:18:21.303448+0000', 'last_peered': '2024-09-11T01:18:21.303448+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:18:21.303448+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "36'3300", 'ondisk_log_start': "36'3300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27889664, 'num_objects': 6809, 'num_object_clones': 0, 'num_object_copies': 13618, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6809, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9976, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2770}, {'shards': '0,2', 'objects': 762}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:25.045 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:18:25.045 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'13159", 'reported_seq': 13296, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:21.519878+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:18:21.519878+0000', 'last_peered': '2024-09-11T01:18:21.519878+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:18:21.519878+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "36'3100", 'ondisk_log_start': "36'3100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10059, 'log_dups_size': 0, 'ondisk_log_size': 10059, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27824745, 'num_objects': 6794, 'num_object_clones': 0, 'num_object_copies': 13588, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6794, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10229, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2877}, {'shards': '0,2', 'objects': 740}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:25.045 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:18:26.064 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:18:26.064 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38469/217824 objects degraded (17.661%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":442,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":108912,"data_bytes":446087828,"bytes_used":2066579456,"bytes_avail":287843713024,"bytes_total":289910292480,"degraded_objects":38469,"degraded_total":217824,"degraded_ratio":0.17660588364918467,"misplaced_objects":709,"misplaced_total":217824,"misplaced_ratio":0.0032549214044366092,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":106457,"recovering_keys_per_sec":0,"num_objects_recovered":312,"num_bytes_recovered":1277952,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":435},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:18:26.159 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:18:26.159 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:26.921 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:18:26.923 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:18:26.971 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":262,"stamp":"2024-09-11T01:18:26.693852+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":440320660,"num_objects":107504,"num_object_clones":0,"num_object_copies":215008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38469,"num_objects_misplaced":709,"num_objects_unfound":0,"num_objects_dirty":107504,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":174787,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8462,"num_bytes_recovered":34627603,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160738,"ondisk_log_size":160738,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2006284,"kb_used_data":1027032,"kb_used_omap":12,"kb_used_meta":979187,"kb_avail":281109236,"statfs":{"total":289910292480,"available":287855857664,"internally_reserved":0,"allocated":1051680768,"data_stored":1050815539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1002687492},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,1,2,5,7,11,3,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23920640,"num_objects":-5840,"num_object_clones":0,"num_object_copies":-11680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":41,"num_objects_misplaced":-263,"num_objects_unfound":0,"num_objects_dirty":-5840,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5840,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":222,"num_bytes_recovered":909312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008911"},"pg_stats":[{"pgid":"2.f","version":"47'13536","reported_seq":13618,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:26.576692+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:18:26.576692+0000","last_peered":"2024-09-11T01:18:26.576692+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:18:26.576692+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27566080,"num_objects":6730,"num_object_clones":0,"num_object_copies":13460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13536,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2658},{"shards":"0,2","objects":669}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13548","reported_seq":13654,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:24.365012+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:18:24.365012+0000","last_peered":"2024-09-11T01:18:24.365012+0000","last_clean":"2024-09-11T01:18:24.365012+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:18:24.365012+0000","last_undegraded":"2024-09-11T01:18:24.365012+0000","last_fullsized":"2024-09-11T01:18:24.365012+0000","mapping_epoch":35,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27869184,"num_objects":6804,"num_object_clones":0,"num_object_copies":13608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11787,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13301","reported_seq":13433,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:24.441775+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:18:24.441775+0000","last_peered":"2024-09-11T01:18:24.441775+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:18:24.441775+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":28024856,"num_objects":6843,"num_object_clones":0,"num_object_copies":13686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6843,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11602,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":669},{"shards":"2","objects":2947}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'13347","reported_seq":13421,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:24.368325+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:18:24.368325+0000","last_peered":"2024-09-11T01:18:24.368325+0000","last_clean":"2024-09-11T01:18:24.368325+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:18:24.368325+0000","last_undegraded":"2024-09-11T01:18:24.368325+0000","last_fullsized":"2024-09-11T01:18:24.368325+0000","mapping_epoch":35,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27799552,"num_objects":6787,"num_object_clones":0,"num_object_copies":13574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6787,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13347,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13313","reported_seq":13445,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:26.550882+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:18:26.550882+0000","last_peered":"2024-09-11T01:18:26.550882+0000","last_clean":"2024-09-11T01:18:26.550882+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:18:26.550882+0000","last_undegraded":"2024-09-11T01:18:26.550882+0000","last_fullsized":"2024-09-11T01:18:26.550882+0000","mapping_epoch":34,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27426816,"num_objects":6696,"num_object_clones":0,"num_object_copies":13392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6696,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10318,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13488","reported_seq":13662,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:26.504012+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:18:26.504012+0000","last_peered":"2024-09-11T01:18:26.504012+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:18:26.504012+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27602944,"num_objects":6739,"num_object_clones":0,"num_object_copies":13478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9829,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2788},{"shards":"0,2","objects":579}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'13291","reported_seq":13426,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:26.469080+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:18:26.469080+0000","last_peered":"2024-09-11T01:18:26.469080+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:18:26.469080+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27193344,"num_objects":6639,"num_object_clones":0,"num_object_copies":13278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10148,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2653},{"shards":"0,2","objects":663}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13529","reported_seq":13664,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:26.517898+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:18:26.517898+0000","last_peered":"2024-09-11T01:18:26.517898+0000","last_clean":"2024-09-11T01:18:26.517898+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:18:26.517898+0000","last_undegraded":"2024-09-11T01:18:26.517898+0000","last_fullsized":"2024-09-11T01:18:26.517898+0000","mapping_epoch":34,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27545600,"num_objects":6725,"num_object_clones":0,"num_object_copies":13450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10405,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934359+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934359+0000","last_peered":"2024-09-11T01:17:58.934359+0000","last_clean":"2024-09-11T01:17:58.934359+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934359+0000","last_undegraded":"2024-09-11T01:17:58.934359+0000","last_fullsized":"2024-09-11T01:17:58.934359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13207","reported_seq":14522,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:24.409919+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:18:24.409919+0000","last_peered":"2024-09-11T01:18:24.409919+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:18:24.409919+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27713536,"num_objects":6766,"num_object_clones":0,"num_object_copies":13532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2635,"num_objects_misplaced":709,"num_objects_unfound":0,"num_objects_dirty":6766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11178,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":591,"num_bytes_recovered":2412544,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":709},{"shards":"2","objects":3422}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13263","reported_seq":13395,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:26.459323+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:18:26.459323+0000","last_peered":"2024-09-11T01:18:26.459323+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:18:26.459323+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27054080,"num_objects":6605,"num_object_clones":0,"num_object_copies":13210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6605,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9850,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2621},{"shards":"0,2","objects":659}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13439","reported_seq":13562,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:24.460464+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:18:24.460464+0000","last_peered":"2024-09-11T01:18:24.460464+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:18:24.460464+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27721728,"num_objects":6768,"num_object_clones":0,"num_object_copies":13536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11256,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":690},{"shards":"2","objects":2744}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935080+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935080+0000","last_peered":"2024-09-11T01:17:58.935080+0000","last_clean":"2024-09-11T01:17:58.935080+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935080+0000","last_undegraded":"2024-09-11T01:17:58.935080+0000","last_fullsized":"2024-09-11T01:17:58.935080+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13159","reported_seq":21145,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:24.386637+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:18:24.386637+0000","last_peered":"2024-09-11T01:18:24.386637+0000","last_clean":"2024-09-11T01:18:24.386637+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:18:24.386637+0000","last_undegraded":"2024-09-11T01:18:24.386637+0000","last_fullsized":"2024-09-11T01:18:24.386637+0000","mapping_epoch":44,"log_start":"36'3100","ondisk_log_start":"36'3100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27185152,"num_objects":6637,"num_object_clones":0,"num_object_copies":13274,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6637,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10944,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13489","reported_seq":13625,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:26.528375+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:18:26.528375+0000","last_peered":"2024-09-11T01:18:26.528375+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:18:26.528375+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27750400,"num_objects":6775,"num_object_clones":0,"num_object_copies":13550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10226,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2825},{"shards":"0,2","objects":594}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938540+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938540+0000","last_peered":"2024-09-11T01:17:58.938540+0000","last_clean":"2024-09-11T01:17:58.938540+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938540+0000","last_undegraded":"2024-09-11T01:17:58.938540+0000","last_fullsized":"2024-09-11T01:17:58.938540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13229","reported_seq":21264,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:25.336697+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:18:25.336697+0000","last_peered":"2024-09-11T01:18:25.336697+0000","last_clean":"2024-09-11T01:18:25.336697+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:18:25.336697+0000","last_undegraded":"2024-09-11T01:18:25.336697+0000","last_fullsized":"2024-09-11T01:18:25.336697+0000","mapping_epoch":46,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27258880,"num_objects":6655,"num_object_clones":0,"num_object_copies":13310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9884,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13515","reported_seq":13655,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:26.535836+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:18:26.535836+0000","last_peered":"2024-09-11T01:18:26.535836+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:18:26.535836+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27283456,"num_objects":6661,"num_object_clones":0,"num_object_copies":13322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10124,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2622},{"shards":"0,2","objects":614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971314+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971314+0000","last_peered":"2024-09-11T01:17:58.971314+0000","last_clean":"2024-09-11T01:17:58.971314+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971314+0000","last_undegraded":"2024-09-11T01:17:58.971314+0000","last_fullsized":"2024-09-11T01:17:58.971314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13281","reported_seq":13420,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:26.505533+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:18:26.505533+0000","last_peered":"2024-09-11T01:18:26.505533+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:18:26.505533+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27325033,"num_objects":6672,"num_object_clones":0,"num_object_copies":13344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10351,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2755},{"shards":"0,2","objects":618}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":440320641,"num_objects":107502,"num_object_clones":0,"num_object_copies":215004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38469,"num_objects_misplaced":709,"num_objects_unfound":0,"num_objects_dirty":107502,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":174785,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8461,"num_bytes_recovered":34627584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1048875008,"data_stored":1048804242,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160735,"ondisk_log_size":160735,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182483,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":764364,"kb_used_data":385660,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93607476,"statfs":{"total":96636764160,"available":95854055424,"internally_reserved":0,"allocated":394915840,"data_stored":394646181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":387772075},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,0,1,3,3,3,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986255,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":647112,"kb_used_data":344760,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93724728,"statfs":{"total":96636764160,"available":95974121472,"internally_reserved":0,"allocated":353034240,"data_stored":352764586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659200,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":594808,"kb_used_data":296612,"kb_used_omap":4,"kb_used_meta":298171,"kb_avail":93777032,"statfs":{"total":96636764160,"available":96027680768,"internally_reserved":0,"allocated":303730688,"data_stored":303404772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":305327791},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,2,1,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":393662464,"data_stored":393658985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":303116288,"data_stored":303056552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":352096256,"data_stored":352088705,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:29.974 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:30.765 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:18:30.767 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:18:30.822 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":264,"stamp":"2024-09-11T01:18:30.696318+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":436261524,"num_objects":106513,"num_object_clones":0,"num_object_copies":213026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38475,"num_objects_misplaced":588,"num_objects_unfound":0,"num_objects_dirty":106513,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":175778,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8577,"num_bytes_recovered":35098643,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160929,"ondisk_log_size":160929,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2009032,"kb_used_data":1021456,"kb_used_omap":12,"kb_used_meta":987507,"kb_avail":281106488,"statfs":{"total":289910292480,"available":287853043712,"internally_reserved":0,"allocated":1045970944,"data_stored":1045105715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1011207172},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,2,1,5,8,13,3,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22151168,"num_objects":-5408,"num_object_clones":0,"num_object_copies":-10816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47,"num_objects_misplaced":-384,"num_objects_unfound":0,"num_objects_dirty":-5408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5408,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":337,"num_bytes_recovered":1380352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009213"},"pg_stats":[{"pgid":"2.f","version":"47'13536","reported_seq":13618,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:26.576692+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:18:26.576692+0000","last_peered":"2024-09-11T01:18:26.576692+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:18:26.576692+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27566080,"num_objects":6730,"num_object_clones":0,"num_object_copies":13460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13536,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2658},{"shards":"0,2","objects":669}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13711","reported_seq":13817,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:29.432038+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:18:29.432038+0000","last_peered":"2024-09-11T01:18:29.432038+0000","last_clean":"2024-09-11T01:18:29.432038+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:18:29.432038+0000","last_undegraded":"2024-09-11T01:18:29.432038+0000","last_fullsized":"2024-09-11T01:18:29.432038+0000","mapping_epoch":35,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27201536,"num_objects":6641,"num_object_clones":0,"num_object_copies":13282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11950,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13444","reported_seq":13577,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:29.473758+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:18:29.473758+0000","last_peered":"2024-09-11T01:18:29.473758+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:18:29.473758+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27439128,"num_objects":6700,"num_object_clones":0,"num_object_copies":13400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6700,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11745,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":526},{"shards":"2","objects":2804}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'13490","reported_seq":13564,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:29.406027+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:18:29.406027+0000","last_peered":"2024-09-11T01:18:29.406027+0000","last_clean":"2024-09-11T01:18:29.406027+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:18:29.406027+0000","last_undegraded":"2024-09-11T01:18:29.406027+0000","last_fullsized":"2024-09-11T01:18:29.406027+0000","mapping_epoch":35,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27213824,"num_objects":6644,"num_object_clones":0,"num_object_copies":13288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13490,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13313","reported_seq":13445,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:26.550882+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:18:26.550882+0000","last_peered":"2024-09-11T01:18:26.550882+0000","last_clean":"2024-09-11T01:18:26.550882+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:18:26.550882+0000","last_undegraded":"2024-09-11T01:18:26.550882+0000","last_fullsized":"2024-09-11T01:18:26.550882+0000","mapping_epoch":34,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27426816,"num_objects":6696,"num_object_clones":0,"num_object_copies":13392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6696,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10318,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13488","reported_seq":13662,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:26.504012+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:18:26.504012+0000","last_peered":"2024-09-11T01:18:26.504012+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:18:26.504012+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27602944,"num_objects":6739,"num_object_clones":0,"num_object_copies":13478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9829,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2788},{"shards":"0,2","objects":579}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'13291","reported_seq":13426,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:26.469080+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:18:26.469080+0000","last_peered":"2024-09-11T01:18:26.469080+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:18:26.469080+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27193344,"num_objects":6639,"num_object_clones":0,"num_object_copies":13278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10148,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2653},{"shards":"0,2","objects":663}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13529","reported_seq":13664,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:26.517898+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:18:26.517898+0000","last_peered":"2024-09-11T01:18:26.517898+0000","last_clean":"2024-09-11T01:18:26.517898+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:18:26.517898+0000","last_undegraded":"2024-09-11T01:18:26.517898+0000","last_fullsized":"2024-09-11T01:18:26.517898+0000","mapping_epoch":34,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27545600,"num_objects":6725,"num_object_clones":0,"num_object_copies":13450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10405,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934359+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934359+0000","last_peered":"2024-09-11T01:17:58.934359+0000","last_clean":"2024-09-11T01:17:58.934359+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934359+0000","last_undegraded":"2024-09-11T01:17:58.934359+0000","last_fullsized":"2024-09-11T01:17:58.934359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13334","reported_seq":14879,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:29.460257+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:18:29.460257+0000","last_peered":"2024-09-11T01:18:29.460257+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:18:29.460257+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27193344,"num_objects":6639,"num_object_clones":0,"num_object_copies":13278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2641,"num_objects_misplaced":588,"num_objects_unfound":0,"num_objects_dirty":6639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11305,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":706,"num_bytes_recovered":2883584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":588},{"shards":"2","objects":3410}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13263","reported_seq":13395,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:26.459323+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:18:26.459323+0000","last_peered":"2024-09-11T01:18:26.459323+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:18:26.459323+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27054080,"num_objects":6605,"num_object_clones":0,"num_object_copies":13210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6605,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9850,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2621},{"shards":"0,2","objects":659}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13578","reported_seq":13701,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:29.412479+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:18:29.412479+0000","last_peered":"2024-09-11T01:18:29.412479+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:18:29.412479+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27152384,"num_objects":6629,"num_object_clones":0,"num_object_copies":13258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11395,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":551},{"shards":"2","objects":2605}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935080+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935080+0000","last_peered":"2024-09-11T01:17:58.935080+0000","last_clean":"2024-09-11T01:17:58.935080+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935080+0000","last_undegraded":"2024-09-11T01:17:58.935080+0000","last_fullsized":"2024-09-11T01:17:58.935080+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13296","reported_seq":21282,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:29.465966+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:18:29.465966+0000","last_peered":"2024-09-11T01:18:29.465966+0000","last_clean":"2024-09-11T01:18:29.465966+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:18:29.465966+0000","last_undegraded":"2024-09-11T01:18:29.465966+0000","last_fullsized":"2024-09-11T01:18:29.465966+0000","mapping_epoch":44,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26624000,"num_objects":6500,"num_object_clones":0,"num_object_copies":13000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11081,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13489","reported_seq":13625,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:26.528375+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:18:26.528375+0000","last_peered":"2024-09-11T01:18:26.528375+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:18:26.528375+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27750400,"num_objects":6775,"num_object_clones":0,"num_object_copies":13550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10226,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2825},{"shards":"0,2","objects":594}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938540+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938540+0000","last_peered":"2024-09-11T01:17:58.938540+0000","last_clean":"2024-09-11T01:17:58.938540+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938540+0000","last_undegraded":"2024-09-11T01:17:58.938540+0000","last_fullsized":"2024-09-11T01:17:58.938540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13368","reported_seq":21404,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:30.372708+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:18:30.372708+0000","last_peered":"2024-09-11T01:18:30.372708+0000","last_clean":"2024-09-11T01:18:30.372708+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:18:30.372708+0000","last_undegraded":"2024-09-11T01:18:30.372708+0000","last_fullsized":"2024-09-11T01:18:30.372708+0000","mapping_epoch":46,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26689536,"num_objects":6516,"num_object_clones":0,"num_object_copies":13032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10023,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13515","reported_seq":13655,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:26.535836+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:18:26.535836+0000","last_peered":"2024-09-11T01:18:26.535836+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:18:26.535836+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27283456,"num_objects":6661,"num_object_clones":0,"num_object_copies":13322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10124,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2622},{"shards":"0,2","objects":614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971314+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971314+0000","last_peered":"2024-09-11T01:17:58.971314+0000","last_clean":"2024-09-11T01:17:58.971314+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971314+0000","last_undegraded":"2024-09-11T01:17:58.971314+0000","last_fullsized":"2024-09-11T01:17:58.971314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13281","reported_seq":13420,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:26.505533+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:18:26.505533+0000","last_peered":"2024-09-11T01:18:26.505533+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:18:26.505533+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27325033,"num_objects":6672,"num_object_clones":0,"num_object_copies":13344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10351,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2755},{"shards":"0,2","objects":618}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":436261505,"num_objects":106511,"num_object_clones":0,"num_object_copies":213022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38475,"num_objects_misplaced":588,"num_objects_unfound":0,"num_objects_dirty":106511,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":175776,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8576,"num_bytes_recovered":35098624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1043197952,"data_stored":1043127186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160926,"ondisk_log_size":160926,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182483,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":764364,"kb_used_data":385660,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93607476,"statfs":{"total":96636764160,"available":95854055424,"internally_reserved":0,"allocated":394915840,"data_stored":394646181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":387772075},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,0,1,3,3,3,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986256,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":646876,"kb_used_data":340360,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93724964,"statfs":{"total":96636764160,"available":95974363136,"internally_reserved":0,"allocated":348528640,"data_stored":348258986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659201,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":597792,"kb_used_data":295436,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93774048,"statfs":{"total":96636764160,"available":96024625152,"internally_reserved":0,"allocated":302526464,"data_stored":302200548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":309587631},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":393662464,"data_stored":393658985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":301969408,"data_stored":301909672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":347566080,"data_stored":347558529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:30.823 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:31.547 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:18:31.549 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:18:31.593 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":264,"stamp":"2024-09-11T01:18:30.696318+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":436261524,"num_objects":106513,"num_object_clones":0,"num_object_copies":213026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38475,"num_objects_misplaced":588,"num_objects_unfound":0,"num_objects_dirty":106513,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":175778,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8577,"num_bytes_recovered":35098643,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160929,"ondisk_log_size":160929,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2009032,"kb_used_data":1021456,"kb_used_omap":12,"kb_used_meta":987507,"kb_avail":281106488,"statfs":{"total":289910292480,"available":287853043712,"internally_reserved":0,"allocated":1045970944,"data_stored":1045105715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1011207172},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,2,1,5,8,13,3,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22151168,"num_objects":-5408,"num_object_clones":0,"num_object_copies":-10816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47,"num_objects_misplaced":-384,"num_objects_unfound":0,"num_objects_dirty":-5408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5408,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":337,"num_bytes_recovered":1380352,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009213"},"pg_stats":[{"pgid":"2.f","version":"47'13536","reported_seq":13618,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:26.576692+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:18:26.576692+0000","last_peered":"2024-09-11T01:18:26.576692+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:18:26.576692+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27566080,"num_objects":6730,"num_object_clones":0,"num_object_copies":13460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13536,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2658},{"shards":"0,2","objects":669}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13711","reported_seq":13817,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:29.432038+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:18:29.432038+0000","last_peered":"2024-09-11T01:18:29.432038+0000","last_clean":"2024-09-11T01:18:29.432038+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:18:29.432038+0000","last_undegraded":"2024-09-11T01:18:29.432038+0000","last_fullsized":"2024-09-11T01:18:29.432038+0000","mapping_epoch":35,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27201536,"num_objects":6641,"num_object_clones":0,"num_object_copies":13282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11950,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13444","reported_seq":13577,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:29.473758+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:18:29.473758+0000","last_peered":"2024-09-11T01:18:29.473758+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:18:29.473758+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27439128,"num_objects":6700,"num_object_clones":0,"num_object_copies":13400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6700,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11745,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":526},{"shards":"2","objects":2804}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'13490","reported_seq":13564,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:29.406027+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:18:29.406027+0000","last_peered":"2024-09-11T01:18:29.406027+0000","last_clean":"2024-09-11T01:18:29.406027+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:18:29.406027+0000","last_undegraded":"2024-09-11T01:18:29.406027+0000","last_fullsized":"2024-09-11T01:18:29.406027+0000","mapping_epoch":35,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27213824,"num_objects":6644,"num_object_clones":0,"num_object_copies":13288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13490,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13313","reported_seq":13445,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:26.550882+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:18:26.550882+0000","last_peered":"2024-09-11T01:18:26.550882+0000","last_clean":"2024-09-11T01:18:26.550882+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:18:26.550882+0000","last_undegraded":"2024-09-11T01:18:26.550882+0000","last_fullsized":"2024-09-11T01:18:26.550882+0000","mapping_epoch":34,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27426816,"num_objects":6696,"num_object_clones":0,"num_object_copies":13392,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6696,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10318,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13488","reported_seq":13662,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:26.504012+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:18:26.504012+0000","last_peered":"2024-09-11T01:18:26.504012+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:18:26.504012+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27602944,"num_objects":6739,"num_object_clones":0,"num_object_copies":13478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9829,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2788},{"shards":"0,2","objects":579}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'13291","reported_seq":13426,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:26.469080+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:18:26.469080+0000","last_peered":"2024-09-11T01:18:26.469080+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:18:26.469080+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27193344,"num_objects":6639,"num_object_clones":0,"num_object_copies":13278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10148,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2653},{"shards":"0,2","objects":663}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13529","reported_seq":13664,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:26.517898+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:18:26.517898+0000","last_peered":"2024-09-11T01:18:26.517898+0000","last_clean":"2024-09-11T01:18:26.517898+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:18:26.517898+0000","last_undegraded":"2024-09-11T01:18:26.517898+0000","last_fullsized":"2024-09-11T01:18:26.517898+0000","mapping_epoch":34,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27545600,"num_objects":6725,"num_object_clones":0,"num_object_copies":13450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10405,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934359+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934359+0000","last_peered":"2024-09-11T01:17:58.934359+0000","last_clean":"2024-09-11T01:17:58.934359+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934359+0000","last_undegraded":"2024-09-11T01:17:58.934359+0000","last_fullsized":"2024-09-11T01:17:58.934359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13334","reported_seq":14879,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:29.460257+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:18:29.460257+0000","last_peered":"2024-09-11T01:18:29.460257+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:18:29.460257+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27193344,"num_objects":6639,"num_object_clones":0,"num_object_copies":13278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2641,"num_objects_misplaced":588,"num_objects_unfound":0,"num_objects_dirty":6639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11305,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":706,"num_bytes_recovered":2883584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":588},{"shards":"2","objects":3410}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13263","reported_seq":13395,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:26.459323+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:18:26.459323+0000","last_peered":"2024-09-11T01:18:26.459323+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:18:26.459323+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27054080,"num_objects":6605,"num_object_clones":0,"num_object_copies":13210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6605,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9850,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2621},{"shards":"0,2","objects":659}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13578","reported_seq":13701,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:29.412479+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:18:29.412479+0000","last_peered":"2024-09-11T01:18:29.412479+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:18:29.412479+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27152384,"num_objects":6629,"num_object_clones":0,"num_object_copies":13258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11395,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":551},{"shards":"2","objects":2605}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935080+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935080+0000","last_peered":"2024-09-11T01:17:58.935080+0000","last_clean":"2024-09-11T01:17:58.935080+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935080+0000","last_undegraded":"2024-09-11T01:17:58.935080+0000","last_fullsized":"2024-09-11T01:17:58.935080+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13296","reported_seq":21282,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:29.465966+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:18:29.465966+0000","last_peered":"2024-09-11T01:18:29.465966+0000","last_clean":"2024-09-11T01:18:29.465966+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:18:29.465966+0000","last_undegraded":"2024-09-11T01:18:29.465966+0000","last_fullsized":"2024-09-11T01:18:29.465966+0000","mapping_epoch":44,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26624000,"num_objects":6500,"num_object_clones":0,"num_object_copies":13000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11081,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13489","reported_seq":13625,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:26.528375+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:18:26.528375+0000","last_peered":"2024-09-11T01:18:26.528375+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:18:26.528375+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27750400,"num_objects":6775,"num_object_clones":0,"num_object_copies":13550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6775,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10226,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2825},{"shards":"0,2","objects":594}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938540+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938540+0000","last_peered":"2024-09-11T01:17:58.938540+0000","last_clean":"2024-09-11T01:17:58.938540+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938540+0000","last_undegraded":"2024-09-11T01:17:58.938540+0000","last_fullsized":"2024-09-11T01:17:58.938540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13368","reported_seq":21404,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:30.372708+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:18:30.372708+0000","last_peered":"2024-09-11T01:18:30.372708+0000","last_clean":"2024-09-11T01:18:30.372708+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:18:30.372708+0000","last_undegraded":"2024-09-11T01:18:30.372708+0000","last_fullsized":"2024-09-11T01:18:30.372708+0000","mapping_epoch":46,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26689536,"num_objects":6516,"num_object_clones":0,"num_object_copies":13032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10023,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13515","reported_seq":13655,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:26.535836+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:18:26.535836+0000","last_peered":"2024-09-11T01:18:26.535836+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:18:26.535836+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27283456,"num_objects":6661,"num_object_clones":0,"num_object_copies":13322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10124,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2622},{"shards":"0,2","objects":614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971314+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971314+0000","last_peered":"2024-09-11T01:17:58.971314+0000","last_clean":"2024-09-11T01:17:58.971314+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971314+0000","last_undegraded":"2024-09-11T01:17:58.971314+0000","last_fullsized":"2024-09-11T01:17:58.971314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13281","reported_seq":13420,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:26.505533+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:18:26.505533+0000","last_peered":"2024-09-11T01:18:26.505533+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:18:26.505533+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27325033,"num_objects":6672,"num_object_clones":0,"num_object_copies":13344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10351,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2755},{"shards":"0,2","objects":618}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":436261505,"num_objects":106511,"num_object_clones":0,"num_object_copies":213022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38475,"num_objects_misplaced":588,"num_objects_unfound":0,"num_objects_dirty":106511,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":175776,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8576,"num_bytes_recovered":35098624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1043197952,"data_stored":1043127186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160926,"ondisk_log_size":160926,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182483,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":764364,"kb_used_data":385660,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93607476,"statfs":{"total":96636764160,"available":95854055424,"internally_reserved":0,"allocated":394915840,"data_stored":394646181,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":387772075},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,0,0,1,3,3,3,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986256,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":646876,"kb_used_data":340360,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93724964,"statfs":{"total":96636764160,"available":95974363136,"internally_reserved":0,"allocated":348528640,"data_stored":348258986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659201,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":597792,"kb_used_data":295436,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93774048,"statfs":{"total":96636764160,"available":96024625152,"internally_reserved":0,"allocated":302526464,"data_stored":302200548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":309587631},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":393662464,"data_stored":393658985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":301969408,"data_stored":301909672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":347566080,"data_stored":347558529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:31.594 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:18:31.594 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'13536", 'reported_seq': 13618, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:26.576692+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:18:26.576692+0000', 'last_peered': '2024-09-11T01:18:26.576692+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:18:26.576692+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "36'3500", 'ondisk_log_start': "36'3500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27566080, 'num_objects': 6730, 'num_object_clones': 0, 'num_object_copies': 13460, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6730, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13536, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2658}, {'shards': '0,2', 'objects': 669}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:31.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:18:31.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'13444", 'reported_seq': 13577, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:29.473758+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:18:29.473758+0000', 'last_peered': '2024-09-11T01:18:29.473758+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:18:29.473758+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "36'3400", 'ondisk_log_start': "36'3400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27439128, 'num_objects': 6700, 'num_object_clones': 0, 'num_object_copies': 13400, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6700, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11745, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 526}, {'shards': '2', 'objects': 2804}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:31.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:18:31.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'13488", 'reported_seq': 13662, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:26.504012+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:18:26.504012+0000', 'last_peered': '2024-09-11T01:18:26.504012+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:18:26.504012+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "36'3400", 'ondisk_log_start': "36'3400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27602944, 'num_objects': 6739, 'num_object_clones': 0, 'num_object_copies': 13478, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6739, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9829, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2788}, {'shards': '0,2', 'objects': 579}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:31.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:18:31.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'13291", 'reported_seq': 13426, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:26.469080+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:18:26.469080+0000', 'last_peered': '2024-09-11T01:18:26.469080+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:18:26.469080+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "36'3200", 'ondisk_log_start': "36'3200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27193344, 'num_objects': 6639, 'num_object_clones': 0, 'num_object_copies': 13278, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6639, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10148, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2653}, {'shards': '0,2', 'objects': 663}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:31.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:18:31.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'13334", 'reported_seq': 14879, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:29.460257+0000', 'last_change': '2024-09-11T01:17:57.355893+0000', 'last_active': '2024-09-11T01:18:29.460257+0000', 'last_peered': '2024-09-11T01:18:29.460257+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:18:29.460257+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "36'3300", 'ondisk_log_start': "36'3300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27193344, 'num_objects': 6639, 'num_object_clones': 0, 'num_object_copies': 13278, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2641, 'num_objects_misplaced': 588, 'num_objects_unfound': 0, 'num_objects_dirty': 6639, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11305, 'num_write_kb': 31844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 706, 'num_bytes_recovered': 2883584, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 588}, {'shards': '2', 'objects': 3410}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:31.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:18:31.595 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'13263", 'reported_seq': 13395, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:26.459323+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:18:26.459323+0000', 'last_peered': '2024-09-11T01:18:26.459323+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:18:26.459323+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "36'3200", 'ondisk_log_start': "36'3200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27054080, 'num_objects': 6605, 'num_object_clones': 0, 'num_object_copies': 13210, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6605, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9850, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2621}, {'shards': '0,2', 'objects': 659}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:31.596 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:18:31.596 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'13578", 'reported_seq': 13701, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:29.412479+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:18:29.412479+0000', 'last_peered': '2024-09-11T01:18:29.412479+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:18:29.412479+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "36'3500", 'ondisk_log_start': "36'3500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27152384, 'num_objects': 6629, 'num_object_clones': 0, 'num_object_copies': 13258, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6629, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11395, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 551}, {'shards': '2', 'objects': 2605}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:31.596 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:18:31.596 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'13489", 'reported_seq': 13625, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:26.528375+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:18:26.528375+0000', 'last_peered': '2024-09-11T01:18:26.528375+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:18:26.528375+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "36'3400", 'ondisk_log_start': "36'3400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27750400, 'num_objects': 6775, 'num_object_clones': 0, 'num_object_copies': 13550, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6775, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10226, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2825}, {'shards': '0,2', 'objects': 594}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:31.596 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:18:31.596 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'13515", 'reported_seq': 13655, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:26.535836+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:18:26.535836+0000', 'last_peered': '2024-09-11T01:18:26.535836+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:18:26.535836+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "36'3500", 'ondisk_log_start': "36'3500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27283456, 'num_objects': 6661, 'num_object_clones': 0, 'num_object_copies': 13322, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6661, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10124, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2622}, {'shards': '0,2', 'objects': 614}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:31.596 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:18:31.596 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'13281", 'reported_seq': 13420, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:26.505533+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:18:26.505533+0000', 'last_peered': '2024-09-11T01:18:26.505533+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:18:26.505533+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "36'3200", 'ondisk_log_start': "36'3200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27325033, 'num_objects': 6672, 'num_object_clones': 0, 'num_object_copies': 13344, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6672, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10351, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2755}, {'shards': '0,2', 'objects': 618}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:31.597 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:18:32.623 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:18:32.623 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38475/213026 objects degraded (18.061%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":449,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":106513,"data_bytes":436261524,"bytes_used":2057248768,"bytes_avail":287853043712,"bytes_total":289910292480,"degraded_objects":38475,"degraded_total":213026,"degraded_ratio":0.18061175631143617,"misplaced_objects":588,"misplaced_total":213026,"misplaced_ratio":0.0027602264512313051,"recovering_objects_per_sec":28,"recovering_bytes_per_sec":114941,"recovering_keys_per_sec":0,"num_objects_recovered":337,"num_bytes_recovered":1380352,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":450},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:18:32.685 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:18:32.685 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:33.514 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:18:33.515 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:18:33.581 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":265,"stamp":"2024-09-11T01:18:32.696832+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":430949012,"num_objects":105216,"num_object_clones":0,"num_object_copies":210432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38475,"num_objects_misplaced":588,"num_objects_unfound":0,"num_objects_dirty":105216,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":177075,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8577,"num_bytes_recovered":35098643,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160726,"ondisk_log_size":160726,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2010512,"kb_used_data":1014616,"kb_used_omap":12,"kb_used_meta":995827,"kb_avail":281105008,"statfs":{"total":289910292480,"available":287851528192,"internally_reserved":0,"allocated":1038966784,"data_stored":1038101555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1019726852},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,3,2,5,6,16,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23760896,"num_objects":-5801,"num_object_clones":0,"num_object_copies":-11602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":28,"num_objects_misplaced":-251,"num_objects_unfound":0,"num_objects_dirty":-5801,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5801,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":913408,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.009335"},"pg_stats":[{"pgid":"2.f","version":"47'13706","reported_seq":13785,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:31.687717+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:18:31.687717+0000","last_peered":"2024-09-11T01:18:31.687717+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:18:31.687717+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26869760,"num_objects":6560,"num_object_clones":0,"num_object_copies":13120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13706,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2488},{"shards":"0,2","objects":499}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13711","reported_seq":13817,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:29.432038+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:18:29.432038+0000","last_peered":"2024-09-11T01:18:29.432038+0000","last_clean":"2024-09-11T01:18:29.432038+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:18:29.432038+0000","last_undegraded":"2024-09-11T01:18:29.432038+0000","last_fullsized":"2024-09-11T01:18:29.432038+0000","mapping_epoch":35,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27201536,"num_objects":6641,"num_object_clones":0,"num_object_copies":13282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6641,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11950,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13444","reported_seq":13577,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:29.473758+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:18:29.473758+0000","last_peered":"2024-09-11T01:18:29.473758+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:18:29.473758+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27439128,"num_objects":6700,"num_object_clones":0,"num_object_copies":13400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6700,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11745,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":526},{"shards":"2","objects":2804}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'13490","reported_seq":13564,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:29.406027+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:18:29.406027+0000","last_peered":"2024-09-11T01:18:29.406027+0000","last_clean":"2024-09-11T01:18:29.406027+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:18:29.406027+0000","last_undegraded":"2024-09-11T01:18:29.406027+0000","last_fullsized":"2024-09-11T01:18:29.406027+0000","mapping_epoch":35,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27213824,"num_objects":6644,"num_object_clones":0,"num_object_copies":13288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13490,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13445","reported_seq":13578,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:31.629942+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:18:31.629942+0000","last_peered":"2024-09-11T01:18:31.629942+0000","last_clean":"2024-09-11T01:18:31.629942+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:18:31.629942+0000","last_undegraded":"2024-09-11T01:18:31.629942+0000","last_fullsized":"2024-09-11T01:18:31.629942+0000","mapping_epoch":34,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26886144,"num_objects":6564,"num_object_clones":0,"num_object_copies":13128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6564,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10450,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13620","reported_seq":13795,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:31.607351+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:18:31.607351+0000","last_peered":"2024-09-11T01:18:31.607351+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:18:31.607351+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"39'3600","ondisk_log_start":"39'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27062272,"num_objects":6607,"num_object_clones":0,"num_object_copies":13214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9961,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2656},{"shards":"0,2","objects":447}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'13439","reported_seq":13574,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:31.599319+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:18:31.599319+0000","last_peered":"2024-09-11T01:18:31.599319+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:18:31.599319+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26587136,"num_objects":6491,"num_object_clones":0,"num_object_copies":12982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10296,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2505},{"shards":"0,2","objects":515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13667","reported_seq":13802,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:31.507256+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:18:31.507256+0000","last_peered":"2024-09-11T01:18:31.507256+0000","last_clean":"2024-09-11T01:18:31.507256+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:18:31.507256+0000","last_undegraded":"2024-09-11T01:18:31.507256+0000","last_fullsized":"2024-09-11T01:18:31.507256+0000","mapping_epoch":34,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26980352,"num_objects":6587,"num_object_clones":0,"num_object_copies":13174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10543,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934359+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934359+0000","last_peered":"2024-09-11T01:17:58.934359+0000","last_clean":"2024-09-11T01:17:58.934359+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934359+0000","last_undegraded":"2024-09-11T01:17:58.934359+0000","last_fullsized":"2024-09-11T01:17:58.934359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13334","reported_seq":14879,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:29.460257+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:18:29.460257+0000","last_peered":"2024-09-11T01:18:29.460257+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:18:29.460257+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27193344,"num_objects":6639,"num_object_clones":0,"num_object_copies":13278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2641,"num_objects_misplaced":588,"num_objects_unfound":0,"num_objects_dirty":6639,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11305,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":706,"num_bytes_recovered":2883584,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":588},{"shards":"2","objects":3410}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13406","reported_seq":13539,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:31.665008+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:18:31.665008+0000","last_peered":"2024-09-11T01:18:31.665008+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:18:31.665008+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26468352,"num_objects":6462,"num_object_clones":0,"num_object_copies":12924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9993,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2478},{"shards":"0,2","objects":516}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13578","reported_seq":13701,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:29.412479+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:18:29.412479+0000","last_peered":"2024-09-11T01:18:29.412479+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:18:29.412479+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27152384,"num_objects":6629,"num_object_clones":0,"num_object_copies":13258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6629,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11395,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":551},{"shards":"2","objects":2605}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935080+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935080+0000","last_peered":"2024-09-11T01:17:58.935080+0000","last_clean":"2024-09-11T01:17:58.935080+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935080+0000","last_undegraded":"2024-09-11T01:17:58.935080+0000","last_fullsized":"2024-09-11T01:17:58.935080+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13296","reported_seq":21282,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:29.465966+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:18:29.465966+0000","last_peered":"2024-09-11T01:18:29.465966+0000","last_clean":"2024-09-11T01:18:29.465966+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:18:29.465966+0000","last_undegraded":"2024-09-11T01:18:29.465966+0000","last_fullsized":"2024-09-11T01:18:29.465966+0000","mapping_epoch":44,"log_start":"36'3200","ondisk_log_start":"36'3200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26624000,"num_objects":6500,"num_object_clones":0,"num_object_copies":13000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11081,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13644","reported_seq":13780,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:31.638010+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:18:31.638010+0000","last_peered":"2024-09-11T01:18:31.638010+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:18:31.638010+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"39'3600","ondisk_log_start":"39'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27115520,"num_objects":6620,"num_object_clones":0,"num_object_copies":13240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10381,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2670},{"shards":"0,2","objects":439}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938540+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938540+0000","last_peered":"2024-09-11T01:17:58.938540+0000","last_clean":"2024-09-11T01:17:58.938540+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938540+0000","last_undegraded":"2024-09-11T01:17:58.938540+0000","last_fullsized":"2024-09-11T01:17:58.938540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13368","reported_seq":21404,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:30.372708+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:18:30.372708+0000","last_peered":"2024-09-11T01:18:30.372708+0000","last_clean":"2024-09-11T01:18:30.372708+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:18:30.372708+0000","last_undegraded":"2024-09-11T01:18:30.372708+0000","last_fullsized":"2024-09-11T01:18:30.372708+0000","mapping_epoch":46,"log_start":"36'3300","ondisk_log_start":"36'3300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26689536,"num_objects":6516,"num_object_clones":0,"num_object_copies":13032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10023,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13650","reported_seq":13790,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:31.487609+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:18:31.487609+0000","last_peered":"2024-09-11T01:18:31.487609+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:18:31.487609+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26730496,"num_objects":6526,"num_object_clones":0,"num_object_copies":13052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10259,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2487},{"shards":"0,2","objects":479}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971314+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971314+0000","last_peered":"2024-09-11T01:17:58.971314+0000","last_clean":"2024-09-11T01:17:58.971314+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971314+0000","last_undegraded":"2024-09-11T01:17:58.971314+0000","last_fullsized":"2024-09-11T01:17:58.971314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13425","reported_seq":13564,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:31.584700+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:18:31.584700+0000","last_peered":"2024-09-11T01:18:31.584700+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:18:31.584700+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26735209,"num_objects":6528,"num_object_clones":0,"num_object_copies":13056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10495,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2611},{"shards":"0,2","objects":474}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":430948993,"num_objects":105214,"num_object_clones":0,"num_object_copies":210428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38475,"num_objects_misplaced":588,"num_objects_unfound":0,"num_objects_dirty":105214,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":177073,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8576,"num_bytes_recovered":35098624,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1036070912,"data_stored":1036000146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160723,"ondisk_log_size":160723,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182484,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":765844,"kb_used_data":378820,"kb_used_omap":4,"kb_used_meta":387003,"kb_avail":93605996,"statfs":{"total":96636764160,"available":95852539904,"internally_reserved":0,"allocated":387911680,"data_stored":387642021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":396291755},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,1,1,1,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986256,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":646876,"kb_used_data":340360,"kb_used_omap":4,"kb_used_meta":306491,"kb_avail":93724964,"statfs":{"total":96636764160,"available":95974363136,"internally_reserved":0,"allocated":348528640,"data_stored":348258986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":313847466},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,3,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659201,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":597792,"kb_used_data":295436,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93774048,"statfs":{"total":96636764160,"available":96024625152,"internally_reserved":0,"allocated":302526464,"data_stored":302200548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":309587631},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":386535424,"data_stored":386531945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":301969408,"data_stored":301909672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":347566080,"data_stored":347558529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:36.582 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:37.356 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:18:37.358 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:18:37.427 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":267,"stamp":"2024-09-11T01:18:36.697834+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":426947220,"num_objects":104239,"num_object_clones":0,"num_object_copies":208478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38508,"num_objects_misplaced":446,"num_objects_unfound":0,"num_objects_dirty":104239,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":178052,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8686,"num_bytes_recovered":35545107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160603,"ondisk_log_size":160603,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2035584,"kb_used_data":1009168,"kb_used_omap":12,"kb_used_meta":1026355,"kb_avail":281079936,"statfs":{"total":289910292480,"available":287825854464,"internally_reserved":0,"allocated":1033388032,"data_stored":1032522803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1050987524},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,5,2,8,10,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19140608,"num_objects":-4673,"num_object_clones":0,"num_object_copies":-9346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39,"num_objects_misplaced":-263,"num_objects_unfound":0,"num_objects_dirty":-4673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4673,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":917504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008974"},"pg_stats":[{"pgid":"2.f","version":"47'13706","reported_seq":13785,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:31.687717+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:18:31.687717+0000","last_peered":"2024-09-11T01:18:31.687717+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:18:31.687717+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26869760,"num_objects":6560,"num_object_clones":0,"num_object_copies":13120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13706,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2488},{"shards":"0,2","objects":499}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13850","reported_seq":13956,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:34.494713+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:18:34.494713+0000","last_peered":"2024-09-11T01:18:34.494713+0000","last_clean":"2024-09-11T01:18:34.494713+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:18:34.494713+0000","last_undegraded":"2024-09-11T01:18:34.494713+0000","last_fullsized":"2024-09-11T01:18:34.494713+0000","mapping_epoch":35,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26632192,"num_objects":6502,"num_object_clones":0,"num_object_copies":13004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12089,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13592","reported_seq":13725,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:34.482351+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:18:34.482351+0000","last_peered":"2024-09-11T01:18:34.482351+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:18:34.482351+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26832920,"num_objects":6552,"num_object_clones":0,"num_object_copies":13104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6552,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11893,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":378},{"shards":"2","objects":2656}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'13612","reported_seq":13686,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:34.494698+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:18:34.494698+0000","last_peered":"2024-09-11T01:18:34.494698+0000","last_clean":"2024-09-11T01:18:34.494698+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:18:34.494698+0000","last_undegraded":"2024-09-11T01:18:34.494698+0000","last_fullsized":"2024-09-11T01:18:34.494698+0000","mapping_epoch":35,"log_start":"39'3600","ondisk_log_start":"39'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26714112,"num_objects":6522,"num_object_clones":0,"num_object_copies":13044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13612,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13445","reported_seq":13578,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:31.629942+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:18:31.629942+0000","last_peered":"2024-09-11T01:18:31.629942+0000","last_clean":"2024-09-11T01:18:31.629942+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:18:31.629942+0000","last_undegraded":"2024-09-11T01:18:31.629942+0000","last_fullsized":"2024-09-11T01:18:31.629942+0000","mapping_epoch":34,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26886144,"num_objects":6564,"num_object_clones":0,"num_object_copies":13128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6564,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10450,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13620","reported_seq":13795,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:31.607351+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:18:31.607351+0000","last_peered":"2024-09-11T01:18:31.607351+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:18:31.607351+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"39'3600","ondisk_log_start":"39'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27062272,"num_objects":6607,"num_object_clones":0,"num_object_copies":13214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9961,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2656},{"shards":"0,2","objects":447}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'13439","reported_seq":13574,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:31.599319+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:18:31.599319+0000","last_peered":"2024-09-11T01:18:31.599319+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:18:31.599319+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26587136,"num_objects":6491,"num_object_clones":0,"num_object_copies":12982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10296,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2505},{"shards":"0,2","objects":515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13667","reported_seq":13802,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:31.507256+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:18:31.507256+0000","last_peered":"2024-09-11T01:18:31.507256+0000","last_clean":"2024-09-11T01:18:31.507256+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:18:31.507256+0000","last_undegraded":"2024-09-11T01:18:31.507256+0000","last_fullsized":"2024-09-11T01:18:31.507256+0000","mapping_epoch":34,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26980352,"num_objects":6587,"num_object_clones":0,"num_object_copies":13174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10543,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934359+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934359+0000","last_peered":"2024-09-11T01:17:58.934359+0000","last_clean":"2024-09-11T01:17:58.934359+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934359+0000","last_undegraded":"2024-09-11T01:17:58.934359+0000","last_fullsized":"2024-09-11T01:17:58.934359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13476","reported_seq":15239,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:34.491932+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:18:34.491932+0000","last_peered":"2024-09-11T01:18:34.491932+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:18:34.491932+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26611712,"num_objects":6497,"num_object_clones":0,"num_object_copies":12994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2674,"num_objects_misplaced":446,"num_objects_unfound":0,"num_objects_dirty":6497,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11447,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":815,"num_bytes_recovered":3330048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":446},{"shards":"2","objects":3377}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13406","reported_seq":13539,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:31.665008+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:18:31.665008+0000","last_peered":"2024-09-11T01:18:31.665008+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:18:31.665008+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26468352,"num_objects":6462,"num_object_clones":0,"num_object_copies":12924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9993,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2478},{"shards":"0,2","objects":516}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13716","reported_seq":13840,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:34.446695+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:18:34.446695+0000","last_peered":"2024-09-11T01:18:34.446695+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:18:34.446695+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26587136,"num_objects":6491,"num_object_clones":0,"num_object_copies":12982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11533,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":413},{"shards":"2","objects":2467}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935080+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935080+0000","last_peered":"2024-09-11T01:17:58.935080+0000","last_clean":"2024-09-11T01:17:58.935080+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935080+0000","last_undegraded":"2024-09-11T01:17:58.935080+0000","last_fullsized":"2024-09-11T01:17:58.935080+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13440","reported_seq":21425,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:34.482983+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:18:34.482983+0000","last_peered":"2024-09-11T01:18:34.482983+0000","last_clean":"2024-09-11T01:18:34.482983+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:18:34.482983+0000","last_undegraded":"2024-09-11T01:18:34.482983+0000","last_fullsized":"2024-09-11T01:18:34.482983+0000","mapping_epoch":44,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26034176,"num_objects":6356,"num_object_clones":0,"num_object_copies":12712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11225,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13644","reported_seq":13780,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:31.638010+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:18:31.638010+0000","last_peered":"2024-09-11T01:18:31.638010+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:18:31.638010+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"39'3600","ondisk_log_start":"39'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27115520,"num_objects":6620,"num_object_clones":0,"num_object_copies":13240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10381,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2670},{"shards":"0,2","objects":439}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938540+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938540+0000","last_peered":"2024-09-11T01:17:58.938540+0000","last_clean":"2024-09-11T01:17:58.938540+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938540+0000","last_undegraded":"2024-09-11T01:17:58.938540+0000","last_fullsized":"2024-09-11T01:17:58.938540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13512","reported_seq":21548,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:35.488819+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:18:35.488819+0000","last_peered":"2024-09-11T01:18:35.488819+0000","last_clean":"2024-09-11T01:18:35.488819+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:18:35.488819+0000","last_undegraded":"2024-09-11T01:18:35.488819+0000","last_fullsized":"2024-09-11T01:18:35.488819+0000","mapping_epoch":46,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26099712,"num_objects":6372,"num_object_clones":0,"num_object_copies":12744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10167,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13650","reported_seq":13790,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:31.487609+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:18:31.487609+0000","last_peered":"2024-09-11T01:18:31.487609+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:18:31.487609+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26730496,"num_objects":6526,"num_object_clones":0,"num_object_copies":13052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10259,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2487},{"shards":"0,2","objects":479}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971314+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971314+0000","last_peered":"2024-09-11T01:17:58.971314+0000","last_clean":"2024-09-11T01:17:58.971314+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971314+0000","last_undegraded":"2024-09-11T01:17:58.971314+0000","last_fullsized":"2024-09-11T01:17:58.971314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13425","reported_seq":13564,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:31.584700+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:18:31.584700+0000","last_peered":"2024-09-11T01:18:31.584700+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:18:31.584700+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26735209,"num_objects":6528,"num_object_clones":0,"num_object_copies":13056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10495,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2611},{"shards":"0,2","objects":474}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":426947201,"num_objects":104237,"num_object_clones":0,"num_object_copies":208474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38508,"num_objects_misplaced":446,"num_objects_unfound":0,"num_objects_dirty":104237,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":178050,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8685,"num_bytes_recovered":35545088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1030410240,"data_stored":1030339474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160600,"ondisk_log_size":160600,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182484,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":765844,"kb_used_data":378820,"kb_used_omap":4,"kb_used_meta":387003,"kb_avail":93605996,"statfs":{"total":96636764160,"available":95852539904,"internally_reserved":0,"allocated":387911680,"data_stored":387642021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":396291755},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,1,1,1,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986257,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":646672,"kb_used_data":336000,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93725168,"statfs":{"total":96636764160,"available":95974572032,"internally_reserved":0,"allocated":344064000,"data_stored":343794346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659202,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":623068,"kb_used_data":294348,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93748772,"statfs":{"total":96636764160,"available":95998742528,"internally_reserved":0,"allocated":301412352,"data_stored":301086436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":336588463},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,5,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":386535424,"data_stored":386531945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":300838912,"data_stored":300779176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":343035904,"data_stored":343028353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:37.428 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:38.205 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:18:38.205 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:18:38.240 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":267,"stamp":"2024-09-11T01:18:36.697834+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":426947220,"num_objects":104239,"num_object_clones":0,"num_object_copies":208478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38508,"num_objects_misplaced":446,"num_objects_unfound":0,"num_objects_dirty":104239,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":178052,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8686,"num_bytes_recovered":35545107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160603,"ondisk_log_size":160603,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2035584,"kb_used_data":1009168,"kb_used_omap":12,"kb_used_meta":1026355,"kb_avail":281079936,"statfs":{"total":289910292480,"available":287825854464,"internally_reserved":0,"allocated":1033388032,"data_stored":1032522803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1050987524},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,5,2,8,10,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19140608,"num_objects":-4673,"num_object_clones":0,"num_object_copies":-9346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39,"num_objects_misplaced":-263,"num_objects_unfound":0,"num_objects_dirty":-4673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4673,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":917504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008974"},"pg_stats":[{"pgid":"2.f","version":"47'13706","reported_seq":13785,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:31.687717+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:18:31.687717+0000","last_peered":"2024-09-11T01:18:31.687717+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:18:31.687717+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26869760,"num_objects":6560,"num_object_clones":0,"num_object_copies":13120,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6560,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13706,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2488},{"shards":"0,2","objects":499}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13850","reported_seq":13956,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:34.494713+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:18:34.494713+0000","last_peered":"2024-09-11T01:18:34.494713+0000","last_clean":"2024-09-11T01:18:34.494713+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:18:34.494713+0000","last_undegraded":"2024-09-11T01:18:34.494713+0000","last_fullsized":"2024-09-11T01:18:34.494713+0000","mapping_epoch":35,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26632192,"num_objects":6502,"num_object_clones":0,"num_object_copies":13004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12089,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13592","reported_seq":13725,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:34.482351+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:18:34.482351+0000","last_peered":"2024-09-11T01:18:34.482351+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:18:34.482351+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26832920,"num_objects":6552,"num_object_clones":0,"num_object_copies":13104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6552,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11893,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":378},{"shards":"2","objects":2656}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'13612","reported_seq":13686,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:34.494698+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:18:34.494698+0000","last_peered":"2024-09-11T01:18:34.494698+0000","last_clean":"2024-09-11T01:18:34.494698+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:18:34.494698+0000","last_undegraded":"2024-09-11T01:18:34.494698+0000","last_fullsized":"2024-09-11T01:18:34.494698+0000","mapping_epoch":35,"log_start":"39'3600","ondisk_log_start":"39'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26714112,"num_objects":6522,"num_object_clones":0,"num_object_copies":13044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13612,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13445","reported_seq":13578,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:31.629942+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:18:31.629942+0000","last_peered":"2024-09-11T01:18:31.629942+0000","last_clean":"2024-09-11T01:18:31.629942+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:18:31.629942+0000","last_undegraded":"2024-09-11T01:18:31.629942+0000","last_fullsized":"2024-09-11T01:18:31.629942+0000","mapping_epoch":34,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26886144,"num_objects":6564,"num_object_clones":0,"num_object_copies":13128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6564,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10450,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13620","reported_seq":13795,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:31.607351+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:18:31.607351+0000","last_peered":"2024-09-11T01:18:31.607351+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:18:31.607351+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"39'3600","ondisk_log_start":"39'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27062272,"num_objects":6607,"num_object_clones":0,"num_object_copies":13214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9961,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2656},{"shards":"0,2","objects":447}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'13439","reported_seq":13574,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:31.599319+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:18:31.599319+0000","last_peered":"2024-09-11T01:18:31.599319+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:18:31.599319+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26587136,"num_objects":6491,"num_object_clones":0,"num_object_copies":12982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10296,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2505},{"shards":"0,2","objects":515}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13667","reported_seq":13802,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:31.507256+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:18:31.507256+0000","last_peered":"2024-09-11T01:18:31.507256+0000","last_clean":"2024-09-11T01:18:31.507256+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:18:31.507256+0000","last_undegraded":"2024-09-11T01:18:31.507256+0000","last_fullsized":"2024-09-11T01:18:31.507256+0000","mapping_epoch":34,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26980352,"num_objects":6587,"num_object_clones":0,"num_object_copies":13174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6587,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10543,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934359+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934359+0000","last_peered":"2024-09-11T01:17:58.934359+0000","last_clean":"2024-09-11T01:17:58.934359+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934359+0000","last_undegraded":"2024-09-11T01:17:58.934359+0000","last_fullsized":"2024-09-11T01:17:58.934359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13476","reported_seq":15239,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:34.491932+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:18:34.491932+0000","last_peered":"2024-09-11T01:18:34.491932+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:18:34.491932+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26611712,"num_objects":6497,"num_object_clones":0,"num_object_copies":12994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2674,"num_objects_misplaced":446,"num_objects_unfound":0,"num_objects_dirty":6497,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11447,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":815,"num_bytes_recovered":3330048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":446},{"shards":"2","objects":3377}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13406","reported_seq":13539,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:31.665008+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:18:31.665008+0000","last_peered":"2024-09-11T01:18:31.665008+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:18:31.665008+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26468352,"num_objects":6462,"num_object_clones":0,"num_object_copies":12924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9993,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2478},{"shards":"0,2","objects":516}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13716","reported_seq":13840,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:34.446695+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:18:34.446695+0000","last_peered":"2024-09-11T01:18:34.446695+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:18:34.446695+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26587136,"num_objects":6491,"num_object_clones":0,"num_object_copies":12982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11533,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":413},{"shards":"2","objects":2467}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935080+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935080+0000","last_peered":"2024-09-11T01:17:58.935080+0000","last_clean":"2024-09-11T01:17:58.935080+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935080+0000","last_undegraded":"2024-09-11T01:17:58.935080+0000","last_fullsized":"2024-09-11T01:17:58.935080+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13440","reported_seq":21425,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:34.482983+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:18:34.482983+0000","last_peered":"2024-09-11T01:18:34.482983+0000","last_clean":"2024-09-11T01:18:34.482983+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:18:34.482983+0000","last_undegraded":"2024-09-11T01:18:34.482983+0000","last_fullsized":"2024-09-11T01:18:34.482983+0000","mapping_epoch":44,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26034176,"num_objects":6356,"num_object_clones":0,"num_object_copies":12712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11225,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13644","reported_seq":13780,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:31.638010+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:18:31.638010+0000","last_peered":"2024-09-11T01:18:31.638010+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:18:31.638010+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"39'3600","ondisk_log_start":"39'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":27115520,"num_objects":6620,"num_object_clones":0,"num_object_copies":13240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10381,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2670},{"shards":"0,2","objects":439}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938540+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938540+0000","last_peered":"2024-09-11T01:17:58.938540+0000","last_clean":"2024-09-11T01:17:58.938540+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938540+0000","last_undegraded":"2024-09-11T01:17:58.938540+0000","last_fullsized":"2024-09-11T01:17:58.938540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13512","reported_seq":21548,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:35.488819+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:18:35.488819+0000","last_peered":"2024-09-11T01:18:35.488819+0000","last_clean":"2024-09-11T01:18:35.488819+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:18:35.488819+0000","last_undegraded":"2024-09-11T01:18:35.488819+0000","last_fullsized":"2024-09-11T01:18:35.488819+0000","mapping_epoch":46,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26099712,"num_objects":6372,"num_object_clones":0,"num_object_copies":12744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10167,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13650","reported_seq":13790,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:31.487609+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:18:31.487609+0000","last_peered":"2024-09-11T01:18:31.487609+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:18:31.487609+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"36'3600","ondisk_log_start":"36'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26730496,"num_objects":6526,"num_object_clones":0,"num_object_copies":13052,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6526,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10259,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2487},{"shards":"0,2","objects":479}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971314+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971314+0000","last_peered":"2024-09-11T01:17:58.971314+0000","last_clean":"2024-09-11T01:17:58.971314+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971314+0000","last_undegraded":"2024-09-11T01:17:58.971314+0000","last_fullsized":"2024-09-11T01:17:58.971314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13425","reported_seq":13564,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:31.584700+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:18:31.584700+0000","last_peered":"2024-09-11T01:18:31.584700+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:18:31.584700+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26735209,"num_objects":6528,"num_object_clones":0,"num_object_copies":13056,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6528,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10495,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2611},{"shards":"0,2","objects":474}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":426947201,"num_objects":104237,"num_object_clones":0,"num_object_copies":208474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38508,"num_objects_misplaced":446,"num_objects_unfound":0,"num_objects_dirty":104237,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":178050,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8685,"num_bytes_recovered":35545088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1030410240,"data_stored":1030339474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160600,"ondisk_log_size":160600,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182484,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":765844,"kb_used_data":378820,"kb_used_omap":4,"kb_used_meta":387003,"kb_avail":93605996,"statfs":{"total":96636764160,"available":95852539904,"internally_reserved":0,"allocated":387911680,"data_stored":387642021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":396291755},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,1,1,1,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986257,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":646672,"kb_used_data":336000,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93725168,"statfs":{"total":96636764160,"available":95974572032,"internally_reserved":0,"allocated":344064000,"data_stored":343794346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659202,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":623068,"kb_used_data":294348,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93748772,"statfs":{"total":96636764160,"available":95998742528,"internally_reserved":0,"allocated":301412352,"data_stored":301086436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":336588463},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,5,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":386535424,"data_stored":386531945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":300838912,"data_stored":300779176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":343035904,"data_stored":343028353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:38.241 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:18:38.241 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'13706", 'reported_seq': 13785, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:31.687717+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:18:31.687717+0000', 'last_peered': '2024-09-11T01:18:31.687717+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:18:31.687717+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "39'3700", 'ondisk_log_start': "39'3700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26869760, 'num_objects': 6560, 'num_object_clones': 0, 'num_object_copies': 13120, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6560, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13706, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2488}, {'shards': '0,2', 'objects': 499}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:38.242 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:18:38.242 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'13592", 'reported_seq': 13725, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:34.482351+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:18:34.482351+0000', 'last_peered': '2024-09-11T01:18:34.482351+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:18:34.482351+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "36'3500", 'ondisk_log_start': "36'3500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26832920, 'num_objects': 6552, 'num_object_clones': 0, 'num_object_copies': 13104, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6552, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 11893, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 378}, {'shards': '2', 'objects': 2656}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:38.242 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:18:38.242 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'13620", 'reported_seq': 13795, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:31.607351+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:18:31.607351+0000', 'last_peered': '2024-09-11T01:18:31.607351+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:18:31.607351+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "39'3600", 'ondisk_log_start': "39'3600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27062272, 'num_objects': 6607, 'num_object_clones': 0, 'num_object_copies': 13214, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6607, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9961, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2656}, {'shards': '0,2', 'objects': 447}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:38.242 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:18:38.242 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'13439", 'reported_seq': 13574, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:31.599319+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:18:31.599319+0000', 'last_peered': '2024-09-11T01:18:31.599319+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:18:31.599319+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "36'3400", 'ondisk_log_start': "36'3400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26587136, 'num_objects': 6491, 'num_object_clones': 0, 'num_object_copies': 12982, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6491, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10296, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2505}, {'shards': '0,2', 'objects': 515}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:38.242 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:18:38.242 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'13476", 'reported_seq': 15239, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:34.491932+0000', 'last_change': '2024-09-11T01:17:57.355893+0000', 'last_active': '2024-09-11T01:18:34.491932+0000', 'last_peered': '2024-09-11T01:18:34.491932+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:18:34.491932+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "36'3400", 'ondisk_log_start': "36'3400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26611712, 'num_objects': 6497, 'num_object_clones': 0, 'num_object_copies': 12994, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2674, 'num_objects_misplaced': 446, 'num_objects_unfound': 0, 'num_objects_dirty': 6497, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11447, 'num_write_kb': 31844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 815, 'num_bytes_recovered': 3330048, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 446}, {'shards': '2', 'objects': 3377}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:38.242 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:18:38.242 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'13406", 'reported_seq': 13539, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:31.665008+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:18:31.665008+0000', 'last_peered': '2024-09-11T01:18:31.665008+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:18:31.665008+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "36'3400", 'ondisk_log_start': "36'3400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26468352, 'num_objects': 6462, 'num_object_clones': 0, 'num_object_copies': 12924, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6462, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 9993, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2478}, {'shards': '0,2', 'objects': 516}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:38.242 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:18:38.243 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'13716", 'reported_seq': 13840, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:34.446695+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:18:34.446695+0000', 'last_peered': '2024-09-11T01:18:34.446695+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:18:34.446695+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "39'3700", 'ondisk_log_start': "39'3700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26587136, 'num_objects': 6491, 'num_object_clones': 0, 'num_object_copies': 12982, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6491, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11533, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 413}, {'shards': '2', 'objects': 2467}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:38.243 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:18:38.243 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'13644", 'reported_seq': 13780, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:31.638010+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:18:31.638010+0000', 'last_peered': '2024-09-11T01:18:31.638010+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:18:31.638010+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "39'3600", 'ondisk_log_start': "39'3600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 27115520, 'num_objects': 6620, 'num_object_clones': 0, 'num_object_copies': 13240, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6620, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10381, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2670}, {'shards': '0,2', 'objects': 439}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:38.243 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:18:38.243 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'13650", 'reported_seq': 13790, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:31.487609+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:18:31.487609+0000', 'last_peered': '2024-09-11T01:18:31.487609+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:18:31.487609+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "36'3600", 'ondisk_log_start': "36'3600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26730496, 'num_objects': 6526, 'num_object_clones': 0, 'num_object_copies': 13052, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6526, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10259, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2487}, {'shards': '0,2', 'objects': 479}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:38.243 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:18:38.243 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'13425", 'reported_seq': 13564, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:31.584700+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:18:31.584700+0000', 'last_peered': '2024-09-11T01:18:31.584700+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:18:31.584700+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "36'3400", 'ondisk_log_start': "36'3400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26735209, 'num_objects': 6528, 'num_object_clones': 0, 'num_object_copies': 13056, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6528, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10495, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2611}, {'shards': '0,2', 'objects': 474}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:38.243 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:18:39.255 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:18:39.255 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38508/205916 objects degraded (18.701%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":455,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":102958,"data_bytes":421700244,"bytes_used":2064908288,"bytes_avail":287845384192,"bytes_total":289910292480,"degraded_objects":38508,"degraded_total":205916,"degraded_ratio":0.18700829464441815,"misplaced_objects":446,"misplaced_total":205916,"misplaced_ratio":0.0021659317391557725,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":76430,"recovering_keys_per_sec":0,"num_objects_recovered":224,"num_bytes_recovered":917504,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":378},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:18:39.309 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:18:39.309 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:40.087 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:18:40.088 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:18:40.140 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":268,"stamp":"2024-09-11T01:18:38.698279+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":421700244,"num_objects":102958,"num_object_clones":0,"num_object_copies":205916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38508,"num_objects_misplaced":446,"num_objects_unfound":0,"num_objects_dirty":102958,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":179333,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8686,"num_bytes_recovered":35545107,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160884,"ondisk_log_size":160884,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2016512,"kb_used_data":1002560,"kb_used_omap":12,"kb_used_meta":1013875,"kb_avail":281099008,"statfs":{"total":289910292480,"available":287845384192,"internally_reserved":0,"allocated":1026621440,"data_stored":1025756211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1038208004},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,5,3,8,9,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18620416,"num_objects":-4546,"num_object_clones":0,"num_object_copies":-9092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":39,"num_objects_misplaced":-263,"num_objects_unfound":0,"num_objects_dirty":-4546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4546,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":224,"num_bytes_recovered":917504,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004426"},"pg_stats":[{"pgid":"2.f","version":"47'13828","reported_seq":13908,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:36.681429+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:18:36.681429+0000","last_peered":"2024-09-11T01:18:36.681429+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:18:36.681429+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26370048,"num_objects":6438,"num_object_clones":0,"num_object_copies":12876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6438,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13828,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2366},{"shards":"0,2","objects":377}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13850","reported_seq":13956,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:34.494713+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:18:34.494713+0000","last_peered":"2024-09-11T01:18:34.494713+0000","last_clean":"2024-09-11T01:18:34.494713+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:18:34.494713+0000","last_undegraded":"2024-09-11T01:18:34.494713+0000","last_fullsized":"2024-09-11T01:18:34.494713+0000","mapping_epoch":35,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26632192,"num_objects":6502,"num_object_clones":0,"num_object_copies":13004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12089,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13592","reported_seq":13725,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:34.482351+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:18:34.482351+0000","last_peered":"2024-09-11T01:18:34.482351+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:18:34.482351+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26832920,"num_objects":6552,"num_object_clones":0,"num_object_copies":13104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6552,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11893,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":378},{"shards":"2","objects":2656}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'13612","reported_seq":13686,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:34.494698+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:18:34.494698+0000","last_peered":"2024-09-11T01:18:34.494698+0000","last_clean":"2024-09-11T01:18:34.494698+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:18:34.494698+0000","last_undegraded":"2024-09-11T01:18:34.494698+0000","last_fullsized":"2024-09-11T01:18:34.494698+0000","mapping_epoch":35,"log_start":"39'3600","ondisk_log_start":"39'3600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26714112,"num_objects":6522,"num_object_clones":0,"num_object_copies":13044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6522,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13612,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13591","reported_seq":13724,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:36.658806+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:18:36.658806+0000","last_peered":"2024-09-11T01:18:36.658806+0000","last_clean":"2024-09-11T01:18:36.658806+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:18:36.658806+0000","last_undegraded":"2024-09-11T01:18:36.658806+0000","last_fullsized":"2024-09-11T01:18:36.658806+0000","mapping_epoch":34,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26288128,"num_objects":6418,"num_object_clones":0,"num_object_copies":12836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6418,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10596,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13754","reported_seq":13929,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:36.640804+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:18:36.640804+0000","last_peered":"2024-09-11T01:18:36.640804+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:18:36.640804+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26513408,"num_objects":6473,"num_object_clones":0,"num_object_copies":12946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6473,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10095,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2522},{"shards":"0,2","objects":313}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'13594","reported_seq":13728,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:36.696412+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:18:36.696412+0000","last_peered":"2024-09-11T01:18:36.696412+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:18:36.696412+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25952256,"num_objects":6336,"num_object_clones":0,"num_object_copies":12672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6336,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10451,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2350},{"shards":"0,2","objects":360}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13810","reported_seq":13945,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:36.658831+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:18:36.658831+0000","last_peered":"2024-09-11T01:18:36.658831+0000","last_clean":"2024-09-11T01:18:36.658831+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:18:36.658831+0000","last_undegraded":"2024-09-11T01:18:36.658831+0000","last_fullsized":"2024-09-11T01:18:36.658831+0000","mapping_epoch":34,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26394624,"num_objects":6444,"num_object_clones":0,"num_object_copies":12888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6444,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10686,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934359+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934359+0000","last_peered":"2024-09-11T01:17:58.934359+0000","last_clean":"2024-09-11T01:17:58.934359+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934359+0000","last_undegraded":"2024-09-11T01:17:58.934359+0000","last_fullsized":"2024-09-11T01:17:58.934359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13476","reported_seq":15239,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:34.491932+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:18:34.491932+0000","last_peered":"2024-09-11T01:18:34.491932+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:18:34.491932+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26611712,"num_objects":6497,"num_object_clones":0,"num_object_copies":12994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2674,"num_objects_misplaced":446,"num_objects_unfound":0,"num_objects_dirty":6497,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11447,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":815,"num_bytes_recovered":3330048,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":446},{"shards":"2","objects":3377}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13560","reported_seq":13693,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:36.641457+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:18:36.641457+0000","last_peered":"2024-09-11T01:18:36.641457+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:18:36.641457+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25837568,"num_objects":6308,"num_object_clones":0,"num_object_copies":12616,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6308,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10147,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2324},{"shards":"0,2","objects":362}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13716","reported_seq":13840,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:34.446695+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:18:34.446695+0000","last_peered":"2024-09-11T01:18:34.446695+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:18:34.446695+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26587136,"num_objects":6491,"num_object_clones":0,"num_object_copies":12982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11533,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":413},{"shards":"2","objects":2467}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935080+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935080+0000","last_peered":"2024-09-11T01:17:58.935080+0000","last_clean":"2024-09-11T01:17:58.935080+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935080+0000","last_undegraded":"2024-09-11T01:17:58.935080+0000","last_fullsized":"2024-09-11T01:17:58.935080+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13440","reported_seq":21425,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:34.482983+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:18:34.482983+0000","last_peered":"2024-09-11T01:18:34.482983+0000","last_clean":"2024-09-11T01:18:34.482983+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:18:34.482983+0000","last_undegraded":"2024-09-11T01:18:34.482983+0000","last_fullsized":"2024-09-11T01:18:34.482983+0000","mapping_epoch":44,"log_start":"36'3400","ondisk_log_start":"36'3400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26034176,"num_objects":6356,"num_object_clones":0,"num_object_copies":12712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11225,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13780","reported_seq":13915,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:36.699204+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:18:36.699204+0000","last_peered":"2024-09-11T01:18:36.699204+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:18:36.699204+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26558464,"num_objects":6484,"num_object_clones":0,"num_object_copies":12968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6484,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10517,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2534},{"shards":"0,2","objects":303}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938540+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938540+0000","last_peered":"2024-09-11T01:17:58.938540+0000","last_clean":"2024-09-11T01:17:58.938540+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938540+0000","last_undegraded":"2024-09-11T01:17:58.938540+0000","last_fullsized":"2024-09-11T01:17:58.938540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13512","reported_seq":21548,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:35.488819+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:18:35.488819+0000","last_peered":"2024-09-11T01:18:35.488819+0000","last_clean":"2024-09-11T01:18:35.488819+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:18:35.488819+0000","last_undegraded":"2024-09-11T01:18:35.488819+0000","last_fullsized":"2024-09-11T01:18:35.488819+0000","mapping_epoch":46,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26099712,"num_objects":6372,"num_object_clones":0,"num_object_copies":12744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10167,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13793","reported_seq":13931,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:36.672447+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:18:36.672447+0000","last_peered":"2024-09-11T01:18:36.672447+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:18:36.672447+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26144768,"num_objects":6383,"num_object_clones":0,"num_object_copies":12766,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6383,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10402,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2344},{"shards":"0,2","objects":336}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971314+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971314+0000","last_peered":"2024-09-11T01:17:58.971314+0000","last_clean":"2024-09-11T01:17:58.971314+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971314+0000","last_undegraded":"2024-09-11T01:17:58.971314+0000","last_fullsized":"2024-09-11T01:17:58.971314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13573","reported_seq":13712,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:36.601544+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:18:36.601544+0000","last_peered":"2024-09-11T01:18:36.601544+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:18:36.601544+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26129001,"num_objects":6380,"num_object_clones":0,"num_object_copies":12760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6380,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10643,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2463},{"shards":"0,2","objects":326}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":421700225,"num_objects":102956,"num_object_clones":0,"num_object_copies":205912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38508,"num_objects_misplaced":446,"num_objects_unfound":0,"num_objects_dirty":102956,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":179331,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8685,"num_bytes_recovered":35545088,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1023479808,"data_stored":1023409042,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160881,"ondisk_log_size":160881,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182485,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":746772,"kb_used_data":372212,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93625068,"statfs":{"total":96636764160,"available":95872069632,"internally_reserved":0,"allocated":381145088,"data_stored":380875429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":383512235},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,1,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986257,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":646672,"kb_used_data":336000,"kb_used_omap":4,"kb_used_meta":310651,"kb_avail":93725168,"statfs":{"total":96636764160,"available":95974572032,"internally_reserved":0,"allocated":344064000,"data_stored":343794346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":318107306},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659202,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":623068,"kb_used_data":294348,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93748772,"statfs":{"total":96636764160,"available":95998742528,"internally_reserved":0,"allocated":301412352,"data_stored":301086436,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":336588463},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,5,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":379604992,"data_stored":379601513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":300838912,"data_stored":300779176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":343035904,"data_stored":343028353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:43.142 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:43.895 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:18:43.897 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:18:43.969 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":270,"stamp":"2024-09-11T01:18:42.700407+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":412578452,"num_objects":100731,"num_object_clones":0,"num_object_copies":201462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38569,"num_objects_misplaced":288,"num_objects_unfound":0,"num_objects_dirty":100731,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":181560,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8783,"num_bytes_recovered":35942419,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160811,"ondisk_log_size":160811,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":2000248,"kb_used_data":990468,"kb_used_omap":12,"kb_used_meta":1009715,"kb_avail":281115272,"statfs":{"total":289910292480,"available":287862038528,"internally_reserved":0,"allocated":1014239232,"data_stored":1013374003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1033948164},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,3,10,9,8,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23683072,"num_objects":-5782,"num_object_clones":0,"num_object_copies":-11564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":94,"num_objects_misplaced":-300,"num_objects_unfound":0,"num_objects_dirty":-5782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5782,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":206,"num_bytes_recovered":843776,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004088"},"pg_stats":[{"pgid":"2.f","version":"47'13976","reported_seq":14059,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:41.661235+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:18:41.661235+0000","last_peered":"2024-09-11T01:18:41.661235+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:18:41.661235+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25763840,"num_objects":6290,"num_object_clones":0,"num_object_copies":12580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13976,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2218},{"shards":"0,2","objects":229}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'13994","reported_seq":14100,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:39.521420+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:18:39.521420+0000","last_peered":"2024-09-11T01:18:39.521420+0000","last_clean":"2024-09-11T01:18:39.521420+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:18:39.521420+0000","last_undegraded":"2024-09-11T01:18:39.521420+0000","last_fullsized":"2024-09-11T01:18:39.521420+0000","mapping_epoch":35,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26042368,"num_objects":6358,"num_object_clones":0,"num_object_copies":12716,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6358,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12233,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13733","reported_seq":13866,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:39.490467+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:18:39.490467+0000","last_peered":"2024-09-11T01:18:39.490467+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:18:39.490467+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26255384,"num_objects":6411,"num_object_clones":0,"num_object_copies":12822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6411,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12034,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":237},{"shards":"2","objects":2515}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'13743","reported_seq":13816,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:39.538199+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:18:39.538199+0000","last_peered":"2024-09-11T01:18:39.538199+0000","last_clean":"2024-09-11T01:18:39.538199+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:18:39.538199+0000","last_undegraded":"2024-09-11T01:18:39.538199+0000","last_fullsized":"2024-09-11T01:18:39.538199+0000","mapping_epoch":35,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26177536,"num_objects":6391,"num_object_clones":0,"num_object_copies":12782,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6391,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13743,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13736","reported_seq":13869,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:41.659668+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:18:41.659668+0000","last_peered":"2024-09-11T01:18:41.659668+0000","last_clean":"2024-09-11T01:18:41.659668+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:18:41.659668+0000","last_undegraded":"2024-09-11T01:18:41.659668+0000","last_fullsized":"2024-09-11T01:18:41.659668+0000","mapping_epoch":34,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25694208,"num_objects":6273,"num_object_clones":0,"num_object_copies":12546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10741,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13884","reported_seq":14058,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:41.661469+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:18:41.661469+0000","last_peered":"2024-09-11T01:18:41.661469+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:18:41.661469+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25980928,"num_objects":6343,"num_object_clones":0,"num_object_copies":12686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10225,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2392},{"shards":"0,2","objects":183}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'13745","reported_seq":13880,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:41.657933+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:18:41.657933+0000","last_peered":"2024-09-11T01:18:41.657933+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:18:41.657933+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25333760,"num_objects":6185,"num_object_clones":0,"num_object_copies":12370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10602,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2199},{"shards":"0,2","objects":209}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13954","reported_seq":14089,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:41.656425+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:18:41.656425+0000","last_peered":"2024-09-11T01:18:41.656425+0000","last_clean":"2024-09-11T01:18:41.656425+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:18:41.656425+0000","last_undegraded":"2024-09-11T01:18:41.656425+0000","last_fullsized":"2024-09-11T01:18:41.656425+0000","mapping_epoch":34,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25804800,"num_objects":6300,"num_object_clones":0,"num_object_copies":12600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10830,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934359+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934359+0000","last_peered":"2024-09-11T01:17:58.934359+0000","last_clean":"2024-09-11T01:17:58.934359+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934359+0000","last_undegraded":"2024-09-11T01:17:58.934359+0000","last_fullsized":"2024-09-11T01:17:58.934359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13633","reported_seq":15591,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:39.531481+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:18:39.531481+0000","last_peered":"2024-09-11T01:18:39.531481+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:18:39.531481+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"39'3600","ondisk_log_start":"39'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25968640,"num_objects":6340,"num_object_clones":0,"num_object_copies":12680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2735,"num_objects_misplaced":288,"num_objects_unfound":0,"num_objects_dirty":6340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11604,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":912,"num_bytes_recovered":3727360,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":288},{"shards":"2","objects":3317}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13698","reported_seq":13831,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:41.625756+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:18:41.625756+0000","last_peered":"2024-09-11T01:18:41.625756+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:18:41.625756+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"39'3600","ondisk_log_start":"39'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25272320,"num_objects":6170,"num_object_clones":0,"num_object_copies":12340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10285,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2186},{"shards":"0,2","objects":224}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13861","reported_seq":13982,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:39.508425+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:18:39.508425+0000","last_peered":"2024-09-11T01:18:39.508425+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:18:39.508425+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25993216,"num_objects":6346,"num_object_clones":0,"num_object_copies":12692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11678,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":268},{"shards":"2","objects":2322}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935080+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935080+0000","last_peered":"2024-09-11T01:17:58.935080+0000","last_clean":"2024-09-11T01:17:58.935080+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935080+0000","last_undegraded":"2024-09-11T01:17:58.935080+0000","last_fullsized":"2024-09-11T01:17:58.935080+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13566","reported_seq":21552,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:39.547340+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:18:39.547340+0000","last_peered":"2024-09-11T01:18:39.547340+0000","last_clean":"2024-09-11T01:18:39.547340+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:18:39.547340+0000","last_undegraded":"2024-09-11T01:18:39.547340+0000","last_fullsized":"2024-09-11T01:18:39.547340+0000","mapping_epoch":44,"log_start":"36'3500","ondisk_log_start":"36'3500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25518080,"num_objects":6230,"num_object_clones":0,"num_object_copies":12460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11351,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13905","reported_seq":14042,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:41.715130+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:18:41.715130+0000","last_peered":"2024-09-11T01:18:41.715130+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:18:41.715130+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"40'3900","ondisk_log_start":"40'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26046464,"num_objects":6359,"num_object_clones":0,"num_object_copies":12718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10642,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2409},{"shards":"0,2","objects":178}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938540+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938540+0000","last_peered":"2024-09-11T01:17:58.938540+0000","last_clean":"2024-09-11T01:17:58.938540+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938540+0000","last_undegraded":"2024-09-11T01:17:58.938540+0000","last_fullsized":"2024-09-11T01:17:58.938540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13654","reported_seq":21690,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:40.593678+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:18:40.593678+0000","last_peered":"2024-09-11T01:18:40.593678+0000","last_clean":"2024-09-11T01:18:40.593678+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:18:40.593678+0000","last_undegraded":"2024-09-11T01:18:40.593678+0000","last_fullsized":"2024-09-11T01:18:40.593678+0000","mapping_epoch":46,"log_start":"39'3600","ondisk_log_start":"39'3600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25518080,"num_objects":6230,"num_object_clones":0,"num_object_copies":12460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10309,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13922","reported_seq":14062,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:41.689894+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:18:41.689894+0000","last_peered":"2024-09-11T01:18:41.689894+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:18:41.689894+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25616384,"num_objects":6254,"num_object_clones":0,"num_object_copies":12508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10531,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2215},{"shards":"0,2","objects":207}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971314+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971314+0000","last_peered":"2024-09-11T01:17:58.971314+0000","last_clean":"2024-09-11T01:17:58.971314+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971314+0000","last_undegraded":"2024-09-11T01:17:58.971314+0000","last_fullsized":"2024-09-11T01:17:58.971314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13704","reported_seq":13843,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:41.672763+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:18:41.672763+0000","last_peered":"2024-09-11T01:18:41.672763+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:18:41.672763+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25592425,"num_objects":6249,"num_object_clones":0,"num_object_copies":12498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10774,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2332},{"shards":"0,2","objects":195}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":412578433,"num_objects":100729,"num_object_clones":0,"num_object_copies":201458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38569,"num_objects_misplaced":288,"num_objects_unfound":0,"num_objects_dirty":100729,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":181558,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8782,"num_bytes_recovered":35942400,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1010954240,"data_stored":1010883474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160808,"ondisk_log_size":160808,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182486,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":744404,"kb_used_data":365696,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93627436,"statfs":{"total":96636764160,"available":95874494464,"internally_reserved":0,"allocated":374472704,"data_stored":374203045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":387772075},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,5,3,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986258,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":646316,"kb_used_data":331484,"kb_used_omap":4,"kb_used_meta":314811,"kb_avail":93725524,"statfs":{"total":96636764160,"available":95974936576,"internally_reserved":0,"allocated":339439616,"data_stored":339169962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":322367146},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,3,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659203,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609528,"kb_used_data":293288,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93762312,"statfs":{"total":96636764160,"available":96012607488,"internally_reserved":0,"allocated":300326912,"data_stored":300000996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":323808943},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":372826112,"data_stored":372822633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":299696128,"data_stored":299636392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":338432000,"data_stored":338424449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:43.970 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:44.973 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:18:44.974 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:18:45.050 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":271,"stamp":"2024-09-11T01:18:44.700903+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":409109140,"num_objects":99884,"num_object_clones":0,"num_object_copies":199768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38641,"num_objects_misplaced":123,"num_objects_unfound":0,"num_objects_dirty":99884,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":182407,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8876,"num_bytes_recovered":36323347,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160858,"ondisk_log_size":160858,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1983116,"kb_used_data":985816,"kb_used_omap":12,"kb_used_meta":997235,"kb_avail":281132404,"statfs":{"total":289910292480,"available":287879581696,"internally_reserved":0,"allocated":1009475584,"data_stored":1008610355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1021168644},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,2,9,8,8,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21839872,"num_objects":-5332,"num_object_clones":0,"num_object_copies":-10664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":166,"num_objects_misplaced":-465,"num_objects_unfound":0,"num_objects_dirty":-5332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5332,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":299,"num_bytes_recovered":1224704,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004070"},"pg_stats":[{"pgid":"2.f","version":"47'13976","reported_seq":14059,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:41.661235+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:18:41.661235+0000","last_peered":"2024-09-11T01:18:41.661235+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:18:41.661235+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25763840,"num_objects":6290,"num_object_clones":0,"num_object_copies":12580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13976,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2218},{"shards":"0,2","objects":229}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14141","reported_seq":14247,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:44.520618+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:18:44.520618+0000","last_peered":"2024-09-11T01:18:44.520618+0000","last_clean":"2024-09-11T01:18:44.520618+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:18:44.520618+0000","last_undegraded":"2024-09-11T01:18:44.520618+0000","last_fullsized":"2024-09-11T01:18:44.520618+0000","mapping_epoch":35,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25440256,"num_objects":6211,"num_object_clones":0,"num_object_copies":12422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6211,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12380,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13864","reported_seq":13995,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:44.509746+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:18:44.509746+0000","last_peered":"2024-09-11T01:18:44.509746+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:18:44.509746+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25718808,"num_objects":6280,"num_object_clones":0,"num_object_copies":12560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6280,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12165,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":106},{"shards":"2","objects":2384}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'13870","reported_seq":13944,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:44.509999+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:18:44.509999+0000","last_peered":"2024-09-11T01:18:44.509999+0000","last_clean":"2024-09-11T01:18:44.509999+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:18:44.509999+0000","last_undegraded":"2024-09-11T01:18:44.509999+0000","last_fullsized":"2024-09-11T01:18:44.509999+0000","mapping_epoch":35,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25657344,"num_objects":6264,"num_object_clones":0,"num_object_copies":12528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13870,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13736","reported_seq":13869,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:41.659668+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:18:41.659668+0000","last_peered":"2024-09-11T01:18:41.659668+0000","last_clean":"2024-09-11T01:18:41.659668+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:18:41.659668+0000","last_undegraded":"2024-09-11T01:18:41.659668+0000","last_fullsized":"2024-09-11T01:18:41.659668+0000","mapping_epoch":34,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25694208,"num_objects":6273,"num_object_clones":0,"num_object_copies":12546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10741,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13884","reported_seq":14058,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:41.661469+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:18:41.661469+0000","last_peered":"2024-09-11T01:18:41.661469+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:18:41.661469+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25980928,"num_objects":6343,"num_object_clones":0,"num_object_copies":12686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10225,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2392},{"shards":"0,2","objects":183}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'13745","reported_seq":13880,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:41.657933+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:18:41.657933+0000","last_peered":"2024-09-11T01:18:41.657933+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:18:41.657933+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25333760,"num_objects":6185,"num_object_clones":0,"num_object_copies":12370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10602,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2199},{"shards":"0,2","objects":209}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13954","reported_seq":14089,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:41.656425+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:18:41.656425+0000","last_peered":"2024-09-11T01:18:41.656425+0000","last_clean":"2024-09-11T01:18:41.656425+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:18:41.656425+0000","last_undegraded":"2024-09-11T01:18:41.656425+0000","last_fullsized":"2024-09-11T01:18:41.656425+0000","mapping_epoch":34,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25804800,"num_objects":6300,"num_object_clones":0,"num_object_copies":12600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10830,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934359+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934359+0000","last_peered":"2024-09-11T01:17:58.934359+0000","last_clean":"2024-09-11T01:17:58.934359+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934359+0000","last_undegraded":"2024-09-11T01:17:58.934359+0000","last_fullsized":"2024-09-11T01:17:58.934359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13798","reported_seq":15941,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:44.572352+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:18:44.572352+0000","last_peered":"2024-09-11T01:18:44.572352+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:18:44.572352+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25292800,"num_objects":6175,"num_object_clones":0,"num_object_copies":12350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2807,"num_objects_misplaced":123,"num_objects_unfound":0,"num_objects_dirty":6175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11769,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1005,"num_bytes_recovered":4108288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":123},{"shards":"2","objects":3245}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13698","reported_seq":13831,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:41.625756+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:18:41.625756+0000","last_peered":"2024-09-11T01:18:41.625756+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:18:41.625756+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"39'3600","ondisk_log_start":"39'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25272320,"num_objects":6170,"num_object_clones":0,"num_object_copies":12340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10285,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2186},{"shards":"0,2","objects":224}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13997","reported_seq":14121,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:44.468453+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:18:44.468453+0000","last_peered":"2024-09-11T01:18:44.468453+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:18:44.468453+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25436160,"num_objects":6210,"num_object_clones":0,"num_object_copies":12420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11814,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":132},{"shards":"2","objects":2186}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935080+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935080+0000","last_peered":"2024-09-11T01:17:58.935080+0000","last_clean":"2024-09-11T01:17:58.935080+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935080+0000","last_undegraded":"2024-09-11T01:17:58.935080+0000","last_fullsized":"2024-09-11T01:17:58.935080+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13707","reported_seq":21693,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:44.548843+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:18:44.548843+0000","last_peered":"2024-09-11T01:18:44.548843+0000","last_clean":"2024-09-11T01:18:44.548843+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:18:44.548843+0000","last_undegraded":"2024-09-11T01:18:44.548843+0000","last_fullsized":"2024-09-11T01:18:44.548843+0000","mapping_epoch":44,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24940544,"num_objects":6089,"num_object_clones":0,"num_object_copies":12178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11492,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13905","reported_seq":14042,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:41.715130+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:18:41.715130+0000","last_peered":"2024-09-11T01:18:41.715130+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:18:41.715130+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"40'3900","ondisk_log_start":"40'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26046464,"num_objects":6359,"num_object_clones":0,"num_object_copies":12718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10642,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2409},{"shards":"0,2","objects":178}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938540+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938540+0000","last_peered":"2024-09-11T01:17:58.938540+0000","last_clean":"2024-09-11T01:17:58.938540+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938540+0000","last_undegraded":"2024-09-11T01:17:58.938540+0000","last_fullsized":"2024-09-11T01:17:58.938540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13654","reported_seq":21690,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:40.593678+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:18:40.593678+0000","last_peered":"2024-09-11T01:18:40.593678+0000","last_clean":"2024-09-11T01:18:40.593678+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:18:40.593678+0000","last_undegraded":"2024-09-11T01:18:40.593678+0000","last_fullsized":"2024-09-11T01:18:40.593678+0000","mapping_epoch":46,"log_start":"39'3600","ondisk_log_start":"39'3600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25518080,"num_objects":6230,"num_object_clones":0,"num_object_copies":12460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10309,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13922","reported_seq":14062,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:41.689894+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:18:41.689894+0000","last_peered":"2024-09-11T01:18:41.689894+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:18:41.689894+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25616384,"num_objects":6254,"num_object_clones":0,"num_object_copies":12508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10531,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2215},{"shards":"0,2","objects":207}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971314+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971314+0000","last_peered":"2024-09-11T01:17:58.971314+0000","last_clean":"2024-09-11T01:17:58.971314+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971314+0000","last_undegraded":"2024-09-11T01:17:58.971314+0000","last_fullsized":"2024-09-11T01:17:58.971314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13704","reported_seq":13843,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:41.672763+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:18:41.672763+0000","last_peered":"2024-09-11T01:18:41.672763+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:18:41.672763+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25592425,"num_objects":6249,"num_object_clones":0,"num_object_copies":12498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10774,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2332},{"shards":"0,2","objects":195}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":409109121,"num_objects":99882,"num_object_clones":0,"num_object_copies":199764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38641,"num_objects_misplaced":123,"num_objects_unfound":0,"num_objects_dirty":99882,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":182405,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8875,"num_bytes_recovered":36323328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1006264320,"data_stored":1006193554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160855,"ondisk_log_size":160855,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182486,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":744404,"kb_used_data":365696,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93627436,"statfs":{"total":96636764160,"available":95874494464,"internally_reserved":0,"allocated":374472704,"data_stored":374203045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":387772075},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,5,3,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986259,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":629184,"kb_used_data":326832,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93742656,"statfs":{"total":96636764160,"available":95992479744,"internally_reserved":0,"allocated":334675968,"data_stored":334406314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,1,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659203,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609528,"kb_used_data":293288,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93762312,"statfs":{"total":96636764160,"available":96012607488,"internally_reserved":0,"allocated":300326912,"data_stored":300000996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":323808943},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,0,2,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":372826112,"data_stored":372822633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":299696128,"data_stored":299636392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":333742080,"data_stored":333734529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:45.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:18:45.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'13976", 'reported_seq': 14059, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:41.661235+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:18:41.661235+0000', 'last_peered': '2024-09-11T01:18:41.661235+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:18:41.661235+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "39'3900", 'ondisk_log_start': "39'3900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25763840, 'num_objects': 6290, 'num_object_clones': 0, 'num_object_copies': 12580, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6290, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13976, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2218}, {'shards': '0,2', 'objects': 229}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:45.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:18:45.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'13864", 'reported_seq': 13995, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:44.509746+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:18:44.509746+0000', 'last_peered': '2024-09-11T01:18:44.509746+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:18:44.509746+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "39'3800", 'ondisk_log_start': "39'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25718808, 'num_objects': 6280, 'num_object_clones': 0, 'num_object_copies': 12560, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6280, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12165, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 106}, {'shards': '2', 'objects': 2384}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:45.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:18:45.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'13884", 'reported_seq': 14058, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:41.661469+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:18:41.661469+0000', 'last_peered': '2024-09-11T01:18:41.661469+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:18:41.661469+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "39'3800", 'ondisk_log_start': "39'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25980928, 'num_objects': 6343, 'num_object_clones': 0, 'num_object_copies': 12686, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6343, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10225, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2392}, {'shards': '0,2', 'objects': 183}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:45.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:18:45.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'13745", 'reported_seq': 13880, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:41.657933+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:18:41.657933+0000', 'last_peered': '2024-09-11T01:18:41.657933+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:18:41.657933+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "39'3700", 'ondisk_log_start': "39'3700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25333760, 'num_objects': 6185, 'num_object_clones': 0, 'num_object_copies': 12370, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6185, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10602, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2199}, {'shards': '0,2', 'objects': 209}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:45.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:18:45.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'13798", 'reported_seq': 15941, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:44.572352+0000', 'last_change': '2024-09-11T01:17:57.355893+0000', 'last_active': '2024-09-11T01:18:44.572352+0000', 'last_peered': '2024-09-11T01:18:44.572352+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:18:44.572352+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "39'3700", 'ondisk_log_start': "39'3700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25292800, 'num_objects': 6175, 'num_object_clones': 0, 'num_object_copies': 12350, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2807, 'num_objects_misplaced': 123, 'num_objects_unfound': 0, 'num_objects_dirty': 6175, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11769, 'num_write_kb': 31844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1005, 'num_bytes_recovered': 4108288, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 123}, {'shards': '2', 'objects': 3245}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:45.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:18:45.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'13698", 'reported_seq': 13831, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:41.625756+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:18:41.625756+0000', 'last_peered': '2024-09-11T01:18:41.625756+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:18:41.625756+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "39'3600", 'ondisk_log_start': "39'3600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10098, 'log_dups_size': 0, 'ondisk_log_size': 10098, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25272320, 'num_objects': 6170, 'num_object_clones': 0, 'num_object_copies': 12340, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6170, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10285, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2186}, {'shards': '0,2', 'objects': 224}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:45.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:18:45.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'13997", 'reported_seq': 14121, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:44.468453+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:18:44.468453+0000', 'last_peered': '2024-09-11T01:18:44.468453+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:18:44.468453+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "39'3900", 'ondisk_log_start': "39'3900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25436160, 'num_objects': 6210, 'num_object_clones': 0, 'num_object_copies': 12420, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6210, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11814, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '0,2', 'objects': 132}, {'shards': '2', 'objects': 2186}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:45.053 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:18:45.053 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'13905", 'reported_seq': 14042, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:41.715130+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:18:41.715130+0000', 'last_peered': '2024-09-11T01:18:41.715130+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:18:41.715130+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "40'3900", 'ondisk_log_start': "40'3900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 26046464, 'num_objects': 6359, 'num_object_clones': 0, 'num_object_copies': 12718, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6359, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10642, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2409}, {'shards': '0,2', 'objects': 178}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:45.053 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:18:45.053 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'13922", 'reported_seq': 14062, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:41.689894+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:18:41.689894+0000', 'last_peered': '2024-09-11T01:18:41.689894+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:18:41.689894+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "39'3900", 'ondisk_log_start': "39'3900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25616384, 'num_objects': 6254, 'num_object_clones': 0, 'num_object_copies': 12508, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6254, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10531, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2215}, {'shards': '0,2', 'objects': 207}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:45.053 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:18:45.053 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'13704", 'reported_seq': 13843, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:41.672763+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:18:41.672763+0000', 'last_peered': '2024-09-11T01:18:41.672763+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:18:41.672763+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "39'3700", 'ondisk_log_start': "39'3700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25592425, 'num_objects': 6249, 'num_object_clones': 0, 'num_object_copies': 12498, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6249, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10774, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2332}, {'shards': '0,2', 'objects': 195}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:45.053 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:18:45.934 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:18:45.934 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38641/199768 objects degraded (19.343%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":462,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":99884,"data_bytes":409109140,"bytes_used":2030710784,"bytes_avail":287879581696,"bytes_total":289910292480,"degraded_objects":38641,"degraded_total":199768,"degraded_ratio":0.19342937807857111,"misplaced_objects":123,"misplaced_total":199768,"misplaced_ratio":0.00061571422850506584,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":102024,"recovering_keys_per_sec":0,"num_objects_recovered":299,"num_bytes_recovered":1224704,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":444},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:18:45.981 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:18:45.981 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:46.802 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:18:46.802 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:18:46.841 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":272,"stamp":"2024-09-11T01:18:46.701374+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":408515220,"num_objects":99739,"num_object_clones":0,"num_object_copies":199478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38641,"num_objects_misplaced":123,"num_objects_unfound":0,"num_objects_dirty":99739,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":182552,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8876,"num_bytes_recovered":36323347,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160903,"ondisk_log_size":160903,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1990284,"kb_used_data":984664,"kb_used_omap":12,"kb_used_meta":1005555,"kb_avail":281125236,"statfs":{"total":289910292480,"available":287872241664,"internally_reserved":0,"allocated":1008295936,"data_stored":1007430707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1029688324},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,8,7,7,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19021824,"num_objects":-4644,"num_object_clones":0,"num_object_copies":-9288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":133,"num_objects_misplaced":-323,"num_objects_unfound":0,"num_objects_dirty":-4644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4644,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":190,"num_bytes_recovered":778240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004054"},"pg_stats":[{"pgid":"2.f","version":"47'13976","reported_seq":14059,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:41.661235+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:18:41.661235+0000","last_peered":"2024-09-11T01:18:41.661235+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:18:41.661235+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25763840,"num_objects":6290,"num_object_clones":0,"num_object_copies":12580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6290,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13976,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2218},{"shards":"0,2","objects":229}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14141","reported_seq":14247,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:44.520618+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:18:44.520618+0000","last_peered":"2024-09-11T01:18:44.520618+0000","last_clean":"2024-09-11T01:18:44.520618+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:18:44.520618+0000","last_undegraded":"2024-09-11T01:18:44.520618+0000","last_fullsized":"2024-09-11T01:18:44.520618+0000","mapping_epoch":35,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25440256,"num_objects":6211,"num_object_clones":0,"num_object_copies":12422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6211,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12380,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13864","reported_seq":13995,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:44.509746+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:18:44.509746+0000","last_peered":"2024-09-11T01:18:44.509746+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:18:44.509746+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25718808,"num_objects":6280,"num_object_clones":0,"num_object_copies":12560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6280,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12165,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":106},{"shards":"2","objects":2384}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'13870","reported_seq":13944,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:44.509999+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:18:44.509999+0000","last_peered":"2024-09-11T01:18:44.509999+0000","last_clean":"2024-09-11T01:18:44.509999+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:18:44.509999+0000","last_undegraded":"2024-09-11T01:18:44.509999+0000","last_fullsized":"2024-09-11T01:18:44.509999+0000","mapping_epoch":35,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25657344,"num_objects":6264,"num_object_clones":0,"num_object_copies":12528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6264,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13870,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13736","reported_seq":13869,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:41.659668+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:18:41.659668+0000","last_peered":"2024-09-11T01:18:41.659668+0000","last_clean":"2024-09-11T01:18:41.659668+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:18:41.659668+0000","last_undegraded":"2024-09-11T01:18:41.659668+0000","last_fullsized":"2024-09-11T01:18:41.659668+0000","mapping_epoch":34,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25694208,"num_objects":6273,"num_object_clones":0,"num_object_copies":12546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6273,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10741,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'13884","reported_seq":14058,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:41.661469+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:18:41.661469+0000","last_peered":"2024-09-11T01:18:41.661469+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:18:41.661469+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25980928,"num_objects":6343,"num_object_clones":0,"num_object_copies":12686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10225,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2392},{"shards":"0,2","objects":183}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'13745","reported_seq":13880,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:41.657933+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:18:41.657933+0000","last_peered":"2024-09-11T01:18:41.657933+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:18:41.657933+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25333760,"num_objects":6185,"num_object_clones":0,"num_object_copies":12370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10602,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2199},{"shards":"0,2","objects":209}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'13954","reported_seq":14089,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:41.656425+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:18:41.656425+0000","last_peered":"2024-09-11T01:18:41.656425+0000","last_clean":"2024-09-11T01:18:41.656425+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:18:41.656425+0000","last_undegraded":"2024-09-11T01:18:41.656425+0000","last_fullsized":"2024-09-11T01:18:41.656425+0000","mapping_epoch":34,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25804800,"num_objects":6300,"num_object_clones":0,"num_object_copies":12600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10830,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934359+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934359+0000","last_peered":"2024-09-11T01:17:58.934359+0000","last_clean":"2024-09-11T01:17:58.934359+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934359+0000","last_undegraded":"2024-09-11T01:17:58.934359+0000","last_fullsized":"2024-09-11T01:17:58.934359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13798","reported_seq":15941,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:44.572352+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:18:44.572352+0000","last_peered":"2024-09-11T01:18:44.572352+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:18:44.572352+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25292800,"num_objects":6175,"num_object_clones":0,"num_object_copies":12350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2807,"num_objects_misplaced":123,"num_objects_unfound":0,"num_objects_dirty":6175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11769,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1005,"num_bytes_recovered":4108288,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":123},{"shards":"2","objects":3245}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13698","reported_seq":13831,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:41.625756+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:18:41.625756+0000","last_peered":"2024-09-11T01:18:41.625756+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:18:41.625756+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"39'3600","ondisk_log_start":"39'3600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25272320,"num_objects":6170,"num_object_clones":0,"num_object_copies":12340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6170,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10285,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2186},{"shards":"0,2","objects":224}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'13997","reported_seq":14121,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:44.468453+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:18:44.468453+0000","last_peered":"2024-09-11T01:18:44.468453+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:18:44.468453+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25436160,"num_objects":6210,"num_object_clones":0,"num_object_copies":12420,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6210,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11814,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"0,2","objects":132},{"shards":"2","objects":2186}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935080+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935080+0000","last_peered":"2024-09-11T01:17:58.935080+0000","last_clean":"2024-09-11T01:17:58.935080+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935080+0000","last_undegraded":"2024-09-11T01:17:58.935080+0000","last_fullsized":"2024-09-11T01:17:58.935080+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13707","reported_seq":21693,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:44.548843+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:18:44.548843+0000","last_peered":"2024-09-11T01:18:44.548843+0000","last_clean":"2024-09-11T01:18:44.548843+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:18:44.548843+0000","last_undegraded":"2024-09-11T01:18:44.548843+0000","last_fullsized":"2024-09-11T01:18:44.548843+0000","mapping_epoch":44,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24940544,"num_objects":6089,"num_object_clones":0,"num_object_copies":12178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6089,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11492,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'13905","reported_seq":14042,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:41.715130+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:18:41.715130+0000","last_peered":"2024-09-11T01:18:41.715130+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:18:41.715130+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"40'3900","ondisk_log_start":"40'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":26046464,"num_objects":6359,"num_object_clones":0,"num_object_copies":12718,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6359,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10642,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2409},{"shards":"0,2","objects":178}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938540+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938540+0000","last_peered":"2024-09-11T01:17:58.938540+0000","last_clean":"2024-09-11T01:17:58.938540+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938540+0000","last_undegraded":"2024-09-11T01:17:58.938540+0000","last_fullsized":"2024-09-11T01:17:58.938540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13799","reported_seq":21835,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:45.643169+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:18:45.643169+0000","last_peered":"2024-09-11T01:18:45.643169+0000","last_clean":"2024-09-11T01:18:45.643169+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:18:45.643169+0000","last_undegraded":"2024-09-11T01:18:45.643169+0000","last_fullsized":"2024-09-11T01:18:45.643169+0000","mapping_epoch":46,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24924160,"num_objects":6085,"num_object_clones":0,"num_object_copies":12170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10454,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'13922","reported_seq":14062,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:41.689894+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:18:41.689894+0000","last_peered":"2024-09-11T01:18:41.689894+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:18:41.689894+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"39'3900","ondisk_log_start":"39'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25616384,"num_objects":6254,"num_object_clones":0,"num_object_copies":12508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10531,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2215},{"shards":"0,2","objects":207}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971314+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971314+0000","last_peered":"2024-09-11T01:17:58.971314+0000","last_clean":"2024-09-11T01:17:58.971314+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971314+0000","last_undegraded":"2024-09-11T01:17:58.971314+0000","last_fullsized":"2024-09-11T01:17:58.971314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13704","reported_seq":13843,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:41.672763+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:18:41.672763+0000","last_peered":"2024-09-11T01:18:41.672763+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:18:41.672763+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25592425,"num_objects":6249,"num_object_clones":0,"num_object_copies":12498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10774,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2332},{"shards":"0,2","objects":195}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":408515201,"num_objects":99737,"num_object_clones":0,"num_object_copies":199474,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38641,"num_objects_misplaced":123,"num_objects_unfound":0,"num_objects_dirty":99737,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":182550,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8875,"num_bytes_recovered":36323328,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1005154304,"data_stored":1005083538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160900,"ondisk_log_size":160900,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182486,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":744404,"kb_used_data":365696,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93627436,"statfs":{"total":96636764160,"available":95874494464,"internally_reserved":0,"allocated":374472704,"data_stored":374203045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":387772075},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,5,3,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986259,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":629184,"kb_used_data":326832,"kb_used_omap":4,"kb_used_meta":302331,"kb_avail":93742656,"statfs":{"total":96636764160,"available":95992479744,"internally_reserved":0,"allocated":334675968,"data_stored":334406314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":309587626},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,1,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659204,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":616696,"kb_used_data":292136,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93755144,"statfs":{"total":96636764160,"available":96005267456,"internally_reserved":0,"allocated":299147264,"data_stored":298821348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":332328623},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,3,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":372826112,"data_stored":372822633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":298586112,"data_stored":298526376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":333742080,"data_stored":333734529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:49.843 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:50.792 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:18:50.794 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:18:50.838 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":274,"stamp":"2024-09-11T01:18:50.702297+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":399811220,"num_objects":97614,"num_object_clones":0,"num_object_copies":195228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38647,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97614,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":184677,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8993,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1992988,"kb_used_data":973484,"kb_used_omap":12,"kb_used_meta":1019443,"kb_avail":281122532,"statfs":{"total":289910292480,"available":287869472768,"internally_reserved":0,"allocated":996847616,"data_stored":995982387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1043909636},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,4,4,10,8,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21889024,"num_objects":-5344,"num_object_clones":0,"num_object_copies":-10688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":139,"num_objects_misplaced":-446,"num_objects_unfound":0,"num_objects_dirty":-5344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5344,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":1175552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004018"},"pg_stats":[{"pgid":"2.f","version":"47'14147","reported_seq":14228,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:46.756400+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:18:46.756400+0000","last_peered":"2024-09-11T01:18:46.756400+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:18:46.756400+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25063424,"num_objects":6119,"num_object_clones":0,"num_object_copies":12238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14147,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2047},{"shards":"0,2","objects":58}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14294","reported_seq":14400,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:49.547336+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:18:49.547336+0000","last_peered":"2024-09-11T01:18:49.547336+0000","last_clean":"2024-09-11T01:18:49.547336+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:18:49.547336+0000","last_undegraded":"2024-09-11T01:18:49.547336+0000","last_fullsized":"2024-09-11T01:18:49.547336+0000","mapping_epoch":35,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24813568,"num_objects":6058,"num_object_clones":0,"num_object_copies":12116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12533,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13987","reported_seq":14120,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:49.494750+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:18:49.494750+0000","last_peered":"2024-09-11T01:18:49.494750+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:18:49.494750+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"40'3900","ondisk_log_start":"40'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25215000,"num_objects":6157,"num_object_clones":0,"num_object_copies":12314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6157,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12288,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2261}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'14018","reported_seq":14092,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:49.483071+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:18:49.483071+0000","last_peered":"2024-09-11T01:18:49.483071+0000","last_clean":"2024-09-11T01:18:49.483071+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:18:49.483071+0000","last_undegraded":"2024-09-11T01:18:49.483071+0000","last_fullsized":"2024-09-11T01:18:49.483071+0000","mapping_epoch":35,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25051136,"num_objects":6116,"num_object_clones":0,"num_object_copies":12232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6116,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14018,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13888","reported_seq":14019,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:46.750089+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:18:46.750089+0000","last_peered":"2024-09-11T01:18:46.750089+0000","last_clean":"2024-09-11T01:18:46.750089+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:18:46.750089+0000","last_undegraded":"2024-09-11T01:18:46.750089+0000","last_fullsized":"2024-09-11T01:18:46.750089+0000","mapping_epoch":34,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25071616,"num_objects":6121,"num_object_clones":0,"num_object_copies":12242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10893,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14019","reported_seq":14194,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:46.588927+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:18:46.588927+0000","last_peered":"2024-09-11T01:18:46.588927+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:18:46.588927+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25427968,"num_objects":6208,"num_object_clones":0,"num_object_copies":12416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10360,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2257},{"shards":"0,2","objects":48}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'13886","reported_seq":14021,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:46.734284+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:18:46.734284+0000","last_peered":"2024-09-11T01:18:46.734284+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:18:46.734284+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24756224,"num_objects":6044,"num_object_clones":0,"num_object_copies":12088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10743,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2058},{"shards":"0,2","objects":68}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14067","reported_seq":14201,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:46.767877+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:18:46.767877+0000","last_peered":"2024-09-11T01:18:46.767877+0000","last_clean":"2024-09-11T01:18:46.767877+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:18:46.767877+0000","last_undegraded":"2024-09-11T01:18:46.767877+0000","last_fullsized":"2024-09-11T01:18:46.767877+0000","mapping_epoch":34,"log_start":"40'4000","ondisk_log_start":"40'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25341952,"num_objects":6187,"num_object_clones":0,"num_object_copies":12374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10943,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934359+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934359+0000","last_peered":"2024-09-11T01:17:58.934359+0000","last_clean":"2024-09-11T01:17:58.934359+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934359+0000","last_undegraded":"2024-09-11T01:17:58.934359+0000","last_fullsized":"2024-09-11T01:17:58.934359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13944","reported_seq":16320,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:49.572109+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:18:49.572109+0000","last_peered":"2024-09-11T01:18:49.572109+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:18:49.572109+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"40'3900","ondisk_log_start":"40'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24694784,"num_objects":6029,"num_object_clones":0,"num_object_copies":12058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2813,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11915,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1122,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3216}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13844","reported_seq":13977,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:46.761777+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:18:46.761777+0000","last_peered":"2024-09-11T01:18:46.761777+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:18:46.761777+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24674304,"num_objects":6024,"num_object_clones":0,"num_object_copies":12048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10431,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2040},{"shards":"0,2","objects":78}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14158","reported_seq":14279,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:49.592644+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:18:49.592644+0000","last_peered":"2024-09-11T01:18:49.592644+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:18:49.592644+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24776704,"num_objects":6049,"num_object_clones":0,"num_object_copies":12098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11975,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2025}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935080+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935080+0000","last_peered":"2024-09-11T01:17:58.935080+0000","last_clean":"2024-09-11T01:17:58.935080+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935080+0000","last_undegraded":"2024-09-11T01:17:58.935080+0000","last_fullsized":"2024-09-11T01:17:58.935080+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13836","reported_seq":21822,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:49.327259+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:18:49.327259+0000","last_peered":"2024-09-11T01:18:49.327259+0000","last_clean":"2024-09-11T01:18:49.327259+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:18:49.327259+0000","last_undegraded":"2024-09-11T01:18:49.327259+0000","last_fullsized":"2024-09-11T01:18:49.327259+0000","mapping_epoch":44,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24412160,"num_objects":5960,"num_object_clones":0,"num_object_copies":11920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11621,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14034","reported_seq":14169,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:46.704625+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:18:46.704625+0000","last_peered":"2024-09-11T01:18:46.704625+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:18:46.704625+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25518080,"num_objects":6230,"num_object_clones":0,"num_object_copies":12460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10771,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2280},{"shards":"0,2","objects":49}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938540+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938540+0000","last_peered":"2024-09-11T01:17:58.938540+0000","last_clean":"2024-09-11T01:17:58.938540+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938540+0000","last_undegraded":"2024-09-11T01:17:58.938540+0000","last_fullsized":"2024-09-11T01:17:58.938540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13799","reported_seq":21835,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:45.643169+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:18:45.643169+0000","last_peered":"2024-09-11T01:18:45.643169+0000","last_clean":"2024-09-11T01:18:45.643169+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:18:45.643169+0000","last_undegraded":"2024-09-11T01:18:45.643169+0000","last_fullsized":"2024-09-11T01:18:45.643169+0000","mapping_epoch":46,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24924160,"num_objects":6085,"num_object_clones":0,"num_object_copies":12170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10454,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14061","reported_seq":14201,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:46.644816+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:18:46.644816+0000","last_peered":"2024-09-11T01:18:46.644816+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:18:46.644816+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"40'4000","ondisk_log_start":"40'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25047040,"num_objects":6115,"num_object_clones":0,"num_object_copies":12230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10670,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2076},{"shards":"0,2","objects":68}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971314+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971314+0000","last_peered":"2024-09-11T01:17:58.971314+0000","last_clean":"2024-09-11T01:17:58.971314+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971314+0000","last_undegraded":"2024-09-11T01:17:58.971314+0000","last_fullsized":"2024-09-11T01:17:58.971314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13843","reported_seq":13982,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:46.763206+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:18:46.763206+0000","last_peered":"2024-09-11T01:18:46.763206+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:18:46.763206+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25023081,"num_objects":6110,"num_object_clones":0,"num_object_copies":12220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10913,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2193},{"shards":"0,2","objects":56}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":399811201,"num_objects":97612,"num_object_clones":0,"num_object_copies":195224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38647,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97612,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":184675,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8992,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":993562624,"data_stored":993491858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160925,"ondisk_log_size":160925,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182487,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":729276,"kb_used_data":358892,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93642564,"statfs":{"total":96636764160,"available":95889985536,"internally_reserved":0,"allocated":367505408,"data_stored":367235749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":379252395},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986260,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":647016,"kb_used_data":322456,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93724824,"statfs":{"total":96636764160,"available":95974219776,"internally_reserved":0,"allocated":330194944,"data_stored":329925290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332328618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,0,0,5,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659204,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":616696,"kb_used_data":292136,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93755144,"statfs":{"total":96636764160,"available":96005267456,"internally_reserved":0,"allocated":299147264,"data_stored":298821348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":332328623},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,3,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":365899776,"data_stored":365896297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":298586112,"data_stored":298526376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":329076736,"data_stored":329069185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:50.839 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:51.764 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:18:51.765 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:18:51.823 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":274,"stamp":"2024-09-11T01:18:50.702297+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":399811220,"num_objects":97614,"num_object_clones":0,"num_object_copies":195228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38647,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97614,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":184677,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8993,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160928,"ondisk_log_size":160928,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1992988,"kb_used_data":973484,"kb_used_omap":12,"kb_used_meta":1019443,"kb_avail":281122532,"statfs":{"total":289910292480,"available":287869472768,"internally_reserved":0,"allocated":996847616,"data_stored":995982387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1043909636},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,4,4,10,8,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21889024,"num_objects":-5344,"num_object_clones":0,"num_object_copies":-10688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":139,"num_objects_misplaced":-446,"num_objects_unfound":0,"num_objects_dirty":-5344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5344,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":307,"num_bytes_recovered":1175552,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004018"},"pg_stats":[{"pgid":"2.f","version":"47'14147","reported_seq":14228,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:46.756400+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:18:46.756400+0000","last_peered":"2024-09-11T01:18:46.756400+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:18:46.756400+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25063424,"num_objects":6119,"num_object_clones":0,"num_object_copies":12238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6119,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14147,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2047},{"shards":"0,2","objects":58}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14294","reported_seq":14400,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:49.547336+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:18:49.547336+0000","last_peered":"2024-09-11T01:18:49.547336+0000","last_clean":"2024-09-11T01:18:49.547336+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:18:49.547336+0000","last_undegraded":"2024-09-11T01:18:49.547336+0000","last_fullsized":"2024-09-11T01:18:49.547336+0000","mapping_epoch":35,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24813568,"num_objects":6058,"num_object_clones":0,"num_object_copies":12116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12533,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13987","reported_seq":14120,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:49.494750+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:18:49.494750+0000","last_peered":"2024-09-11T01:18:49.494750+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:18:49.494750+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"40'3900","ondisk_log_start":"40'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25215000,"num_objects":6157,"num_object_clones":0,"num_object_copies":12314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6157,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12288,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2261}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'14018","reported_seq":14092,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:49.483071+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:18:49.483071+0000","last_peered":"2024-09-11T01:18:49.483071+0000","last_clean":"2024-09-11T01:18:49.483071+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:18:49.483071+0000","last_undegraded":"2024-09-11T01:18:49.483071+0000","last_fullsized":"2024-09-11T01:18:49.483071+0000","mapping_epoch":35,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25051136,"num_objects":6116,"num_object_clones":0,"num_object_copies":12232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6116,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14018,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'13888","reported_seq":14019,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:46.750089+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:18:46.750089+0000","last_peered":"2024-09-11T01:18:46.750089+0000","last_clean":"2024-09-11T01:18:46.750089+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:18:46.750089+0000","last_undegraded":"2024-09-11T01:18:46.750089+0000","last_fullsized":"2024-09-11T01:18:46.750089+0000","mapping_epoch":34,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25071616,"num_objects":6121,"num_object_clones":0,"num_object_copies":12242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6121,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10893,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14019","reported_seq":14194,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:46.588927+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:18:46.588927+0000","last_peered":"2024-09-11T01:18:46.588927+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:18:46.588927+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25427968,"num_objects":6208,"num_object_clones":0,"num_object_copies":12416,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6208,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10360,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2257},{"shards":"0,2","objects":48}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'13886","reported_seq":14021,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:46.734284+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:18:46.734284+0000","last_peered":"2024-09-11T01:18:46.734284+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:18:46.734284+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24756224,"num_objects":6044,"num_object_clones":0,"num_object_copies":12088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10743,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2058},{"shards":"0,2","objects":68}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14067","reported_seq":14201,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:46.767877+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:18:46.767877+0000","last_peered":"2024-09-11T01:18:46.767877+0000","last_clean":"2024-09-11T01:18:46.767877+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:18:46.767877+0000","last_undegraded":"2024-09-11T01:18:46.767877+0000","last_fullsized":"2024-09-11T01:18:46.767877+0000","mapping_epoch":34,"log_start":"40'4000","ondisk_log_start":"40'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25341952,"num_objects":6187,"num_object_clones":0,"num_object_copies":12374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10943,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934359+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934359+0000","last_peered":"2024-09-11T01:17:58.934359+0000","last_clean":"2024-09-11T01:17:58.934359+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934359+0000","last_undegraded":"2024-09-11T01:17:58.934359+0000","last_fullsized":"2024-09-11T01:17:58.934359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13944","reported_seq":16320,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:49.572109+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:18:49.572109+0000","last_peered":"2024-09-11T01:18:49.572109+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:18:49.572109+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"40'3900","ondisk_log_start":"40'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24694784,"num_objects":6029,"num_object_clones":0,"num_object_copies":12058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2813,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11915,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1122,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3216}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'13844","reported_seq":13977,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:46.761777+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:18:46.761777+0000","last_peered":"2024-09-11T01:18:46.761777+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:18:46.761777+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24674304,"num_objects":6024,"num_object_clones":0,"num_object_copies":12048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6024,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10431,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2040},{"shards":"0,2","objects":78}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14158","reported_seq":14279,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:49.592644+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:18:49.592644+0000","last_peered":"2024-09-11T01:18:49.592644+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:18:49.592644+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24776704,"num_objects":6049,"num_object_clones":0,"num_object_copies":12098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11975,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2025}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935080+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935080+0000","last_peered":"2024-09-11T01:17:58.935080+0000","last_clean":"2024-09-11T01:17:58.935080+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935080+0000","last_undegraded":"2024-09-11T01:17:58.935080+0000","last_fullsized":"2024-09-11T01:17:58.935080+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13836","reported_seq":21822,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:49.327259+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:18:49.327259+0000","last_peered":"2024-09-11T01:18:49.327259+0000","last_clean":"2024-09-11T01:18:49.327259+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:18:49.327259+0000","last_undegraded":"2024-09-11T01:18:49.327259+0000","last_fullsized":"2024-09-11T01:18:49.327259+0000","mapping_epoch":44,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24412160,"num_objects":5960,"num_object_clones":0,"num_object_copies":11920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11621,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14034","reported_seq":14169,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:46.704625+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:18:46.704625+0000","last_peered":"2024-09-11T01:18:46.704625+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:18:46.704625+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25518080,"num_objects":6230,"num_object_clones":0,"num_object_copies":12460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10771,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2280},{"shards":"0,2","objects":49}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938540+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938540+0000","last_peered":"2024-09-11T01:17:58.938540+0000","last_clean":"2024-09-11T01:17:58.938540+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938540+0000","last_undegraded":"2024-09-11T01:17:58.938540+0000","last_fullsized":"2024-09-11T01:17:58.938540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13799","reported_seq":21835,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:45.643169+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:18:45.643169+0000","last_peered":"2024-09-11T01:18:45.643169+0000","last_clean":"2024-09-11T01:18:45.643169+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:18:45.643169+0000","last_undegraded":"2024-09-11T01:18:45.643169+0000","last_fullsized":"2024-09-11T01:18:45.643169+0000","mapping_epoch":46,"log_start":"39'3700","ondisk_log_start":"39'3700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24924160,"num_objects":6085,"num_object_clones":0,"num_object_copies":12170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10454,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14061","reported_seq":14201,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:46.644816+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:18:46.644816+0000","last_peered":"2024-09-11T01:18:46.644816+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:18:46.644816+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"40'4000","ondisk_log_start":"40'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25047040,"num_objects":6115,"num_object_clones":0,"num_object_copies":12230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6115,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10670,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2076},{"shards":"0,2","objects":68}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971314+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971314+0000","last_peered":"2024-09-11T01:17:58.971314+0000","last_clean":"2024-09-11T01:17:58.971314+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971314+0000","last_undegraded":"2024-09-11T01:17:58.971314+0000","last_fullsized":"2024-09-11T01:17:58.971314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13843","reported_seq":13982,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:46.763206+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:18:46.763206+0000","last_peered":"2024-09-11T01:18:46.763206+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:18:46.763206+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25023081,"num_objects":6110,"num_object_clones":0,"num_object_copies":12220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10913,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2193},{"shards":"0,2","objects":56}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":399811201,"num_objects":97612,"num_object_clones":0,"num_object_copies":195224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38647,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97612,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":184675,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8992,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":993562624,"data_stored":993491858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160925,"ondisk_log_size":160925,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182487,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":729276,"kb_used_data":358892,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93642564,"statfs":{"total":96636764160,"available":95889985536,"internally_reserved":0,"allocated":367505408,"data_stored":367235749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":379252395},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986260,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":647016,"kb_used_data":322456,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93724824,"statfs":{"total":96636764160,"available":95974219776,"internally_reserved":0,"allocated":330194944,"data_stored":329925290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332328618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,0,0,5,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659204,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":616696,"kb_used_data":292136,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93755144,"statfs":{"total":96636764160,"available":96005267456,"internally_reserved":0,"allocated":299147264,"data_stored":298821348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":332328623},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,3,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":365899776,"data_stored":365896297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":298586112,"data_stored":298526376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":329076736,"data_stored":329069185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:51.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:18:51.824 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'14147", 'reported_seq': 14228, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:46.756400+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:18:46.756400+0000', 'last_peered': '2024-09-11T01:18:46.756400+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:18:46.756400+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "43'4100", 'ondisk_log_start': "43'4100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25063424, 'num_objects': 6119, 'num_object_clones': 0, 'num_object_copies': 12238, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6119, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14147, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2047}, {'shards': '0,2', 'objects': 58}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:51.825 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:18:51.825 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'13987", 'reported_seq': 14120, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:49.494750+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:18:49.494750+0000', 'last_peered': '2024-09-11T01:18:49.494750+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:18:49.494750+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "40'3900", 'ondisk_log_start': "40'3900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25215000, 'num_objects': 6157, 'num_object_clones': 0, 'num_object_copies': 12314, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6157, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12288, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2261}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:51.825 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:18:51.825 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'14019", 'reported_seq': 14194, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:46.588927+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:18:46.588927+0000', 'last_peered': '2024-09-11T01:18:46.588927+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:18:46.588927+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "43'4000", 'ondisk_log_start': "43'4000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25427968, 'num_objects': 6208, 'num_object_clones': 0, 'num_object_copies': 12416, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6208, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10360, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2257}, {'shards': '0,2', 'objects': 48}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:51.825 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:18:51.825 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'13886", 'reported_seq': 14021, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:46.734284+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:18:46.734284+0000', 'last_peered': '2024-09-11T01:18:46.734284+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:18:46.734284+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "39'3800", 'ondisk_log_start': "39'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24756224, 'num_objects': 6044, 'num_object_clones': 0, 'num_object_copies': 12088, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6044, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10743, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2058}, {'shards': '0,2', 'objects': 68}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:51.825 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:18:51.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'13944", 'reported_seq': 16320, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:49.572109+0000', 'last_change': '2024-09-11T01:17:57.355893+0000', 'last_active': '2024-09-11T01:18:49.572109+0000', 'last_peered': '2024-09-11T01:18:49.572109+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:18:49.572109+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "40'3900", 'ondisk_log_start': "40'3900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24694784, 'num_objects': 6029, 'num_object_clones': 0, 'num_object_copies': 12058, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2813, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6029, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11915, 'num_write_kb': 31844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1122, 'num_bytes_recovered': 4505600, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3216}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:51.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:18:51.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'13844", 'reported_seq': 13977, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:46.761777+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:18:46.761777+0000', 'last_peered': '2024-09-11T01:18:46.761777+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:18:46.761777+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "39'3800", 'ondisk_log_start': "39'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24674304, 'num_objects': 6024, 'num_object_clones': 0, 'num_object_copies': 12048, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6024, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10431, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2040}, {'shards': '0,2', 'objects': 78}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:51.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:18:51.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'14158", 'reported_seq': 14279, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:49.592644+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:18:49.592644+0000', 'last_peered': '2024-09-11T01:18:49.592644+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:18:49.592644+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "43'4100", 'ondisk_log_start': "43'4100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24776704, 'num_objects': 6049, 'num_object_clones': 0, 'num_object_copies': 12098, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6049, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11975, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2025}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:51.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:18:51.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'14034", 'reported_seq': 14169, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:46.704625+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:18:46.704625+0000', 'last_peered': '2024-09-11T01:18:46.704625+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:18:46.704625+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "43'4000", 'ondisk_log_start': "43'4000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25518080, 'num_objects': 6230, 'num_object_clones': 0, 'num_object_copies': 12460, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6230, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10771, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2280}, {'shards': '0,2', 'objects': 49}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:51.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:18:51.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'14061", 'reported_seq': 14201, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:46.644816+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:18:46.644816+0000', 'last_peered': '2024-09-11T01:18:46.644816+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:18:46.644816+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "40'4000", 'ondisk_log_start': "40'4000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25047040, 'num_objects': 6115, 'num_object_clones': 0, 'num_object_copies': 12230, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6115, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10670, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2076}, {'shards': '0,2', 'objects': 68}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:51.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:18:51.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'13843", 'reported_seq': 13982, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:46.763206+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:18:46.763206+0000', 'last_peered': '2024-09-11T01:18:46.763206+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:18:46.763206+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "39'3800", 'ondisk_log_start': "39'3800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 25023081, 'num_objects': 6110, 'num_object_clones': 0, 'num_object_copies': 12220, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6110, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10913, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2193}, {'shards': '0,2', 'objects': 56}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:51.827 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:18:52.903 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:18:52.903 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38647/195228 objects degraded (19.796%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":469,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":97614,"data_bytes":399811220,"bytes_used":2040819712,"bytes_avail":287869472768,"bytes_total":289910292480,"degraded_objects":38647,"degraded_total":195228,"degraded_ratio":0.1979582846722806,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":97929,"recovering_keys_per_sec":0,"num_objects_recovered":307,"num_bytes_recovered":1175552,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":445},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:18:53.009 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:18:53.009 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:53.841 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:18:53.842 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:18:53.896 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":275,"stamp":"2024-09-11T01:18:52.704400+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":393802388,"num_objects":96147,"num_object_clones":0,"num_object_copies":192294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38647,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96147,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":186144,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8993,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160795,"ondisk_log_size":160795,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1996128,"kb_used_data":964144,"kb_used_omap":12,"kb_used_meta":1031923,"kb_avail":281119392,"statfs":{"total":289910292480,"available":287866257408,"internally_reserved":0,"allocated":987283456,"data_stored":986418227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1056689156},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,6,1,3,3,8,6,9],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23859200,"num_objects":-5825,"num_object_clones":0,"num_object_copies":-11650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":78,"num_objects_misplaced":-288,"num_objects_unfound":0,"num_objects_dirty":-5825,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5825,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":210,"num_bytes_recovered":778240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005653"},"pg_stats":[{"pgid":"2.f","version":"47'14282","reported_seq":14365,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:51.761595+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:18:51.761595+0000","last_peered":"2024-09-11T01:18:51.761595+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:18:51.761595+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24510464,"num_objects":5984,"num_object_clones":0,"num_object_copies":11968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14282,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1912}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14294","reported_seq":14400,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:49.547336+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:18:49.547336+0000","last_peered":"2024-09-11T01:18:49.547336+0000","last_clean":"2024-09-11T01:18:49.547336+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:18:49.547336+0000","last_undegraded":"2024-09-11T01:18:49.547336+0000","last_fullsized":"2024-09-11T01:18:49.547336+0000","mapping_epoch":35,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24813568,"num_objects":6058,"num_object_clones":0,"num_object_copies":12116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12533,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'13987","reported_seq":14120,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:49.494750+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:18:49.494750+0000","last_peered":"2024-09-11T01:18:49.494750+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:18:49.494750+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"40'3900","ondisk_log_start":"40'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25215000,"num_objects":6157,"num_object_clones":0,"num_object_copies":12314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6157,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12288,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2261}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'14018","reported_seq":14092,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:49.483071+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:18:49.483071+0000","last_peered":"2024-09-11T01:18:49.483071+0000","last_clean":"2024-09-11T01:18:49.483071+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:18:49.483071+0000","last_undegraded":"2024-09-11T01:18:49.483071+0000","last_fullsized":"2024-09-11T01:18:49.483071+0000","mapping_epoch":35,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":25051136,"num_objects":6116,"num_object_clones":0,"num_object_copies":12232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6116,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14018,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14053","reported_seq":14186,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:51.748662+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:18:51.748662+0000","last_peered":"2024-09-11T01:18:51.748662+0000","last_clean":"2024-09-11T01:18:51.748662+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:18:51.748662+0000","last_undegraded":"2024-09-11T01:18:51.748662+0000","last_fullsized":"2024-09-11T01:18:51.748662+0000","mapping_epoch":34,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24395776,"num_objects":5956,"num_object_clones":0,"num_object_copies":11912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5956,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11058,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14141","reported_seq":14316,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:51.762306+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:18:51.762306+0000","last_peered":"2024-09-11T01:18:51.762306+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:18:51.762306+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24928256,"num_objects":6086,"num_object_clones":0,"num_object_copies":12172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10482,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2135}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'14042","reported_seq":14177,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:51.716241+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:18:51.716241+0000","last_peered":"2024-09-11T01:18:51.716241+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:18:51.716241+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24117248,"num_objects":5888,"num_object_clones":0,"num_object_copies":11776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10899,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1902}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14217","reported_seq":14352,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:51.784225+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:18:51.784225+0000","last_peered":"2024-09-11T01:18:51.784225+0000","last_clean":"2024-09-11T01:18:51.784225+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:18:51.784225+0000","last_undegraded":"2024-09-11T01:18:51.784225+0000","last_fullsized":"2024-09-11T01:18:51.784225+0000","mapping_epoch":34,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24727552,"num_objects":6037,"num_object_clones":0,"num_object_copies":12074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11093,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934359+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934359+0000","last_peered":"2024-09-11T01:17:58.934359+0000","last_clean":"2024-09-11T01:17:58.934359+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934359+0000","last_undegraded":"2024-09-11T01:17:58.934359+0000","last_fullsized":"2024-09-11T01:17:58.934359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'13944","reported_seq":16320,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:49.572109+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:18:49.572109+0000","last_peered":"2024-09-11T01:18:49.572109+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:18:49.572109+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"40'3900","ondisk_log_start":"40'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24694784,"num_objects":6029,"num_object_clones":0,"num_object_copies":12058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2813,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11915,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1122,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3216}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14007","reported_seq":14137,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:51.809320+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:18:51.809320+0000","last_peered":"2024-09-11T01:18:51.809320+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:18:51.809320+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24006656,"num_objects":5861,"num_object_clones":0,"num_object_copies":11722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10594,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1877}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14158","reported_seq":14279,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:49.592644+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:18:49.592644+0000","last_peered":"2024-09-11T01:18:49.592644+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:18:49.592644+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24776704,"num_objects":6049,"num_object_clones":0,"num_object_copies":12098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11975,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2025}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935080+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935080+0000","last_peered":"2024-09-11T01:17:58.935080+0000","last_clean":"2024-09-11T01:17:58.935080+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935080+0000","last_undegraded":"2024-09-11T01:17:58.935080+0000","last_fullsized":"2024-09-11T01:17:58.935080+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13836","reported_seq":21822,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:49.327259+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:18:49.327259+0000","last_peered":"2024-09-11T01:18:49.327259+0000","last_clean":"2024-09-11T01:18:49.327259+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:18:49.327259+0000","last_undegraded":"2024-09-11T01:18:49.327259+0000","last_fullsized":"2024-09-11T01:18:49.327259+0000","mapping_epoch":44,"log_start":"39'3800","ondisk_log_start":"39'3800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24412160,"num_objects":5960,"num_object_clones":0,"num_object_copies":11920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11621,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14174","reported_seq":14309,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:51.761977+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:18:51.761977+0000","last_peered":"2024-09-11T01:18:51.761977+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:18:51.761977+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24944640,"num_objects":6090,"num_object_clones":0,"num_object_copies":12180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10911,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2140}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938540+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938540+0000","last_peered":"2024-09-11T01:17:58.938540+0000","last_clean":"2024-09-11T01:17:58.938540+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938540+0000","last_undegraded":"2024-09-11T01:17:58.938540+0000","last_fullsized":"2024-09-11T01:17:58.938540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'13947","reported_seq":21983,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:50.716154+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:18:50.716154+0000","last_peered":"2024-09-11T01:18:50.716154+0000","last_clean":"2024-09-11T01:18:50.716154+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:18:50.716154+0000","last_undegraded":"2024-09-11T01:18:50.716154+0000","last_fullsized":"2024-09-11T01:18:50.716154+0000","mapping_epoch":46,"log_start":"40'3900","ondisk_log_start":"40'3900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24317952,"num_objects":5937,"num_object_clones":0,"num_object_copies":11874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5937,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10602,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14217","reported_seq":14356,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:51.756092+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:18:51.756092+0000","last_peered":"2024-09-11T01:18:51.756092+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:18:51.756092+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24408064,"num_objects":5959,"num_object_clones":0,"num_object_copies":11918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5959,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10826,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1920}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971314+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971314+0000","last_peered":"2024-09-11T01:17:58.971314+0000","last_clean":"2024-09-11T01:17:58.971314+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971314+0000","last_undegraded":"2024-09-11T01:17:58.971314+0000","last_fullsized":"2024-09-11T01:17:58.971314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13975","reported_seq":14114,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:51.820619+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:18:51.820619+0000","last_peered":"2024-09-11T01:18:51.820619+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:18:51.820619+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"40'3900","ondisk_log_start":"40'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24482409,"num_objects":5978,"num_object_clones":0,"num_object_copies":11956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11045,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2061}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":393802369,"num_objects":96145,"num_object_clones":0,"num_object_copies":192290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38647,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96145,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":186142,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8992,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":982990848,"data_stored":982920082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160792,"ondisk_log_size":160792,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182488,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":726448,"kb_used_data":351904,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93645392,"statfs":{"total":96636764160,"available":95892881408,"internally_reserved":0,"allocated":360349696,"data_stored":360080037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":383512235},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3,0,2,3,2,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986260,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":647016,"kb_used_data":322456,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93724824,"statfs":{"total":96636764160,"available":95974219776,"internally_reserved":0,"allocated":330194944,"data_stored":329925290,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332328618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,0,0,5,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659205,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":622664,"kb_used_data":289784,"kb_used_omap":4,"kb_used_meta":332859,"kb_avail":93749176,"statfs":{"total":96636764160,"available":95999156224,"internally_reserved":0,"allocated":296738816,"data_stored":296412900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":340848303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,0,1,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":358662144,"data_stored":358658665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":295251968,"data_stored":295192232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":329076736,"data_stored":329069185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:56.898 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:57.608 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:18:57.609 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:18:57.672 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":277,"stamp":"2024-09-11T01:18:56.706217+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":389812884,"num_objects":95173,"num_object_clones":0,"num_object_copies":190346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95173,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":187118,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9099,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160869,"ondisk_log_size":160869,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1976564,"kb_used_data":952896,"kb_used_omap":12,"kb_used_meta":1023603,"kb_avail":281138956,"statfs":{"total":289910292480,"available":287886290944,"internally_reserved":0,"allocated":975765504,"data_stored":974900275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1048169476},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3,2,4,5,10,7,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19296256,"num_objects":-4711,"num_object_clones":0,"num_object_copies":-9422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-100,"num_objects_misplaced":-123,"num_objects_unfound":0,"num_objects_dirty":-4711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4711,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":397312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005314"},"pg_stats":[{"pgid":"2.f","version":"47'14282","reported_seq":14365,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:51.761595+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:18:51.761595+0000","last_peered":"2024-09-11T01:18:51.761595+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:18:51.761595+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24510464,"num_objects":5984,"num_object_clones":0,"num_object_copies":11968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14282,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1912}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14423","reported_seq":14530,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:54.604760+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:18:54.604760+0000","last_peered":"2024-09-11T01:18:54.604760+0000","last_clean":"2024-09-11T01:18:54.604760+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:18:54.604760+0000","last_undegraded":"2024-09-11T01:18:54.604760+0000","last_fullsized":"2024-09-11T01:18:54.604760+0000","mapping_epoch":35,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24285184,"num_objects":5929,"num_object_clones":0,"num_object_copies":11858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12662,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14134","reported_seq":14266,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:54.526136+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:18:54.526136+0000","last_peered":"2024-09-11T01:18:54.526136+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:18:54.526136+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24612888,"num_objects":6010,"num_object_clones":0,"num_object_copies":12020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6010,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12435,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2114}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'14165","reported_seq":14238,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:54.556682+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:18:54.556682+0000","last_peered":"2024-09-11T01:18:54.556682+0000","last_clean":"2024-09-11T01:18:54.556682+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:18:54.556682+0000","last_undegraded":"2024-09-11T01:18:54.556682+0000","last_fullsized":"2024-09-11T01:18:54.556682+0000","mapping_epoch":35,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24449024,"num_objects":5969,"num_object_clones":0,"num_object_copies":11938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14165,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14053","reported_seq":14186,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:51.748662+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:18:51.748662+0000","last_peered":"2024-09-11T01:18:51.748662+0000","last_clean":"2024-09-11T01:18:51.748662+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:18:51.748662+0000","last_undegraded":"2024-09-11T01:18:51.748662+0000","last_fullsized":"2024-09-11T01:18:51.748662+0000","mapping_epoch":34,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24395776,"num_objects":5956,"num_object_clones":0,"num_object_copies":11912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5956,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11058,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14141","reported_seq":14316,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:51.762306+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:18:51.762306+0000","last_peered":"2024-09-11T01:18:51.762306+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:18:51.762306+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24928256,"num_objects":6086,"num_object_clones":0,"num_object_copies":12172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10482,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2135}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'14042","reported_seq":14177,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:51.716241+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:18:51.716241+0000","last_peered":"2024-09-11T01:18:51.716241+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:18:51.716241+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24117248,"num_objects":5888,"num_object_clones":0,"num_object_copies":11776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10899,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1902}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14217","reported_seq":14352,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:51.784225+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:18:51.784225+0000","last_peered":"2024-09-11T01:18:51.784225+0000","last_clean":"2024-09-11T01:18:51.784225+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:18:51.784225+0000","last_undegraded":"2024-09-11T01:18:51.784225+0000","last_fullsized":"2024-09-11T01:18:51.784225+0000","mapping_epoch":34,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24727552,"num_objects":6037,"num_object_clones":0,"num_object_copies":12074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11093,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934359+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934359+0000","last_peered":"2024-09-11T01:17:58.934359+0000","last_clean":"2024-09-11T01:17:58.934359+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934359+0000","last_undegraded":"2024-09-11T01:17:58.934359+0000","last_fullsized":"2024-09-11T01:17:58.934359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14091","reported_seq":16681,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:54.594306+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:18:54.594306+0000","last_peered":"2024-09-11T01:18:54.594306+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:18:54.594306+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24092672,"num_objects":5882,"num_object_clones":0,"num_object_copies":11764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2707,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12062,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1228,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3175}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14007","reported_seq":14137,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:51.809320+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:18:51.809320+0000","last_peered":"2024-09-11T01:18:51.809320+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:18:51.809320+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24006656,"num_objects":5861,"num_object_clones":0,"num_object_copies":11722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10594,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1877}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14287","reported_seq":14411,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:54.472555+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:18:54.472555+0000","last_peered":"2024-09-11T01:18:54.472555+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:18:54.472555+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24248320,"num_objects":5920,"num_object_clones":0,"num_object_copies":11840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12104,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1896}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935080+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935080+0000","last_peered":"2024-09-11T01:17:58.935080+0000","last_clean":"2024-09-11T01:17:58.935080+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935080+0000","last_undegraded":"2024-09-11T01:17:58.935080+0000","last_fullsized":"2024-09-11T01:17:58.935080+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13959","reported_seq":21945,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:54.583851+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:18:54.583851+0000","last_peered":"2024-09-11T01:18:54.583851+0000","last_clean":"2024-09-11T01:18:54.583851+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:18:54.583851+0000","last_undegraded":"2024-09-11T01:18:54.583851+0000","last_fullsized":"2024-09-11T01:18:54.583851+0000","mapping_epoch":44,"log_start":"40'3900","ondisk_log_start":"40'3900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23908352,"num_objects":5837,"num_object_clones":0,"num_object_copies":11674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11744,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14174","reported_seq":14309,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:51.761977+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:18:51.761977+0000","last_peered":"2024-09-11T01:18:51.761977+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:18:51.761977+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24944640,"num_objects":6090,"num_object_clones":0,"num_object_copies":12180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10911,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2140}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938540+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938540+0000","last_peered":"2024-09-11T01:17:58.938540+0000","last_clean":"2024-09-11T01:17:58.938540+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938540+0000","last_undegraded":"2024-09-11T01:17:58.938540+0000","last_fullsized":"2024-09-11T01:17:58.938540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14099","reported_seq":22135,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:55.778360+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:18:55.778360+0000","last_peered":"2024-09-11T01:18:55.778360+0000","last_clean":"2024-09-11T01:18:55.778360+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:18:55.778360+0000","last_undegraded":"2024-09-11T01:18:55.778360+0000","last_fullsized":"2024-09-11T01:18:55.778360+0000","mapping_epoch":46,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23695360,"num_objects":5785,"num_object_clones":0,"num_object_copies":11570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10754,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14217","reported_seq":14356,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:51.756092+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:18:51.756092+0000","last_peered":"2024-09-11T01:18:51.756092+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:18:51.756092+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24408064,"num_objects":5959,"num_object_clones":0,"num_object_copies":11918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5959,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10826,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1920}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971314+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971314+0000","last_peered":"2024-09-11T01:17:58.971314+0000","last_clean":"2024-09-11T01:17:58.971314+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971314+0000","last_undegraded":"2024-09-11T01:17:58.971314+0000","last_fullsized":"2024-09-11T01:17:58.971314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13975","reported_seq":14114,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:51.820619+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:18:51.820619+0000","last_peered":"2024-09-11T01:18:51.820619+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:18:51.820619+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"40'3900","ondisk_log_start":"40'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24482409,"num_objects":5978,"num_object_clones":0,"num_object_copies":11956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11045,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2061}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":389812865,"num_objects":95171,"num_object_clones":0,"num_object_copies":190342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95171,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":187116,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9098,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":971169792,"data_stored":971099026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160866,"ondisk_log_size":160866,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182488,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":726448,"kb_used_data":351904,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93645392,"statfs":{"total":96636764160,"available":95892881408,"internally_reserved":0,"allocated":360349696,"data_stored":360080037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":383512235},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3,0,2,3,2,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986261,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":646660,"kb_used_data":317940,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93725180,"statfs":{"total":96636764160,"available":95974584320,"internally_reserved":0,"allocated":325570560,"data_stored":325300906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336588458},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659206,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":603456,"kb_used_data":283052,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93768384,"statfs":{"total":96636764160,"available":96018825216,"internally_reserved":0,"allocated":289845248,"data_stored":289519332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":328068783},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,6,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":358662144,"data_stored":358658665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":288092160,"data_stored":288032424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":324415488,"data_stored":324407937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:57.673 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:18:58.607 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:18:58.608 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:18:58.679 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":277,"stamp":"2024-09-11T01:18:56.706217+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":389812884,"num_objects":95173,"num_object_clones":0,"num_object_copies":190346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95173,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":187118,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9099,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160869,"ondisk_log_size":160869,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1976564,"kb_used_data":952896,"kb_used_omap":12,"kb_used_meta":1023603,"kb_avail":281138956,"statfs":{"total":289910292480,"available":287886290944,"internally_reserved":0,"allocated":975765504,"data_stored":974900275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1048169476},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3,2,4,5,10,7,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19296256,"num_objects":-4711,"num_object_clones":0,"num_object_copies":-9422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-100,"num_objects_misplaced":-123,"num_objects_unfound":0,"num_objects_dirty":-4711,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4711,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":397312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005314"},"pg_stats":[{"pgid":"2.f","version":"47'14282","reported_seq":14365,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:51.761595+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:18:51.761595+0000","last_peered":"2024-09-11T01:18:51.761595+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:18:51.761595+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24510464,"num_objects":5984,"num_object_clones":0,"num_object_copies":11968,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5984,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14282,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1912}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14423","reported_seq":14530,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:54.604760+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:18:54.604760+0000","last_peered":"2024-09-11T01:18:54.604760+0000","last_clean":"2024-09-11T01:18:54.604760+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:18:54.604760+0000","last_undegraded":"2024-09-11T01:18:54.604760+0000","last_fullsized":"2024-09-11T01:18:54.604760+0000","mapping_epoch":35,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24285184,"num_objects":5929,"num_object_clones":0,"num_object_copies":11858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12662,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14134","reported_seq":14266,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:54.526136+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:18:54.526136+0000","last_peered":"2024-09-11T01:18:54.526136+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:18:54.526136+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24612888,"num_objects":6010,"num_object_clones":0,"num_object_copies":12020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6010,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12435,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2114}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'14165","reported_seq":14238,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:54.556682+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:18:54.556682+0000","last_peered":"2024-09-11T01:18:54.556682+0000","last_clean":"2024-09-11T01:18:54.556682+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:18:54.556682+0000","last_undegraded":"2024-09-11T01:18:54.556682+0000","last_fullsized":"2024-09-11T01:18:54.556682+0000","mapping_epoch":35,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24449024,"num_objects":5969,"num_object_clones":0,"num_object_copies":11938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14165,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14053","reported_seq":14186,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:51.748662+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:18:51.748662+0000","last_peered":"2024-09-11T01:18:51.748662+0000","last_clean":"2024-09-11T01:18:51.748662+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:18:51.748662+0000","last_undegraded":"2024-09-11T01:18:51.748662+0000","last_fullsized":"2024-09-11T01:18:51.748662+0000","mapping_epoch":34,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24395776,"num_objects":5956,"num_object_clones":0,"num_object_copies":11912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5956,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11058,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14141","reported_seq":14316,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:51.762306+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:18:51.762306+0000","last_peered":"2024-09-11T01:18:51.762306+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:18:51.762306+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24928256,"num_objects":6086,"num_object_clones":0,"num_object_copies":12172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6086,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10482,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2135}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'14042","reported_seq":14177,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:51.716241+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:18:51.716241+0000","last_peered":"2024-09-11T01:18:51.716241+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:18:51.716241+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24117248,"num_objects":5888,"num_object_clones":0,"num_object_copies":11776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10899,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1902}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14217","reported_seq":14352,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:51.784225+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:18:51.784225+0000","last_peered":"2024-09-11T01:18:51.784225+0000","last_clean":"2024-09-11T01:18:51.784225+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:18:51.784225+0000","last_undegraded":"2024-09-11T01:18:51.784225+0000","last_fullsized":"2024-09-11T01:18:51.784225+0000","mapping_epoch":34,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24727552,"num_objects":6037,"num_object_clones":0,"num_object_copies":12074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11093,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934359+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934359+0000","last_peered":"2024-09-11T01:17:58.934359+0000","last_clean":"2024-09-11T01:17:58.934359+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934359+0000","last_undegraded":"2024-09-11T01:17:58.934359+0000","last_fullsized":"2024-09-11T01:17:58.934359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14091","reported_seq":16681,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:54.594306+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:18:54.594306+0000","last_peered":"2024-09-11T01:18:54.594306+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:18:54.594306+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24092672,"num_objects":5882,"num_object_clones":0,"num_object_copies":11764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2707,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12062,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1228,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3175}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14007","reported_seq":14137,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:51.809320+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:18:51.809320+0000","last_peered":"2024-09-11T01:18:51.809320+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:18:51.809320+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24006656,"num_objects":5861,"num_object_clones":0,"num_object_copies":11722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10594,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1877}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14287","reported_seq":14411,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:54.472555+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:18:54.472555+0000","last_peered":"2024-09-11T01:18:54.472555+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:18:54.472555+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24248320,"num_objects":5920,"num_object_clones":0,"num_object_copies":11840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12104,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1896}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935080+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935080+0000","last_peered":"2024-09-11T01:17:58.935080+0000","last_clean":"2024-09-11T01:17:58.935080+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935080+0000","last_undegraded":"2024-09-11T01:17:58.935080+0000","last_fullsized":"2024-09-11T01:17:58.935080+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13959","reported_seq":21945,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:54.583851+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:18:54.583851+0000","last_peered":"2024-09-11T01:18:54.583851+0000","last_clean":"2024-09-11T01:18:54.583851+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:18:54.583851+0000","last_undegraded":"2024-09-11T01:18:54.583851+0000","last_fullsized":"2024-09-11T01:18:54.583851+0000","mapping_epoch":44,"log_start":"40'3900","ondisk_log_start":"40'3900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23908352,"num_objects":5837,"num_object_clones":0,"num_object_copies":11674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11744,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14174","reported_seq":14309,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:51.761977+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:18:51.761977+0000","last_peered":"2024-09-11T01:18:51.761977+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:18:51.761977+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24944640,"num_objects":6090,"num_object_clones":0,"num_object_copies":12180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6090,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10911,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2140}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938540+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938540+0000","last_peered":"2024-09-11T01:17:58.938540+0000","last_clean":"2024-09-11T01:17:58.938540+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938540+0000","last_undegraded":"2024-09-11T01:17:58.938540+0000","last_fullsized":"2024-09-11T01:17:58.938540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14099","reported_seq":22135,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:55.778360+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:18:55.778360+0000","last_peered":"2024-09-11T01:18:55.778360+0000","last_clean":"2024-09-11T01:18:55.778360+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:18:55.778360+0000","last_undegraded":"2024-09-11T01:18:55.778360+0000","last_fullsized":"2024-09-11T01:18:55.778360+0000","mapping_epoch":46,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23695360,"num_objects":5785,"num_object_clones":0,"num_object_copies":11570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10754,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14217","reported_seq":14356,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:51.756092+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:18:51.756092+0000","last_peered":"2024-09-11T01:18:51.756092+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:18:51.756092+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24408064,"num_objects":5959,"num_object_clones":0,"num_object_copies":11918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5959,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10826,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1920}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971314+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971314+0000","last_peered":"2024-09-11T01:17:58.971314+0000","last_clean":"2024-09-11T01:17:58.971314+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971314+0000","last_undegraded":"2024-09-11T01:17:58.971314+0000","last_fullsized":"2024-09-11T01:17:58.971314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'13975","reported_seq":14114,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:51.820619+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:18:51.820619+0000","last_peered":"2024-09-11T01:18:51.820619+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:18:51.820619+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"40'3900","ondisk_log_start":"40'3900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24482409,"num_objects":5978,"num_object_clones":0,"num_object_copies":11956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5978,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11045,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2061}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":389812865,"num_objects":95171,"num_object_clones":0,"num_object_copies":190342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95171,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":187116,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9098,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":971169792,"data_stored":971099026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160866,"ondisk_log_size":160866,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182488,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":726448,"kb_used_data":351904,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93645392,"statfs":{"total":96636764160,"available":95892881408,"internally_reserved":0,"allocated":360349696,"data_stored":360080037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":383512235},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,3,0,2,3,2,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986261,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":646660,"kb_used_data":317940,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93725180,"statfs":{"total":96636764160,"available":95974584320,"internally_reserved":0,"allocated":325570560,"data_stored":325300906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336588458},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659206,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":603456,"kb_used_data":283052,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93768384,"statfs":{"total":96636764160,"available":96018825216,"internally_reserved":0,"allocated":289845248,"data_stored":289519332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":328068783},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,6,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":358662144,"data_stored":358658665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":288092160,"data_stored":288032424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":324415488,"data_stored":324407937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:18:58.680 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:18:58.681 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'14282", 'reported_seq': 14365, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:51.761595+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:18:51.761595+0000', 'last_peered': '2024-09-11T01:18:51.761595+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:18:51.761595+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "43'4200", 'ondisk_log_start': "43'4200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24510464, 'num_objects': 5984, 'num_object_clones': 0, 'num_object_copies': 11968, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5984, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14282, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1912}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:58.681 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:18:58.681 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'14134", 'reported_seq': 14266, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:54.526136+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:18:54.526136+0000', 'last_peered': '2024-09-11T01:18:54.526136+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:18:54.526136+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "43'4100", 'ondisk_log_start': "43'4100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24612888, 'num_objects': 6010, 'num_object_clones': 0, 'num_object_copies': 12020, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6010, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12435, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 2114}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:58.681 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:18:58.681 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'14141", 'reported_seq': 14316, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:51.762306+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:18:51.762306+0000', 'last_peered': '2024-09-11T01:18:51.762306+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:18:51.762306+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "43'4100", 'ondisk_log_start': "43'4100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24928256, 'num_objects': 6086, 'num_object_clones': 0, 'num_object_copies': 12172, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6086, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10482, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2135}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:58.681 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:18:58.681 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'14042", 'reported_seq': 14177, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:51.716241+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:18:51.716241+0000', 'last_peered': '2024-09-11T01:18:51.716241+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:18:51.716241+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "43'4000", 'ondisk_log_start': "43'4000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24117248, 'num_objects': 5888, 'num_object_clones': 0, 'num_object_copies': 11776, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5888, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10899, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1902}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:58.681 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:18:58.681 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'14091", 'reported_seq': 16681, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:54.594306+0000', 'last_change': '2024-09-11T01:17:57.355893+0000', 'last_active': '2024-09-11T01:18:54.594306+0000', 'last_peered': '2024-09-11T01:18:54.594306+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:18:54.594306+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "43'4000", 'ondisk_log_start': "43'4000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24092672, 'num_objects': 5882, 'num_object_clones': 0, 'num_object_copies': 11764, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2707, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5882, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12062, 'num_write_kb': 31844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1228, 'num_bytes_recovered': 4505600, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3175}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:58.682 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:18:58.682 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'14007", 'reported_seq': 14137, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:51.809320+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:18:51.809320+0000', 'last_peered': '2024-09-11T01:18:51.809320+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:18:51.809320+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "43'4000", 'ondisk_log_start': "43'4000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24006656, 'num_objects': 5861, 'num_object_clones': 0, 'num_object_copies': 11722, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5861, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10594, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1877}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:58.682 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:18:58.682 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'14287", 'reported_seq': 14411, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:54.472555+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:18:54.472555+0000', 'last_peered': '2024-09-11T01:18:54.472555+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:18:54.472555+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "43'4200", 'ondisk_log_start': "43'4200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24248320, 'num_objects': 5920, 'num_object_clones': 0, 'num_object_copies': 11840, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5920, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12104, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1896}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:18:58.682 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:18:58.682 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'14174", 'reported_seq': 14309, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:51.761977+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:18:51.761977+0000', 'last_peered': '2024-09-11T01:18:51.761977+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:18:51.761977+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "43'4100", 'ondisk_log_start': "43'4100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24944640, 'num_objects': 6090, 'num_object_clones': 0, 'num_object_copies': 12180, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 6090, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10911, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2140}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:58.682 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:18:58.682 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'14217", 'reported_seq': 14356, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:51.756092+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:18:51.756092+0000', 'last_peered': '2024-09-11T01:18:51.756092+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:18:51.756092+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "43'4200", 'ondisk_log_start': "43'4200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24408064, 'num_objects': 5959, 'num_object_clones': 0, 'num_object_copies': 11918, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5959, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10826, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1920}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:58.682 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:18:58.682 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'13975", 'reported_seq': 14114, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:18:51.820619+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:18:51.820619+0000', 'last_peered': '2024-09-11T01:18:51.820619+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:18:51.820619+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "40'3900", 'ondisk_log_start': "40'3900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 24482409, 'num_objects': 5978, 'num_object_clones': 0, 'num_object_copies': 11956, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5978, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11045, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 2061}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:18:58.683 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:18:59.791 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:18:59.791 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38541/187644 objects degraded (20.539%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":476,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":93822,"data_bytes":384279188,"bytes_used":2024009728,"bytes_avail":287886282752,"bytes_total":289910292480,"degraded_objects":38541,"degraded_total":187644,"degraded_ratio":0.20539425721046237,"recovering_objects_per_sec":18,"recovering_bytes_per_sec":33094,"recovering_keys_per_sec":0,"num_objects_recovered":223,"num_bytes_recovered":397312,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":492},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:18:59.835 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:18:59.835 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:00.488 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:19:00.488 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:19:00.550 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":278,"stamp":"2024-09-11T01:18:58.706667+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":384279188,"num_objects":93822,"num_object_clones":0,"num_object_copies":187644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93822,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":188469,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9099,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160820,"ondisk_log_size":160820,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1976572,"kb_used_data":944584,"kb_used_omap":12,"kb_used_meta":1031923,"kb_avail":281138948,"statfs":{"total":289910292480,"available":287886282752,"internally_reserved":0,"allocated":967254016,"data_stored":966388787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1056689156},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,5,6,9,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24236032,"num_objects":-5917,"num_object_clones":0,"num_object_copies":-11834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-100,"num_objects_misplaced":-123,"num_objects_unfound":0,"num_objects_dirty":-5917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5917,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":223,"num_bytes_recovered":397312,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005292"},"pg_stats":[{"pgid":"2.f","version":"47'14443","reported_seq":14526,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:56.852606+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:18:56.852606+0000","last_peered":"2024-09-11T01:18:56.852606+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:18:56.852606+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23851008,"num_objects":5823,"num_object_clones":0,"num_object_copies":11646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14443,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1751}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14423","reported_seq":14530,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:54.604760+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:18:54.604760+0000","last_peered":"2024-09-11T01:18:54.604760+0000","last_clean":"2024-09-11T01:18:54.604760+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:18:54.604760+0000","last_undegraded":"2024-09-11T01:18:54.604760+0000","last_fullsized":"2024-09-11T01:18:54.604760+0000","mapping_epoch":35,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24285184,"num_objects":5929,"num_object_clones":0,"num_object_copies":11858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12662,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14134","reported_seq":14266,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:54.526136+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:18:54.526136+0000","last_peered":"2024-09-11T01:18:54.526136+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:18:54.526136+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24612888,"num_objects":6010,"num_object_clones":0,"num_object_copies":12020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6010,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12435,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":2114}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'14165","reported_seq":14238,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:54.556682+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:18:54.556682+0000","last_peered":"2024-09-11T01:18:54.556682+0000","last_clean":"2024-09-11T01:18:54.556682+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:18:54.556682+0000","last_undegraded":"2024-09-11T01:18:54.556682+0000","last_fullsized":"2024-09-11T01:18:54.556682+0000","mapping_epoch":35,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24449024,"num_objects":5969,"num_object_clones":0,"num_object_copies":11938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5969,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14165,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14209","reported_seq":14342,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:56.832486+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:18:56.832486+0000","last_peered":"2024-09-11T01:18:56.832486+0000","last_clean":"2024-09-11T01:18:56.832486+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:18:56.832486+0000","last_undegraded":"2024-09-11T01:18:56.832486+0000","last_fullsized":"2024-09-11T01:18:56.832486+0000","mapping_epoch":34,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23756800,"num_objects":5800,"num_object_clones":0,"num_object_copies":11600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5800,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11214,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14309","reported_seq":14482,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:56.791079+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:18:56.791079+0000","last_peered":"2024-09-11T01:18:56.791079+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:18:56.791079+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24240128,"num_objects":5918,"num_object_clones":0,"num_object_copies":11836,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5918,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10650,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1967}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'14179","reported_seq":14314,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:56.830294+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:18:56.830294+0000","last_peered":"2024-09-11T01:18:56.830294+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:18:56.830294+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23556096,"num_objects":5751,"num_object_clones":0,"num_object_copies":11502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5751,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11036,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1765}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14378","reported_seq":14511,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:56.814454+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:18:56.814454+0000","last_peered":"2024-09-11T01:18:56.814454+0000","last_clean":"2024-09-11T01:18:56.814454+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:18:56.814454+0000","last_undegraded":"2024-09-11T01:18:56.814454+0000","last_fullsized":"2024-09-11T01:18:56.814454+0000","mapping_epoch":34,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24068096,"num_objects":5876,"num_object_clones":0,"num_object_copies":11752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5876,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11254,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934359+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934359+0000","last_peered":"2024-09-11T01:17:58.934359+0000","last_clean":"2024-09-11T01:17:58.934359+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934359+0000","last_undegraded":"2024-09-11T01:17:58.934359+0000","last_fullsized":"2024-09-11T01:17:58.934359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14091","reported_seq":16681,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:54.594306+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:18:54.594306+0000","last_peered":"2024-09-11T01:18:54.594306+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:18:54.594306+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24092672,"num_objects":5882,"num_object_clones":0,"num_object_copies":11764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2707,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12062,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1228,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3175}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14138","reported_seq":14269,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:56.866870+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:18:56.866870+0000","last_peered":"2024-09-11T01:18:56.866870+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:18:56.866870+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23470080,"num_objects":5730,"num_object_clones":0,"num_object_copies":11460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5730,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10725,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1746}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14287","reported_seq":14411,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:54.472555+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:18:54.472555+0000","last_peered":"2024-09-11T01:18:54.472555+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:18:54.472555+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24248320,"num_objects":5920,"num_object_clones":0,"num_object_copies":11840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5920,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12104,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1896}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935080+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935080+0000","last_peered":"2024-09-11T01:17:58.935080+0000","last_clean":"2024-09-11T01:17:58.935080+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935080+0000","last_undegraded":"2024-09-11T01:17:58.935080+0000","last_fullsized":"2024-09-11T01:17:58.935080+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'13959","reported_seq":21945,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:54.583851+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:18:54.583851+0000","last_peered":"2024-09-11T01:18:54.583851+0000","last_clean":"2024-09-11T01:18:54.583851+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:18:54.583851+0000","last_undegraded":"2024-09-11T01:18:54.583851+0000","last_fullsized":"2024-09-11T01:18:54.583851+0000","mapping_epoch":44,"log_start":"40'3900","ondisk_log_start":"40'3900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23908352,"num_objects":5837,"num_object_clones":0,"num_object_copies":11674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11744,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14306","reported_seq":14443,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:56.839631+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:18:56.839631+0000","last_peered":"2024-09-11T01:18:56.839631+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:18:56.839631+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24403968,"num_objects":5958,"num_object_clones":0,"num_object_copies":11916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5958,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11043,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":2008}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938540+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938540+0000","last_peered":"2024-09-11T01:17:58.938540+0000","last_clean":"2024-09-11T01:17:58.938540+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938540+0000","last_undegraded":"2024-09-11T01:17:58.938540+0000","last_fullsized":"2024-09-11T01:17:58.938540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14099","reported_seq":22135,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:55.778360+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:18:55.778360+0000","last_peered":"2024-09-11T01:18:55.778360+0000","last_clean":"2024-09-11T01:18:55.778360+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:18:55.778360+0000","last_undegraded":"2024-09-11T01:18:55.778360+0000","last_fullsized":"2024-09-11T01:18:55.778360+0000","mapping_epoch":46,"log_start":"43'4000","ondisk_log_start":"43'4000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23695360,"num_objects":5785,"num_object_clones":0,"num_object_copies":11570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10754,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14367","reported_seq":14506,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:56.782081+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:18:56.782081+0000","last_peered":"2024-09-11T01:18:56.782081+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:18:56.782081+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23793664,"num_objects":5809,"num_object_clones":0,"num_object_copies":11618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5809,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10976,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1770}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971314+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971314+0000","last_peered":"2024-09-11T01:17:58.971314+0000","last_clean":"2024-09-11T01:17:58.971314+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971314+0000","last_undegraded":"2024-09-11T01:17:58.971314+0000","last_fullsized":"2024-09-11T01:17:58.971314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14130","reported_seq":14268,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:56.844995+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:18:56.844995+0000","last_peered":"2024-09-11T01:18:56.844995+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:18:56.844995+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23847529,"num_objects":5823,"num_object_clones":0,"num_object_copies":11646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11200,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1906}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":384279169,"num_objects":93820,"num_object_clones":0,"num_object_copies":187640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38541,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93820,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":188467,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9098,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":963866624,"data_stored":963795858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160817,"ondisk_log_size":160817,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182489,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":726456,"kb_used_data":343592,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93645384,"statfs":{"total":96636764160,"available":95892873216,"internally_reserved":0,"allocated":351838208,"data_stored":351568549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":392031915},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,3,4,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986261,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":646660,"kb_used_data":317940,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93725180,"statfs":{"total":96636764160,"available":95974584320,"internally_reserved":0,"allocated":325570560,"data_stored":325300906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336588458},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659206,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":603456,"kb_used_data":283052,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93768384,"statfs":{"total":96636764160,"available":96018825216,"internally_reserved":0,"allocated":289845248,"data_stored":289519332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":328068783},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,6,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":351358976,"data_stored":351355497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":288092160,"data_stored":288032424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":324415488,"data_stored":324407937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:03.553 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:04.498 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:19:04.499 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:19:04.551 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":280,"stamp":"2024-09-11T01:19:02.707572+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":374858388,"num_objects":91522,"num_object_clones":0,"num_object_copies":183044,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38404,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91522,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":190769,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9236,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160720,"ondisk_log_size":160720,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1945668,"kb_used_data":926164,"kb_used_omap":12,"kb_used_meta":1019443,"kb_avail":281169852,"statfs":{"total":289910292480,"available":287917928448,"internally_reserved":0,"allocated":948391936,"data_stored":947526707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1043909636},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,8,1,9,12,3,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24952832,"num_objects":-6092,"num_object_clones":0,"num_object_copies":-12184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-243,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6092,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6092,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":243,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005274"},"pg_stats":[{"pgid":"2.f","version":"47'14574","reported_seq":14656,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.829249+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:19:01.829249+0000","last_peered":"2024-09-11T01:19:01.829249+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:19:01.829249+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23314432,"num_objects":5692,"num_object_clones":0,"num_object_copies":11384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14574,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1620}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14570","reported_seq":14677,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:59.600952+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:18:59.600952+0000","last_peered":"2024-09-11T01:18:59.600952+0000","last_clean":"2024-09-11T01:18:59.600952+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:18:59.600952+0000","last_undegraded":"2024-09-11T01:18:59.600952+0000","last_fullsized":"2024-09-11T01:18:59.600952+0000","mapping_epoch":35,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23683072,"num_objects":5782,"num_object_clones":0,"num_object_copies":11564,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5782,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12809,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14279","reported_seq":14412,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:59.559228+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:18:59.559228+0000","last_peered":"2024-09-11T01:18:59.559228+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:18:59.559228+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":24018968,"num_objects":5865,"num_object_clones":0,"num_object_copies":11730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5865,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12580,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1969}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'14306","reported_seq":14381,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:59.556248+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:18:59.556248+0000","last_peered":"2024-09-11T01:18:59.556248+0000","last_clean":"2024-09-11T01:18:59.556248+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:18:59.556248+0000","last_undegraded":"2024-09-11T01:18:59.556248+0000","last_fullsized":"2024-09-11T01:18:59.556248+0000","mapping_epoch":35,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23871488,"num_objects":5828,"num_object_clones":0,"num_object_copies":11656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5828,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14306,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14347","reported_seq":14480,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:01.842809+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:19:01.842809+0000","last_peered":"2024-09-11T01:19:01.842809+0000","last_clean":"2024-09-11T01:19:01.842809+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:19:01.842809+0000","last_undegraded":"2024-09-11T01:19:01.842809+0000","last_fullsized":"2024-09-11T01:19:01.842809+0000","mapping_epoch":34,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23191552,"num_objects":5662,"num_object_clones":0,"num_object_copies":11324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11352,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14440","reported_seq":14614,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.843511+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:19:01.843511+0000","last_peered":"2024-09-11T01:19:01.843511+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:19:01.843511+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23703552,"num_objects":5787,"num_object_clones":0,"num_object_copies":11574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5787,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10781,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1836}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'14326","reported_seq":14461,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.924495+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:19:01.924495+0000","last_peered":"2024-09-11T01:19:01.924495+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:19:01.924495+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22953984,"num_objects":5604,"num_object_clones":0,"num_object_copies":11208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11183,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1618}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14531","reported_seq":14666,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:01.931095+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:19:01.931095+0000","last_peered":"2024-09-11T01:19:01.931095+0000","last_clean":"2024-09-11T01:19:01.931095+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:19:01.931095+0000","last_undegraded":"2024-09-11T01:19:01.931095+0000","last_fullsized":"2024-09-11T01:19:01.931095+0000","mapping_epoch":34,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23441408,"num_objects":5723,"num_object_clones":0,"num_object_copies":11446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11407,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934359+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934359+0000","last_peered":"2024-09-11T01:17:58.934359+0000","last_clean":"2024-09-11T01:17:58.934359+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934359+0000","last_undegraded":"2024-09-11T01:17:58.934359+0000","last_fullsized":"2024-09-11T01:17:58.934359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14228","reported_seq":17091,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:59.600864+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:18:59.600864+0000","last_peered":"2024-09-11T01:18:59.600864+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:18:59.600864+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23531520,"num_objects":5745,"num_object_clones":0,"num_object_copies":11490,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2570,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5745,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12199,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1365,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3175}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14284","reported_seq":14417,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.847907+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:19:01.847907+0000","last_peered":"2024-09-11T01:19:01.847907+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:19:01.847907+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22872064,"num_objects":5584,"num_object_clones":0,"num_object_copies":11168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10871,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1600}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14415","reported_seq":14539,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:18:59.474098+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:18:59.474098+0000","last_peered":"2024-09-11T01:18:59.474098+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:18:59.474098+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23724032,"num_objects":5792,"num_object_clones":0,"num_object_copies":11584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12232,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1768}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935080+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935080+0000","last_peered":"2024-09-11T01:17:58.935080+0000","last_clean":"2024-09-11T01:17:58.935080+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935080+0000","last_undegraded":"2024-09-11T01:17:58.935080+0000","last_fullsized":"2024-09-11T01:17:58.935080+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14109","reported_seq":22095,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:59.405259+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:18:59.405259+0000","last_peered":"2024-09-11T01:18:59.405259+0000","last_clean":"2024-09-11T01:18:59.405259+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:18:59.405259+0000","last_undegraded":"2024-09-11T01:18:59.405259+0000","last_fullsized":"2024-09-11T01:18:59.405259+0000","mapping_epoch":44,"log_start":"43'4100","ondisk_log_start":"43'4100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23293952,"num_objects":5687,"num_object_clones":0,"num_object_copies":11374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5687,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11894,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14460","reported_seq":14593,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.879982+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:19:01.879982+0000","last_peered":"2024-09-11T01:19:01.879982+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:19:01.879982+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23773184,"num_objects":5804,"num_object_clones":0,"num_object_copies":11608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11197,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1854}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938540+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938540+0000","last_peered":"2024-09-11T01:17:58.938540+0000","last_clean":"2024-09-11T01:17:58.938540+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938540+0000","last_undegraded":"2024-09-11T01:17:58.938540+0000","last_fullsized":"2024-09-11T01:17:58.938540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14248","reported_seq":22284,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:00.820980+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:19:00.820980+0000","last_peered":"2024-09-11T01:19:00.820980+0000","last_clean":"2024-09-11T01:19:00.820980+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:19:00.820980+0000","last_undegraded":"2024-09-11T01:19:00.820980+0000","last_fullsized":"2024-09-11T01:19:00.820980+0000","mapping_epoch":46,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23085056,"num_objects":5636,"num_object_clones":0,"num_object_copies":11272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10903,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14512","reported_seq":14652,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.876521+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:19:01.876521+0000","last_peered":"2024-09-11T01:19:01.876521+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:19:01.876521+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23199744,"num_objects":5664,"num_object_clones":0,"num_object_copies":11328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11121,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1625}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971314+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971314+0000","last_peered":"2024-09-11T01:17:58.971314+0000","last_clean":"2024-09-11T01:17:58.971314+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971314+0000","last_undegraded":"2024-09-11T01:17:58.971314+0000","last_fullsized":"2024-09-11T01:17:58.971314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14288","reported_seq":14427,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.880130+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:19:01.880130+0000","last_peered":"2024-09-11T01:19:01.880130+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:19:01.880130+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23200361,"num_objects":5665,"num_object_clones":0,"num_object_copies":11330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11358,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1748}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":374858369,"num_objects":91520,"num_object_clones":0,"num_object_copies":183040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38404,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91520,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":190767,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9235,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":945049600,"data_stored":944978834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160717,"ondisk_log_size":160717,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182490,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":711220,"kb_used_data":336676,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93660620,"statfs":{"total":96636764160,"available":95908474880,"internally_reserved":0,"allocated":344756224,"data_stored":344486565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":383512235},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,0,2,7,3,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986262,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":629528,"kb_used_data":313288,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93742312,"statfs":{"total":96636764160,"available":95992127488,"internally_reserved":0,"allocated":320806912,"data_stored":320537258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323808938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,4,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659207,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":604920,"kb_used_data":276200,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93766920,"statfs":{"total":96636764160,"available":96017326080,"internally_reserved":0,"allocated":282828800,"data_stored":282502884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":336588463},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,5,0,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":344248320,"data_stored":344244841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":281108480,"data_stored":281048744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":319692800,"data_stored":319685249,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:04.553 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:05.333 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:19:05.336 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:19:05.422 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":281,"stamp":"2024-09-11T01:19:04.708040+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":371290772,"num_objects":90651,"num_object_clones":0,"num_object_copies":181302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38270,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90651,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":191640,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9370,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160791,"ondisk_log_size":160791,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1945236,"kb_used_data":921572,"kb_used_omap":12,"kb_used_meta":1023603,"kb_avail":281170284,"statfs":{"total":289910292480,"available":287918370816,"internally_reserved":0,"allocated":943689728,"data_stored":942824499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1048169476},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,8,5,6,9,3,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22511616,"num_objects":-5496,"num_object_clones":0,"num_object_copies":-10992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-377,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5496,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":377,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003640"},"pg_stats":[{"pgid":"2.f","version":"47'14574","reported_seq":14656,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.829249+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:19:01.829249+0000","last_peered":"2024-09-11T01:19:01.829249+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:19:01.829249+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23314432,"num_objects":5692,"num_object_clones":0,"num_object_copies":11384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14574,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1620}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14724","reported_seq":14831,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:04.562780+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:19:04.562780+0000","last_peered":"2024-09-11T01:19:04.562780+0000","last_clean":"2024-09-11T01:19:04.562780+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:19:04.562780+0000","last_undegraded":"2024-09-11T01:19:04.562780+0000","last_fullsized":"2024-09-11T01:19:04.562780+0000","mapping_epoch":35,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23052288,"num_objects":5628,"num_object_clones":0,"num_object_copies":11256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12963,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14428","reported_seq":14560,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:04.628749+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:19:04.628749+0000","last_peered":"2024-09-11T01:19:04.628749+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:19:04.628749+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23408664,"num_objects":5716,"num_object_clones":0,"num_object_copies":11432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5716,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12729,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1820}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'14446","reported_seq":14519,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:04.596546+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:19:04.596546+0000","last_peered":"2024-09-11T01:19:04.596546+0000","last_clean":"2024-09-11T01:19:04.596546+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:19:04.596546+0000","last_undegraded":"2024-09-11T01:19:04.596546+0000","last_fullsized":"2024-09-11T01:19:04.596546+0000","mapping_epoch":35,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23298048,"num_objects":5688,"num_object_clones":0,"num_object_copies":11376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14446,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14347","reported_seq":14480,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:01.842809+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:19:01.842809+0000","last_peered":"2024-09-11T01:19:01.842809+0000","last_clean":"2024-09-11T01:19:01.842809+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:19:01.842809+0000","last_undegraded":"2024-09-11T01:19:01.842809+0000","last_fullsized":"2024-09-11T01:19:01.842809+0000","mapping_epoch":34,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23191552,"num_objects":5662,"num_object_clones":0,"num_object_copies":11324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11352,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14440","reported_seq":14614,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.843511+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:19:01.843511+0000","last_peered":"2024-09-11T01:19:01.843511+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:19:01.843511+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23703552,"num_objects":5787,"num_object_clones":0,"num_object_copies":11574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5787,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10781,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1836}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'14326","reported_seq":14461,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.924495+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:19:01.924495+0000","last_peered":"2024-09-11T01:19:01.924495+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:19:01.924495+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22953984,"num_objects":5604,"num_object_clones":0,"num_object_copies":11208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11183,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1618}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14531","reported_seq":14666,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:01.931095+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:19:01.931095+0000","last_peered":"2024-09-11T01:19:01.931095+0000","last_clean":"2024-09-11T01:19:01.931095+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:19:01.931095+0000","last_undegraded":"2024-09-11T01:19:01.931095+0000","last_fullsized":"2024-09-11T01:19:01.931095+0000","mapping_epoch":34,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23441408,"num_objects":5723,"num_object_clones":0,"num_object_copies":11446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11407,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934359+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934359+0000","last_peered":"2024-09-11T01:17:58.934359+0000","last_clean":"2024-09-11T01:17:58.934359+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934359+0000","last_undegraded":"2024-09-11T01:17:58.934359+0000","last_fullsized":"2024-09-11T01:17:58.934359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14368","reported_seq":17500,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:04.600627+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:19:04.600627+0000","last_peered":"2024-09-11T01:19:04.600627+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:19:04.600627+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22958080,"num_objects":5605,"num_object_clones":0,"num_object_copies":11210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2436,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5605,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12339,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1499,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3169}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14284","reported_seq":14417,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.847907+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:19:01.847907+0000","last_peered":"2024-09-11T01:19:01.847907+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:19:01.847907+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22872064,"num_objects":5584,"num_object_clones":0,"num_object_copies":11168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10871,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1600}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14556","reported_seq":14680,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:04.560739+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:19:04.560739+0000","last_peered":"2024-09-11T01:19:04.560739+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:19:04.560739+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23146496,"num_objects":5651,"num_object_clones":0,"num_object_copies":11302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12373,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1627}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935080+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935080+0000","last_peered":"2024-09-11T01:17:58.935080+0000","last_clean":"2024-09-11T01:17:58.935080+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935080+0000","last_undegraded":"2024-09-11T01:17:58.935080+0000","last_fullsized":"2024-09-11T01:17:58.935080+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14256","reported_seq":22242,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:04.600985+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:19:04.600985+0000","last_peered":"2024-09-11T01:19:04.600985+0000","last_clean":"2024-09-11T01:19:04.600985+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:19:04.600985+0000","last_undegraded":"2024-09-11T01:19:04.600985+0000","last_fullsized":"2024-09-11T01:19:04.600985+0000","mapping_epoch":44,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22691840,"num_objects":5540,"num_object_clones":0,"num_object_copies":11080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12041,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14460","reported_seq":14593,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.879982+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:19:01.879982+0000","last_peered":"2024-09-11T01:19:01.879982+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:19:01.879982+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23773184,"num_objects":5804,"num_object_clones":0,"num_object_copies":11608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11197,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1854}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938540+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938540+0000","last_peered":"2024-09-11T01:17:58.938540+0000","last_clean":"2024-09-11T01:17:58.938540+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938540+0000","last_undegraded":"2024-09-11T01:17:58.938540+0000","last_fullsized":"2024-09-11T01:17:58.938540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14248","reported_seq":22284,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:00.820980+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:19:00.820980+0000","last_peered":"2024-09-11T01:19:00.820980+0000","last_clean":"2024-09-11T01:19:00.820980+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:19:00.820980+0000","last_undegraded":"2024-09-11T01:19:00.820980+0000","last_fullsized":"2024-09-11T01:19:00.820980+0000","mapping_epoch":46,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23085056,"num_objects":5636,"num_object_clones":0,"num_object_copies":11272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10903,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14512","reported_seq":14652,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.876521+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:19:01.876521+0000","last_peered":"2024-09-11T01:19:01.876521+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:19:01.876521+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23199744,"num_objects":5664,"num_object_clones":0,"num_object_copies":11328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11121,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1625}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971314+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971314+0000","last_peered":"2024-09-11T01:17:58.971314+0000","last_clean":"2024-09-11T01:17:58.971314+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971314+0000","last_undegraded":"2024-09-11T01:17:58.971314+0000","last_fullsized":"2024-09-11T01:17:58.971314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14288","reported_seq":14427,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.880130+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:19:01.880130+0000","last_peered":"2024-09-11T01:19:01.880130+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:19:01.880130+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23200361,"num_objects":5665,"num_object_clones":0,"num_object_copies":11330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11358,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1748}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":371290753,"num_objects":90649,"num_object_clones":0,"num_object_copies":181298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38270,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90649,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":191638,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9369,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":940298240,"data_stored":940227474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160788,"ondisk_log_size":160788,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182490,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":711220,"kb_used_data":336676,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93660620,"statfs":{"total":96636764160,"available":95908474880,"internally_reserved":0,"allocated":344756224,"data_stored":344486565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":383512235},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,0,2,7,3,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986263,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":629096,"kb_used_data":308696,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93742744,"statfs":{"total":96636764160,"available":95992569856,"internally_reserved":0,"allocated":316104704,"data_stored":315835050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328068778},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,5,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659207,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":604920,"kb_used_data":276200,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93766920,"statfs":{"total":96636764160,"available":96017326080,"internally_reserved":0,"allocated":282828800,"data_stored":282502884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":336588463},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,5,0,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":344248320,"data_stored":344244841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":281108480,"data_stored":281048744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":314941440,"data_stored":314933889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:05.423 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:19:05.423 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'14574", 'reported_seq': 14656, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:01.829249+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:19:01.829249+0000', 'last_peered': '2024-09-11T01:19:01.829249+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:19:01.829249+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "43'4500", 'ondisk_log_start': "43'4500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23314432, 'num_objects': 5692, 'num_object_clones': 0, 'num_object_copies': 11384, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5692, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14574, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1620}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:05.424 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:19:05.424 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'14428", 'reported_seq': 14560, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:04.628749+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:19:04.628749+0000', 'last_peered': '2024-09-11T01:19:04.628749+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:19:04.628749+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "43'4400", 'ondisk_log_start': "43'4400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10028, 'log_dups_size': 0, 'ondisk_log_size': 10028, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23408664, 'num_objects': 5716, 'num_object_clones': 0, 'num_object_copies': 11432, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5716, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12729, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1820}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:05.424 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:19:05.424 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'14440", 'reported_seq': 14614, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:01.843511+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:19:01.843511+0000', 'last_peered': '2024-09-11T01:19:01.843511+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:19:01.843511+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "43'4400", 'ondisk_log_start': "43'4400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23703552, 'num_objects': 5787, 'num_object_clones': 0, 'num_object_copies': 11574, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5787, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10781, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1836}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:05.424 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:19:05.424 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'14326", 'reported_seq': 14461, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:01.924495+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:19:01.924495+0000', 'last_peered': '2024-09-11T01:19:01.924495+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:19:01.924495+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "43'4300", 'ondisk_log_start': "43'4300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22953984, 'num_objects': 5604, 'num_object_clones': 0, 'num_object_copies': 11208, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5604, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11183, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1618}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:05.424 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:19:05.424 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'14368", 'reported_seq': 17500, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:04.600627+0000', 'last_change': '2024-09-11T01:17:57.355893+0000', 'last_active': '2024-09-11T01:19:04.600627+0000', 'last_peered': '2024-09-11T01:19:04.600627+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:19:04.600627+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "43'4300", 'ondisk_log_start': "43'4300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22958080, 'num_objects': 5605, 'num_object_clones': 0, 'num_object_copies': 11210, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2436, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5605, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12339, 'num_write_kb': 31844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1499, 'num_bytes_recovered': 4505600, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3169}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:05.424 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:19:05.425 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'14284", 'reported_seq': 14417, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:01.847907+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:19:01.847907+0000', 'last_peered': '2024-09-11T01:19:01.847907+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:19:01.847907+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "43'4200", 'ondisk_log_start': "43'4200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22872064, 'num_objects': 5584, 'num_object_clones': 0, 'num_object_copies': 11168, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5584, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10871, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1600}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:05.425 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:19:05.425 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'14556", 'reported_seq': 14680, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:04.560739+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:19:04.560739+0000', 'last_peered': '2024-09-11T01:19:04.560739+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:19:04.560739+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "43'4500", 'ondisk_log_start': "43'4500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23146496, 'num_objects': 5651, 'num_object_clones': 0, 'num_object_copies': 11302, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5651, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12373, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1627}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:05.425 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:19:05.425 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'14460", 'reported_seq': 14593, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:01.879982+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:19:01.879982+0000', 'last_peered': '2024-09-11T01:19:01.879982+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:19:01.879982+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "43'4400", 'ondisk_log_start': "43'4400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23773184, 'num_objects': 5804, 'num_object_clones': 0, 'num_object_copies': 11608, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5804, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11197, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1854}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:05.425 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:19:05.425 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'14512", 'reported_seq': 14652, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:01.876521+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:19:01.876521+0000', 'last_peered': '2024-09-11T01:19:01.876521+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:19:01.876521+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "43'4500", 'ondisk_log_start': "43'4500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23199744, 'num_objects': 5664, 'num_object_clones': 0, 'num_object_copies': 11328, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5664, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11121, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1625}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:05.425 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:19:05.425 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'14288", 'reported_seq': 14427, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:01.880130+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:19:01.880130+0000', 'last_peered': '2024-09-11T01:19:01.880130+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:19:01.880130+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "43'4200", 'ondisk_log_start': "43'4200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23200361, 'num_objects': 5665, 'num_object_clones': 0, 'num_object_copies': 11330, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5665, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11358, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1748}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:05.426 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:19:06.349 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:19:06.349 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38270/181302 objects degraded (21.108%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":482,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":90651,"data_bytes":371290772,"bytes_used":1991921664,"bytes_avail":287918370816,"bytes_total":289910292480,"degraded_objects":38270,"degraded_total":181302,"degraded_ratio":0.21108426823752635,"recovering_objects_per_sec":31,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":377,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":457},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:19:06.382 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:19:06.382 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:07.144 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:19:07.145 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:19:07.204 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":282,"stamp":"2024-09-11T01:19:06.708490+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":370631316,"num_objects":90490,"num_object_clones":0,"num_object_copies":180980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38270,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90490,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":191801,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9370,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160752,"ondisk_log_size":160752,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1929900,"kb_used_data":914556,"kb_used_omap":12,"kb_used_meta":1015283,"kb_avail":281185620,"statfs":{"total":289910292480,"available":287934074880,"internally_reserved":0,"allocated":936505344,"data_stored":935640115,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1039649796},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,5,7,7,11,4,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19804160,"num_objects":-4835,"num_object_clones":0,"num_object_copies":-9670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4835,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003608"},"pg_stats":[{"pgid":"2.f","version":"47'14574","reported_seq":14656,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.829249+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:19:01.829249+0000","last_peered":"2024-09-11T01:19:01.829249+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:19:01.829249+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23314432,"num_objects":5692,"num_object_clones":0,"num_object_copies":11384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14574,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1620}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14724","reported_seq":14831,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:04.562780+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:19:04.562780+0000","last_peered":"2024-09-11T01:19:04.562780+0000","last_clean":"2024-09-11T01:19:04.562780+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:19:04.562780+0000","last_undegraded":"2024-09-11T01:19:04.562780+0000","last_fullsized":"2024-09-11T01:19:04.562780+0000","mapping_epoch":35,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23052288,"num_objects":5628,"num_object_clones":0,"num_object_copies":11256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12963,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14428","reported_seq":14560,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:04.628749+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:19:04.628749+0000","last_peered":"2024-09-11T01:19:04.628749+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:19:04.628749+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23408664,"num_objects":5716,"num_object_clones":0,"num_object_copies":11432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5716,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12729,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1820}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'14446","reported_seq":14519,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:04.596546+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:19:04.596546+0000","last_peered":"2024-09-11T01:19:04.596546+0000","last_clean":"2024-09-11T01:19:04.596546+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:19:04.596546+0000","last_undegraded":"2024-09-11T01:19:04.596546+0000","last_fullsized":"2024-09-11T01:19:04.596546+0000","mapping_epoch":35,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23298048,"num_objects":5688,"num_object_clones":0,"num_object_copies":11376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14446,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14347","reported_seq":14480,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:01.842809+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:19:01.842809+0000","last_peered":"2024-09-11T01:19:01.842809+0000","last_clean":"2024-09-11T01:19:01.842809+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:19:01.842809+0000","last_undegraded":"2024-09-11T01:19:01.842809+0000","last_fullsized":"2024-09-11T01:19:01.842809+0000","mapping_epoch":34,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23191552,"num_objects":5662,"num_object_clones":0,"num_object_copies":11324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11352,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14440","reported_seq":14614,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.843511+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:19:01.843511+0000","last_peered":"2024-09-11T01:19:01.843511+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:19:01.843511+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23703552,"num_objects":5787,"num_object_clones":0,"num_object_copies":11574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5787,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10781,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1836}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'14326","reported_seq":14461,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.924495+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:19:01.924495+0000","last_peered":"2024-09-11T01:19:01.924495+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:19:01.924495+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22953984,"num_objects":5604,"num_object_clones":0,"num_object_copies":11208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5604,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11183,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1618}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14531","reported_seq":14666,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:01.931095+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:19:01.931095+0000","last_peered":"2024-09-11T01:19:01.931095+0000","last_clean":"2024-09-11T01:19:01.931095+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:19:01.931095+0000","last_undegraded":"2024-09-11T01:19:01.931095+0000","last_fullsized":"2024-09-11T01:19:01.931095+0000","mapping_epoch":34,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23441408,"num_objects":5723,"num_object_clones":0,"num_object_copies":11446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5723,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11407,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934359+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934359+0000","last_peered":"2024-09-11T01:17:58.934359+0000","last_clean":"2024-09-11T01:17:58.934359+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934359+0000","last_undegraded":"2024-09-11T01:17:58.934359+0000","last_fullsized":"2024-09-11T01:17:58.934359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14368","reported_seq":17500,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:04.600627+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:19:04.600627+0000","last_peered":"2024-09-11T01:19:04.600627+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:19:04.600627+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22958080,"num_objects":5605,"num_object_clones":0,"num_object_copies":11210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2436,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5605,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12339,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1499,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3169}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14284","reported_seq":14417,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.847907+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:19:01.847907+0000","last_peered":"2024-09-11T01:19:01.847907+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:19:01.847907+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22872064,"num_objects":5584,"num_object_clones":0,"num_object_copies":11168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10871,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1600}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14556","reported_seq":14680,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:04.560739+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:19:04.560739+0000","last_peered":"2024-09-11T01:19:04.560739+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:19:04.560739+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23146496,"num_objects":5651,"num_object_clones":0,"num_object_copies":11302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12373,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1627}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935080+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935080+0000","last_peered":"2024-09-11T01:17:58.935080+0000","last_clean":"2024-09-11T01:17:58.935080+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935080+0000","last_undegraded":"2024-09-11T01:17:58.935080+0000","last_fullsized":"2024-09-11T01:17:58.935080+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14256","reported_seq":22242,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:04.600985+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:19:04.600985+0000","last_peered":"2024-09-11T01:19:04.600985+0000","last_clean":"2024-09-11T01:19:04.600985+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:19:04.600985+0000","last_undegraded":"2024-09-11T01:19:04.600985+0000","last_fullsized":"2024-09-11T01:19:04.600985+0000","mapping_epoch":44,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22691840,"num_objects":5540,"num_object_clones":0,"num_object_copies":11080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12041,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14460","reported_seq":14593,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.879982+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:19:01.879982+0000","last_peered":"2024-09-11T01:19:01.879982+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:19:01.879982+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23773184,"num_objects":5804,"num_object_clones":0,"num_object_copies":11608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11197,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1854}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938540+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938540+0000","last_peered":"2024-09-11T01:17:58.938540+0000","last_clean":"2024-09-11T01:17:58.938540+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938540+0000","last_undegraded":"2024-09-11T01:17:58.938540+0000","last_fullsized":"2024-09-11T01:17:58.938540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14409","reported_seq":22445,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:05.830486+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:19:05.830486+0000","last_peered":"2024-09-11T01:19:05.830486+0000","last_clean":"2024-09-11T01:19:05.830486+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:19:05.830486+0000","last_undegraded":"2024-09-11T01:19:05.830486+0000","last_fullsized":"2024-09-11T01:19:05.830486+0000","mapping_epoch":46,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22425600,"num_objects":5475,"num_object_clones":0,"num_object_copies":10950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11064,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14512","reported_seq":14652,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.876521+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:19:01.876521+0000","last_peered":"2024-09-11T01:19:01.876521+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:19:01.876521+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23199744,"num_objects":5664,"num_object_clones":0,"num_object_copies":11328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11121,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1625}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971314+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971314+0000","last_peered":"2024-09-11T01:17:58.971314+0000","last_clean":"2024-09-11T01:17:58.971314+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971314+0000","last_undegraded":"2024-09-11T01:17:58.971314+0000","last_fullsized":"2024-09-11T01:17:58.971314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14288","reported_seq":14427,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:01.880130+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:19:01.880130+0000","last_peered":"2024-09-11T01:19:01.880130+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:19:01.880130+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'4200","ondisk_log_start":"43'4200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23200361,"num_objects":5665,"num_object_clones":0,"num_object_copies":11330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5665,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11358,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1748}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":370631297,"num_objects":90488,"num_object_clones":0,"num_object_copies":180976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38270,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90488,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":191799,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9369,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":933175296,"data_stored":933104530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160749,"ondisk_log_size":160749,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182490,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":711220,"kb_used_data":336676,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93660620,"statfs":{"total":96636764160,"available":95908474880,"internally_reserved":0,"allocated":344756224,"data_stored":344486565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":383512235},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,0,2,7,3,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986263,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":629096,"kb_used_data":308696,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93742744,"statfs":{"total":96636764160,"available":95992569856,"internally_reserved":0,"allocated":316104704,"data_stored":315835050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328068778},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,5,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659208,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589584,"kb_used_data":269184,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93782256,"statfs":{"total":96636764160,"available":96033030144,"internally_reserved":0,"allocated":275644416,"data_stored":275318500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":328068783},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":344248320,"data_stored":344244841,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":273985536,"data_stored":273925800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":314941440,"data_stored":314933889,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:10.206 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:11.094 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:19:11.094 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:19:11.152 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":284,"stamp":"2024-09-11T01:19:10.709367+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":361935508,"num_objects":88367,"num_object_clones":0,"num_object_copies":176734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88367,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":193924,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9483,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160575,"ondisk_log_size":160575,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1946072,"kb_used_data":904360,"kb_used_omap":12,"kb_used_meta":1041651,"kb_avail":281169448,"statfs":{"total":289910292480,"available":287917514752,"internally_reserved":0,"allocated":926064640,"data_stored":925199411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1066650628},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,8,12,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22343680,"num_objects":-5455,"num_object_clones":0,"num_object_copies":-10910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-384,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5455,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":384,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002700"},"pg_stats":[{"pgid":"2.f","version":"47'14714","reported_seq":14797,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:06.846829+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:19:06.846829+0000","last_peered":"2024-09-11T01:19:06.846829+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:19:06.846829+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22740992,"num_objects":5552,"num_object_clones":0,"num_object_copies":11104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14714,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1480}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14874","reported_seq":14981,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:09.622058+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:19:09.622058+0000","last_peered":"2024-09-11T01:19:09.622058+0000","last_clean":"2024-09-11T01:19:09.622058+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:19:09.622058+0000","last_undegraded":"2024-09-11T01:19:09.622058+0000","last_fullsized":"2024-09-11T01:19:09.622058+0000","mapping_epoch":35,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22437888,"num_objects":5478,"num_object_clones":0,"num_object_copies":10956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13113,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14601","reported_seq":14735,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:09.581840+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:19:09.581840+0000","last_peered":"2024-09-11T01:19:09.581840+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:19:09.581840+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22700056,"num_objects":5543,"num_object_clones":0,"num_object_copies":11086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5543,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12902,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1647}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'14601","reported_seq":14676,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:09.630307+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:19:09.630307+0000","last_peered":"2024-09-11T01:19:09.630307+0000","last_clean":"2024-09-11T01:19:09.630307+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:19:09.630307+0000","last_undegraded":"2024-09-11T01:19:09.630307+0000","last_fullsized":"2024-09-11T01:19:09.630307+0000","mapping_epoch":35,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22663168,"num_objects":5533,"num_object_clones":0,"num_object_copies":11066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14601,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14485","reported_seq":14618,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:06.846715+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:19:06.846715+0000","last_peered":"2024-09-11T01:19:06.846715+0000","last_clean":"2024-09-11T01:19:06.846715+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:19:06.846715+0000","last_undegraded":"2024-09-11T01:19:06.846715+0000","last_fullsized":"2024-09-11T01:19:06.846715+0000","mapping_epoch":34,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22626304,"num_objects":5524,"num_object_clones":0,"num_object_copies":11048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11490,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14589","reported_seq":14764,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:06.918127+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:19:06.918127+0000","last_peered":"2024-09-11T01:19:06.918127+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:19:06.918127+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23093248,"num_objects":5638,"num_object_clones":0,"num_object_copies":11276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10930,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1687}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'14434","reported_seq":14569,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:06.852976+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:19:06.852976+0000","last_peered":"2024-09-11T01:19:06.852976+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:19:06.852976+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22511616,"num_objects":5496,"num_object_clones":0,"num_object_copies":10992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11291,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1510}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14679","reported_seq":14814,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:06.829532+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:19:06.829532+0000","last_peered":"2024-09-11T01:19:06.829532+0000","last_clean":"2024-09-11T01:19:06.829532+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:19:06.829532+0000","last_undegraded":"2024-09-11T01:19:06.829532+0000","last_fullsized":"2024-09-11T01:19:06.829532+0000","mapping_epoch":34,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22835200,"num_objects":5575,"num_object_clones":0,"num_object_copies":11150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11555,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934359+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934359+0000","last_peered":"2024-09-11T01:17:58.934359+0000","last_clean":"2024-09-11T01:17:58.934359+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934359+0000","last_undegraded":"2024-09-11T01:17:58.934359+0000","last_fullsized":"2024-09-11T01:17:58.934359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14533","reported_seq":17891,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:09.657728+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:19:09.657728+0000","last_peered":"2024-09-11T01:19:09.657728+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:19:09.657728+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22282240,"num_objects":5440,"num_object_clones":0,"num_object_copies":10880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12504,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1612,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3117}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14413","reported_seq":14546,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:06.879983+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:19:06.879983+0000","last_peered":"2024-09-11T01:19:06.879983+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:19:06.879983+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22343680,"num_objects":5455,"num_object_clones":0,"num_object_copies":10910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11000,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1471}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14702","reported_seq":14826,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:09.621200+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:19:09.621200+0000","last_peered":"2024-09-11T01:19:09.621200+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:19:09.621200+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22548480,"num_objects":5505,"num_object_clones":0,"num_object_copies":11010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12519,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1481}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935080+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935080+0000","last_peered":"2024-09-11T01:17:58.935080+0000","last_clean":"2024-09-11T01:17:58.935080+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935080+0000","last_undegraded":"2024-09-11T01:17:58.935080+0000","last_fullsized":"2024-09-11T01:17:58.935080+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14356","reported_seq":22342,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:09.569337+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:19:09.569337+0000","last_peered":"2024-09-11T01:19:09.569337+0000","last_clean":"2024-09-11T01:19:09.569337+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:19:09.569337+0000","last_undegraded":"2024-09-11T01:19:09.569337+0000","last_fullsized":"2024-09-11T01:19:09.569337+0000","mapping_epoch":44,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22282240,"num_objects":5440,"num_object_clones":0,"num_object_copies":10880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12141,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14606","reported_seq":14743,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:06.882224+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:19:06.882224+0000","last_peered":"2024-09-11T01:19:06.882224+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:19:06.882224+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23175168,"num_objects":5658,"num_object_clones":0,"num_object_copies":11316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11343,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1708}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938540+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938540+0000","last_peered":"2024-09-11T01:17:58.938540+0000","last_clean":"2024-09-11T01:17:58.938540+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938540+0000","last_undegraded":"2024-09-11T01:17:58.938540+0000","last_fullsized":"2024-09-11T01:17:58.938540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14409","reported_seq":22445,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:05.830486+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:19:05.830486+0000","last_peered":"2024-09-11T01:19:05.830486+0000","last_clean":"2024-09-11T01:19:05.830486+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:19:05.830486+0000","last_undegraded":"2024-09-11T01:19:05.830486+0000","last_fullsized":"2024-09-11T01:19:05.830486+0000","mapping_epoch":46,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22425600,"num_objects":5475,"num_object_clones":0,"num_object_copies":10950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11064,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14643","reported_seq":14783,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:06.802285+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:19:06.802285+0000","last_peered":"2024-09-11T01:19:06.802285+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:19:06.802285+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22663168,"num_objects":5533,"num_object_clones":0,"num_object_copies":11066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11252,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1494}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971314+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971314+0000","last_peered":"2024-09-11T01:17:58.971314+0000","last_clean":"2024-09-11T01:17:58.971314+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971314+0000","last_undegraded":"2024-09-11T01:17:58.971314+0000","last_fullsized":"2024-09-11T01:17:58.971314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14433","reported_seq":14572,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:06.928813+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:19:06.928813+0000","last_peered":"2024-09-11T01:19:06.928813+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:19:06.928813+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22606441,"num_objects":5520,"num_object_clones":0,"num_object_copies":11040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11503,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1603}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":361935489,"num_objects":88365,"num_object_clones":0,"num_object_copies":176730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88365,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":193922,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9482,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":921464832,"data_stored":921394066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160572,"ondisk_log_size":160572,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182491,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":727764,"kb_used_data":331012,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93644076,"statfs":{"total":96636764160,"available":95891533824,"internally_reserved":0,"allocated":338956288,"data_stored":338686629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":406253227},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,7,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986264,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":628724,"kb_used_data":304164,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93743116,"statfs":{"total":96636764160,"available":95992950784,"internally_reserved":0,"allocated":311463936,"data_stored":311194282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332328618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,5,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659208,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589584,"kb_used_data":269184,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93782256,"statfs":{"total":96636764160,"available":96033030144,"internally_reserved":0,"allocated":275644416,"data_stored":275318500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":328068783},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":337321984,"data_stored":337318505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":273985536,"data_stored":273925800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":310157312,"data_stored":310149761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:11.154 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:12.009 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:19:12.010 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:19:12.074 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":284,"stamp":"2024-09-11T01:19:10.709367+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":361935508,"num_objects":88367,"num_object_clones":0,"num_object_copies":176734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88367,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":193924,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9483,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160575,"ondisk_log_size":160575,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1946072,"kb_used_data":904360,"kb_used_omap":12,"kb_used_meta":1041651,"kb_avail":281169448,"statfs":{"total":289910292480,"available":287917514752,"internally_reserved":0,"allocated":926064640,"data_stored":925199411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1066650628},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,8,12,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22343680,"num_objects":-5455,"num_object_clones":0,"num_object_copies":-10910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-384,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5455,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":384,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002700"},"pg_stats":[{"pgid":"2.f","version":"47'14714","reported_seq":14797,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:06.846829+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:19:06.846829+0000","last_peered":"2024-09-11T01:19:06.846829+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:19:06.846829+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22740992,"num_objects":5552,"num_object_clones":0,"num_object_copies":11104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14714,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1480}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14874","reported_seq":14981,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:09.622058+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:19:09.622058+0000","last_peered":"2024-09-11T01:19:09.622058+0000","last_clean":"2024-09-11T01:19:09.622058+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:19:09.622058+0000","last_undegraded":"2024-09-11T01:19:09.622058+0000","last_fullsized":"2024-09-11T01:19:09.622058+0000","mapping_epoch":35,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22437888,"num_objects":5478,"num_object_clones":0,"num_object_copies":10956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13113,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14601","reported_seq":14735,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:09.581840+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:19:09.581840+0000","last_peered":"2024-09-11T01:19:09.581840+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:19:09.581840+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22700056,"num_objects":5543,"num_object_clones":0,"num_object_copies":11086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5543,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12902,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1647}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'14601","reported_seq":14676,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:09.630307+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:19:09.630307+0000","last_peered":"2024-09-11T01:19:09.630307+0000","last_clean":"2024-09-11T01:19:09.630307+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:19:09.630307+0000","last_undegraded":"2024-09-11T01:19:09.630307+0000","last_fullsized":"2024-09-11T01:19:09.630307+0000","mapping_epoch":35,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22663168,"num_objects":5533,"num_object_clones":0,"num_object_copies":11066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14601,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14485","reported_seq":14618,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:06.846715+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:19:06.846715+0000","last_peered":"2024-09-11T01:19:06.846715+0000","last_clean":"2024-09-11T01:19:06.846715+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:19:06.846715+0000","last_undegraded":"2024-09-11T01:19:06.846715+0000","last_fullsized":"2024-09-11T01:19:06.846715+0000","mapping_epoch":34,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22626304,"num_objects":5524,"num_object_clones":0,"num_object_copies":11048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11490,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14589","reported_seq":14764,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:06.918127+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:19:06.918127+0000","last_peered":"2024-09-11T01:19:06.918127+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:19:06.918127+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23093248,"num_objects":5638,"num_object_clones":0,"num_object_copies":11276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10930,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1687}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'14434","reported_seq":14569,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:06.852976+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:19:06.852976+0000","last_peered":"2024-09-11T01:19:06.852976+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:19:06.852976+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22511616,"num_objects":5496,"num_object_clones":0,"num_object_copies":10992,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5496,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11291,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1510}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14679","reported_seq":14814,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:06.829532+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:19:06.829532+0000","last_peered":"2024-09-11T01:19:06.829532+0000","last_clean":"2024-09-11T01:19:06.829532+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:19:06.829532+0000","last_undegraded":"2024-09-11T01:19:06.829532+0000","last_fullsized":"2024-09-11T01:19:06.829532+0000","mapping_epoch":34,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22835200,"num_objects":5575,"num_object_clones":0,"num_object_copies":11150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11555,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934359+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934359+0000","last_peered":"2024-09-11T01:17:58.934359+0000","last_clean":"2024-09-11T01:17:58.934359+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934359+0000","last_undegraded":"2024-09-11T01:17:58.934359+0000","last_fullsized":"2024-09-11T01:17:58.934359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14533","reported_seq":17891,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:09.657728+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:19:09.657728+0000","last_peered":"2024-09-11T01:19:09.657728+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:19:09.657728+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22282240,"num_objects":5440,"num_object_clones":0,"num_object_copies":10880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12504,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1612,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3117}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14413","reported_seq":14546,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:06.879983+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:19:06.879983+0000","last_peered":"2024-09-11T01:19:06.879983+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:19:06.879983+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22343680,"num_objects":5455,"num_object_clones":0,"num_object_copies":10910,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5455,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11000,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1471}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14702","reported_seq":14826,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:09.621200+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:19:09.621200+0000","last_peered":"2024-09-11T01:19:09.621200+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:19:09.621200+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22548480,"num_objects":5505,"num_object_clones":0,"num_object_copies":11010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12519,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1481}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935080+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935080+0000","last_peered":"2024-09-11T01:17:58.935080+0000","last_clean":"2024-09-11T01:17:58.935080+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935080+0000","last_undegraded":"2024-09-11T01:17:58.935080+0000","last_fullsized":"2024-09-11T01:17:58.935080+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14356","reported_seq":22342,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:09.569337+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:19:09.569337+0000","last_peered":"2024-09-11T01:19:09.569337+0000","last_clean":"2024-09-11T01:19:09.569337+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:19:09.569337+0000","last_undegraded":"2024-09-11T01:19:09.569337+0000","last_fullsized":"2024-09-11T01:19:09.569337+0000","mapping_epoch":44,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22282240,"num_objects":5440,"num_object_clones":0,"num_object_copies":10880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12141,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14606","reported_seq":14743,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:06.882224+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:19:06.882224+0000","last_peered":"2024-09-11T01:19:06.882224+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:19:06.882224+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":23175168,"num_objects":5658,"num_object_clones":0,"num_object_copies":11316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5658,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11343,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1708}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938540+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938540+0000","last_peered":"2024-09-11T01:17:58.938540+0000","last_clean":"2024-09-11T01:17:58.938540+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938540+0000","last_undegraded":"2024-09-11T01:17:58.938540+0000","last_fullsized":"2024-09-11T01:17:58.938540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14409","reported_seq":22445,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:05.830486+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:19:05.830486+0000","last_peered":"2024-09-11T01:19:05.830486+0000","last_clean":"2024-09-11T01:19:05.830486+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:19:05.830486+0000","last_undegraded":"2024-09-11T01:19:05.830486+0000","last_fullsized":"2024-09-11T01:19:05.830486+0000","mapping_epoch":46,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22425600,"num_objects":5475,"num_object_clones":0,"num_object_copies":10950,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5475,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11064,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14643","reported_seq":14783,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:06.802285+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:19:06.802285+0000","last_peered":"2024-09-11T01:19:06.802285+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:19:06.802285+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22663168,"num_objects":5533,"num_object_clones":0,"num_object_copies":11066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11252,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1494}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971314+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971314+0000","last_peered":"2024-09-11T01:17:58.971314+0000","last_clean":"2024-09-11T01:17:58.971314+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971314+0000","last_undegraded":"2024-09-11T01:17:58.971314+0000","last_fullsized":"2024-09-11T01:17:58.971314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14433","reported_seq":14572,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:06.928813+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:19:06.928813+0000","last_peered":"2024-09-11T01:19:06.928813+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:19:06.928813+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'4400","ondisk_log_start":"43'4400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22606441,"num_objects":5520,"num_object_clones":0,"num_object_copies":11040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11503,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1603}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":361935489,"num_objects":88365,"num_object_clones":0,"num_object_copies":176730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88365,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":193922,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9482,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":921464832,"data_stored":921394066,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160572,"ondisk_log_size":160572,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182491,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":727764,"kb_used_data":331012,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93644076,"statfs":{"total":96636764160,"available":95891533824,"internally_reserved":0,"allocated":338956288,"data_stored":338686629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":406253227},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,7,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986264,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":628724,"kb_used_data":304164,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93743116,"statfs":{"total":96636764160,"available":95992950784,"internally_reserved":0,"allocated":311463936,"data_stored":311194282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332328618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,5,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659208,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589584,"kb_used_data":269184,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93782256,"statfs":{"total":96636764160,"available":96033030144,"internally_reserved":0,"allocated":275644416,"data_stored":275318500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":328068783},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,2,4,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":337321984,"data_stored":337318505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":273985536,"data_stored":273925800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":310157312,"data_stored":310149761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:12.076 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:19:12.076 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'14714", 'reported_seq': 14797, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:06.846829+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:19:06.846829+0000', 'last_peered': '2024-09-11T01:19:06.846829+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:19:06.846829+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "43'4700", 'ondisk_log_start': "43'4700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22740992, 'num_objects': 5552, 'num_object_clones': 0, 'num_object_copies': 11104, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5552, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14714, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1480}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:12.076 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:19:12.076 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'14601", 'reported_seq': 14735, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:09.581840+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:19:09.581840+0000', 'last_peered': '2024-09-11T01:19:09.581840+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:19:09.581840+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "43'4600", 'ondisk_log_start': "43'4600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22700056, 'num_objects': 5543, 'num_object_clones': 0, 'num_object_copies': 11086, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5543, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 12902, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1647}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:12.076 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:19:12.076 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'14589", 'reported_seq': 14764, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:06.918127+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:19:06.918127+0000', 'last_peered': '2024-09-11T01:19:06.918127+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:19:06.918127+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "43'4500", 'ondisk_log_start': "43'4500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23093248, 'num_objects': 5638, 'num_object_clones': 0, 'num_object_copies': 11276, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5638, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 10930, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1687}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:12.077 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:19:12.077 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'14434", 'reported_seq': 14569, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:06.852976+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:19:06.852976+0000', 'last_peered': '2024-09-11T01:19:06.852976+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:19:06.852976+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "43'4400", 'ondisk_log_start': "43'4400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22511616, 'num_objects': 5496, 'num_object_clones': 0, 'num_object_copies': 10992, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5496, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11291, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1510}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:12.077 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:19:12.077 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'14533", 'reported_seq': 17891, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:09.657728+0000', 'last_change': '2024-09-11T01:17:57.355893+0000', 'last_active': '2024-09-11T01:19:09.657728+0000', 'last_peered': '2024-09-11T01:19:09.657728+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:19:09.657728+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "43'4500", 'ondisk_log_start': "43'4500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22282240, 'num_objects': 5440, 'num_object_clones': 0, 'num_object_copies': 10880, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2323, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5440, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12504, 'num_write_kb': 31844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1612, 'num_bytes_recovered': 4505600, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3117}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:12.077 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:19:12.077 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'14413", 'reported_seq': 14546, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:06.879983+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:19:06.879983+0000', 'last_peered': '2024-09-11T01:19:06.879983+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:19:06.879983+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "43'4400", 'ondisk_log_start': "43'4400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22343680, 'num_objects': 5455, 'num_object_clones': 0, 'num_object_copies': 10910, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5455, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11000, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1471}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:12.077 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:19:12.077 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'14702", 'reported_seq': 14826, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:09.621200+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:19:09.621200+0000', 'last_peered': '2024-09-11T01:19:09.621200+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:19:09.621200+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "43'4700", 'ondisk_log_start': "43'4700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22548480, 'num_objects': 5505, 'num_object_clones': 0, 'num_object_copies': 11010, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5505, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12519, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1481}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:12.077 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:19:12.077 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'14606", 'reported_seq': 14743, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:06.882224+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:19:06.882224+0000', 'last_peered': '2024-09-11T01:19:06.882224+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:19:06.882224+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "43'4600", 'ondisk_log_start': "43'4600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 23175168, 'num_objects': 5658, 'num_object_clones': 0, 'num_object_copies': 11316, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5658, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11343, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1708}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:12.078 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:19:12.078 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'14643", 'reported_seq': 14783, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:06.802285+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:19:06.802285+0000', 'last_peered': '2024-09-11T01:19:06.802285+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:19:06.802285+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "43'4600", 'ondisk_log_start': "43'4600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22663168, 'num_objects': 5533, 'num_object_clones': 0, 'num_object_copies': 11066, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5533, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11252, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1494}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:12.078 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:19:12.078 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'14433", 'reported_seq': 14572, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:06.928813+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:19:06.928813+0000', 'last_peered': '2024-09-11T01:19:06.928813+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:19:06.928813+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "43'4400", 'ondisk_log_start': "43'4400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22606441, 'num_objects': 5520, 'num_object_clones': 0, 'num_object_copies': 11040, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5520, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11503, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1603}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:12.078 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:19:13.045 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:19:13.045 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38157/173870 objects degraded (21.946%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":489,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":86935,"data_bytes":356070036,"bytes_used":1994313728,"bytes_avail":287915978752,"bytes_total":289910292480,"degraded_objects":38157,"degraded_total":173870,"degraded_ratio":0.21945706562374187,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":247,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":503},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:19:13.147 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:19:13.147 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:13.984 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:19:13.986 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:19:14.052 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":285,"stamp":"2024-09-11T01:19:12.709716+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":356070036,"num_objects":86935,"num_object_clones":0,"num_object_copies":173870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86935,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":195356,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9483,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160707,"ondisk_log_size":160707,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1947572,"kb_used_data":889220,"kb_used_omap":12,"kb_used_meta":1058291,"kb_avail":281167948,"statfs":{"total":289910292480,"available":287915978752,"internally_reserved":0,"allocated":910561280,"data_stored":909696051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1083689988},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,4,1,10,10,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24735744,"num_objects":-6039,"num_object_clones":0,"num_object_copies":-12078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-247,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6039,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":247,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002588"},"pg_stats":[{"pgid":"2.f","version":"47'14829","reported_seq":14912,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:11.939101+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:19:11.939101+0000","last_peered":"2024-09-11T01:19:11.939101+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:19:11.939101+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22269952,"num_objects":5437,"num_object_clones":0,"num_object_copies":10874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14829,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1365}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'14874","reported_seq":14981,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:09.622058+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:19:09.622058+0000","last_peered":"2024-09-11T01:19:09.622058+0000","last_clean":"2024-09-11T01:19:09.622058+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:19:09.622058+0000","last_undegraded":"2024-09-11T01:19:09.622058+0000","last_fullsized":"2024-09-11T01:19:09.622058+0000","mapping_epoch":35,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22437888,"num_objects":5478,"num_object_clones":0,"num_object_copies":10956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13113,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14601","reported_seq":14735,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:09.581840+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:19:09.581840+0000","last_peered":"2024-09-11T01:19:09.581840+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:19:09.581840+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22700056,"num_objects":5543,"num_object_clones":0,"num_object_copies":11086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5543,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":12902,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1647}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'14601","reported_seq":14676,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:09.630307+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:19:09.630307+0000","last_peered":"2024-09-11T01:19:09.630307+0000","last_clean":"2024-09-11T01:19:09.630307+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:19:09.630307+0000","last_undegraded":"2024-09-11T01:19:09.630307+0000","last_fullsized":"2024-09-11T01:19:09.630307+0000","mapping_epoch":35,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22663168,"num_objects":5533,"num_object_clones":0,"num_object_copies":11066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5533,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14601,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14654","reported_seq":14787,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:11.948772+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:19:11.948772+0000","last_peered":"2024-09-11T01:19:11.948772+0000","last_clean":"2024-09-11T01:19:11.948772+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:19:11.948772+0000","last_undegraded":"2024-09-11T01:19:11.948772+0000","last_fullsized":"2024-09-11T01:19:11.948772+0000","mapping_epoch":34,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21934080,"num_objects":5355,"num_object_clones":0,"num_object_copies":10710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5355,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11659,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14735","reported_seq":14907,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:11.960063+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:19:11.960063+0000","last_peered":"2024-09-11T01:19:11.960063+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:19:11.960063+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22495232,"num_objects":5492,"num_object_clones":0,"num_object_copies":10984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11076,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1541}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'14570","reported_seq":14705,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:11.914830+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:19:11.914830+0000","last_peered":"2024-09-11T01:19:11.914830+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:19:11.914830+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21954560,"num_objects":5360,"num_object_clones":0,"num_object_copies":10720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11427,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1374}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14800","reported_seq":14935,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:11.846894+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:19:11.846894+0000","last_peered":"2024-09-11T01:19:11.846894+0000","last_clean":"2024-09-11T01:19:11.846894+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:19:11.846894+0000","last_undegraded":"2024-09-11T01:19:11.846894+0000","last_fullsized":"2024-09-11T01:19:11.846894+0000","mapping_epoch":34,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22339584,"num_objects":5454,"num_object_clones":0,"num_object_copies":10908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11676,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934737+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934737+0000","last_peered":"2024-09-11T01:17:58.934737+0000","last_clean":"2024-09-11T01:17:58.934737+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934737+0000","last_undegraded":"2024-09-11T01:17:58.934737+0000","last_fullsized":"2024-09-11T01:17:58.934737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14533","reported_seq":17891,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:09.657728+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:19:09.657728+0000","last_peered":"2024-09-11T01:19:09.657728+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:19:09.657728+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22282240,"num_objects":5440,"num_object_clones":0,"num_object_copies":10880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2323,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12504,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1612,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3117}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14556","reported_seq":14687,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:11.922112+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:19:11.922112+0000","last_peered":"2024-09-11T01:19:11.922112+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:19:11.922112+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21757952,"num_objects":5312,"num_object_clones":0,"num_object_copies":10624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11143,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1328}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14702","reported_seq":14826,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:09.621200+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:19:09.621200+0000","last_peered":"2024-09-11T01:19:09.621200+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:19:09.621200+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22548480,"num_objects":5505,"num_object_clones":0,"num_object_copies":11010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12519,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1481}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935080+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935080+0000","last_peered":"2024-09-11T01:17:58.935080+0000","last_clean":"2024-09-11T01:17:58.935080+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935080+0000","last_undegraded":"2024-09-11T01:17:58.935080+0000","last_fullsized":"2024-09-11T01:17:58.935080+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14356","reported_seq":22342,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:09.569337+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:19:09.569337+0000","last_peered":"2024-09-11T01:19:09.569337+0000","last_clean":"2024-09-11T01:19:09.569337+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:19:09.569337+0000","last_undegraded":"2024-09-11T01:19:09.569337+0000","last_fullsized":"2024-09-11T01:19:09.569337+0000","mapping_epoch":44,"log_start":"43'4300","ondisk_log_start":"43'4300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22282240,"num_objects":5440,"num_object_clones":0,"num_object_copies":10880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12141,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14767","reported_seq":14904,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:11.845113+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:19:11.845113+0000","last_peered":"2024-09-11T01:19:11.845113+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:19:11.845113+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22515712,"num_objects":5497,"num_object_clones":0,"num_object_copies":10994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5497,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11504,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1547}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938540+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938540+0000","last_peered":"2024-09-11T01:17:58.938540+0000","last_clean":"2024-09-11T01:17:58.938540+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938540+0000","last_undegraded":"2024-09-11T01:17:58.938540+0000","last_fullsized":"2024-09-11T01:17:58.938540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14551","reported_seq":22587,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:11.032786+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:19:11.032786+0000","last_peered":"2024-09-11T01:19:11.032786+0000","last_clean":"2024-09-11T01:19:11.032786+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:19:11.032786+0000","last_undegraded":"2024-09-11T01:19:11.032786+0000","last_fullsized":"2024-09-11T01:19:11.032786+0000","mapping_epoch":46,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21843968,"num_objects":5333,"num_object_clones":0,"num_object_copies":10666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11206,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14769","reported_seq":14909,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:11.822419+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:19:11.822419+0000","last_peered":"2024-09-11T01:19:11.822419+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:19:11.822419+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22147072,"num_objects":5407,"num_object_clones":0,"num_object_copies":10814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11378,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1368}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971314+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971314+0000","last_peered":"2024-09-11T01:17:58.971314+0000","last_clean":"2024-09-11T01:17:58.971314+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971314+0000","last_undegraded":"2024-09-11T01:17:58.971314+0000","last_fullsized":"2024-09-11T01:17:58.971314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14606","reported_seq":14744,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:11.949215+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:19:11.949215+0000","last_peered":"2024-09-11T01:19:11.949215+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:19:11.949215+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21897833,"num_objects":5347,"num_object_clones":0,"num_object_copies":10694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11676,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1430}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":356070017,"num_objects":86933,"num_object_clones":0,"num_object_copies":173866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38157,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86933,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":195354,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9482,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":907182080,"data_stored":907111314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160704,"ondisk_log_size":160704,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182492,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":727980,"kb_used_data":322908,"kb_used_omap":4,"kb_used_meta":405051,"kb_avail":93643860,"statfs":{"total":96636764160,"available":95891312640,"internally_reserved":0,"allocated":330657792,"data_stored":330388133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":414772907},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,7,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986264,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":628724,"kb_used_data":304164,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93743116,"statfs":{"total":96636764160,"available":95992950784,"internally_reserved":0,"allocated":311463936,"data_stored":311194282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332328618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,0,5,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659209,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":590868,"kb_used_data":262148,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93780972,"statfs":{"total":96636764160,"available":96031715328,"internally_reserved":0,"allocated":268439552,"data_stored":268113636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":336588463},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,1,4,2,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":330186752,"data_stored":330183273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":266838016,"data_stored":266778280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":310157312,"data_stored":310149761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:17.054 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:17.993 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:19:17.996 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:19:18.031 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":287,"stamp":"2024-09-11T01:19:16.710685+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":351818388,"num_objects":85897,"num_object_clones":0,"num_object_copies":171794,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38024,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85897,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":196394,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9616,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160845,"ondisk_log_size":160845,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1945836,"kb_used_data":877752,"kb_used_omap":12,"kb_used_meta":1068019,"kb_avail":281169684,"statfs":{"total":289910292480,"available":287917756416,"internally_reserved":0,"allocated":898818048,"data_stored":897952819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1093651460},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,4,3,4,16,9,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19472384,"num_objects":-4754,"num_object_clones":0,"num_object_copies":-9508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-246,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4754,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4754,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002644"},"pg_stats":[{"pgid":"2.f","version":"47'14829","reported_seq":14912,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:11.939101+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:19:11.939101+0000","last_peered":"2024-09-11T01:19:11.939101+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:19:11.939101+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22269952,"num_objects":5437,"num_object_clones":0,"num_object_copies":10874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5437,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14829,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1365}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15012","reported_seq":15119,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:14.630293+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:19:14.630293+0000","last_peered":"2024-09-11T01:19:14.630293+0000","last_clean":"2024-09-11T01:19:14.630293+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:19:14.630293+0000","last_undegraded":"2024-09-11T01:19:14.630293+0000","last_fullsized":"2024-09-11T01:19:14.630293+0000","mapping_epoch":35,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21872640,"num_objects":5340,"num_object_clones":0,"num_object_copies":10680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13251,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14733","reported_seq":14867,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:14.564936+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:19:14.564936+0000","last_peered":"2024-09-11T01:19:14.564936+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:19:14.564936+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22159384,"num_objects":5411,"num_object_clones":0,"num_object_copies":10822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5411,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13034,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1515}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'14763","reported_seq":14838,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:14.510881+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:19:14.510881+0000","last_peered":"2024-09-11T01:19:14.510881+0000","last_clean":"2024-09-11T01:19:14.510881+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:19:14.510881+0000","last_undegraded":"2024-09-11T01:19:14.510881+0000","last_fullsized":"2024-09-11T01:19:14.510881+0000","mapping_epoch":35,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21999616,"num_objects":5371,"num_object_clones":0,"num_object_copies":10742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14763,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14654","reported_seq":14787,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:11.948772+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:19:11.948772+0000","last_peered":"2024-09-11T01:19:11.948772+0000","last_clean":"2024-09-11T01:19:11.948772+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:19:11.948772+0000","last_undegraded":"2024-09-11T01:19:11.948772+0000","last_fullsized":"2024-09-11T01:19:11.948772+0000","mapping_epoch":34,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21934080,"num_objects":5355,"num_object_clones":0,"num_object_copies":10710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5355,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11659,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14735","reported_seq":14907,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:11.960063+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:19:11.960063+0000","last_peered":"2024-09-11T01:19:11.960063+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:19:11.960063+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22495232,"num_objects":5492,"num_object_clones":0,"num_object_copies":10984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11076,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1541}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'14570","reported_seq":14705,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:11.914830+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:19:11.914830+0000","last_peered":"2024-09-11T01:19:11.914830+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:19:11.914830+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21954560,"num_objects":5360,"num_object_clones":0,"num_object_copies":10720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11427,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1374}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14800","reported_seq":14935,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:11.846894+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:19:11.846894+0000","last_peered":"2024-09-11T01:19:11.846894+0000","last_clean":"2024-09-11T01:19:11.846894+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:19:11.846894+0000","last_undegraded":"2024-09-11T01:19:11.846894+0000","last_fullsized":"2024-09-11T01:19:11.846894+0000","mapping_epoch":34,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22339584,"num_objects":5454,"num_object_clones":0,"num_object_copies":10908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11676,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934737+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934737+0000","last_peered":"2024-09-11T01:17:58.934737+0000","last_clean":"2024-09-11T01:17:58.934737+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934737+0000","last_undegraded":"2024-09-11T01:17:58.934737+0000","last_fullsized":"2024-09-11T01:17:58.934737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14671","reported_seq":18292,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:14.682468+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:19:14.682468+0000","last_peered":"2024-09-11T01:19:14.682468+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:19:14.682468+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21716992,"num_objects":5302,"num_object_clones":0,"num_object_copies":10604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2190,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12642,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1745,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3112}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14556","reported_seq":14687,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:11.922112+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:19:11.922112+0000","last_peered":"2024-09-11T01:19:11.922112+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:19:11.922112+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21757952,"num_objects":5312,"num_object_clones":0,"num_object_copies":10624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11143,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1328}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14855","reported_seq":14979,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:14.515143+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:19:14.515143+0000","last_peered":"2024-09-11T01:19:14.515143+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:19:14.515143+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21921792,"num_objects":5352,"num_object_clones":0,"num_object_copies":10704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12672,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1328}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935080+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935080+0000","last_peered":"2024-09-11T01:17:58.935080+0000","last_clean":"2024-09-11T01:17:58.935080+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935080+0000","last_undegraded":"2024-09-11T01:17:58.935080+0000","last_fullsized":"2024-09-11T01:17:58.935080+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14525","reported_seq":22510,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:14.674186+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:19:14.674186+0000","last_peered":"2024-09-11T01:19:14.674186+0000","last_clean":"2024-09-11T01:19:14.674186+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:19:14.674186+0000","last_undegraded":"2024-09-11T01:19:14.674186+0000","last_fullsized":"2024-09-11T01:19:14.674186+0000","mapping_epoch":44,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21590016,"num_objects":5271,"num_object_clones":0,"num_object_copies":10542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12310,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14767","reported_seq":14904,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:11.845113+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:19:11.845113+0000","last_peered":"2024-09-11T01:19:11.845113+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:19:11.845113+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22515712,"num_objects":5497,"num_object_clones":0,"num_object_copies":10994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5497,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11504,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1547}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938540+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938540+0000","last_peered":"2024-09-11T01:17:58.938540+0000","last_clean":"2024-09-11T01:17:58.938540+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938540+0000","last_undegraded":"2024-09-11T01:17:58.938540+0000","last_fullsized":"2024-09-11T01:17:58.938540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14697","reported_seq":22733,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:16.081514+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:19:16.081514+0000","last_peered":"2024-09-11T01:19:16.081514+0000","last_clean":"2024-09-11T01:19:16.081514+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:19:16.081514+0000","last_undegraded":"2024-09-11T01:19:16.081514+0000","last_fullsized":"2024-09-11T01:19:16.081514+0000","mapping_epoch":46,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21245952,"num_objects":5187,"num_object_clones":0,"num_object_copies":10374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11352,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14769","reported_seq":14909,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:11.822419+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:19:11.822419+0000","last_peered":"2024-09-11T01:19:11.822419+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:19:11.822419+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22147072,"num_objects":5407,"num_object_clones":0,"num_object_copies":10814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5407,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11378,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1368}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971314+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971314+0000","last_peered":"2024-09-11T01:17:58.971314+0000","last_clean":"2024-09-11T01:17:58.971314+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971314+0000","last_undegraded":"2024-09-11T01:17:58.971314+0000","last_fullsized":"2024-09-11T01:17:58.971314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14606","reported_seq":14744,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:11.949215+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:19:11.949215+0000","last_peered":"2024-09-11T01:19:11.949215+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:19:11.949215+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21897833,"num_objects":5347,"num_object_clones":0,"num_object_copies":10694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11676,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1430}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":351818369,"num_objects":85895,"num_object_clones":0,"num_object_copies":171790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38024,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":85895,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":196392,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9615,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":895299584,"data_stored":895228818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160842,"ondisk_log_size":160842,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182492,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":727980,"kb_used_data":322908,"kb_used_omap":4,"kb_used_meta":405051,"kb_avail":93643860,"statfs":{"total":96636764160,"available":95891312640,"internally_reserved":0,"allocated":330657792,"data_stored":330388133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":414772907},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,7,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986265,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":628148,"kb_used_data":299428,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93743692,"statfs":{"total":96636764160,"available":95993540608,"internally_reserved":0,"allocated":306614272,"data_stored":306344618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336588458},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,4,0,3,5,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659210,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589708,"kb_used_data":255416,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93782132,"statfs":{"total":96636764160,"available":96032903168,"internally_reserved":0,"allocated":261545984,"data_stored":261220068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":342290095},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,0,4,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":330186752,"data_stored":330183273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":259792896,"data_stored":259733160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":305319936,"data_stored":305312385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:18.033 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:18.852 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:19:18.853 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:19:18.938 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":288,"stamp":"2024-09-11T01:19:18.712560+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":346620564,"num_objects":84628,"num_object_clones":0,"num_object_copies":169256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38024,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84628,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":197663,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9616,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160814,"ondisk_log_size":160814,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1926420,"kb_used_data":870812,"kb_used_omap":12,"kb_used_meta":1055539,"kb_avail":281189100,"statfs":{"total":289910292480,"available":287937638400,"internally_reserved":0,"allocated":891711488,"data_stored":890846259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1080871940},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,5,3,5,17,7,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24010752,"num_objects":-5862,"num_object_clones":0,"num_object_copies":-11724,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-246,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5862,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5862,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":246,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004069"},"pg_stats":[{"pgid":"2.f","version":"47'14955","reported_seq":15038,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:16.626028+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:19:16.626028+0000","last_peered":"2024-09-11T01:19:16.626028+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:19:16.626028+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21753856,"num_objects":5311,"num_object_clones":0,"num_object_copies":10622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14955,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1239}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15012","reported_seq":15119,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:14.630293+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:19:14.630293+0000","last_peered":"2024-09-11T01:19:14.630293+0000","last_clean":"2024-09-11T01:19:14.630293+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:19:14.630293+0000","last_undegraded":"2024-09-11T01:19:14.630293+0000","last_fullsized":"2024-09-11T01:19:14.630293+0000","mapping_epoch":35,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21872640,"num_objects":5340,"num_object_clones":0,"num_object_copies":10680,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5340,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13251,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14733","reported_seq":14867,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:14.564936+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:19:14.564936+0000","last_peered":"2024-09-11T01:19:14.564936+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:19:14.564936+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22159384,"num_objects":5411,"num_object_clones":0,"num_object_copies":10822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5411,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13034,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1515}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'14763","reported_seq":14838,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:14.510881+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:19:14.510881+0000","last_peered":"2024-09-11T01:19:14.510881+0000","last_clean":"2024-09-11T01:19:14.510881+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:19:14.510881+0000","last_undegraded":"2024-09-11T01:19:14.510881+0000","last_fullsized":"2024-09-11T01:19:14.510881+0000","mapping_epoch":35,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21999616,"num_objects":5371,"num_object_clones":0,"num_object_copies":10742,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5371,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14763,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14800","reported_seq":14933,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:16.958754+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:19:16.958754+0000","last_peered":"2024-09-11T01:19:16.958754+0000","last_clean":"2024-09-11T01:19:16.958754+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:19:16.958754+0000","last_undegraded":"2024-09-11T01:19:16.958754+0000","last_fullsized":"2024-09-11T01:19:16.958754+0000","mapping_epoch":34,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21336064,"num_objects":5209,"num_object_clones":0,"num_object_copies":10418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11805,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14858","reported_seq":15033,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:16.902382+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:19:16.902382+0000","last_peered":"2024-09-11T01:19:16.902382+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:19:16.902382+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21991424,"num_objects":5369,"num_object_clones":0,"num_object_copies":10738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11199,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1418}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'14723","reported_seq":14858,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:16.950243+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:19:16.950243+0000","last_peered":"2024-09-11T01:19:16.950243+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:19:16.950243+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21327872,"num_objects":5207,"num_object_clones":0,"num_object_copies":10414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11580,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1221}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14952","reported_seq":15087,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:16.972437+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:19:16.972437+0000","last_peered":"2024-09-11T01:19:16.972437+0000","last_clean":"2024-09-11T01:19:16.972437+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:19:16.972437+0000","last_undegraded":"2024-09-11T01:19:16.972437+0000","last_fullsized":"2024-09-11T01:19:16.972437+0000","mapping_epoch":34,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21716992,"num_objects":5302,"num_object_clones":0,"num_object_copies":10604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11828,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934737+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934737+0000","last_peered":"2024-09-11T01:17:58.934737+0000","last_clean":"2024-09-11T01:17:58.934737+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934737+0000","last_undegraded":"2024-09-11T01:17:58.934737+0000","last_fullsized":"2024-09-11T01:17:58.934737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14671","reported_seq":18292,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:14.682468+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:19:14.682468+0000","last_peered":"2024-09-11T01:19:14.682468+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:19:14.682468+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21716992,"num_objects":5302,"num_object_clones":0,"num_object_copies":10604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2190,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12642,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1745,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3112}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14682","reported_seq":14816,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:16.987522+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:19:16.987522+0000","last_peered":"2024-09-11T01:19:16.987522+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:19:16.987522+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21241856,"num_objects":5186,"num_object_clones":0,"num_object_copies":10372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11269,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1202}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14855","reported_seq":14979,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:14.515143+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:19:14.515143+0000","last_peered":"2024-09-11T01:19:14.515143+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:19:14.515143+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21921792,"num_objects":5352,"num_object_clones":0,"num_object_copies":10704,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5352,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12672,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1328}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935080+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935080+0000","last_peered":"2024-09-11T01:17:58.935080+0000","last_clean":"2024-09-11T01:17:58.935080+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935080+0000","last_undegraded":"2024-09-11T01:17:58.935080+0000","last_fullsized":"2024-09-11T01:17:58.935080+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14525","reported_seq":22510,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:14.674186+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:19:14.674186+0000","last_peered":"2024-09-11T01:19:14.674186+0000","last_clean":"2024-09-11T01:19:14.674186+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:19:14.674186+0000","last_undegraded":"2024-09-11T01:19:14.674186+0000","last_fullsized":"2024-09-11T01:19:14.674186+0000","mapping_epoch":44,"log_start":"43'4500","ondisk_log_start":"43'4500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21590016,"num_objects":5271,"num_object_clones":0,"num_object_copies":10542,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5271,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12310,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14914","reported_seq":15049,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:16.995314+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:19:16.995314+0000","last_peered":"2024-09-11T01:19:16.995314+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:19:16.995314+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21913600,"num_objects":5350,"num_object_clones":0,"num_object_copies":10700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11651,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1400}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938540+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938540+0000","last_peered":"2024-09-11T01:17:58.938540+0000","last_clean":"2024-09-11T01:17:58.938540+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938540+0000","last_undegraded":"2024-09-11T01:17:58.938540+0000","last_fullsized":"2024-09-11T01:17:58.938540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14697","reported_seq":22733,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:16.081514+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:19:16.081514+0000","last_peered":"2024-09-11T01:19:16.081514+0000","last_clean":"2024-09-11T01:19:16.081514+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:19:16.081514+0000","last_undegraded":"2024-09-11T01:19:16.081514+0000","last_fullsized":"2024-09-11T01:19:16.081514+0000","mapping_epoch":46,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21245952,"num_objects":5187,"num_object_clones":0,"num_object_copies":10374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11352,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14915","reported_seq":15052,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:16.934874+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:19:16.934874+0000","last_peered":"2024-09-11T01:19:16.934874+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:19:16.934874+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21549056,"num_objects":5261,"num_object_clones":0,"num_object_copies":10522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11524,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1222}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971314+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971314+0000","last_peered":"2024-09-11T01:17:58.971314+0000","last_clean":"2024-09-11T01:17:58.971314+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971314+0000","last_undegraded":"2024-09-11T01:17:58.971314+0000","last_fullsized":"2024-09-11T01:17:58.971314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14756","reported_seq":14895,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:16.929683+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:19:16.929683+0000","last_peered":"2024-09-11T01:19:16.929683+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:19:16.929683+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21283433,"num_objects":5197,"num_object_clones":0,"num_object_copies":10394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5197,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11826,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1280}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":346620545,"num_objects":84626,"num_object_clones":0,"num_object_copies":169252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":38024,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84626,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":197661,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9615,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":888250368,"data_stored":888179602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160811,"ondisk_log_size":160811,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182493,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":708564,"kb_used_data":315968,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93663276,"statfs":{"total":96636764160,"available":95911194624,"internally_reserved":0,"allocated":323551232,"data_stored":323281573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":401993387},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,0,2,8,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986265,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":628148,"kb_used_data":299428,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93743692,"statfs":{"total":96636764160,"available":95993540608,"internally_reserved":0,"allocated":306614272,"data_stored":306344618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336588458},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,4,0,3,5,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659210,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589708,"kb_used_data":255416,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93782132,"statfs":{"total":96636764160,"available":96032903168,"internally_reserved":0,"allocated":261545984,"data_stored":261220068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":342290095},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,0,4,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":323137536,"data_stored":323134057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":259792896,"data_stored":259733160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":305319936,"data_stored":305312385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:18.939 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:19:18.939 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'14955", 'reported_seq': 15038, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:16.626028+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:19:16.626028+0000', 'last_peered': '2024-09-11T01:19:16.626028+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:19:16.626028+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "43'4900", 'ondisk_log_start': "43'4900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21753856, 'num_objects': 5311, 'num_object_clones': 0, 'num_object_copies': 10622, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5311, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14955, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1239}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:18.939 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:19:18.939 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'14733", 'reported_seq': 14867, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:14.564936+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:19:14.564936+0000', 'last_peered': '2024-09-11T01:19:14.564936+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:19:14.564936+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "43'4700", 'ondisk_log_start': "43'4700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 22159384, 'num_objects': 5411, 'num_object_clones': 0, 'num_object_copies': 10822, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5411, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13034, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1515}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:18.939 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:19:18.939 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'14858", 'reported_seq': 15033, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:16.902382+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:19:16.902382+0000', 'last_peered': '2024-09-11T01:19:16.902382+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:19:16.902382+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "43'4800", 'ondisk_log_start': "43'4800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21991424, 'num_objects': 5369, 'num_object_clones': 0, 'num_object_copies': 10738, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5369, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11199, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1418}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:18.940 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:19:18.940 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'14723", 'reported_seq': 14858, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:16.950243+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:19:16.950243+0000', 'last_peered': '2024-09-11T01:19:16.950243+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:19:16.950243+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "43'4700", 'ondisk_log_start': "43'4700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21327872, 'num_objects': 5207, 'num_object_clones': 0, 'num_object_copies': 10414, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5207, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11580, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1221}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:18.940 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:19:18.940 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'14671", 'reported_seq': 18292, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:14.682468+0000', 'last_change': '2024-09-11T01:17:57.355893+0000', 'last_active': '2024-09-11T01:19:14.682468+0000', 'last_peered': '2024-09-11T01:19:14.682468+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:19:14.682468+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "43'4600", 'ondisk_log_start': "43'4600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21716992, 'num_objects': 5302, 'num_object_clones': 0, 'num_object_copies': 10604, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2190, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5302, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12642, 'num_write_kb': 31844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1745, 'num_bytes_recovered': 4505600, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3112}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:18.940 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:19:18.940 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'14682", 'reported_seq': 14816, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:16.987522+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:19:16.987522+0000', 'last_peered': '2024-09-11T01:19:16.987522+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:19:16.987522+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "43'4600", 'ondisk_log_start': "43'4600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21241856, 'num_objects': 5186, 'num_object_clones': 0, 'num_object_copies': 10372, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5186, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11269, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1202}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:18.940 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:19:18.940 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'14855", 'reported_seq': 14979, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:14.515143+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:19:14.515143+0000', 'last_peered': '2024-09-11T01:19:14.515143+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:19:14.515143+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "43'4800", 'ondisk_log_start': "43'4800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21921792, 'num_objects': 5352, 'num_object_clones': 0, 'num_object_copies': 10704, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5352, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12672, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1328}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:18.940 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:19:18.940 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'14914", 'reported_seq': 15049, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:16.995314+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:19:16.995314+0000', 'last_peered': '2024-09-11T01:19:16.995314+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:19:16.995314+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "43'4900", 'ondisk_log_start': "43'4900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21913600, 'num_objects': 5350, 'num_object_clones': 0, 'num_object_copies': 10700, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5350, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11651, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1400}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:18.941 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:19:18.941 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'14915", 'reported_seq': 15052, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:16.934874+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:19:16.934874+0000', 'last_peered': '2024-09-11T01:19:16.934874+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:19:16.934874+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "43'4900", 'ondisk_log_start': "43'4900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21549056, 'num_objects': 5261, 'num_object_clones': 0, 'num_object_copies': 10522, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5261, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11524, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1222}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:18.941 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:19:18.941 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'14756", 'reported_seq': 14895, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:16.929683+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:19:16.929683+0000', 'last_peered': '2024-09-11T01:19:16.929683+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:19:16.929683+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "43'4700", 'ondisk_log_start': "43'4700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21283433, 'num_objects': 5197, 'num_object_clones': 0, 'num_object_copies': 10394, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5197, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11826, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1280}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:18.941 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:19:20.082 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:19:20.082 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 38024/169256 objects degraded (22.465%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":496,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":84628,"data_bytes":346620564,"bytes_used":1972654080,"bytes_avail":287937638400,"bytes_total":289910292480,"degraded_objects":38024,"degraded_total":169256,"degraded_ratio":0.22465377889114713,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":246,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":488},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:19:20.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:19:20.173 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:20.973 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:19:20.974 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:19:21.013 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":289,"stamp":"2024-09-11T01:19:20.713035+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":343052948,"num_objects":83757,"num_object_clones":0,"num_object_copies":167514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37889,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83757,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":198534,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9751,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160885,"ondisk_log_size":160885,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1909172,"kb_used_data":866044,"kb_used_omap":12,"kb_used_meta":1043059,"kb_avail":281206348,"statfs":{"total":289910292480,"available":287955300352,"internally_reserved":0,"allocated":886829056,"data_stored":885963827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1068092420},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,5,4,16,8,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22523904,"num_objects":-5499,"num_object_clones":0,"num_object_copies":-10998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-381,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5499,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5499,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":381,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004104"},"pg_stats":[{"pgid":"2.f","version":"47'14955","reported_seq":15038,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:16.626028+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:19:16.626028+0000","last_peered":"2024-09-11T01:19:16.626028+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:19:16.626028+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21753856,"num_objects":5311,"num_object_clones":0,"num_object_copies":10622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14955,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1239}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15168","reported_seq":15275,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:19.686856+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:19:19.686856+0000","last_peered":"2024-09-11T01:19:19.686856+0000","last_clean":"2024-09-11T01:19:19.686856+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:19:19.686856+0000","last_undegraded":"2024-09-11T01:19:19.686856+0000","last_fullsized":"2024-09-11T01:19:19.686856+0000","mapping_epoch":35,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21233664,"num_objects":5184,"num_object_clones":0,"num_object_copies":10368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13407,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14881","reported_seq":15015,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:19.574380+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:19:19.574380+0000","last_peered":"2024-09-11T01:19:19.574380+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:19:19.574380+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21553176,"num_objects":5263,"num_object_clones":0,"num_object_copies":10526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5263,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13182,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1367}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'14904","reported_seq":14978,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:19.672257+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:19:19.672257+0000","last_peered":"2024-09-11T01:19:19.672257+0000","last_clean":"2024-09-11T01:19:19.672257+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:19:19.672257+0000","last_undegraded":"2024-09-11T01:19:19.672257+0000","last_fullsized":"2024-09-11T01:19:19.672257+0000","mapping_epoch":35,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21422080,"num_objects":5230,"num_object_clones":0,"num_object_copies":10460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14904,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14800","reported_seq":14933,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:16.958754+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:19:16.958754+0000","last_peered":"2024-09-11T01:19:16.958754+0000","last_clean":"2024-09-11T01:19:16.958754+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:19:16.958754+0000","last_undegraded":"2024-09-11T01:19:16.958754+0000","last_fullsized":"2024-09-11T01:19:16.958754+0000","mapping_epoch":34,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21336064,"num_objects":5209,"num_object_clones":0,"num_object_copies":10418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11805,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14858","reported_seq":15033,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:16.902382+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:19:16.902382+0000","last_peered":"2024-09-11T01:19:16.902382+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:19:16.902382+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21991424,"num_objects":5369,"num_object_clones":0,"num_object_copies":10738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11199,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1418}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'14723","reported_seq":14858,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:16.950243+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:19:16.950243+0000","last_peered":"2024-09-11T01:19:16.950243+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:19:16.950243+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21327872,"num_objects":5207,"num_object_clones":0,"num_object_copies":10414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11580,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1221}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'14952","reported_seq":15087,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:16.972437+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:19:16.972437+0000","last_peered":"2024-09-11T01:19:16.972437+0000","last_clean":"2024-09-11T01:19:16.972437+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:19:16.972437+0000","last_undegraded":"2024-09-11T01:19:16.972437+0000","last_fullsized":"2024-09-11T01:19:16.972437+0000","mapping_epoch":34,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21716992,"num_objects":5302,"num_object_clones":0,"num_object_copies":10604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5302,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11828,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934737+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934737+0000","last_peered":"2024-09-11T01:17:58.934737+0000","last_clean":"2024-09-11T01:17:58.934737+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934737+0000","last_undegraded":"2024-09-11T01:17:58.934737+0000","last_fullsized":"2024-09-11T01:17:58.934737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14813","reported_seq":18705,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:19.687759+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:19:19.687759+0000","last_peered":"2024-09-11T01:19:19.687759+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:19:19.687759+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21135360,"num_objects":5160,"num_object_clones":0,"num_object_copies":10320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2055,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12784,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1880,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3105}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14682","reported_seq":14816,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:16.987522+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:19:16.987522+0000","last_peered":"2024-09-11T01:19:16.987522+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:19:16.987522+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21241856,"num_objects":5186,"num_object_clones":0,"num_object_copies":10372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11269,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1202}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14995","reported_seq":15118,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:19.637359+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:19:19.637359+0000","last_peered":"2024-09-11T01:19:19.637359+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:19:19.637359+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21348352,"num_objects":5212,"num_object_clones":0,"num_object_copies":10424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12812,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1188}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935080+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935080+0000","last_peered":"2024-09-11T01:17:58.935080+0000","last_clean":"2024-09-11T01:17:58.935080+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935080+0000","last_undegraded":"2024-09-11T01:17:58.935080+0000","last_fullsized":"2024-09-11T01:17:58.935080+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14669","reported_seq":22655,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:19.666194+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:19:19.666194+0000","last_peered":"2024-09-11T01:19:19.666194+0000","last_clean":"2024-09-11T01:19:19.666194+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:19:19.666194+0000","last_undegraded":"2024-09-11T01:19:19.666194+0000","last_fullsized":"2024-09-11T01:19:19.666194+0000","mapping_epoch":44,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21000192,"num_objects":5127,"num_object_clones":0,"num_object_copies":10254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12454,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'14914","reported_seq":15049,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:16.995314+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:19:16.995314+0000","last_peered":"2024-09-11T01:19:16.995314+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:19:16.995314+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21913600,"num_objects":5350,"num_object_clones":0,"num_object_copies":10700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11651,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1400}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938540+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938540+0000","last_peered":"2024-09-11T01:17:58.938540+0000","last_clean":"2024-09-11T01:17:58.938540+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938540+0000","last_undegraded":"2024-09-11T01:17:58.938540+0000","last_fullsized":"2024-09-11T01:17:58.938540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14697","reported_seq":22733,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:16.081514+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:19:16.081514+0000","last_peered":"2024-09-11T01:19:16.081514+0000","last_clean":"2024-09-11T01:19:16.081514+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:19:16.081514+0000","last_undegraded":"2024-09-11T01:19:16.081514+0000","last_fullsized":"2024-09-11T01:19:16.081514+0000","mapping_epoch":46,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21245952,"num_objects":5187,"num_object_clones":0,"num_object_copies":10374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11352,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'14915","reported_seq":15052,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:16.934874+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:19:16.934874+0000","last_peered":"2024-09-11T01:19:16.934874+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:19:16.934874+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21549056,"num_objects":5261,"num_object_clones":0,"num_object_copies":10522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11524,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1222}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971314+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971314+0000","last_peered":"2024-09-11T01:17:58.971314+0000","last_clean":"2024-09-11T01:17:58.971314+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971314+0000","last_undegraded":"2024-09-11T01:17:58.971314+0000","last_fullsized":"2024-09-11T01:17:58.971314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14756","reported_seq":14895,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:16.929683+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:19:16.929683+0000","last_peered":"2024-09-11T01:19:16.929683+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:19:16.929683+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'4700","ondisk_log_start":"43'4700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21283433,"num_objects":5197,"num_object_clones":0,"num_object_copies":10394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5197,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11826,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1280}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":343052929,"num_objects":83755,"num_object_clones":0,"num_object_copies":167510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37889,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83755,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":198532,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9750,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":883478528,"data_stored":883407762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160882,"ondisk_log_size":160882,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182493,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":708564,"kb_used_data":315968,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93663276,"statfs":{"total":96636764160,"available":95911194624,"internally_reserved":0,"allocated":323551232,"data_stored":323281573,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":401993387},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,0,2,8,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986266,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610900,"kb_used_data":294660,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93760940,"statfs":{"total":96636764160,"available":96011202560,"internally_reserved":0,"allocated":301731840,"data_stored":301462186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323808938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659210,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589708,"kb_used_data":255416,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93782132,"statfs":{"total":96636764160,"available":96032903168,"internally_reserved":0,"allocated":261545984,"data_stored":261220068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":342290095},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,0,4,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":323137536,"data_stored":323134057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":259792896,"data_stored":259733160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":300548096,"data_stored":300540545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:24.016 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:24.884 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:19:24.884 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:19:24.999 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2024-09-11T01:19:24.714002+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":337293972,"num_objects":82351,"num_object_clones":0,"num_object_copies":164702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37889,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82351,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":199940,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9751,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160791,"ondisk_log_size":160791,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1906884,"kb_used_data":851284,"kb_used_omap":12,"kb_used_meta":1055539,"kb_avail":281208636,"statfs":{"total":289910292480,"available":287957643264,"internally_reserved":0,"allocated":871714816,"data_stored":870849587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1080871940},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,7,7,7,12],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18776064,"num_objects":-4584,"num_object_clones":0,"num_object_copies":-9168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4584,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004286"},"pg_stats":[{"pgid":"2.f","version":"47'15107","reported_seq":15190,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:21.987289+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:19:21.987289+0000","last_peered":"2024-09-11T01:19:21.987289+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:19:21.987289+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21131264,"num_objects":5159,"num_object_clones":0,"num_object_copies":10318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15107,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1087}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15168","reported_seq":15275,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:19.686856+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:19:19.686856+0000","last_peered":"2024-09-11T01:19:19.686856+0000","last_clean":"2024-09-11T01:19:19.686856+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:19:19.686856+0000","last_undegraded":"2024-09-11T01:19:19.686856+0000","last_fullsized":"2024-09-11T01:19:19.686856+0000","mapping_epoch":35,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21233664,"num_objects":5184,"num_object_clones":0,"num_object_copies":10368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13407,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14881","reported_seq":15015,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:19.574380+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:19:19.574380+0000","last_peered":"2024-09-11T01:19:19.574380+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:19:19.574380+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21553176,"num_objects":5263,"num_object_clones":0,"num_object_copies":10526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5263,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13182,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1367}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'14904","reported_seq":14978,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:19.672257+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:19:19.672257+0000","last_peered":"2024-09-11T01:19:19.672257+0000","last_clean":"2024-09-11T01:19:19.672257+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:19:19.672257+0000","last_undegraded":"2024-09-11T01:19:19.672257+0000","last_fullsized":"2024-09-11T01:19:19.672257+0000","mapping_epoch":35,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21422080,"num_objects":5230,"num_object_clones":0,"num_object_copies":10460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14904,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14927","reported_seq":15060,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:21.953973+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:19:21.953973+0000","last_peered":"2024-09-11T01:19:21.953973+0000","last_clean":"2024-09-11T01:19:21.953973+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:19:21.953973+0000","last_undegraded":"2024-09-11T01:19:21.953973+0000","last_fullsized":"2024-09-11T01:19:21.953973+0000","mapping_epoch":34,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20815872,"num_objects":5082,"num_object_clones":0,"num_object_copies":10164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11932,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14987","reported_seq":15162,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:21.998912+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:19:21.998912+0000","last_peered":"2024-09-11T01:19:21.998912+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:19:21.998912+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21463040,"num_objects":5240,"num_object_clones":0,"num_object_copies":10480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11328,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1289}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'14858","reported_seq":14993,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:21.927457+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:19:21.927457+0000","last_peered":"2024-09-11T01:19:21.927457+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:19:21.927457+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20774912,"num_objects":5072,"num_object_clones":0,"num_object_copies":10144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11715,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1086}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15105","reported_seq":15240,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:21.928014+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:19:21.928014+0000","last_peered":"2024-09-11T01:19:21.928014+0000","last_clean":"2024-09-11T01:19:21.928014+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:19:21.928014+0000","last_undegraded":"2024-09-11T01:19:21.928014+0000","last_fullsized":"2024-09-11T01:19:21.928014+0000","mapping_epoch":34,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21090304,"num_objects":5149,"num_object_clones":0,"num_object_copies":10298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11981,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934737+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934737+0000","last_peered":"2024-09-11T01:17:58.934737+0000","last_clean":"2024-09-11T01:17:58.934737+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934737+0000","last_undegraded":"2024-09-11T01:17:58.934737+0000","last_fullsized":"2024-09-11T01:17:58.934737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14813","reported_seq":18705,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:19.687759+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:19:19.687759+0000","last_peered":"2024-09-11T01:19:19.687759+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:19:19.687759+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21135360,"num_objects":5160,"num_object_clones":0,"num_object_copies":10320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2055,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12784,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1880,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3105}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14819","reported_seq":14953,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:21.973942+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:19:21.973942+0000","last_peered":"2024-09-11T01:19:21.973942+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:19:21.973942+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20680704,"num_objects":5049,"num_object_clones":0,"num_object_copies":10098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11406,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1065}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14995","reported_seq":15118,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:19.637359+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:19:19.637359+0000","last_peered":"2024-09-11T01:19:19.637359+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:19:19.637359+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21348352,"num_objects":5212,"num_object_clones":0,"num_object_copies":10424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12812,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1188}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935080+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935080+0000","last_peered":"2024-09-11T01:17:58.935080+0000","last_clean":"2024-09-11T01:17:58.935080+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935080+0000","last_undegraded":"2024-09-11T01:17:58.935080+0000","last_fullsized":"2024-09-11T01:17:58.935080+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14669","reported_seq":22655,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:19.666194+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:19:19.666194+0000","last_peered":"2024-09-11T01:19:19.666194+0000","last_clean":"2024-09-11T01:19:19.666194+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:19:19.666194+0000","last_undegraded":"2024-09-11T01:19:19.666194+0000","last_fullsized":"2024-09-11T01:19:19.666194+0000","mapping_epoch":44,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21000192,"num_objects":5127,"num_object_clones":0,"num_object_copies":10254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12454,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15051","reported_seq":15188,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:21.921370+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:19:21.921370+0000","last_peered":"2024-09-11T01:19:21.921370+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:19:21.921370+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21352448,"num_objects":5213,"num_object_clones":0,"num_object_copies":10426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5213,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11788,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1263}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938540+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938540+0000","last_peered":"2024-09-11T01:17:58.938540+0000","last_clean":"2024-09-11T01:17:58.938540+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938540+0000","last_undegraded":"2024-09-11T01:17:58.938540+0000","last_fullsized":"2024-09-11T01:17:58.938540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14834","reported_seq":22870,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:21.102737+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:19:21.102737+0000","last_peered":"2024-09-11T01:19:21.102737+0000","last_clean":"2024-09-11T01:19:21.102737+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:19:21.102737+0000","last_undegraded":"2024-09-11T01:19:21.102737+0000","last_fullsized":"2024-09-11T01:19:21.102737+0000","mapping_epoch":46,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20684800,"num_objects":5050,"num_object_clones":0,"num_object_copies":10100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11489,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15071","reported_seq":15211,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:22.021977+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:19:22.021977+0000","last_peered":"2024-09-11T01:19:22.021977+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:19:22.021977+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20910080,"num_objects":5105,"num_object_clones":0,"num_object_copies":10210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11680,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1066}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971314+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971314+0000","last_peered":"2024-09-11T01:17:58.971314+0000","last_clean":"2024-09-11T01:17:58.971314+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971314+0000","last_undegraded":"2024-09-11T01:17:58.971314+0000","last_fullsized":"2024-09-11T01:17:58.971314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14899","reported_seq":15037,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:21.966905+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:19:21.966905+0000","last_peered":"2024-09-11T01:19:21.966905+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:19:21.966905+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20697705,"num_objects":5054,"num_object_clones":0,"num_object_copies":10108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5054,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11969,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1137}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":337293953,"num_objects":82349,"num_object_clones":0,"num_object_copies":164698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37889,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82349,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":199938,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9750,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":869580800,"data_stored":869510034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160788,"ondisk_log_size":160788,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182494,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":706040,"kb_used_data":309288,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93665800,"statfs":{"total":96636764160,"available":95913779200,"internally_reserved":0,"allocated":316710912,"data_stored":316441253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":406253227},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,1,3,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986266,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610900,"kb_used_data":294660,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93760940,"statfs":{"total":96636764160,"available":96011202560,"internally_reserved":0,"allocated":301731840,"data_stored":301462186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323808938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659211,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589944,"kb_used_data":247336,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93781896,"statfs":{"total":96636764160,"available":96032661504,"internally_reserved":0,"allocated":253272064,"data_stored":252946148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":350809775},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2,0,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":316207104,"data_stored":316203625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":252825600,"data_stored":252765864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":300548096,"data_stored":300540545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:25.000 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:25.788 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:19:25.791 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:19:25.907 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":291,"stamp":"2024-09-11T01:19:24.714002+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":337293972,"num_objects":82351,"num_object_clones":0,"num_object_copies":164702,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37889,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82351,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":199940,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9751,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160791,"ondisk_log_size":160791,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1906884,"kb_used_data":851284,"kb_used_omap":12,"kb_used_meta":1055539,"kb_avail":281208636,"statfs":{"total":289910292480,"available":287957643264,"internally_reserved":0,"allocated":871714816,"data_stored":870849587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1080871940},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,7,7,7,12],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18776064,"num_objects":-4584,"num_object_clones":0,"num_object_copies":-9168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4584,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4584,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004286"},"pg_stats":[{"pgid":"2.f","version":"47'15107","reported_seq":15190,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:21.987289+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:19:21.987289+0000","last_peered":"2024-09-11T01:19:21.987289+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:19:21.987289+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21131264,"num_objects":5159,"num_object_clones":0,"num_object_copies":10318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15107,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1087}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15168","reported_seq":15275,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:19.686856+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:19:19.686856+0000","last_peered":"2024-09-11T01:19:19.686856+0000","last_clean":"2024-09-11T01:19:19.686856+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:19:19.686856+0000","last_undegraded":"2024-09-11T01:19:19.686856+0000","last_fullsized":"2024-09-11T01:19:19.686856+0000","mapping_epoch":35,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21233664,"num_objects":5184,"num_object_clones":0,"num_object_copies":10368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13407,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14881","reported_seq":15015,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:19.574380+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:19:19.574380+0000","last_peered":"2024-09-11T01:19:19.574380+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:19:19.574380+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21553176,"num_objects":5263,"num_object_clones":0,"num_object_copies":10526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5263,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13182,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1367}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'14904","reported_seq":14978,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:19.672257+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:19:19.672257+0000","last_peered":"2024-09-11T01:19:19.672257+0000","last_clean":"2024-09-11T01:19:19.672257+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:19:19.672257+0000","last_undegraded":"2024-09-11T01:19:19.672257+0000","last_fullsized":"2024-09-11T01:19:19.672257+0000","mapping_epoch":35,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21422080,"num_objects":5230,"num_object_clones":0,"num_object_copies":10460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14904,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14927","reported_seq":15060,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:21.953973+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:19:21.953973+0000","last_peered":"2024-09-11T01:19:21.953973+0000","last_clean":"2024-09-11T01:19:21.953973+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:19:21.953973+0000","last_undegraded":"2024-09-11T01:19:21.953973+0000","last_fullsized":"2024-09-11T01:19:21.953973+0000","mapping_epoch":34,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20815872,"num_objects":5082,"num_object_clones":0,"num_object_copies":10164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11932,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14987","reported_seq":15162,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:21.998912+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:19:21.998912+0000","last_peered":"2024-09-11T01:19:21.998912+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:19:21.998912+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21463040,"num_objects":5240,"num_object_clones":0,"num_object_copies":10480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11328,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1289}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'14858","reported_seq":14993,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:21.927457+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:19:21.927457+0000","last_peered":"2024-09-11T01:19:21.927457+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:19:21.927457+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20774912,"num_objects":5072,"num_object_clones":0,"num_object_copies":10144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11715,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1086}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15105","reported_seq":15240,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:21.928014+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:19:21.928014+0000","last_peered":"2024-09-11T01:19:21.928014+0000","last_clean":"2024-09-11T01:19:21.928014+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:19:21.928014+0000","last_undegraded":"2024-09-11T01:19:21.928014+0000","last_fullsized":"2024-09-11T01:19:21.928014+0000","mapping_epoch":34,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21090304,"num_objects":5149,"num_object_clones":0,"num_object_copies":10298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11981,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934737+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934737+0000","last_peered":"2024-09-11T01:17:58.934737+0000","last_clean":"2024-09-11T01:17:58.934737+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934737+0000","last_undegraded":"2024-09-11T01:17:58.934737+0000","last_fullsized":"2024-09-11T01:17:58.934737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14813","reported_seq":18705,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:19.687759+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:19:19.687759+0000","last_peered":"2024-09-11T01:19:19.687759+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:19:19.687759+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21135360,"num_objects":5160,"num_object_clones":0,"num_object_copies":10320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2055,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12784,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1880,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3105}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14819","reported_seq":14953,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:21.973942+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:19:21.973942+0000","last_peered":"2024-09-11T01:19:21.973942+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:19:21.973942+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20680704,"num_objects":5049,"num_object_clones":0,"num_object_copies":10098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11406,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1065}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'14995","reported_seq":15118,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:19.637359+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:19:19.637359+0000","last_peered":"2024-09-11T01:19:19.637359+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:19:19.637359+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21348352,"num_objects":5212,"num_object_clones":0,"num_object_copies":10424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12812,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1188}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":84,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935080+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935080+0000","last_peered":"2024-09-11T01:17:58.935080+0000","last_clean":"2024-09-11T01:17:58.935080+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935080+0000","last_undegraded":"2024-09-11T01:17:58.935080+0000","last_fullsized":"2024-09-11T01:17:58.935080+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14669","reported_seq":22655,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:19.666194+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:19:19.666194+0000","last_peered":"2024-09-11T01:19:19.666194+0000","last_clean":"2024-09-11T01:19:19.666194+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:19:19.666194+0000","last_undegraded":"2024-09-11T01:19:19.666194+0000","last_fullsized":"2024-09-11T01:19:19.666194+0000","mapping_epoch":44,"log_start":"43'4600","ondisk_log_start":"43'4600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21000192,"num_objects":5127,"num_object_clones":0,"num_object_copies":10254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5127,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12454,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15051","reported_seq":15188,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:21.921370+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:19:21.921370+0000","last_peered":"2024-09-11T01:19:21.921370+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:19:21.921370+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21352448,"num_objects":5213,"num_object_clones":0,"num_object_copies":10426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5213,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11788,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1263}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938540+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938540+0000","last_peered":"2024-09-11T01:17:58.938540+0000","last_clean":"2024-09-11T01:17:58.938540+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938540+0000","last_undegraded":"2024-09-11T01:17:58.938540+0000","last_fullsized":"2024-09-11T01:17:58.938540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14834","reported_seq":22870,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:21.102737+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:19:21.102737+0000","last_peered":"2024-09-11T01:19:21.102737+0000","last_clean":"2024-09-11T01:19:21.102737+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:19:21.102737+0000","last_undegraded":"2024-09-11T01:19:21.102737+0000","last_fullsized":"2024-09-11T01:19:21.102737+0000","mapping_epoch":46,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20684800,"num_objects":5050,"num_object_clones":0,"num_object_copies":10100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11489,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15071","reported_seq":15211,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:22.021977+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:19:22.021977+0000","last_peered":"2024-09-11T01:19:22.021977+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:19:22.021977+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20910080,"num_objects":5105,"num_object_clones":0,"num_object_copies":10210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11680,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1066}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971314+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971314+0000","last_peered":"2024-09-11T01:17:58.971314+0000","last_clean":"2024-09-11T01:17:58.971314+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971314+0000","last_undegraded":"2024-09-11T01:17:58.971314+0000","last_fullsized":"2024-09-11T01:17:58.971314+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14899","reported_seq":15037,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:21.966905+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:19:21.966905+0000","last_peered":"2024-09-11T01:19:21.966905+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:19:21.966905+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20697705,"num_objects":5054,"num_object_clones":0,"num_object_copies":10108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5054,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11969,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1137}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":337293953,"num_objects":82349,"num_object_clones":0,"num_object_copies":164698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37889,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82349,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":199938,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9750,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":869580800,"data_stored":869510034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160788,"ondisk_log_size":160788,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182494,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":706040,"kb_used_data":309288,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93665800,"statfs":{"total":96636764160,"available":95913779200,"internally_reserved":0,"allocated":316710912,"data_stored":316441253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":406253227},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,1,3,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986266,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610900,"kb_used_data":294660,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93760940,"statfs":{"total":96636764160,"available":96011202560,"internally_reserved":0,"allocated":301731840,"data_stored":301462186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323808938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659211,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":589944,"kb_used_data":247336,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93781896,"statfs":{"total":96636764160,"available":96032661504,"internally_reserved":0,"allocated":253272064,"data_stored":252946148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":350809775},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,2,0,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":316207104,"data_stored":316203625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":252825600,"data_stored":252765864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":300548096,"data_stored":300540545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:25.908 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:19:25.908 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'15107", 'reported_seq': 15190, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:21.987289+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:19:21.987289+0000', 'last_peered': '2024-09-11T01:19:21.987289+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:19:21.987289+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "43'5100", 'ondisk_log_start': "43'5100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21131264, 'num_objects': 5159, 'num_object_clones': 0, 'num_object_copies': 10318, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5159, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15107, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1087}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:25.908 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:19:25.908 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'14881", 'reported_seq': 15015, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:19.574380+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:19:19.574380+0000', 'last_peered': '2024-09-11T01:19:19.574380+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:19:19.574380+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "43'4800", 'ondisk_log_start': "43'4800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21553176, 'num_objects': 5263, 'num_object_clones': 0, 'num_object_copies': 10526, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5263, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13182, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1367}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:25.909 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:19:25.909 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'14987", 'reported_seq': 15162, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:21.998912+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:19:21.998912+0000', 'last_peered': '2024-09-11T01:19:21.998912+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:19:21.998912+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "43'4900", 'ondisk_log_start': "43'4900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21463040, 'num_objects': 5240, 'num_object_clones': 0, 'num_object_copies': 10480, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5240, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11328, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1289}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:25.909 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:19:25.909 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'14858", 'reported_seq': 14993, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:21.927457+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:19:21.927457+0000', 'last_peered': '2024-09-11T01:19:21.927457+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:19:21.927457+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "43'4800", 'ondisk_log_start': "43'4800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20774912, 'num_objects': 5072, 'num_object_clones': 0, 'num_object_copies': 10144, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5072, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11715, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1086}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:25.909 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:19:25.909 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'14813", 'reported_seq': 18705, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:19.687759+0000', 'last_change': '2024-09-11T01:17:57.355893+0000', 'last_active': '2024-09-11T01:19:19.687759+0000', 'last_peered': '2024-09-11T01:19:19.687759+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:19:19.687759+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "43'4800", 'ondisk_log_start': "43'4800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21135360, 'num_objects': 5160, 'num_object_clones': 0, 'num_object_copies': 10320, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2055, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5160, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12784, 'num_write_kb': 31844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1880, 'num_bytes_recovered': 4505600, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3105}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:25.909 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:19:25.909 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'14819", 'reported_seq': 14953, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:21.973942+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:19:21.973942+0000', 'last_peered': '2024-09-11T01:19:21.973942+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:19:21.973942+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "43'4800", 'ondisk_log_start': "43'4800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20680704, 'num_objects': 5049, 'num_object_clones': 0, 'num_object_copies': 10098, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5049, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11406, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1065}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:25.909 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:19:25.910 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'14995", 'reported_seq': 15118, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:19.637359+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:19:19.637359+0000', 'last_peered': '2024-09-11T01:19:19.637359+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:19:19.637359+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "43'4900", 'ondisk_log_start': "43'4900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21348352, 'num_objects': 5212, 'num_object_clones': 0, 'num_object_copies': 10424, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5212, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12812, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1188}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:25.910 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:19:25.910 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'15051", 'reported_seq': 15188, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:21.921370+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:19:21.921370+0000', 'last_peered': '2024-09-11T01:19:21.921370+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:19:21.921370+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "43'5000", 'ondisk_log_start': "43'5000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 21352448, 'num_objects': 5213, 'num_object_clones': 0, 'num_object_copies': 10426, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5213, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11788, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1263}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:25.910 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:19:25.910 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'15071", 'reported_seq': 15211, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:22.021977+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:19:22.021977+0000', 'last_peered': '2024-09-11T01:19:22.021977+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:19:22.021977+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "43'5000", 'ondisk_log_start': "43'5000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20910080, 'num_objects': 5105, 'num_object_clones': 0, 'num_object_copies': 10210, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5105, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11680, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1066}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:25.910 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:19:25.910 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'14899", 'reported_seq': 15037, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:21.966905+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:19:21.966905+0000', 'last_peered': '2024-09-11T01:19:21.966905+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:19:21.966905+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "43'4800", 'ondisk_log_start': "43'4800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20697705, 'num_objects': 5054, 'num_object_clones': 0, 'num_object_copies': 10108, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5054, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11969, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1137}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:25.910 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:19:26.969 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:19:26.969 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37754/162744 objects degraded (23.198%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":503,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":81372,"data_bytes":333283988,"bytes_used":1954979840,"bytes_avail":287955312640,"bytes_total":289910292480,"degraded_objects":37754,"degraded_total":162744,"degraded_ratio":0.23198397483163741,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":270,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":389},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:19:27.054 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:19:27.054 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:27.782 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:19:27.785 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:19:27.844 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":292,"stamp":"2024-09-11T01:19:26.714428+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":333283988,"num_objects":81372,"num_object_clones":0,"num_object_copies":162744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37754,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81372,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":200919,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9886,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160770,"ondisk_log_size":160770,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1909160,"kb_used_data":841076,"kb_used_omap":12,"kb_used_meta":1068019,"kb_avail":281206360,"statfs":{"total":289910292480,"available":287955312640,"internally_reserved":0,"allocated":861261824,"data_stored":860396595,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1093651460},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,6,4,3,10,9],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19132416,"num_objects":-4671,"num_object_clones":0,"num_object_copies":-9342,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-270,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4671,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4671,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":270,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004215"},"pg_stats":[{"pgid":"2.f","version":"47'15107","reported_seq":15190,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:21.987289+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:19:21.987289+0000","last_peered":"2024-09-11T01:19:21.987289+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:19:21.987289+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21131264,"num_objects":5159,"num_object_clones":0,"num_object_copies":10318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15107,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1087}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15310","reported_seq":15417,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:24.662759+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:19:24.662759+0000","last_peered":"2024-09-11T01:19:24.662759+0000","last_clean":"2024-09-11T01:19:24.662759+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:19:24.662759+0000","last_undegraded":"2024-09-11T01:19:24.662759+0000","last_fullsized":"2024-09-11T01:19:24.662759+0000","mapping_epoch":35,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20652032,"num_objects":5042,"num_object_clones":0,"num_object_copies":10084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13549,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'14998","reported_seq":15125,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:24.629340+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:19:24.629340+0000","last_peered":"2024-09-11T01:19:24.629340+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:19:24.629340+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21073944,"num_objects":5146,"num_object_clones":0,"num_object_copies":10292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5146,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13299,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1250}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15054","reported_seq":15129,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:24.675595+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:19:24.675595+0000","last_peered":"2024-09-11T01:19:24.675595+0000","last_clean":"2024-09-11T01:19:24.675595+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:19:24.675595+0000","last_undegraded":"2024-09-11T01:19:24.675595+0000","last_fullsized":"2024-09-11T01:19:24.675595+0000","mapping_epoch":35,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20807680,"num_objects":5080,"num_object_clones":0,"num_object_copies":10160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15054,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'14927","reported_seq":15060,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:21.953973+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:19:21.953973+0000","last_peered":"2024-09-11T01:19:21.953973+0000","last_clean":"2024-09-11T01:19:21.953973+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:19:21.953973+0000","last_undegraded":"2024-09-11T01:19:21.953973+0000","last_fullsized":"2024-09-11T01:19:21.953973+0000","mapping_epoch":34,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20815872,"num_objects":5082,"num_object_clones":0,"num_object_copies":10164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11932,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'14987","reported_seq":15162,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:21.998912+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:19:21.998912+0000","last_peered":"2024-09-11T01:19:21.998912+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:19:21.998912+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21463040,"num_objects":5240,"num_object_clones":0,"num_object_copies":10480,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5240,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11328,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1289}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'14858","reported_seq":14993,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:21.927457+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:19:21.927457+0000","last_peered":"2024-09-11T01:19:21.927457+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:19:21.927457+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20774912,"num_objects":5072,"num_object_clones":0,"num_object_copies":10144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11715,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1086}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15105","reported_seq":15240,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:21.928014+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:19:21.928014+0000","last_peered":"2024-09-11T01:19:21.928014+0000","last_clean":"2024-09-11T01:19:21.928014+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:19:21.928014+0000","last_undegraded":"2024-09-11T01:19:21.928014+0000","last_fullsized":"2024-09-11T01:19:21.928014+0000","mapping_epoch":34,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21090304,"num_objects":5149,"num_object_clones":0,"num_object_copies":10298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5149,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11981,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934737+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934737+0000","last_peered":"2024-09-11T01:17:58.934737+0000","last_clean":"2024-09-11T01:17:58.934737+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934737+0000","last_undegraded":"2024-09-11T01:17:58.934737+0000","last_fullsized":"2024-09-11T01:17:58.934737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'14973","reported_seq":19137,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:24.722339+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:19:24.722339+0000","last_peered":"2024-09-11T01:19:24.722339+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:19:24.722339+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20480000,"num_objects":5000,"num_object_clones":0,"num_object_copies":10000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5000,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2015,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3080}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14819","reported_seq":14953,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:21.973942+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:19:21.973942+0000","last_peered":"2024-09-11T01:19:21.973942+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:19:21.973942+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20680704,"num_objects":5049,"num_object_clones":0,"num_object_copies":10098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11406,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1065}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15137","reported_seq":15261,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:24.584983+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:19:24.584983+0000","last_peered":"2024-09-11T01:19:24.584983+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:19:24.584983+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20766720,"num_objects":5070,"num_object_clones":0,"num_object_copies":10140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5070,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12954,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1046}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935141+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935141+0000","last_peered":"2024-09-11T01:17:58.935141+0000","last_clean":"2024-09-11T01:17:58.935141+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935141+0000","last_undegraded":"2024-09-11T01:17:58.935141+0000","last_fullsized":"2024-09-11T01:17:58.935141+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14813","reported_seq":22796,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:24.624809+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:19:24.624809+0000","last_peered":"2024-09-11T01:19:24.624809+0000","last_clean":"2024-09-11T01:19:24.624809+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:19:24.624809+0000","last_undegraded":"2024-09-11T01:19:24.624809+0000","last_fullsized":"2024-09-11T01:19:24.624809+0000","mapping_epoch":44,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20410368,"num_objects":4983,"num_object_clones":0,"num_object_copies":9966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4983,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12598,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15051","reported_seq":15188,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:21.921370+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:19:21.921370+0000","last_peered":"2024-09-11T01:19:21.921370+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:19:21.921370+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":21352448,"num_objects":5213,"num_object_clones":0,"num_object_copies":10426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5213,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11788,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1263}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938599+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938599+0000","last_peered":"2024-09-11T01:17:58.938599+0000","last_clean":"2024-09-11T01:17:58.938599+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938599+0000","last_undegraded":"2024-09-11T01:17:58.938599+0000","last_fullsized":"2024-09-11T01:17:58.938599+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14958","reported_seq":22992,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:26.239676+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:19:26.239676+0000","last_peered":"2024-09-11T01:19:26.239676+0000","last_clean":"2024-09-11T01:19:26.239676+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:19:26.239676+0000","last_undegraded":"2024-09-11T01:19:26.239676+0000","last_fullsized":"2024-09-11T01:19:26.239676+0000","mapping_epoch":46,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20176896,"num_objects":4926,"num_object_clones":0,"num_object_copies":9852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11613,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15071","reported_seq":15211,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:22.021977+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:19:22.021977+0000","last_peered":"2024-09-11T01:19:22.021977+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:19:22.021977+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20910080,"num_objects":5105,"num_object_clones":0,"num_object_copies":10210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11680,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1066}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971371+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971371+0000","last_peered":"2024-09-11T01:17:58.971371+0000","last_clean":"2024-09-11T01:17:58.971371+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971371+0000","last_undegraded":"2024-09-11T01:17:58.971371+0000","last_fullsized":"2024-09-11T01:17:58.971371+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'14899","reported_seq":15037,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:21.966905+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:19:21.966905+0000","last_peered":"2024-09-11T01:19:21.966905+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:19:21.966905+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'4800","ondisk_log_start":"43'4800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20697705,"num_objects":5054,"num_object_clones":0,"num_object_copies":10108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5054,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11969,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1137}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":333283969,"num_objects":81370,"num_object_clones":0,"num_object_copies":162740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37754,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81370,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":200917,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9885,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":857653248,"data_stored":857582482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160767,"ondisk_log_size":160767,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182494,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":706040,"kb_used_data":309288,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93665800,"statfs":{"total":96636764160,"available":95913779200,"internally_reserved":0,"allocated":316710912,"data_stored":316441253,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":406253227},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,1,3,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986267,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610488,"kb_used_data":290084,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93761352,"statfs":{"total":96636764160,"available":96011624448,"internally_reserved":0,"allocated":297046016,"data_stored":296776362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328068778},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,0,0,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659212,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592632,"kb_used_data":241704,"kb_used_omap":4,"kb_used_meta":350907,"kb_avail":93779208,"statfs":{"total":96636764160,"available":96029908992,"internally_reserved":0,"allocated":247504896,"data_stored":247178980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":359329455},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,2,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":316207104,"data_stored":316203625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":245633024,"data_stored":245573288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":295813120,"data_stored":295805569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:30.846 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:31.642 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:19:31.642 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:19:31.694 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-09-11T01:19:30.715396+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":324244116,"num_objects":79165,"num_object_clones":0,"num_object_copies":158330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37614,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79165,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":203126,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10026,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160977,"ondisk_log_size":160977,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1910292,"kb_used_data":829732,"kb_used_omap":12,"kb_used_meta":1080499,"kb_avail":281205228,"statfs":{"total":289910292480,"available":287954153472,"internally_reserved":0,"allocated":849645568,"data_stored":848780339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1106430980},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,1,2,3,14,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22376448,"num_objects":-5463,"num_object_clones":0,"num_object_copies":-10926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-410,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5463,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":410,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002835"},"pg_stats":[{"pgid":"2.f","version":"47'15261","reported_seq":15344,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:27.094394+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:19:27.094394+0000","last_peered":"2024-09-11T01:19:27.094394+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:19:27.094394+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20500480,"num_objects":5005,"num_object_clones":0,"num_object_copies":10010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15261,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":933}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15456","reported_seq":15563,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:29.750676+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:19:29.750676+0000","last_peered":"2024-09-11T01:19:29.750676+0000","last_clean":"2024-09-11T01:19:29.750676+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:19:29.750676+0000","last_undegraded":"2024-09-11T01:19:29.750676+0000","last_fullsized":"2024-09-11T01:19:29.750676+0000","mapping_epoch":35,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20054016,"num_objects":4896,"num_object_clones":0,"num_object_copies":9792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13695,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15130","reported_seq":15264,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:29.763612+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:19:29.763612+0000","last_peered":"2024-09-11T01:19:29.763612+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:19:29.763612+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20533272,"num_objects":5014,"num_object_clones":0,"num_object_copies":10028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5014,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13431,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1118}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15198","reported_seq":15273,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:29.801160+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:19:29.801160+0000","last_peered":"2024-09-11T01:19:29.801160+0000","last_clean":"2024-09-11T01:19:29.801160+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:19:29.801160+0000","last_undegraded":"2024-09-11T01:19:29.801160+0000","last_fullsized":"2024-09-11T01:19:29.801160+0000","mapping_epoch":35,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20217856,"num_objects":4936,"num_object_clones":0,"num_object_copies":9872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15198,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15065","reported_seq":15198,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:27.004869+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:19:27.004869+0000","last_peered":"2024-09-11T01:19:27.004869+0000","last_clean":"2024-09-11T01:19:27.004869+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:19:27.004869+0000","last_undegraded":"2024-09-11T01:19:27.004869+0000","last_fullsized":"2024-09-11T01:19:27.004869+0000","mapping_epoch":34,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20250624,"num_objects":4944,"num_object_clones":0,"num_object_copies":9888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4944,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12070,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15134","reported_seq":15308,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:27.045989+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:19:27.045989+0000","last_peered":"2024-09-11T01:19:27.045989+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:19:27.045989+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20860928,"num_objects":5093,"num_object_clones":0,"num_object_copies":10186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11475,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1142}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'14999","reported_seq":15133,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:27.056859+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:19:27.056859+0000","last_peered":"2024-09-11T01:19:27.056859+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:19:27.056859+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20197376,"num_objects":4931,"num_object_clones":0,"num_object_copies":9862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11856,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":945}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15266","reported_seq":15401,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:26.866278+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:19:26.866278+0000","last_peered":"2024-09-11T01:19:26.866278+0000","last_clean":"2024-09-11T01:19:26.866278+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:19:26.866278+0000","last_undegraded":"2024-09-11T01:19:26.866278+0000","last_fullsized":"2024-09-11T01:19:26.866278+0000","mapping_epoch":34,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20430848,"num_objects":4988,"num_object_clones":0,"num_object_copies":9976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12142,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934737+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934737+0000","last_peered":"2024-09-11T01:17:58.934737+0000","last_clean":"2024-09-11T01:17:58.934737+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934737+0000","last_undegraded":"2024-09-11T01:17:58.934737+0000","last_fullsized":"2024-09-11T01:17:58.934737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15112","reported_seq":19555,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:29.763201+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:19:29.763201+0000","last_peered":"2024-09-11T01:19:29.763201+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:19:29.763201+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19910656,"num_objects":4861,"num_object_clones":0,"num_object_copies":9722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1780,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13083,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2155,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3081}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14965","reported_seq":15094,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:27.046944+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:19:27.046944+0000","last_peered":"2024-09-11T01:19:27.046944+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:19:27.046944+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20082688,"num_objects":4903,"num_object_clones":0,"num_object_copies":9806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11552,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":919}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15290","reported_seq":15412,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:29.714122+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:19:29.714122+0000","last_peered":"2024-09-11T01:19:29.714122+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:19:29.714122+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20140032,"num_objects":4917,"num_object_clones":0,"num_object_copies":9834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13107,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":893}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935141+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935141+0000","last_peered":"2024-09-11T01:17:58.935141+0000","last_clean":"2024-09-11T01:17:58.935141+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935141+0000","last_undegraded":"2024-09-11T01:17:58.935141+0000","last_fullsized":"2024-09-11T01:17:58.935141+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14984","reported_seq":22970,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:29.731756+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:19:29.731756+0000","last_peered":"2024-09-11T01:19:29.731756+0000","last_clean":"2024-09-11T01:19:29.731756+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:19:29.731756+0000","last_undegraded":"2024-09-11T01:19:29.731756+0000","last_fullsized":"2024-09-11T01:19:29.731756+0000","mapping_epoch":44,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19709952,"num_objects":4812,"num_object_clones":0,"num_object_copies":9624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4812,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12769,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15187","reported_seq":15324,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:27.018373+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:19:27.018373+0000","last_peered":"2024-09-11T01:19:27.018373+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:19:27.018373+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20795392,"num_objects":5077,"num_object_clones":0,"num_object_copies":10154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11924,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1127}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938599+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938599+0000","last_peered":"2024-09-11T01:17:58.938599+0000","last_clean":"2024-09-11T01:17:58.938599+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938599+0000","last_undegraded":"2024-09-11T01:17:58.938599+0000","last_fullsized":"2024-09-11T01:17:58.938599+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14958","reported_seq":22992,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:26.239676+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:19:26.239676+0000","last_peered":"2024-09-11T01:19:26.239676+0000","last_clean":"2024-09-11T01:19:26.239676+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:19:26.239676+0000","last_undegraded":"2024-09-11T01:19:26.239676+0000","last_fullsized":"2024-09-11T01:19:26.239676+0000","mapping_epoch":46,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20176896,"num_objects":4926,"num_object_clones":0,"num_object_copies":9852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11613,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15216","reported_seq":15356,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:27.057374+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:19:27.057374+0000","last_peered":"2024-09-11T01:19:27.057374+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:19:27.057374+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20316160,"num_objects":4960,"num_object_clones":0,"num_object_copies":9920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11825,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":921}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971371+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971371+0000","last_peered":"2024-09-11T01:17:58.971371+0000","last_clean":"2024-09-11T01:17:58.971371+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971371+0000","last_undegraded":"2024-09-11T01:17:58.971371+0000","last_fullsized":"2024-09-11T01:17:58.971371+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15053","reported_seq":15192,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:27.055968+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:19:27.055968+0000","last_peered":"2024-09-11T01:19:27.055968+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:19:27.055968+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20066921,"num_objects":4900,"num_object_clones":0,"num_object_copies":9800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12123,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":983}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":324244097,"num_objects":79163,"num_object_clones":0,"num_object_copies":158326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37614,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79163,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":203124,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10025,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":845750272,"data_stored":845679506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160974,"ondisk_log_size":160974,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182495,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":707688,"kb_used_data":302616,"kb_used_omap":4,"kb_used_meta":405051,"kb_avail":93664152,"statfs":{"total":96636764160,"available":95912091648,"internally_reserved":0,"allocated":309878784,"data_stored":309609125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":414772907},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,0,0,2,8,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986268,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609972,"kb_used_data":285412,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93761868,"statfs":{"total":96636764160,"available":96012152832,"internally_reserved":0,"allocated":292261888,"data_stored":291992234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332328618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,4,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659212,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592632,"kb_used_data":241704,"kb_used_omap":4,"kb_used_meta":350907,"kb_avail":93779208,"statfs":{"total":96636764160,"available":96029908992,"internally_reserved":0,"allocated":247504896,"data_stored":247178980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":359329455},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,2,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":309121024,"data_stored":309117545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":245633024,"data_stored":245573288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":290996224,"data_stored":290988673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:31.696 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:32.662 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:19:32.665 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:19:32.707 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-09-11T01:19:30.715396+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":324244116,"num_objects":79165,"num_object_clones":0,"num_object_copies":158330,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37614,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79165,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":203126,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10026,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160977,"ondisk_log_size":160977,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1910292,"kb_used_data":829732,"kb_used_omap":12,"kb_used_meta":1080499,"kb_avail":281205228,"statfs":{"total":289910292480,"available":287954153472,"internally_reserved":0,"allocated":849645568,"data_stored":848780339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1106430980},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,1,2,3,14,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22376448,"num_objects":-5463,"num_object_clones":0,"num_object_copies":-10926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-410,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5463,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":410,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002835"},"pg_stats":[{"pgid":"2.f","version":"47'15261","reported_seq":15344,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:27.094394+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:19:27.094394+0000","last_peered":"2024-09-11T01:19:27.094394+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:19:27.094394+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20500480,"num_objects":5005,"num_object_clones":0,"num_object_copies":10010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5005,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15261,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":933}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15456","reported_seq":15563,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:29.750676+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:19:29.750676+0000","last_peered":"2024-09-11T01:19:29.750676+0000","last_clean":"2024-09-11T01:19:29.750676+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:19:29.750676+0000","last_undegraded":"2024-09-11T01:19:29.750676+0000","last_fullsized":"2024-09-11T01:19:29.750676+0000","mapping_epoch":35,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20054016,"num_objects":4896,"num_object_clones":0,"num_object_copies":9792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13695,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15130","reported_seq":15264,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:29.763612+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:19:29.763612+0000","last_peered":"2024-09-11T01:19:29.763612+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:19:29.763612+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20533272,"num_objects":5014,"num_object_clones":0,"num_object_copies":10028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5014,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13431,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1118}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15198","reported_seq":15273,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:29.801160+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:19:29.801160+0000","last_peered":"2024-09-11T01:19:29.801160+0000","last_clean":"2024-09-11T01:19:29.801160+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:19:29.801160+0000","last_undegraded":"2024-09-11T01:19:29.801160+0000","last_fullsized":"2024-09-11T01:19:29.801160+0000","mapping_epoch":35,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20217856,"num_objects":4936,"num_object_clones":0,"num_object_copies":9872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15198,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15065","reported_seq":15198,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:27.004869+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:19:27.004869+0000","last_peered":"2024-09-11T01:19:27.004869+0000","last_clean":"2024-09-11T01:19:27.004869+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:19:27.004869+0000","last_undegraded":"2024-09-11T01:19:27.004869+0000","last_fullsized":"2024-09-11T01:19:27.004869+0000","mapping_epoch":34,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20250624,"num_objects":4944,"num_object_clones":0,"num_object_copies":9888,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4944,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12070,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15134","reported_seq":15308,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:27.045989+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:19:27.045989+0000","last_peered":"2024-09-11T01:19:27.045989+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:19:27.045989+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20860928,"num_objects":5093,"num_object_clones":0,"num_object_copies":10186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5093,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11475,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1142}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'14999","reported_seq":15133,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:27.056859+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:19:27.056859+0000","last_peered":"2024-09-11T01:19:27.056859+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:19:27.056859+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20197376,"num_objects":4931,"num_object_clones":0,"num_object_copies":9862,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4931,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11856,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":945}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15266","reported_seq":15401,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:26.866278+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:19:26.866278+0000","last_peered":"2024-09-11T01:19:26.866278+0000","last_clean":"2024-09-11T01:19:26.866278+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:19:26.866278+0000","last_undegraded":"2024-09-11T01:19:26.866278+0000","last_fullsized":"2024-09-11T01:19:26.866278+0000","mapping_epoch":34,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20430848,"num_objects":4988,"num_object_clones":0,"num_object_copies":9976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4988,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12142,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934737+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934737+0000","last_peered":"2024-09-11T01:17:58.934737+0000","last_clean":"2024-09-11T01:17:58.934737+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934737+0000","last_undegraded":"2024-09-11T01:17:58.934737+0000","last_fullsized":"2024-09-11T01:17:58.934737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15112","reported_seq":19555,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:29.763201+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:19:29.763201+0000","last_peered":"2024-09-11T01:19:29.763201+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:19:29.763201+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19910656,"num_objects":4861,"num_object_clones":0,"num_object_copies":9722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1780,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13083,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2155,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3081}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'14965","reported_seq":15094,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:27.046944+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:19:27.046944+0000","last_peered":"2024-09-11T01:19:27.046944+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:19:27.046944+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20082688,"num_objects":4903,"num_object_clones":0,"num_object_copies":9806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11552,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":919}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15290","reported_seq":15412,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:29.714122+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:19:29.714122+0000","last_peered":"2024-09-11T01:19:29.714122+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:19:29.714122+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20140032,"num_objects":4917,"num_object_clones":0,"num_object_copies":9834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13107,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":893}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935141+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935141+0000","last_peered":"2024-09-11T01:17:58.935141+0000","last_clean":"2024-09-11T01:17:58.935141+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935141+0000","last_undegraded":"2024-09-11T01:17:58.935141+0000","last_fullsized":"2024-09-11T01:17:58.935141+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14984","reported_seq":22970,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:29.731756+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:19:29.731756+0000","last_peered":"2024-09-11T01:19:29.731756+0000","last_clean":"2024-09-11T01:19:29.731756+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:19:29.731756+0000","last_undegraded":"2024-09-11T01:19:29.731756+0000","last_fullsized":"2024-09-11T01:19:29.731756+0000","mapping_epoch":44,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19709952,"num_objects":4812,"num_object_clones":0,"num_object_copies":9624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4812,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12769,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15187","reported_seq":15324,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:27.018373+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:19:27.018373+0000","last_peered":"2024-09-11T01:19:27.018373+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:19:27.018373+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20795392,"num_objects":5077,"num_object_clones":0,"num_object_copies":10154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11924,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":1127}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938599+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938599+0000","last_peered":"2024-09-11T01:17:58.938599+0000","last_clean":"2024-09-11T01:17:58.938599+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938599+0000","last_undegraded":"2024-09-11T01:17:58.938599+0000","last_fullsized":"2024-09-11T01:17:58.938599+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'14958","reported_seq":22992,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:26.239676+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:19:26.239676+0000","last_peered":"2024-09-11T01:19:26.239676+0000","last_clean":"2024-09-11T01:19:26.239676+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:19:26.239676+0000","last_undegraded":"2024-09-11T01:19:26.239676+0000","last_fullsized":"2024-09-11T01:19:26.239676+0000","mapping_epoch":46,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20176896,"num_objects":4926,"num_object_clones":0,"num_object_copies":9852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11613,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15216","reported_seq":15356,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:27.057374+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:19:27.057374+0000","last_peered":"2024-09-11T01:19:27.057374+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:19:27.057374+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20316160,"num_objects":4960,"num_object_clones":0,"num_object_copies":9920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11825,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":921}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971371+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971371+0000","last_peered":"2024-09-11T01:17:58.971371+0000","last_clean":"2024-09-11T01:17:58.971371+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971371+0000","last_undegraded":"2024-09-11T01:17:58.971371+0000","last_fullsized":"2024-09-11T01:17:58.971371+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15053","reported_seq":15192,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:27.055968+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:19:27.055968+0000","last_peered":"2024-09-11T01:19:27.055968+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:19:27.055968+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'5000","ondisk_log_start":"43'5000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20066921,"num_objects":4900,"num_object_clones":0,"num_object_copies":9800,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4900,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12123,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":983}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":324244097,"num_objects":79163,"num_object_clones":0,"num_object_copies":158326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37614,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79163,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":203124,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10025,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":845750272,"data_stored":845679506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160974,"ondisk_log_size":160974,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182495,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":707688,"kb_used_data":302616,"kb_used_omap":4,"kb_used_meta":405051,"kb_avail":93664152,"statfs":{"total":96636764160,"available":95912091648,"internally_reserved":0,"allocated":309878784,"data_stored":309609125,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":414772907},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,0,0,2,8,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986268,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609972,"kb_used_data":285412,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93761868,"statfs":{"total":96636764160,"available":96012152832,"internally_reserved":0,"allocated":292261888,"data_stored":291992234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332328618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,4,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659212,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592632,"kb_used_data":241704,"kb_used_omap":4,"kb_used_meta":350907,"kb_avail":93779208,"statfs":{"total":96636764160,"available":96029908992,"internally_reserved":0,"allocated":247504896,"data_stored":247178980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":359329455},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,2,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":309121024,"data_stored":309117545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":245633024,"data_stored":245573288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":290996224,"data_stored":290988673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:32.709 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:19:32.709 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'15261", 'reported_seq': 15344, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:27.094394+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:19:27.094394+0000', 'last_peered': '2024-09-11T01:19:27.094394+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:19:27.094394+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "43'5200", 'ondisk_log_start': "43'5200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20500480, 'num_objects': 5005, 'num_object_clones': 0, 'num_object_copies': 10010, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5005, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15261, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 933}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:32.709 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:19:32.709 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'15130", 'reported_seq': 15264, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:29.763612+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:19:29.763612+0000', 'last_peered': '2024-09-11T01:19:29.763612+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:19:29.763612+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "43'5100", 'ondisk_log_start': "43'5100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20533272, 'num_objects': 5014, 'num_object_clones': 0, 'num_object_copies': 10028, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5014, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13431, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 1118}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:32.709 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:19:32.710 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'15134", 'reported_seq': 15308, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:27.045989+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:19:27.045989+0000', 'last_peered': '2024-09-11T01:19:27.045989+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:19:27.045989+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "43'5100", 'ondisk_log_start': "43'5100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20860928, 'num_objects': 5093, 'num_object_clones': 0, 'num_object_copies': 10186, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5093, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11475, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1142}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:32.710 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:19:32.710 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'14999", 'reported_seq': 15133, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:27.056859+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:19:27.056859+0000', 'last_peered': '2024-09-11T01:19:27.056859+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:19:27.056859+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "43'4900", 'ondisk_log_start': "43'4900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20197376, 'num_objects': 4931, 'num_object_clones': 0, 'num_object_copies': 9862, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4931, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11856, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 945}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:32.710 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:19:32.710 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'15112", 'reported_seq': 19555, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:29.763201+0000', 'last_change': '2024-09-11T01:17:57.355893+0000', 'last_active': '2024-09-11T01:19:29.763201+0000', 'last_peered': '2024-09-11T01:19:29.763201+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:19:29.763201+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "43'5100", 'ondisk_log_start': "43'5100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19910656, 'num_objects': 4861, 'num_object_clones': 0, 'num_object_copies': 9722, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1780, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4861, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13083, 'num_write_kb': 31844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2155, 'num_bytes_recovered': 4505600, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3081}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:32.710 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:19:32.710 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'14965", 'reported_seq': 15094, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:27.046944+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:19:27.046944+0000', 'last_peered': '2024-09-11T01:19:27.046944+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:19:27.046944+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "43'4900", 'ondisk_log_start': "43'4900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20082688, 'num_objects': 4903, 'num_object_clones': 0, 'num_object_copies': 9806, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4903, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11552, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 919}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:32.710 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:19:32.710 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'15290", 'reported_seq': 15412, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:29.714122+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:19:29.714122+0000', 'last_peered': '2024-09-11T01:19:29.714122+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:19:29.714122+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "43'5200", 'ondisk_log_start': "43'5200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20140032, 'num_objects': 4917, 'num_object_clones': 0, 'num_object_copies': 9834, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4917, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13107, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 893}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:32.710 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:19:32.710 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'15187", 'reported_seq': 15324, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:27.018373+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:19:27.018373+0000', 'last_peered': '2024-09-11T01:19:27.018373+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:19:27.018373+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "43'5100", 'ondisk_log_start': "43'5100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20795392, 'num_objects': 5077, 'num_object_clones': 0, 'num_object_copies': 10154, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 5077, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11924, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 1127}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:32.711 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:19:32.711 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'15216", 'reported_seq': 15356, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:27.057374+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:19:27.057374+0000', 'last_peered': '2024-09-11T01:19:27.057374+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:19:27.057374+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "43'5200", 'ondisk_log_start': "43'5200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20316160, 'num_objects': 4960, 'num_object_clones': 0, 'num_object_copies': 9920, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4960, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11825, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 921}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:32.711 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:19:32.711 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'15053", 'reported_seq': 15192, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:27.055968+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:19:27.055968+0000', 'last_peered': '2024-09-11T01:19:27.055968+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:19:27.055968+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "43'5000", 'ondisk_log_start': "43'5000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20066921, 'num_objects': 4900, 'num_object_clones': 0, 'num_object_copies': 9800, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4900, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12123, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 983}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:32.711 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:19:33.539 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:19:33.539 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37614/155432 objects degraded (24.200%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":510,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":77716,"data_bytes":318309012,"bytes_used":1919139840,"bytes_avail":287991152640,"bytes_total":289910292480,"degraded_objects":37614,"degraded_total":155432,"degraded_ratio":0.24199650007720419,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":275,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":503},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:19:33.590 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:19:33.590 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:34.335 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:19:34.337 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:19:34.385 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":295,"stamp":"2024-09-11T01:19:32.715861+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":318309012,"num_objects":77716,"num_object_clones":0,"num_object_copies":155432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37614,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77716,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":204575,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10026,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160626,"ondisk_log_size":160626,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1874160,"kb_used_data":814392,"kb_used_omap":12,"kb_used_meta":1059699,"kb_avail":281241360,"statfs":{"total":289910292480,"available":287991152640,"internally_reserved":0,"allocated":833937408,"data_stored":833072179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1085131780},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,5,10,8,7,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24743936,"num_objects":-6041,"num_object_clones":0,"num_object_copies":-12082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-275,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6041,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":275,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002825"},"pg_stats":[{"pgid":"2.f","version":"47'15411","reported_seq":15494,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:32.043274+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:19:32.043274+0000","last_peered":"2024-09-11T01:19:32.043274+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:19:32.043274+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19886080,"num_objects":4855,"num_object_clones":0,"num_object_copies":9710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15411,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":783}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15456","reported_seq":15563,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:29.750676+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:19:29.750676+0000","last_peered":"2024-09-11T01:19:29.750676+0000","last_clean":"2024-09-11T01:19:29.750676+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:19:29.750676+0000","last_undegraded":"2024-09-11T01:19:29.750676+0000","last_fullsized":"2024-09-11T01:19:29.750676+0000","mapping_epoch":35,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20054016,"num_objects":4896,"num_object_clones":0,"num_object_copies":9792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13695,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15130","reported_seq":15264,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:29.763612+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:19:29.763612+0000","last_peered":"2024-09-11T01:19:29.763612+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:19:29.763612+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20533272,"num_objects":5014,"num_object_clones":0,"num_object_copies":10028,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5014,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13431,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":1118}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15198","reported_seq":15273,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:29.801160+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:19:29.801160+0000","last_peered":"2024-09-11T01:19:29.801160+0000","last_clean":"2024-09-11T01:19:29.801160+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:19:29.801160+0000","last_undegraded":"2024-09-11T01:19:29.801160+0000","last_fullsized":"2024-09-11T01:19:29.801160+0000","mapping_epoch":35,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20217856,"num_objects":4936,"num_object_clones":0,"num_object_copies":9872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15198,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15219","reported_seq":15352,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:32.091906+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:19:32.091906+0000","last_peered":"2024-09-11T01:19:32.091906+0000","last_clean":"2024-09-11T01:19:32.091906+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:19:32.091906+0000","last_undegraded":"2024-09-11T01:19:32.091906+0000","last_fullsized":"2024-09-11T01:19:32.091906+0000","mapping_epoch":34,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19619840,"num_objects":4790,"num_object_clones":0,"num_object_copies":9580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4790,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12224,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15277","reported_seq":15452,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:32.135809+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:19:32.135809+0000","last_peered":"2024-09-11T01:19:32.135809+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:19:32.135809+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20275200,"num_objects":4950,"num_object_clones":0,"num_object_copies":9900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11618,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":999}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15135","reported_seq":15269,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:32.115114+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:19:32.115114+0000","last_peered":"2024-09-11T01:19:32.115114+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:19:32.115114+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19640320,"num_objects":4795,"num_object_clones":0,"num_object_copies":9590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11992,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":809}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15409","reported_seq":15544,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:32.072918+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:19:32.072918+0000","last_peered":"2024-09-11T01:19:32.072918+0000","last_clean":"2024-09-11T01:19:32.072918+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:19:32.072918+0000","last_undegraded":"2024-09-11T01:19:32.072918+0000","last_fullsized":"2024-09-11T01:19:32.072918+0000","mapping_epoch":34,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19845120,"num_objects":4845,"num_object_clones":0,"num_object_copies":9690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12285,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934737+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934737+0000","last_peered":"2024-09-11T01:17:58.934737+0000","last_clean":"2024-09-11T01:17:58.934737+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934737+0000","last_undegraded":"2024-09-11T01:17:58.934737+0000","last_fullsized":"2024-09-11T01:17:58.934737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15112","reported_seq":19555,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:29.763201+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:19:29.763201+0000","last_peered":"2024-09-11T01:19:29.763201+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:19:29.763201+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19910656,"num_objects":4861,"num_object_clones":0,"num_object_copies":9722,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1780,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4861,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13083,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2155,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3081}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15108","reported_seq":15242,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:32.034465+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:19:32.034465+0000","last_peered":"2024-09-11T01:19:32.034465+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:19:32.034465+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19496960,"num_objects":4760,"num_object_clones":0,"num_object_copies":9520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11695,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":776}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15290","reported_seq":15412,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:29.714122+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:19:29.714122+0000","last_peered":"2024-09-11T01:19:29.714122+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:19:29.714122+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20140032,"num_objects":4917,"num_object_clones":0,"num_object_copies":9834,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4917,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13107,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":893}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935141+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935141+0000","last_peered":"2024-09-11T01:17:58.935141+0000","last_clean":"2024-09-11T01:17:58.935141+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935141+0000","last_undegraded":"2024-09-11T01:17:58.935141+0000","last_fullsized":"2024-09-11T01:17:58.935141+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'14984","reported_seq":22970,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:29.731756+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:19:29.731756+0000","last_peered":"2024-09-11T01:19:29.731756+0000","last_clean":"2024-09-11T01:19:29.731756+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:19:29.731756+0000","last_undegraded":"2024-09-11T01:19:29.731756+0000","last_fullsized":"2024-09-11T01:19:29.731756+0000","mapping_epoch":44,"log_start":"43'4900","ondisk_log_start":"43'4900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19709952,"num_objects":4812,"num_object_clones":0,"num_object_copies":9624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4812,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12769,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15326","reported_seq":15463,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:32.091258+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:19:32.091258+0000","last_peered":"2024-09-11T01:19:32.091258+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:19:32.091258+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20226048,"num_objects":4938,"num_object_clones":0,"num_object_copies":9876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12063,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":988}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938599+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938599+0000","last_peered":"2024-09-11T01:17:58.938599+0000","last_clean":"2024-09-11T01:17:58.938599+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938599+0000","last_undegraded":"2024-09-11T01:17:58.938599+0000","last_fullsized":"2024-09-11T01:17:58.938599+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15103","reported_seq":23139,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:31.278371+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:19:31.278371+0000","last_peered":"2024-09-11T01:19:31.278371+0000","last_clean":"2024-09-11T01:19:31.278371+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:19:31.278371+0000","last_undegraded":"2024-09-11T01:19:31.278371+0000","last_fullsized":"2024-09-11T01:19:31.278371+0000","mapping_epoch":46,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19582976,"num_objects":4781,"num_object_clones":0,"num_object_copies":9562,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4781,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11758,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15351","reported_seq":15491,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:32.041654+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:19:32.041654+0000","last_peered":"2024-09-11T01:19:32.041654+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:19:32.041654+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19763200,"num_objects":4825,"num_object_clones":0,"num_object_copies":9650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4825,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11960,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":786}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971371+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971371+0000","last_peered":"2024-09-11T01:17:58.971371+0000","last_clean":"2024-09-11T01:17:58.971371+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971371+0000","last_undegraded":"2024-09-11T01:17:58.971371+0000","last_fullsized":"2024-09-11T01:17:58.971371+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15214","reported_seq":15351,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:32.058090+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:19:32.058090+0000","last_peered":"2024-09-11T01:19:32.058090+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:19:32.058090+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19407465,"num_objects":4739,"num_object_clones":0,"num_object_copies":9478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12284,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":822}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":318308993,"num_objects":77714,"num_object_clones":0,"num_object_copies":155428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37614,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77714,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":204573,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10025,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":831373312,"data_stored":831302546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160623,"ondisk_log_size":160623,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182496,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":688304,"kb_used_data":295712,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93683536,"statfs":{"total":96636764160,"available":95931940864,"internally_reserved":0,"allocated":302809088,"data_stored":302539429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":401993387},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,5,2,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986268,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609972,"kb_used_data":285412,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93761868,"statfs":{"total":96636764160,"available":96012152832,"internally_reserved":0,"allocated":292261888,"data_stored":291992234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332328618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,4,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659213,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":575884,"kb_used_data":233268,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93795956,"statfs":{"total":96636764160,"available":96047058944,"internally_reserved":0,"allocated":238866432,"data_stored":238540516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":350809775},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,5,2,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":301998080,"data_stored":301994601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":238379008,"data_stored":238319272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":290996224,"data_stored":290988673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:37.387 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:38.257 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:19:38.257 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:19:38.294 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":297,"stamp":"2024-09-11T01:19:36.716798+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":314282644,"num_objects":76733,"num_object_clones":0,"num_object_copies":153466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37476,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76733,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":205558,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10164,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160609,"ondisk_log_size":160609,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1871068,"kb_used_data":802984,"kb_used_omap":12,"kb_used_meta":1068019,"kb_avail":281244452,"statfs":{"total":289910292480,"available":287994318848,"internally_reserved":0,"allocated":822255616,"data_stored":821390387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1093651460},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,4,6,4,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-23011328,"num_objects":-5618,"num_object_clones":0,"num_object_copies":-11236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-413,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5618,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5618,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":413,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002796"},"pg_stats":[{"pgid":"2.f","version":"47'15411","reported_seq":15494,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:32.043274+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:19:32.043274+0000","last_peered":"2024-09-11T01:19:32.043274+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:19:32.043274+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19886080,"num_objects":4855,"num_object_clones":0,"num_object_copies":9710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4855,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15411,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":783}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15582","reported_seq":15689,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:34.651398+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:19:34.651398+0000","last_peered":"2024-09-11T01:19:34.651398+0000","last_clean":"2024-09-11T01:19:34.651398+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:19:34.651398+0000","last_undegraded":"2024-09-11T01:19:34.651398+0000","last_fullsized":"2024-09-11T01:19:34.651398+0000","mapping_epoch":35,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19537920,"num_objects":4770,"num_object_clones":0,"num_object_copies":9540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13821,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15256","reported_seq":15390,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:34.709877+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:19:34.709877+0000","last_peered":"2024-09-11T01:19:34.709877+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:19:34.709877+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20017176,"num_objects":4888,"num_object_clones":0,"num_object_copies":9776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4888,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13557,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":992}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15341","reported_seq":15415,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:34.782980+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:19:34.782980+0000","last_peered":"2024-09-11T01:19:34.782980+0000","last_clean":"2024-09-11T01:19:34.782980+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:19:34.782980+0000","last_undegraded":"2024-09-11T01:19:34.782980+0000","last_fullsized":"2024-09-11T01:19:34.782980+0000","mapping_epoch":35,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19632128,"num_objects":4793,"num_object_clones":0,"num_object_copies":9586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15341,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15219","reported_seq":15352,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:32.091906+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:19:32.091906+0000","last_peered":"2024-09-11T01:19:32.091906+0000","last_clean":"2024-09-11T01:19:32.091906+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:19:32.091906+0000","last_undegraded":"2024-09-11T01:19:32.091906+0000","last_fullsized":"2024-09-11T01:19:32.091906+0000","mapping_epoch":34,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19619840,"num_objects":4790,"num_object_clones":0,"num_object_copies":9580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4790,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12224,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15277","reported_seq":15452,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:32.135809+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:19:32.135809+0000","last_peered":"2024-09-11T01:19:32.135809+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:19:32.135809+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20275200,"num_objects":4950,"num_object_clones":0,"num_object_copies":9900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11618,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":999}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15135","reported_seq":15269,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:32.115114+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:19:32.115114+0000","last_peered":"2024-09-11T01:19:32.115114+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:19:32.115114+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19640320,"num_objects":4795,"num_object_clones":0,"num_object_copies":9590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4795,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11992,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":809}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15409","reported_seq":15544,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:32.072918+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:19:32.072918+0000","last_peered":"2024-09-11T01:19:32.072918+0000","last_clean":"2024-09-11T01:19:32.072918+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:19:32.072918+0000","last_undegraded":"2024-09-11T01:19:32.072918+0000","last_fullsized":"2024-09-11T01:19:32.072918+0000","mapping_epoch":34,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19845120,"num_objects":4845,"num_object_clones":0,"num_object_copies":9690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4845,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12285,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934737+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934737+0000","last_peered":"2024-09-11T01:17:58.934737+0000","last_clean":"2024-09-11T01:17:58.934737+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934737+0000","last_undegraded":"2024-09-11T01:17:58.934737+0000","last_fullsized":"2024-09-11T01:17:58.934737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15258","reported_seq":19978,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:34.789212+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:19:34.789212+0000","last_peered":"2024-09-11T01:19:34.789212+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:19:34.789212+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19312640,"num_objects":4715,"num_object_clones":0,"num_object_copies":9430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1642,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13229,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2293,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3073}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15108","reported_seq":15242,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:32.034465+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:19:32.034465+0000","last_peered":"2024-09-11T01:19:32.034465+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:19:32.034465+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19496960,"num_objects":4760,"num_object_clones":0,"num_object_copies":9520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4760,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11695,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":776}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15429","reported_seq":15553,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:34.729134+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:19:34.729134+0000","last_peered":"2024-09-11T01:19:34.729134+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:19:34.729134+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19570688,"num_objects":4778,"num_object_clones":0,"num_object_copies":9556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4778,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13246,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":754}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935141+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935141+0000","last_peered":"2024-09-11T01:17:58.935141+0000","last_clean":"2024-09-11T01:17:58.935141+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935141+0000","last_undegraded":"2024-09-11T01:17:58.935141+0000","last_fullsized":"2024-09-11T01:17:58.935141+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15134","reported_seq":23120,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:34.770868+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:19:34.770868+0000","last_peered":"2024-09-11T01:19:34.770868+0000","last_clean":"2024-09-11T01:19:34.770868+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:19:34.770868+0000","last_undegraded":"2024-09-11T01:19:34.770868+0000","last_fullsized":"2024-09-11T01:19:34.770868+0000","mapping_epoch":44,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19095552,"num_objects":4662,"num_object_clones":0,"num_object_copies":9324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12919,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15326","reported_seq":15463,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:32.091258+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:19:32.091258+0000","last_peered":"2024-09-11T01:19:32.091258+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:19:32.091258+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20226048,"num_objects":4938,"num_object_clones":0,"num_object_copies":9876,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4938,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12063,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":988}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938599+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938599+0000","last_peered":"2024-09-11T01:17:58.938599+0000","last_clean":"2024-09-11T01:17:58.938599+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938599+0000","last_undegraded":"2024-09-11T01:17:58.938599+0000","last_fullsized":"2024-09-11T01:17:58.938599+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15256","reported_seq":23292,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:36.428057+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:19:36.428057+0000","last_peered":"2024-09-11T01:19:36.428057+0000","last_clean":"2024-09-11T01:19:36.428057+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:19:36.428057+0000","last_undegraded":"2024-09-11T01:19:36.428057+0000","last_fullsized":"2024-09-11T01:19:36.428057+0000","mapping_epoch":46,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18956288,"num_objects":4628,"num_object_clones":0,"num_object_copies":9256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11911,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15351","reported_seq":15491,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:32.041654+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:19:32.041654+0000","last_peered":"2024-09-11T01:19:32.041654+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:19:32.041654+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19763200,"num_objects":4825,"num_object_clones":0,"num_object_copies":9650,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4825,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11960,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":786}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971371+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971371+0000","last_peered":"2024-09-11T01:17:58.971371+0000","last_clean":"2024-09-11T01:17:58.971371+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971371+0000","last_undegraded":"2024-09-11T01:17:58.971371+0000","last_fullsized":"2024-09-11T01:17:58.971371+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15214","reported_seq":15351,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:32.058090+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:19:32.058090+0000","last_peered":"2024-09-11T01:19:32.058090+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:19:32.058090+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19407465,"num_objects":4739,"num_object_clones":0,"num_object_copies":9478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12284,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":822}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":314282625,"num_objects":76731,"num_object_clones":0,"num_object_copies":153462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37476,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76731,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":205556,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10163,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":819634176,"data_stored":819563410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160606,"ondisk_log_size":160606,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182496,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":688304,"kb_used_data":295712,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93683536,"statfs":{"total":96636764160,"available":95931940864,"internally_reserved":0,"allocated":302809088,"data_stored":302539429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":401993387},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,5,2,1,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986269,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609520,"kb_used_data":280796,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93762320,"statfs":{"total":96636764160,"available":96012615680,"internally_reserved":0,"allocated":287535104,"data_stored":287265450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336588458},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659214,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":573244,"kb_used_data":226476,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93798596,"statfs":{"total":96636764160,"available":96049762304,"internally_reserved":0,"allocated":231911424,"data_stored":231585508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":355069615},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":301998080,"data_stored":301994601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":231235584,"data_stored":231175848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":286400512,"data_stored":286392961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:38.296 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:39.017 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:19:39.019 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:19:39.110 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":298,"stamp":"2024-09-11T01:19:38.717257+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":308970132,"num_objects":75436,"num_object_clones":0,"num_object_copies":150872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37476,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75436,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":206855,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10164,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160906,"ondisk_log_size":160906,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1868448,"kb_used_data":796192,"kb_used_omap":12,"kb_used_meta":1072179,"kb_avail":281247072,"statfs":{"total":289910292480,"available":287997001728,"internally_reserved":0,"allocated":815300608,"data_stored":814435379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1097911300},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,4,3,5,5,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24313856,"num_objects":-5936,"num_object_clones":0,"num_object_copies":-11872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-278,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5936,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":278,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002828"},"pg_stats":[{"pgid":"2.f","version":"47'15553","reported_seq":15636,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:37.060621+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:19:37.060621+0000","last_peered":"2024-09-11T01:19:37.060621+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:19:37.060621+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19304448,"num_objects":4713,"num_object_clones":0,"num_object_copies":9426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15553,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":641}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15582","reported_seq":15689,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:34.651398+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:19:34.651398+0000","last_peered":"2024-09-11T01:19:34.651398+0000","last_clean":"2024-09-11T01:19:34.651398+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:19:34.651398+0000","last_undegraded":"2024-09-11T01:19:34.651398+0000","last_fullsized":"2024-09-11T01:19:34.651398+0000","mapping_epoch":35,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19537920,"num_objects":4770,"num_object_clones":0,"num_object_copies":9540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13821,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15256","reported_seq":15390,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:34.709877+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:19:34.709877+0000","last_peered":"2024-09-11T01:19:34.709877+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:19:34.709877+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":20017176,"num_objects":4888,"num_object_clones":0,"num_object_copies":9776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4888,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13557,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":992}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15341","reported_seq":15415,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:34.782980+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:19:34.782980+0000","last_peered":"2024-09-11T01:19:34.782980+0000","last_clean":"2024-09-11T01:19:34.782980+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:19:34.782980+0000","last_undegraded":"2024-09-11T01:19:34.782980+0000","last_fullsized":"2024-09-11T01:19:34.782980+0000","mapping_epoch":35,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19632128,"num_objects":4793,"num_object_clones":0,"num_object_copies":9586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15341,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15359","reported_seq":15492,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:37.109686+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:19:37.109686+0000","last_peered":"2024-09-11T01:19:37.109686+0000","last_clean":"2024-09-11T01:19:37.109686+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:19:37.109686+0000","last_undegraded":"2024-09-11T01:19:37.109686+0000","last_fullsized":"2024-09-11T01:19:37.109686+0000","mapping_epoch":34,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19046400,"num_objects":4650,"num_object_clones":0,"num_object_copies":9300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12364,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15422","reported_seq":15595,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:37.099533+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:19:37.099533+0000","last_peered":"2024-09-11T01:19:37.099533+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:19:37.099533+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19681280,"num_objects":4805,"num_object_clones":0,"num_object_copies":9610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11763,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":854}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15281","reported_seq":15417,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:37.100587+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:19:37.100587+0000","last_peered":"2024-09-11T01:19:37.100587+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:19:37.100587+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19042304,"num_objects":4649,"num_object_clones":0,"num_object_copies":9298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12138,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":663}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15551","reported_seq":15684,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:36.956761+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:19:36.956761+0000","last_peered":"2024-09-11T01:19:36.956761+0000","last_clean":"2024-09-11T01:19:36.956761+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:19:36.956761+0000","last_undegraded":"2024-09-11T01:19:36.956761+0000","last_fullsized":"2024-09-11T01:19:36.956761+0000","mapping_epoch":34,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19263488,"num_objects":4703,"num_object_clones":0,"num_object_copies":9406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12427,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934737+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934737+0000","last_peered":"2024-09-11T01:17:58.934737+0000","last_clean":"2024-09-11T01:17:58.934737+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934737+0000","last_undegraded":"2024-09-11T01:17:58.934737+0000","last_fullsized":"2024-09-11T01:17:58.934737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15258","reported_seq":19978,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:34.789212+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:19:34.789212+0000","last_peered":"2024-09-11T01:19:34.789212+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:19:34.789212+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19312640,"num_objects":4715,"num_object_clones":0,"num_object_copies":9430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1642,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4715,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13229,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2293,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3073}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15252","reported_seq":15386,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:37.044121+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:19:37.044121+0000","last_peered":"2024-09-11T01:19:37.044121+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:19:37.044121+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18907136,"num_objects":4616,"num_object_clones":0,"num_object_copies":9232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11839,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":632}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15429","reported_seq":15553,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:34.729134+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:19:34.729134+0000","last_peered":"2024-09-11T01:19:34.729134+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:19:34.729134+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10029,"log_dups_size":0,"ondisk_log_size":10029,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19570688,"num_objects":4778,"num_object_clones":0,"num_object_copies":9556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4778,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13246,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":754}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935141+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935141+0000","last_peered":"2024-09-11T01:17:58.935141+0000","last_clean":"2024-09-11T01:17:58.935141+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935141+0000","last_undegraded":"2024-09-11T01:17:58.935141+0000","last_fullsized":"2024-09-11T01:17:58.935141+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15134","reported_seq":23120,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:34.770868+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:19:34.770868+0000","last_peered":"2024-09-11T01:19:34.770868+0000","last_clean":"2024-09-11T01:19:34.770868+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:19:34.770868+0000","last_undegraded":"2024-09-11T01:19:34.770868+0000","last_fullsized":"2024-09-11T01:19:34.770868+0000","mapping_epoch":44,"log_start":"43'5100","ondisk_log_start":"43'5100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19095552,"num_objects":4662,"num_object_clones":0,"num_object_copies":9324,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4662,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12919,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15471","reported_seq":15608,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:37.141735+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:19:37.141735+0000","last_peered":"2024-09-11T01:19:37.141735+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:19:37.141735+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19632128,"num_objects":4793,"num_object_clones":0,"num_object_copies":9586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12208,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":843}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938599+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938599+0000","last_peered":"2024-09-11T01:17:58.938599+0000","last_clean":"2024-09-11T01:17:58.938599+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938599+0000","last_undegraded":"2024-09-11T01:17:58.938599+0000","last_fullsized":"2024-09-11T01:17:58.938599+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15256","reported_seq":23292,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:36.428057+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:19:36.428057+0000","last_peered":"2024-09-11T01:19:36.428057+0000","last_clean":"2024-09-11T01:19:36.428057+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:19:36.428057+0000","last_undegraded":"2024-09-11T01:19:36.428057+0000","last_fullsized":"2024-09-11T01:19:36.428057+0000","mapping_epoch":46,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18956288,"num_objects":4628,"num_object_clones":0,"num_object_copies":9256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11911,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15493","reported_seq":15632,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:37.031828+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:19:37.031828+0000","last_peered":"2024-09-11T01:19:37.031828+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:19:37.031828+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19181568,"num_objects":4683,"num_object_clones":0,"num_object_copies":9366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12102,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":644}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971371+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971371+0000","last_peered":"2024-09-11T01:17:58.971371+0000","last_clean":"2024-09-11T01:17:58.971371+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971371+0000","last_undegraded":"2024-09-11T01:17:58.971371+0000","last_fullsized":"2024-09-11T01:17:58.971371+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15365","reported_seq":15503,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:37.105268+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:19:37.105268+0000","last_peered":"2024-09-11T01:19:37.105268+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:19:37.105268+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18788969,"num_objects":4588,"num_object_clones":0,"num_object_copies":9176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12435,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":671}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":308970113,"num_objects":75434,"num_object_clones":0,"num_object_copies":150868,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37476,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75434,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":206853,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10163,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":812572672,"data_stored":812501906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160903,"ondisk_log_size":160903,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182497,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":685684,"kb_used_data":288920,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93686156,"statfs":{"total":96636764160,"available":95934623744,"internally_reserved":0,"allocated":295854080,"data_stored":295584421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":406253227},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,2,3,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986269,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":609520,"kb_used_data":280796,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93762320,"statfs":{"total":96636764160,"available":96012615680,"internally_reserved":0,"allocated":287535104,"data_stored":287265450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336588458},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659214,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":573244,"kb_used_data":226476,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93798596,"statfs":{"total":96636764160,"available":96049762304,"internally_reserved":0,"allocated":231911424,"data_stored":231585508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":355069615},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":294936576,"data_stored":294933097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":231235584,"data_stored":231175848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":286400512,"data_stored":286392961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:39.112 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:19:39.112 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'15553", 'reported_seq': 15636, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:37.060621+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:19:37.060621+0000', 'last_peered': '2024-09-11T01:19:37.060621+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:19:37.060621+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "43'5500", 'ondisk_log_start': "43'5500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19304448, 'num_objects': 4713, 'num_object_clones': 0, 'num_object_copies': 9426, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4713, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15553, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 641}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:39.112 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:19:39.112 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'15256", 'reported_seq': 15390, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:34.709877+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:19:34.709877+0000', 'last_peered': '2024-09-11T01:19:34.709877+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:19:34.709877+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "43'5200", 'ondisk_log_start': "43'5200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 20017176, 'num_objects': 4888, 'num_object_clones': 0, 'num_object_copies': 9776, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4888, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13557, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 992}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:39.112 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:19:39.112 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'15422", 'reported_seq': 15595, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:37.099533+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:19:37.099533+0000', 'last_peered': '2024-09-11T01:19:37.099533+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:19:37.099533+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "43'5400", 'ondisk_log_start': "43'5400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19681280, 'num_objects': 4805, 'num_object_clones': 0, 'num_object_copies': 9610, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4805, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11763, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 854}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:39.112 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:19:39.112 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'15281", 'reported_seq': 15417, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:37.100587+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:19:37.100587+0000', 'last_peered': '2024-09-11T01:19:37.100587+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:19:37.100587+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "43'5200", 'ondisk_log_start': "43'5200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19042304, 'num_objects': 4649, 'num_object_clones': 0, 'num_object_copies': 9298, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4649, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12138, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 663}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:39.113 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:19:39.113 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'15258", 'reported_seq': 19978, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:34.789212+0000', 'last_change': '2024-09-11T01:17:57.355893+0000', 'last_active': '2024-09-11T01:19:34.789212+0000', 'last_peered': '2024-09-11T01:19:34.789212+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:19:34.789212+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "43'5200", 'ondisk_log_start': "43'5200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19312640, 'num_objects': 4715, 'num_object_clones': 0, 'num_object_copies': 9430, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1642, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4715, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13229, 'num_write_kb': 31844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2293, 'num_bytes_recovered': 4505600, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3073}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:39.113 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:19:39.113 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'15252", 'reported_seq': 15386, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:37.044121+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:19:37.044121+0000', 'last_peered': '2024-09-11T01:19:37.044121+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:19:37.044121+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "43'5200", 'ondisk_log_start': "43'5200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18907136, 'num_objects': 4616, 'num_object_clones': 0, 'num_object_copies': 9232, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4616, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11839, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 632}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:39.113 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:19:39.113 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'15429", 'reported_seq': 15553, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:34.729134+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:19:34.729134+0000', 'last_peered': '2024-09-11T01:19:34.729134+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:19:34.729134+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "43'5400", 'ondisk_log_start': "43'5400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10029, 'log_dups_size': 0, 'ondisk_log_size': 10029, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19570688, 'num_objects': 4778, 'num_object_clones': 0, 'num_object_copies': 9556, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4778, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13246, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 754}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:39.113 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:19:39.113 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'15471", 'reported_seq': 15608, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:37.141735+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:19:37.141735+0000', 'last_peered': '2024-09-11T01:19:37.141735+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:19:37.141735+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "43'5400", 'ondisk_log_start': "43'5400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19632128, 'num_objects': 4793, 'num_object_clones': 0, 'num_object_copies': 9586, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4793, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12208, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 843}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:39.113 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:19:39.113 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'15493", 'reported_seq': 15632, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:37.031828+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:19:37.031828+0000', 'last_peered': '2024-09-11T01:19:37.031828+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:19:37.031828+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "43'5400", 'ondisk_log_start': "43'5400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19181568, 'num_objects': 4683, 'num_object_clones': 0, 'num_object_copies': 9366, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4683, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12102, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 644}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:39.114 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:19:39.114 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'15365", 'reported_seq': 15503, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:37.105268+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:19:37.105268+0000', 'last_peered': '2024-09-11T01:19:37.105268+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:19:37.105268+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "43'5300", 'ondisk_log_start': "43'5300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10065, 'log_dups_size': 0, 'ondisk_log_size': 10065, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18788969, 'num_objects': 4588, 'num_object_clones': 0, 'num_object_copies': 9176, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4588, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12435, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 671}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:39.114 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:19:40.139 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:19:40.139 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37476/150872 objects degraded (24.840%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":516,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":75436,"data_bytes":308970132,"bytes_used":1913290752,"bytes_avail":287997001728,"bytes_total":289910292480,"degraded_objects":37476,"degraded_total":150872,"degraded_ratio":0.24839599130388673,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":278,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":494},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:19:40.234 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:19:40.234 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:41.005 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:19:41.006 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:19:41.079 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":299,"stamp":"2024-09-11T01:19:40.717760+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":305369748,"num_objects":74557,"num_object_clones":0,"num_object_copies":149114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37317,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74557,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":207734,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10323,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160785,"ondisk_log_size":160785,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1851440,"kb_used_data":791664,"kb_used_omap":12,"kb_used_meta":1059699,"kb_avail":281264080,"statfs":{"total":289910292480,"available":288014417920,"internally_reserved":0,"allocated":810663936,"data_stored":809798707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1085131780},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,0,2,6,3,5,8,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22499328,"num_objects":-5493,"num_object_clones":0,"num_object_copies":-10986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-437,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5493,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":437,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002847"},"pg_stats":[{"pgid":"2.f","version":"47'15553","reported_seq":15636,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:37.060621+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:19:37.060621+0000","last_peered":"2024-09-11T01:19:37.060621+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:19:37.060621+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19304448,"num_objects":4713,"num_object_clones":0,"num_object_copies":9426,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4713,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15553,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":641}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15719","reported_seq":15826,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:39.742844+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:19:39.742844+0000","last_peered":"2024-09-11T01:19:39.742844+0000","last_clean":"2024-09-11T01:19:39.742844+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:19:39.742844+0000","last_undegraded":"2024-09-11T01:19:39.742844+0000","last_fullsized":"2024-09-11T01:19:39.742844+0000","mapping_epoch":35,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18976768,"num_objects":4633,"num_object_clones":0,"num_object_copies":9266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13958,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15413","reported_seq":15547,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:39.787355+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:19:39.787355+0000","last_peered":"2024-09-11T01:19:39.787355+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:19:39.787355+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19374104,"num_objects":4731,"num_object_clones":0,"num_object_copies":9462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4731,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13714,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":835}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15502","reported_seq":15577,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:39.789533+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:19:39.789533+0000","last_peered":"2024-09-11T01:19:39.789533+0000","last_clean":"2024-09-11T01:19:39.789533+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:19:39.789533+0000","last_undegraded":"2024-09-11T01:19:39.789533+0000","last_fullsized":"2024-09-11T01:19:39.789533+0000","mapping_epoch":35,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18972672,"num_objects":4632,"num_object_clones":0,"num_object_copies":9264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15502,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15359","reported_seq":15492,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:37.109686+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:19:37.109686+0000","last_peered":"2024-09-11T01:19:37.109686+0000","last_clean":"2024-09-11T01:19:37.109686+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:19:37.109686+0000","last_undegraded":"2024-09-11T01:19:37.109686+0000","last_fullsized":"2024-09-11T01:19:37.109686+0000","mapping_epoch":34,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19046400,"num_objects":4650,"num_object_clones":0,"num_object_copies":9300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4650,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12364,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15422","reported_seq":15595,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:37.099533+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:19:37.099533+0000","last_peered":"2024-09-11T01:19:37.099533+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:19:37.099533+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19681280,"num_objects":4805,"num_object_clones":0,"num_object_copies":9610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4805,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11763,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":854}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15281","reported_seq":15417,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:37.100587+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:19:37.100587+0000","last_peered":"2024-09-11T01:19:37.100587+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:19:37.100587+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19042304,"num_objects":4649,"num_object_clones":0,"num_object_copies":9298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12138,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":663}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15551","reported_seq":15684,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:36.956761+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:19:36.956761+0000","last_peered":"2024-09-11T01:19:36.956761+0000","last_clean":"2024-09-11T01:19:36.956761+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:19:36.956761+0000","last_undegraded":"2024-09-11T01:19:36.956761+0000","last_fullsized":"2024-09-11T01:19:36.956761+0000","mapping_epoch":34,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19263488,"num_objects":4703,"num_object_clones":0,"num_object_copies":9406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4703,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12427,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934737+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934737+0000","last_peered":"2024-09-11T01:17:58.934737+0000","last_clean":"2024-09-11T01:17:58.934737+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934737+0000","last_undegraded":"2024-09-11T01:17:58.934737+0000","last_fullsized":"2024-09-11T01:17:58.934737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15408","reported_seq":20443,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:39.818954+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:19:39.818954+0000","last_peered":"2024-09-11T01:19:39.818954+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:19:39.818954+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18698240,"num_objects":4565,"num_object_clones":0,"num_object_copies":9130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1483,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13379,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2452,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3082}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15252","reported_seq":15386,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:37.044121+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:19:37.044121+0000","last_peered":"2024-09-11T01:19:37.044121+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:19:37.044121+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18907136,"num_objects":4616,"num_object_clones":0,"num_object_copies":9232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11839,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":632}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15583","reported_seq":15707,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:39.794495+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:19:39.794495+0000","last_peered":"2024-09-11T01:19:39.794495+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:19:39.794495+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18939904,"num_objects":4624,"num_object_clones":0,"num_object_copies":9248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13400,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":600}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935141+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935141+0000","last_peered":"2024-09-11T01:17:58.935141+0000","last_clean":"2024-09-11T01:17:58.935141+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935141+0000","last_undegraded":"2024-09-11T01:17:58.935141+0000","last_fullsized":"2024-09-11T01:17:58.935141+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15254","reported_seq":23240,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:39.660992+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:19:39.660992+0000","last_peered":"2024-09-11T01:19:39.660992+0000","last_clean":"2024-09-11T01:19:39.660992+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:19:39.660992+0000","last_undegraded":"2024-09-11T01:19:39.660992+0000","last_fullsized":"2024-09-11T01:19:39.660992+0000","mapping_epoch":44,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18604032,"num_objects":4542,"num_object_clones":0,"num_object_copies":9084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13039,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15471","reported_seq":15608,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:37.141735+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:19:37.141735+0000","last_peered":"2024-09-11T01:19:37.141735+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:19:37.141735+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19632128,"num_objects":4793,"num_object_clones":0,"num_object_copies":9586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12208,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":843}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938599+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938599+0000","last_peered":"2024-09-11T01:17:58.938599+0000","last_clean":"2024-09-11T01:17:58.938599+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938599+0000","last_undegraded":"2024-09-11T01:17:58.938599+0000","last_fullsized":"2024-09-11T01:17:58.938599+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15256","reported_seq":23292,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:36.428057+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:19:36.428057+0000","last_peered":"2024-09-11T01:19:36.428057+0000","last_clean":"2024-09-11T01:19:36.428057+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:19:36.428057+0000","last_undegraded":"2024-09-11T01:19:36.428057+0000","last_fullsized":"2024-09-11T01:19:36.428057+0000","mapping_epoch":46,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18956288,"num_objects":4628,"num_object_clones":0,"num_object_copies":9256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4628,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11911,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15493","reported_seq":15632,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:37.031828+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:19:37.031828+0000","last_peered":"2024-09-11T01:19:37.031828+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:19:37.031828+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19181568,"num_objects":4683,"num_object_clones":0,"num_object_copies":9366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4683,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12102,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":644}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971371+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971371+0000","last_peered":"2024-09-11T01:17:58.971371+0000","last_clean":"2024-09-11T01:17:58.971371+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971371+0000","last_undegraded":"2024-09-11T01:17:58.971371+0000","last_fullsized":"2024-09-11T01:17:58.971371+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":79,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938260+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938260+0000","last_peered":"2024-09-11T01:17:58.938260+0000","last_clean":"2024-09-11T01:17:58.938260+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938260+0000","last_undegraded":"2024-09-11T01:17:58.938260+0000","last_fullsized":"2024-09-11T01:17:58.938260+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15365","reported_seq":15503,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:37.105268+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:19:37.105268+0000","last_peered":"2024-09-11T01:19:37.105268+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:19:37.105268+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18788969,"num_objects":4588,"num_object_clones":0,"num_object_copies":9176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4588,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12435,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":671}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":305369729,"num_objects":74555,"num_object_clones":0,"num_object_copies":149110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37317,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74555,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":207732,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10322,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":807768064,"data_stored":807697298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160782,"ondisk_log_size":160782,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182497,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":685684,"kb_used_data":288920,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93686156,"statfs":{"total":96636764160,"available":95934623744,"internally_reserved":0,"allocated":295854080,"data_stored":295584421,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":406253227},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,1,2,3,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986270,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592512,"kb_used_data":276268,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93779328,"statfs":{"total":96636764160,"available":96030031872,"internally_reserved":0,"allocated":282898432,"data_stored":282628778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323808938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,3,0,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659214,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":573244,"kb_used_data":226476,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93798596,"statfs":{"total":96636764160,"available":96049762304,"internally_reserved":0,"allocated":231911424,"data_stored":231585508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":355069615},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":294936576,"data_stored":294933097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":231235584,"data_stored":231175848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":281595904,"data_stored":281588353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:44.082 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:44.845 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:19:44.847 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:19:44.905 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2024-09-11T01:19:44.718784+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":299565716,"num_objects":73140,"num_object_clones":0,"num_object_copies":146280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37317,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73140,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":209151,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10323,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160702,"ondisk_log_size":160702,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1821244,"kb_used_data":778116,"kb_used_omap":12,"kb_used_meta":1043059,"kb_avail":281294276,"statfs":{"total":289910292480,"available":288045338624,"internally_reserved":0,"allocated":796790784,"data_stored":795925555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1068092420},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,3,7,4,5,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18743296,"num_objects":-4576,"num_object_clones":0,"num_object_copies":-9152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-297,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4576,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":297,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002922"},"pg_stats":[{"pgid":"2.f","version":"47'15686","reported_seq":15767,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:42.162576+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:19:42.162576+0000","last_peered":"2024-09-11T01:19:42.162576+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:19:42.162576+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18759680,"num_objects":4580,"num_object_clones":0,"num_object_copies":9160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15686,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":508}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15719","reported_seq":15826,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:39.742844+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:19:39.742844+0000","last_peered":"2024-09-11T01:19:39.742844+0000","last_clean":"2024-09-11T01:19:39.742844+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:19:39.742844+0000","last_undegraded":"2024-09-11T01:19:39.742844+0000","last_fullsized":"2024-09-11T01:19:39.742844+0000","mapping_epoch":35,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18976768,"num_objects":4633,"num_object_clones":0,"num_object_copies":9266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13958,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15413","reported_seq":15547,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:39.787355+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:19:39.787355+0000","last_peered":"2024-09-11T01:19:39.787355+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:19:39.787355+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19374104,"num_objects":4731,"num_object_clones":0,"num_object_copies":9462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4731,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13714,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":835}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15502","reported_seq":15577,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:39.789533+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:19:39.789533+0000","last_peered":"2024-09-11T01:19:39.789533+0000","last_clean":"2024-09-11T01:19:39.789533+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:19:39.789533+0000","last_undegraded":"2024-09-11T01:19:39.789533+0000","last_fullsized":"2024-09-11T01:19:39.789533+0000","mapping_epoch":35,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18972672,"num_objects":4632,"num_object_clones":0,"num_object_copies":9264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15502,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15526","reported_seq":15659,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:42.114091+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:19:42.114091+0000","last_peered":"2024-09-11T01:19:42.114091+0000","last_clean":"2024-09-11T01:19:42.114091+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:19:42.114091+0000","last_undegraded":"2024-09-11T01:19:42.114091+0000","last_fullsized":"2024-09-11T01:19:42.114091+0000","mapping_epoch":34,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18362368,"num_objects":4483,"num_object_clones":0,"num_object_copies":8966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12531,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15575","reported_seq":15746,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:42.119617+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:19:42.119617+0000","last_peered":"2024-09-11T01:19:42.119617+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:19:42.119617+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19054592,"num_objects":4652,"num_object_clones":0,"num_object_copies":9304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11916,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":701}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15401","reported_seq":15537,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:42.048485+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:19:42.048485+0000","last_peered":"2024-09-11T01:19:42.048485+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:19:42.048485+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18550784,"num_objects":4529,"num_object_clones":0,"num_object_copies":9058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4529,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12258,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15685","reported_seq":15820,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:42.048948+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:19:42.048948+0000","last_peered":"2024-09-11T01:19:42.048948+0000","last_clean":"2024-09-11T01:19:42.048948+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:19:42.048948+0000","last_undegraded":"2024-09-11T01:19:42.048948+0000","last_fullsized":"2024-09-11T01:19:42.048948+0000","mapping_epoch":34,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18714624,"num_objects":4569,"num_object_clones":0,"num_object_copies":9138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12561,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934737+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934737+0000","last_peered":"2024-09-11T01:17:58.934737+0000","last_clean":"2024-09-11T01:17:58.934737+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934737+0000","last_undegraded":"2024-09-11T01:17:58.934737+0000","last_fullsized":"2024-09-11T01:17:58.934737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15408","reported_seq":20443,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:39.818954+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:19:39.818954+0000","last_peered":"2024-09-11T01:19:39.818954+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:19:39.818954+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18698240,"num_objects":4565,"num_object_clones":0,"num_object_copies":9130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1483,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13379,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2452,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3082}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15414","reported_seq":15547,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:42.120821+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:19:42.120821+0000","last_peered":"2024-09-11T01:19:42.120821+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:19:42.120821+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18243584,"num_objects":4454,"num_object_clones":0,"num_object_copies":8908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12001,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":470}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15583","reported_seq":15707,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:39.794495+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:19:39.794495+0000","last_peered":"2024-09-11T01:19:39.794495+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:19:39.794495+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18939904,"num_objects":4624,"num_object_clones":0,"num_object_copies":9248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13400,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":600}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935141+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935141+0000","last_peered":"2024-09-11T01:17:58.935141+0000","last_clean":"2024-09-11T01:17:58.935141+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935141+0000","last_undegraded":"2024-09-11T01:17:58.935141+0000","last_fullsized":"2024-09-11T01:17:58.935141+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15254","reported_seq":23240,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:39.660992+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:19:39.660992+0000","last_peered":"2024-09-11T01:19:39.660992+0000","last_clean":"2024-09-11T01:19:39.660992+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:19:39.660992+0000","last_undegraded":"2024-09-11T01:19:39.660992+0000","last_fullsized":"2024-09-11T01:19:39.660992+0000","mapping_epoch":44,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18604032,"num_objects":4542,"num_object_clones":0,"num_object_copies":9084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13039,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15608","reported_seq":15743,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:42.127952+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:19:42.127952+0000","last_peered":"2024-09-11T01:19:42.127952+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:19:42.127952+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19070976,"num_objects":4656,"num_object_clones":0,"num_object_copies":9312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4656,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12345,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":706}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938599+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938599+0000","last_peered":"2024-09-11T01:17:58.938599+0000","last_clean":"2024-09-11T01:17:58.938599+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938599+0000","last_undegraded":"2024-09-11T01:17:58.938599+0000","last_fullsized":"2024-09-11T01:17:58.938599+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15383","reported_seq":23419,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:41.415955+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:19:41.415955+0000","last_peered":"2024-09-11T01:19:41.415955+0000","last_clean":"2024-09-11T01:19:41.415955+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:19:41.415955+0000","last_undegraded":"2024-09-11T01:19:41.415955+0000","last_fullsized":"2024-09-11T01:19:41.415955+0000","mapping_epoch":46,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18436096,"num_objects":4501,"num_object_clones":0,"num_object_copies":9002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12038,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15651","reported_seq":15791,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:42.133056+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:19:42.133056+0000","last_peered":"2024-09-11T01:19:42.133056+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:19:42.133056+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18534400,"num_objects":4525,"num_object_clones":0,"num_object_copies":9050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12260,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971371+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971371+0000","last_peered":"2024-09-11T01:17:58.971371+0000","last_clean":"2024-09-11T01:17:58.971371+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971371+0000","last_undegraded":"2024-09-11T01:17:58.971371+0000","last_fullsized":"2024-09-11T01:17:58.971371+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938303+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938303+0000","last_peered":"2024-09-11T01:17:58.938303+0000","last_clean":"2024-09-11T01:17:58.938303+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938303+0000","last_undegraded":"2024-09-11T01:17:58.938303+0000","last_fullsized":"2024-09-11T01:17:58.938303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15491","reported_seq":15630,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:42.144466+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:19:42.144466+0000","last_peered":"2024-09-11T01:19:42.144466+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:19:42.144466+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18272873,"num_objects":4462,"num_object_clones":0,"num_object_copies":8924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12561,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":545}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":299565697,"num_objects":73138,"num_object_clones":0,"num_object_copies":146276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37317,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73138,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":209149,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10322,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":793911296,"data_stored":793840530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160699,"ondisk_log_size":160699,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182498,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":670568,"kb_used_data":282136,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93701272,"statfs":{"total":96636764160,"available":95950102528,"internally_reserved":0,"allocated":288907264,"data_stored":288637605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":397733547},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,2,3,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986270,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592512,"kb_used_data":276268,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93779328,"statfs":{"total":96636764160,"available":96030031872,"internally_reserved":0,"allocated":282898432,"data_stored":282628778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323808938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,3,0,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659215,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":558164,"kb_used_data":219712,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93813676,"statfs":{"total":96636764160,"available":96065204224,"internally_reserved":0,"allocated":224985088,"data_stored":224659172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":346549935},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":288030720,"data_stored":288027241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":224284672,"data_stored":224224936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":281595904,"data_stored":281588353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:44.906 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:45.600 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:19:45.600 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:19:45.662 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2024-09-11T01:19:44.718784+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":299565716,"num_objects":73140,"num_object_clones":0,"num_object_copies":146280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37317,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73140,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":209151,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10323,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160702,"ondisk_log_size":160702,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1821244,"kb_used_data":778116,"kb_used_omap":12,"kb_used_meta":1043059,"kb_avail":281294276,"statfs":{"total":289910292480,"available":288045338624,"internally_reserved":0,"allocated":796790784,"data_stored":795925555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1068092420},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[4,0,3,7,4,5,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18743296,"num_objects":-4576,"num_object_clones":0,"num_object_copies":-9152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-297,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4576,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":297,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002922"},"pg_stats":[{"pgid":"2.f","version":"47'15686","reported_seq":15767,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:42.162576+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:19:42.162576+0000","last_peered":"2024-09-11T01:19:42.162576+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:19:42.162576+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18759680,"num_objects":4580,"num_object_clones":0,"num_object_copies":9160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15686,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":508}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15719","reported_seq":15826,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:39.742844+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:19:39.742844+0000","last_peered":"2024-09-11T01:19:39.742844+0000","last_clean":"2024-09-11T01:19:39.742844+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:19:39.742844+0000","last_undegraded":"2024-09-11T01:19:39.742844+0000","last_fullsized":"2024-09-11T01:19:39.742844+0000","mapping_epoch":35,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18976768,"num_objects":4633,"num_object_clones":0,"num_object_copies":9266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13958,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15413","reported_seq":15547,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:39.787355+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:19:39.787355+0000","last_peered":"2024-09-11T01:19:39.787355+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:19:39.787355+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19374104,"num_objects":4731,"num_object_clones":0,"num_object_copies":9462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4731,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13714,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":835}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15502","reported_seq":15577,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:39.789533+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:19:39.789533+0000","last_peered":"2024-09-11T01:19:39.789533+0000","last_clean":"2024-09-11T01:19:39.789533+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:19:39.789533+0000","last_undegraded":"2024-09-11T01:19:39.789533+0000","last_fullsized":"2024-09-11T01:19:39.789533+0000","mapping_epoch":35,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18972672,"num_objects":4632,"num_object_clones":0,"num_object_copies":9264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4632,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15502,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15526","reported_seq":15659,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:42.114091+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:19:42.114091+0000","last_peered":"2024-09-11T01:19:42.114091+0000","last_clean":"2024-09-11T01:19:42.114091+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:19:42.114091+0000","last_undegraded":"2024-09-11T01:19:42.114091+0000","last_fullsized":"2024-09-11T01:19:42.114091+0000","mapping_epoch":34,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18362368,"num_objects":4483,"num_object_clones":0,"num_object_copies":8966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12531,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15575","reported_seq":15746,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:42.119617+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:19:42.119617+0000","last_peered":"2024-09-11T01:19:42.119617+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:19:42.119617+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19054592,"num_objects":4652,"num_object_clones":0,"num_object_copies":9304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11916,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":701}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15401","reported_seq":15537,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:42.048485+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:19:42.048485+0000","last_peered":"2024-09-11T01:19:42.048485+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:19:42.048485+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18550784,"num_objects":4529,"num_object_clones":0,"num_object_copies":9058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4529,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12258,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15685","reported_seq":15820,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:42.048948+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:19:42.048948+0000","last_peered":"2024-09-11T01:19:42.048948+0000","last_clean":"2024-09-11T01:19:42.048948+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:19:42.048948+0000","last_undegraded":"2024-09-11T01:19:42.048948+0000","last_fullsized":"2024-09-11T01:19:42.048948+0000","mapping_epoch":34,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18714624,"num_objects":4569,"num_object_clones":0,"num_object_copies":9138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12561,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934737+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934737+0000","last_peered":"2024-09-11T01:17:58.934737+0000","last_clean":"2024-09-11T01:17:58.934737+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934737+0000","last_undegraded":"2024-09-11T01:17:58.934737+0000","last_fullsized":"2024-09-11T01:17:58.934737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15408","reported_seq":20443,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:39.818954+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:19:39.818954+0000","last_peered":"2024-09-11T01:19:39.818954+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:19:39.818954+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18698240,"num_objects":4565,"num_object_clones":0,"num_object_copies":9130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1483,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4565,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13379,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2452,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3082}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15414","reported_seq":15547,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:42.120821+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:19:42.120821+0000","last_peered":"2024-09-11T01:19:42.120821+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:19:42.120821+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18243584,"num_objects":4454,"num_object_clones":0,"num_object_copies":8908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12001,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":470}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15583","reported_seq":15707,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:39.794495+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:19:39.794495+0000","last_peered":"2024-09-11T01:19:39.794495+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:19:39.794495+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18939904,"num_objects":4624,"num_object_clones":0,"num_object_copies":9248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13400,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":600}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935141+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935141+0000","last_peered":"2024-09-11T01:17:58.935141+0000","last_clean":"2024-09-11T01:17:58.935141+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935141+0000","last_undegraded":"2024-09-11T01:17:58.935141+0000","last_fullsized":"2024-09-11T01:17:58.935141+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15254","reported_seq":23240,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:39.660992+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:19:39.660992+0000","last_peered":"2024-09-11T01:19:39.660992+0000","last_clean":"2024-09-11T01:19:39.660992+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:19:39.660992+0000","last_undegraded":"2024-09-11T01:19:39.660992+0000","last_fullsized":"2024-09-11T01:19:39.660992+0000","mapping_epoch":44,"log_start":"43'5200","ondisk_log_start":"43'5200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18604032,"num_objects":4542,"num_object_clones":0,"num_object_copies":9084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4542,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13039,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15608","reported_seq":15743,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:42.127952+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:19:42.127952+0000","last_peered":"2024-09-11T01:19:42.127952+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:19:42.127952+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19070976,"num_objects":4656,"num_object_clones":0,"num_object_copies":9312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4656,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12345,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":706}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938599+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938599+0000","last_peered":"2024-09-11T01:17:58.938599+0000","last_clean":"2024-09-11T01:17:58.938599+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938599+0000","last_undegraded":"2024-09-11T01:17:58.938599+0000","last_fullsized":"2024-09-11T01:17:58.938599+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15383","reported_seq":23419,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:41.415955+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:19:41.415955+0000","last_peered":"2024-09-11T01:19:41.415955+0000","last_clean":"2024-09-11T01:19:41.415955+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:19:41.415955+0000","last_undegraded":"2024-09-11T01:19:41.415955+0000","last_fullsized":"2024-09-11T01:19:41.415955+0000","mapping_epoch":46,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18436096,"num_objects":4501,"num_object_clones":0,"num_object_copies":9002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4501,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12038,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15651","reported_seq":15791,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:42.133056+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:19:42.133056+0000","last_peered":"2024-09-11T01:19:42.133056+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:19:42.133056+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18534400,"num_objects":4525,"num_object_clones":0,"num_object_copies":9050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12260,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971371+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971371+0000","last_peered":"2024-09-11T01:17:58.971371+0000","last_clean":"2024-09-11T01:17:58.971371+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971371+0000","last_undegraded":"2024-09-11T01:17:58.971371+0000","last_fullsized":"2024-09-11T01:17:58.971371+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938303+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938303+0000","last_peered":"2024-09-11T01:17:58.938303+0000","last_clean":"2024-09-11T01:17:58.938303+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938303+0000","last_undegraded":"2024-09-11T01:17:58.938303+0000","last_fullsized":"2024-09-11T01:17:58.938303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15491","reported_seq":15630,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:42.144466+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:19:42.144466+0000","last_peered":"2024-09-11T01:19:42.144466+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:19:42.144466+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18272873,"num_objects":4462,"num_object_clones":0,"num_object_copies":8924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12561,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":545}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":299565697,"num_objects":73138,"num_object_clones":0,"num_object_copies":146276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37317,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73138,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":209149,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10322,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":793911296,"data_stored":793840530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160699,"ondisk_log_size":160699,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182498,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":670568,"kb_used_data":282136,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93701272,"statfs":{"total":96636764160,"available":95950102528,"internally_reserved":0,"allocated":288907264,"data_stored":288637605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":397733547},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,2,3,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986270,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":592512,"kb_used_data":276268,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93779328,"statfs":{"total":96636764160,"available":96030031872,"internally_reserved":0,"allocated":282898432,"data_stored":282628778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323808938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,3,0,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659215,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":558164,"kb_used_data":219712,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93813676,"statfs":{"total":96636764160,"available":96065204224,"internally_reserved":0,"allocated":224985088,"data_stored":224659172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":346549935},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":288030720,"data_stored":288027241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":224284672,"data_stored":224224936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":281595904,"data_stored":281588353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:45.663 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:19:45.663 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'15686", 'reported_seq': 15767, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:42.162576+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:19:42.162576+0000', 'last_peered': '2024-09-11T01:19:42.162576+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:19:42.162576+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "43'5600", 'ondisk_log_start': "43'5600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18759680, 'num_objects': 4580, 'num_object_clones': 0, 'num_object_copies': 9160, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4580, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15686, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 508}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:45.664 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:19:45.664 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'15413", 'reported_seq': 15547, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:39.787355+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:19:39.787355+0000', 'last_peered': '2024-09-11T01:19:39.787355+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:19:39.787355+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "43'5400", 'ondisk_log_start': "43'5400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19374104, 'num_objects': 4731, 'num_object_clones': 0, 'num_object_copies': 9462, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4731, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 13714, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 835}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:45.664 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:19:45.664 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'15575", 'reported_seq': 15746, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:42.119617+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:19:42.119617+0000', 'last_peered': '2024-09-11T01:19:42.119617+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:19:42.119617+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "43'5500", 'ondisk_log_start': "43'5500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19054592, 'num_objects': 4652, 'num_object_clones': 0, 'num_object_copies': 9304, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4652, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 11916, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 701}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:45.664 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:19:45.664 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'15401", 'reported_seq': 15537, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:42.048485+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:19:42.048485+0000', 'last_peered': '2024-09-11T01:19:42.048485+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:19:42.048485+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "43'5400", 'ondisk_log_start': "43'5400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18550784, 'num_objects': 4529, 'num_object_clones': 0, 'num_object_copies': 9058, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4529, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12258, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 543}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:45.664 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:19:45.664 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'15408", 'reported_seq': 20443, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:39.818954+0000', 'last_change': '2024-09-11T01:17:57.355893+0000', 'last_active': '2024-09-11T01:19:39.818954+0000', 'last_peered': '2024-09-11T01:19:39.818954+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:19:39.818954+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "43'5400", 'ondisk_log_start': "43'5400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18698240, 'num_objects': 4565, 'num_object_clones': 0, 'num_object_copies': 9130, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1483, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4565, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13379, 'num_write_kb': 31844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2452, 'num_bytes_recovered': 4505600, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3082}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:45.664 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:19:45.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'15414", 'reported_seq': 15547, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:42.120821+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:19:42.120821+0000', 'last_peered': '2024-09-11T01:19:42.120821+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:19:42.120821+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "43'5400", 'ondisk_log_start': "43'5400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18243584, 'num_objects': 4454, 'num_object_clones': 0, 'num_object_copies': 8908, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4454, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12001, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 470}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:45.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:19:45.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'15583", 'reported_seq': 15707, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:39.794495+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:19:39.794495+0000', 'last_peered': '2024-09-11T01:19:39.794495+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:19:39.794495+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "43'5500", 'ondisk_log_start': "43'5500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18939904, 'num_objects': 4624, 'num_object_clones': 0, 'num_object_copies': 9248, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4624, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13400, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 600}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:45.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:19:45.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'15608", 'reported_seq': 15743, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:42.127952+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:19:42.127952+0000', 'last_peered': '2024-09-11T01:19:42.127952+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:19:42.127952+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "43'5600", 'ondisk_log_start': "43'5600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 19070976, 'num_objects': 4656, 'num_object_clones': 0, 'num_object_copies': 9312, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4656, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12345, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 706}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:45.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:19:45.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'15651", 'reported_seq': 15791, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:42.133056+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:19:42.133056+0000', 'last_peered': '2024-09-11T01:19:42.133056+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:19:42.133056+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "43'5600", 'ondisk_log_start': "43'5600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18534400, 'num_objects': 4525, 'num_object_clones': 0, 'num_object_copies': 9050, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4525, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12260, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 486}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:45.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:19:45.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'15491", 'reported_seq': 15630, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:42.144466+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:19:42.144466+0000', 'last_peered': '2024-09-11T01:19:42.144466+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:19:42.144466+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "43'5400", 'ondisk_log_start': "43'5400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18272873, 'num_objects': 4462, 'num_object_clones': 0, 'num_object_copies': 8924, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4462, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12561, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 545}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:45.666 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:19:46.668 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:19:46.668 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37317/146280 objects degraded (25.511%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":523,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":73140,"data_bytes":299565716,"bytes_used":1864953856,"bytes_avail":288045338624,"bytes_total":289910292480,"degraded_objects":37317,"degraded_total":146280,"degraded_ratio":0.25510664479081213,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":297,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":381},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:19:46.742 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:19:46.742 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:47.365 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:19:47.366 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:19:47.431 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-09-11T01:19:46.719200+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":295404180,"num_objects":72124,"num_object_clones":0,"num_object_copies":144248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37153,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72124,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":210167,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10487,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160818,"ondisk_log_size":160818,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1821932,"kb_used_data":766316,"kb_used_omap":12,"kb_used_meta":1055539,"kb_avail":281293588,"statfs":{"total":289910292480,"available":288044634112,"internally_reserved":0,"allocated":784707584,"data_stored":783842355,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1080871940},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,2,2,3,5,6,7,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22904832,"num_objects":-5592,"num_object_clones":0,"num_object_copies":-11184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-461,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5592,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":461,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002840"},"pg_stats":[{"pgid":"2.f","version":"47'15686","reported_seq":15767,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:42.162576+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:19:42.162576+0000","last_peered":"2024-09-11T01:19:42.162576+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:19:42.162576+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18759680,"num_objects":4580,"num_object_clones":0,"num_object_copies":9160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4580,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15686,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":508}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'15862","reported_seq":15969,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:44.681097+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:19:44.681097+0000","last_peered":"2024-09-11T01:19:44.681097+0000","last_clean":"2024-09-11T01:19:44.681097+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:19:44.681097+0000","last_undegraded":"2024-09-11T01:19:44.681097+0000","last_fullsized":"2024-09-11T01:19:44.681097+0000","mapping_epoch":35,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18391040,"num_objects":4490,"num_object_clones":0,"num_object_copies":8980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14101,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15590","reported_seq":15722,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:44.786661+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:19:44.786661+0000","last_peered":"2024-09-11T01:19:44.786661+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:19:44.786661+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18649112,"num_objects":4554,"num_object_clones":0,"num_object_copies":9108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4554,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13891,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":658}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15647","reported_seq":15722,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:44.744979+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:19:44.744979+0000","last_peered":"2024-09-11T01:19:44.744979+0000","last_clean":"2024-09-11T01:19:44.744979+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:19:44.744979+0000","last_undegraded":"2024-09-11T01:19:44.744979+0000","last_fullsized":"2024-09-11T01:19:44.744979+0000","mapping_epoch":35,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18378752,"num_objects":4487,"num_object_clones":0,"num_object_copies":8974,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4487,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15647,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15526","reported_seq":15659,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:42.114091+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:19:42.114091+0000","last_peered":"2024-09-11T01:19:42.114091+0000","last_clean":"2024-09-11T01:19:42.114091+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:19:42.114091+0000","last_undegraded":"2024-09-11T01:19:42.114091+0000","last_fullsized":"2024-09-11T01:19:42.114091+0000","mapping_epoch":34,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18362368,"num_objects":4483,"num_object_clones":0,"num_object_copies":8966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4483,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12531,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15575","reported_seq":15746,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:42.119617+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:19:42.119617+0000","last_peered":"2024-09-11T01:19:42.119617+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:19:42.119617+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19054592,"num_objects":4652,"num_object_clones":0,"num_object_copies":9304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4652,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11916,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":701}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15401","reported_seq":15537,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:42.048485+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:19:42.048485+0000","last_peered":"2024-09-11T01:19:42.048485+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:19:42.048485+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18550784,"num_objects":4529,"num_object_clones":0,"num_object_copies":9058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4529,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12258,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15685","reported_seq":15820,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:42.048948+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:19:42.048948+0000","last_peered":"2024-09-11T01:19:42.048948+0000","last_clean":"2024-09-11T01:19:42.048948+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:19:42.048948+0000","last_undegraded":"2024-09-11T01:19:42.048948+0000","last_fullsized":"2024-09-11T01:19:42.048948+0000","mapping_epoch":34,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18714624,"num_objects":4569,"num_object_clones":0,"num_object_copies":9138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4569,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12561,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934737+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934737+0000","last_peered":"2024-09-11T01:17:58.934737+0000","last_clean":"2024-09-11T01:17:58.934737+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934737+0000","last_undegraded":"2024-09-11T01:17:58.934737+0000","last_fullsized":"2024-09-11T01:17:58.934737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15545","reported_seq":20911,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:44.814762+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:19:44.814762+0000","last_peered":"2024-09-11T01:19:44.814762+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:19:44.814762+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18137088,"num_objects":4428,"num_object_clones":0,"num_object_copies":8856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1319,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13516,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2616,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3109}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15414","reported_seq":15547,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:42.120821+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:19:42.120821+0000","last_peered":"2024-09-11T01:19:42.120821+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:19:42.120821+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18243584,"num_objects":4454,"num_object_clones":0,"num_object_copies":8908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12001,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":470}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15739","reported_seq":15863,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:44.742175+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:19:44.742175+0000","last_peered":"2024-09-11T01:19:44.742175+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:19:44.742175+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18300928,"num_objects":4468,"num_object_clones":0,"num_object_copies":8936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4468,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13556,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":444}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935141+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935141+0000","last_peered":"2024-09-11T01:17:58.935141+0000","last_clean":"2024-09-11T01:17:58.935141+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935141+0000","last_undegraded":"2024-09-11T01:17:58.935141+0000","last_fullsized":"2024-09-11T01:17:58.935141+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15376","reported_seq":23362,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:44.781005+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:19:44.781005+0000","last_peered":"2024-09-11T01:19:44.781005+0000","last_clean":"2024-09-11T01:19:44.781005+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:19:44.781005+0000","last_undegraded":"2024-09-11T01:19:44.781005+0000","last_fullsized":"2024-09-11T01:19:44.781005+0000","mapping_epoch":44,"log_start":"43'5300","ondisk_log_start":"43'5300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18104320,"num_objects":4420,"num_object_clones":0,"num_object_copies":8840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13161,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15608","reported_seq":15743,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:42.127952+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:19:42.127952+0000","last_peered":"2024-09-11T01:19:42.127952+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:19:42.127952+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19070976,"num_objects":4656,"num_object_clones":0,"num_object_copies":9312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4656,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12345,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":706}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938599+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938599+0000","last_peered":"2024-09-11T01:17:58.938599+0000","last_clean":"2024-09-11T01:17:58.938599+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938599+0000","last_undegraded":"2024-09-11T01:17:58.938599+0000","last_fullsized":"2024-09-11T01:17:58.938599+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15519","reported_seq":23554,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:46.566851+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:19:46.566851+0000","last_peered":"2024-09-11T01:19:46.566851+0000","last_clean":"2024-09-11T01:19:46.566851+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:19:46.566851+0000","last_undegraded":"2024-09-11T01:19:46.566851+0000","last_fullsized":"2024-09-11T01:19:46.566851+0000","mapping_epoch":46,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17879040,"num_objects":4365,"num_object_clones":0,"num_object_copies":8730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12174,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15651","reported_seq":15791,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:42.133056+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:19:42.133056+0000","last_peered":"2024-09-11T01:19:42.133056+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:19:42.133056+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18534400,"num_objects":4525,"num_object_clones":0,"num_object_copies":9050,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4525,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12260,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":486}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971371+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971371+0000","last_peered":"2024-09-11T01:17:58.971371+0000","last_clean":"2024-09-11T01:17:58.971371+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971371+0000","last_undegraded":"2024-09-11T01:17:58.971371+0000","last_fullsized":"2024-09-11T01:17:58.971371+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938303+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938303+0000","last_peered":"2024-09-11T01:17:58.938303+0000","last_clean":"2024-09-11T01:17:58.938303+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938303+0000","last_undegraded":"2024-09-11T01:17:58.938303+0000","last_fullsized":"2024-09-11T01:17:58.938303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15491","reported_seq":15630,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:42.144466+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:19:42.144466+0000","last_peered":"2024-09-11T01:19:42.144466+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:19:42.144466+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'5400","ondisk_log_start":"43'5400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18272873,"num_objects":4462,"num_object_clones":0,"num_object_copies":8924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12561,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":545}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":295404161,"num_objects":72122,"num_object_clones":0,"num_object_copies":144244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37153,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72122,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":210165,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10486,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":782090240,"data_stored":782019474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160815,"ondisk_log_size":160815,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182498,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":670568,"kb_used_data":282136,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93701272,"statfs":{"total":96636764160,"available":95950102528,"internally_reserved":0,"allocated":288907264,"data_stored":288637605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":397733547},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,0,2,2,3,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986271,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591768,"kb_used_data":271364,"kb_used_omap":4,"kb_used_meta":320379,"kb_avail":93780072,"statfs":{"total":96636764160,"available":96030793728,"internally_reserved":0,"allocated":277876736,"data_stored":277607082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":328068778},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,1,2,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659216,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":559596,"kb_used_data":212816,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93812244,"statfs":{"total":96636764160,"available":96063737856,"internally_reserved":0,"allocated":217923584,"data_stored":217597668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":355069615},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":288030720,"data_stored":288027241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":217300992,"data_stored":217241256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":276758528,"data_stored":276750977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:50.433 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:51.339 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:19:51.340 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:19:51.412 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":304,"stamp":"2024-09-11T01:19:50.720111+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":286663316,"num_objects":69990,"num_object_clones":0,"num_object_copies":139980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37008,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69990,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":212301,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10632,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160652,"ondisk_log_size":160652,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1818752,"kb_used_data":754804,"kb_used_omap":12,"kb_used_meta":1063859,"kb_avail":281296768,"statfs":{"total":289910292480,"available":288047890432,"internally_reserved":0,"allocated":772919296,"data_stored":772054067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1089391620},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,2,2,1,4,9,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22306816,"num_objects":-5446,"num_object_clones":0,"num_object_copies":-10892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-468,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5446,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":468,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002854"},"pg_stats":[{"pgid":"2.f","version":"47'15817","reported_seq":15900,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:47.156034+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:19:47.156034+0000","last_peered":"2024-09-11T01:19:47.156034+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:19:47.156034+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18223104,"num_objects":4449,"num_object_clones":0,"num_object_copies":8898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15817,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":377}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16008","reported_seq":16115,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:49.611940+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:19:49.611940+0000","last_peered":"2024-09-11T01:19:49.611940+0000","last_clean":"2024-09-11T01:19:49.611940+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:19:49.611940+0000","last_undegraded":"2024-09-11T01:19:49.611940+0000","last_fullsized":"2024-09-11T01:19:49.611940+0000","mapping_epoch":35,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17793024,"num_objects":4344,"num_object_clones":0,"num_object_copies":8688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14247,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15743","reported_seq":15877,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:49.836985+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:19:49.836985+0000","last_peered":"2024-09-11T01:19:49.836985+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:19:49.836985+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18022424,"num_objects":4401,"num_object_clones":0,"num_object_copies":8802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4401,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14044,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":505}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15791","reported_seq":15866,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:49.763081+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:19:49.763081+0000","last_peered":"2024-09-11T01:19:49.763081+0000","last_clean":"2024-09-11T01:19:49.763081+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:19:49.763081+0000","last_undegraded":"2024-09-11T01:19:49.763081+0000","last_fullsized":"2024-09-11T01:19:49.763081+0000","mapping_epoch":35,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17788928,"num_objects":4343,"num_object_clones":0,"num_object_copies":8686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15791,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15663","reported_seq":15796,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:47.177108+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:19:47.177108+0000","last_peered":"2024-09-11T01:19:47.177108+0000","last_clean":"2024-09-11T01:19:47.177108+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:19:47.177108+0000","last_undegraded":"2024-09-11T01:19:47.177108+0000","last_fullsized":"2024-09-11T01:19:47.177108+0000","mapping_epoch":34,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17801216,"num_objects":4346,"num_object_clones":0,"num_object_copies":8692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12668,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15720","reported_seq":15896,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:47.140060+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:19:47.140060+0000","last_peered":"2024-09-11T01:19:47.140060+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:19:47.140060+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18460672,"num_objects":4507,"num_object_clones":0,"num_object_copies":9014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12061,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":556}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15532","reported_seq":15668,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:47.116480+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:19:47.116480+0000","last_peered":"2024-09-11T01:19:47.116480+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:19:47.116480+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18014208,"num_objects":4398,"num_object_clones":0,"num_object_copies":8796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4398,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12389,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":412}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15849","reported_seq":15984,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:47.002549+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:19:47.002549+0000","last_peered":"2024-09-11T01:19:47.002549+0000","last_clean":"2024-09-11T01:19:47.002549+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:19:47.002549+0000","last_undegraded":"2024-09-11T01:19:47.002549+0000","last_fullsized":"2024-09-11T01:19:47.002549+0000","mapping_epoch":34,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18042880,"num_objects":4405,"num_object_clones":0,"num_object_copies":8810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12725,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934737+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934737+0000","last_peered":"2024-09-11T01:17:58.934737+0000","last_clean":"2024-09-11T01:17:58.934737+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934737+0000","last_undegraded":"2024-09-11T01:17:58.934737+0000","last_fullsized":"2024-09-11T01:17:58.934737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15705","reported_seq":21361,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:49.833006+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:19:49.833006+0000","last_peered":"2024-09-11T01:19:49.833006+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:19:49.833006+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17481728,"num_objects":4268,"num_object_clones":0,"num_object_copies":8536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13676,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2761,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3094}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15556","reported_seq":15690,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:47.141372+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:19:47.141372+0000","last_peered":"2024-09-11T01:19:47.141372+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:19:47.141372+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17661952,"num_objects":4312,"num_object_clones":0,"num_object_copies":8624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12143,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":328}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15876","reported_seq":16000,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:49.765146+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:19:49.765146+0000","last_peered":"2024-09-11T01:19:49.765146+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:19:49.765146+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17739776,"num_objects":4331,"num_object_clones":0,"num_object_copies":8662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13693,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":307}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935141+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935141+0000","last_peered":"2024-09-11T01:17:58.935141+0000","last_clean":"2024-09-11T01:17:58.935141+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935141+0000","last_undegraded":"2024-09-11T01:17:58.935141+0000","last_fullsized":"2024-09-11T01:17:58.935141+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15516","reported_seq":23502,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:49.812557+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:19:49.812557+0000","last_peered":"2024-09-11T01:19:49.812557+0000","last_clean":"2024-09-11T01:19:49.812557+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:19:49.812557+0000","last_undegraded":"2024-09-11T01:19:49.812557+0000","last_fullsized":"2024-09-11T01:19:49.812557+0000","mapping_epoch":44,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17530880,"num_objects":4280,"num_object_clones":0,"num_object_copies":8560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13301,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15745","reported_seq":15882,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:47.072560+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:19:47.072560+0000","last_peered":"2024-09-11T01:19:47.072560+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:19:47.072560+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18509824,"num_objects":4519,"num_object_clones":0,"num_object_copies":9038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12482,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":569}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938599+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938599+0000","last_peered":"2024-09-11T01:17:58.938599+0000","last_clean":"2024-09-11T01:17:58.938599+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938599+0000","last_undegraded":"2024-09-11T01:17:58.938599+0000","last_fullsized":"2024-09-11T01:17:58.938599+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15519","reported_seq":23554,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:46.566851+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:19:46.566851+0000","last_peered":"2024-09-11T01:19:46.566851+0000","last_clean":"2024-09-11T01:19:46.566851+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:19:46.566851+0000","last_undegraded":"2024-09-11T01:19:46.566851+0000","last_fullsized":"2024-09-11T01:19:46.566851+0000","mapping_epoch":46,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17879040,"num_objects":4365,"num_object_clones":0,"num_object_copies":8730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12174,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15794","reported_seq":15934,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:47.093297+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:19:47.093297+0000","last_peered":"2024-09-11T01:19:47.093297+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:19:47.093297+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17948672,"num_objects":4382,"num_object_clones":0,"num_object_copies":8764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4382,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12403,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":343}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971371+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971371+0000","last_peered":"2024-09-11T01:17:58.971371+0000","last_clean":"2024-09-11T01:17:58.971371+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971371+0000","last_undegraded":"2024-09-11T01:17:58.971371+0000","last_fullsized":"2024-09-11T01:17:58.971371+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938303+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938303+0000","last_peered":"2024-09-11T01:17:58.938303+0000","last_clean":"2024-09-11T01:17:58.938303+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938303+0000","last_undegraded":"2024-09-11T01:17:58.938303+0000","last_fullsized":"2024-09-11T01:17:58.938303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15615","reported_seq":15753,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:47.115294+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:19:47.115294+0000","last_peered":"2024-09-11T01:19:47.115294+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:19:47.115294+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17764969,"num_objects":4338,"num_object_clones":0,"num_object_copies":8676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12685,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":421}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":286663297,"num_objects":69988,"num_object_clones":0,"num_object_copies":139976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37008,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69988,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":212299,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10631,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":770260992,"data_stored":770190226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160649,"ondisk_log_size":160649,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182499,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":667980,"kb_used_data":275376,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93703860,"statfs":{"total":96636764160,"available":95952752640,"internally_reserved":0,"allocated":281985024,"data_stored":281715365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":401993387},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,0,0,0,1,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986272,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591176,"kb_used_data":266612,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93780664,"statfs":{"total":96636764160,"available":96031399936,"internally_reserved":0,"allocated":273010688,"data_stored":272741034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332328618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,1,2,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659216,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":559596,"kb_used_data":212816,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93812244,"statfs":{"total":96636764160,"available":96063737856,"internally_reserved":0,"allocated":217923584,"data_stored":217597668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":355069615},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":281063424,"data_stored":281059945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":217300992,"data_stored":217241256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":271896576,"data_stored":271889025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:51.414 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:52.274 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:19:52.275 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:19:52.336 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":304,"stamp":"2024-09-11T01:19:50.720111+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":286663316,"num_objects":69990,"num_object_clones":0,"num_object_copies":139980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37008,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69990,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":212301,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10632,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160652,"ondisk_log_size":160652,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1818752,"kb_used_data":754804,"kb_used_omap":12,"kb_used_meta":1063859,"kb_avail":281296768,"statfs":{"total":289910292480,"available":288047890432,"internally_reserved":0,"allocated":772919296,"data_stored":772054067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1089391620},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,2,2,1,4,9,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22306816,"num_objects":-5446,"num_object_clones":0,"num_object_copies":-10892,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-468,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5446,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5446,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":468,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002854"},"pg_stats":[{"pgid":"2.f","version":"47'15817","reported_seq":15900,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:47.156034+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:19:47.156034+0000","last_peered":"2024-09-11T01:19:47.156034+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:19:47.156034+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18223104,"num_objects":4449,"num_object_clones":0,"num_object_copies":8898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4449,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15817,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":377}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16008","reported_seq":16115,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:49.611940+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:19:49.611940+0000","last_peered":"2024-09-11T01:19:49.611940+0000","last_clean":"2024-09-11T01:19:49.611940+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:19:49.611940+0000","last_undegraded":"2024-09-11T01:19:49.611940+0000","last_fullsized":"2024-09-11T01:19:49.611940+0000","mapping_epoch":35,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17793024,"num_objects":4344,"num_object_clones":0,"num_object_copies":8688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14247,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15743","reported_seq":15877,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:49.836985+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:19:49.836985+0000","last_peered":"2024-09-11T01:19:49.836985+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:19:49.836985+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18022424,"num_objects":4401,"num_object_clones":0,"num_object_copies":8802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4401,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14044,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":505}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15791","reported_seq":15866,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:49.763081+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:19:49.763081+0000","last_peered":"2024-09-11T01:19:49.763081+0000","last_clean":"2024-09-11T01:19:49.763081+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:19:49.763081+0000","last_undegraded":"2024-09-11T01:19:49.763081+0000","last_fullsized":"2024-09-11T01:19:49.763081+0000","mapping_epoch":35,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17788928,"num_objects":4343,"num_object_clones":0,"num_object_copies":8686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15791,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15663","reported_seq":15796,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:47.177108+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:19:47.177108+0000","last_peered":"2024-09-11T01:19:47.177108+0000","last_clean":"2024-09-11T01:19:47.177108+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:19:47.177108+0000","last_undegraded":"2024-09-11T01:19:47.177108+0000","last_fullsized":"2024-09-11T01:19:47.177108+0000","mapping_epoch":34,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17801216,"num_objects":4346,"num_object_clones":0,"num_object_copies":8692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12668,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15720","reported_seq":15896,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:47.140060+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:19:47.140060+0000","last_peered":"2024-09-11T01:19:47.140060+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:19:47.140060+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18460672,"num_objects":4507,"num_object_clones":0,"num_object_copies":9014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12061,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":556}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15532","reported_seq":15668,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:47.116480+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:19:47.116480+0000","last_peered":"2024-09-11T01:19:47.116480+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:19:47.116480+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18014208,"num_objects":4398,"num_object_clones":0,"num_object_copies":8796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4398,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12389,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":412}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'15849","reported_seq":15984,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:47.002549+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:19:47.002549+0000","last_peered":"2024-09-11T01:19:47.002549+0000","last_clean":"2024-09-11T01:19:47.002549+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:19:47.002549+0000","last_undegraded":"2024-09-11T01:19:47.002549+0000","last_fullsized":"2024-09-11T01:19:47.002549+0000","mapping_epoch":34,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18042880,"num_objects":4405,"num_object_clones":0,"num_object_copies":8810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4405,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12725,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934737+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934737+0000","last_peered":"2024-09-11T01:17:58.934737+0000","last_clean":"2024-09-11T01:17:58.934737+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934737+0000","last_undegraded":"2024-09-11T01:17:58.934737+0000","last_fullsized":"2024-09-11T01:17:58.934737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15705","reported_seq":21361,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:49.833006+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:19:49.833006+0000","last_peered":"2024-09-11T01:19:49.833006+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:19:49.833006+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17481728,"num_objects":4268,"num_object_clones":0,"num_object_copies":8536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13676,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2761,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3094}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15556","reported_seq":15690,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:47.141372+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:19:47.141372+0000","last_peered":"2024-09-11T01:19:47.141372+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:19:47.141372+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17661952,"num_objects":4312,"num_object_clones":0,"num_object_copies":8624,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4312,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12143,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":328}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15876","reported_seq":16000,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:49.765146+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:19:49.765146+0000","last_peered":"2024-09-11T01:19:49.765146+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:19:49.765146+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17739776,"num_objects":4331,"num_object_clones":0,"num_object_copies":8662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13693,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":307}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935141+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935141+0000","last_peered":"2024-09-11T01:17:58.935141+0000","last_clean":"2024-09-11T01:17:58.935141+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935141+0000","last_undegraded":"2024-09-11T01:17:58.935141+0000","last_fullsized":"2024-09-11T01:17:58.935141+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15516","reported_seq":23502,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:49.812557+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:19:49.812557+0000","last_peered":"2024-09-11T01:19:49.812557+0000","last_clean":"2024-09-11T01:19:49.812557+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:19:49.812557+0000","last_undegraded":"2024-09-11T01:19:49.812557+0000","last_fullsized":"2024-09-11T01:19:49.812557+0000","mapping_epoch":44,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17530880,"num_objects":4280,"num_object_clones":0,"num_object_copies":8560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13301,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15745","reported_seq":15882,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:47.072560+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:19:47.072560+0000","last_peered":"2024-09-11T01:19:47.072560+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:19:47.072560+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18509824,"num_objects":4519,"num_object_clones":0,"num_object_copies":9038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12482,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":569}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938599+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938599+0000","last_peered":"2024-09-11T01:17:58.938599+0000","last_clean":"2024-09-11T01:17:58.938599+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938599+0000","last_undegraded":"2024-09-11T01:17:58.938599+0000","last_fullsized":"2024-09-11T01:17:58.938599+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15519","reported_seq":23554,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:46.566851+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:19:46.566851+0000","last_peered":"2024-09-11T01:19:46.566851+0000","last_clean":"2024-09-11T01:19:46.566851+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:19:46.566851+0000","last_undegraded":"2024-09-11T01:19:46.566851+0000","last_fullsized":"2024-09-11T01:19:46.566851+0000","mapping_epoch":46,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17879040,"num_objects":4365,"num_object_clones":0,"num_object_copies":8730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12174,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15794","reported_seq":15934,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:47.093297+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:19:47.093297+0000","last_peered":"2024-09-11T01:19:47.093297+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:19:47.093297+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17948672,"num_objects":4382,"num_object_clones":0,"num_object_copies":8764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4382,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12403,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":343}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971371+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971371+0000","last_peered":"2024-09-11T01:17:58.971371+0000","last_clean":"2024-09-11T01:17:58.971371+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971371+0000","last_undegraded":"2024-09-11T01:17:58.971371+0000","last_fullsized":"2024-09-11T01:17:58.971371+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938303+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938303+0000","last_peered":"2024-09-11T01:17:58.938303+0000","last_clean":"2024-09-11T01:17:58.938303+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938303+0000","last_undegraded":"2024-09-11T01:17:58.938303+0000","last_fullsized":"2024-09-11T01:17:58.938303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15615","reported_seq":15753,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:47.115294+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:19:47.115294+0000","last_peered":"2024-09-11T01:19:47.115294+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:19:47.115294+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17764969,"num_objects":4338,"num_object_clones":0,"num_object_copies":8676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12685,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":421}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":286663297,"num_objects":69988,"num_object_clones":0,"num_object_copies":139976,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37008,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69988,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":212299,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10631,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":770260992,"data_stored":770190226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160649,"ondisk_log_size":160649,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182499,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":667980,"kb_used_data":275376,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93703860,"statfs":{"total":96636764160,"available":95952752640,"internally_reserved":0,"allocated":281985024,"data_stored":281715365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":401993387},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[6,0,0,0,1,6,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986272,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591176,"kb_used_data":266612,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93780664,"statfs":{"total":96636764160,"available":96031399936,"internally_reserved":0,"allocated":273010688,"data_stored":272741034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332328618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,1,2,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659216,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":559596,"kb_used_data":212816,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93812244,"statfs":{"total":96636764160,"available":96063737856,"internally_reserved":0,"allocated":217923584,"data_stored":217597668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":355069615},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":281063424,"data_stored":281059945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":217300992,"data_stored":217241256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":271896576,"data_stored":271889025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:52.337 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:19:52.337 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'15817", 'reported_seq': 15900, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:47.156034+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:19:47.156034+0000', 'last_peered': '2024-09-11T01:19:47.156034+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:19:47.156034+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "43'5800", 'ondisk_log_start': "43'5800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18223104, 'num_objects': 4449, 'num_object_clones': 0, 'num_object_copies': 8898, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4449, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15817, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 377}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:52.337 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:19:52.337 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'15743", 'reported_seq': 15877, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:49.836985+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:19:49.836985+0000', 'last_peered': '2024-09-11T01:19:49.836985+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:19:49.836985+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "43'5700", 'ondisk_log_start': "43'5700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18022424, 'num_objects': 4401, 'num_object_clones': 0, 'num_object_copies': 8802, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4401, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14044, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 505}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:52.338 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:19:52.338 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'15720", 'reported_seq': 15896, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:47.140060+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:19:47.140060+0000', 'last_peered': '2024-09-11T01:19:47.140060+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:19:47.140060+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "43'5700", 'ondisk_log_start': "43'5700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18460672, 'num_objects': 4507, 'num_object_clones': 0, 'num_object_copies': 9014, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4507, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12061, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 556}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:52.338 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:19:52.338 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'15532", 'reported_seq': 15668, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:47.116480+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:19:47.116480+0000', 'last_peered': '2024-09-11T01:19:47.116480+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:19:47.116480+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "43'5500", 'ondisk_log_start': "43'5500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10032, 'log_dups_size': 0, 'ondisk_log_size': 10032, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18014208, 'num_objects': 4398, 'num_object_clones': 0, 'num_object_copies': 8796, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4398, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12389, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 412}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:52.338 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:19:52.338 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'15705", 'reported_seq': 21361, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:49.833006+0000', 'last_change': '2024-09-11T01:17:57.355893+0000', 'last_active': '2024-09-11T01:19:49.833006+0000', 'last_peered': '2024-09-11T01:19:49.833006+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:19:49.833006+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "43'5700", 'ondisk_log_start': "43'5700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 10005, 'log_dups_size': 0, 'ondisk_log_size': 10005, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17481728, 'num_objects': 4268, 'num_object_clones': 0, 'num_object_copies': 8536, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1174, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4268, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13676, 'num_write_kb': 31844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2761, 'num_bytes_recovered': 4505600, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:52.338 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:19:52.338 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'15556", 'reported_seq': 15690, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:47.141372+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:19:47.141372+0000', 'last_peered': '2024-09-11T01:19:47.141372+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:19:47.141372+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "43'5500", 'ondisk_log_start': "43'5500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17661952, 'num_objects': 4312, 'num_object_clones': 0, 'num_object_copies': 8624, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4312, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12143, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 328}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:52.338 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:19:52.338 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'15876", 'reported_seq': 16000, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:49.765146+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:19:49.765146+0000', 'last_peered': '2024-09-11T01:19:49.765146+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:19:49.765146+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "43'5800", 'ondisk_log_start': "43'5800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17739776, 'num_objects': 4331, 'num_object_clones': 0, 'num_object_copies': 8662, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4331, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13693, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 307}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:52.339 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:19:52.339 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'15745", 'reported_seq': 15882, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:47.072560+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:19:47.072560+0000', 'last_peered': '2024-09-11T01:19:47.072560+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:19:47.072560+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "43'5700", 'ondisk_log_start': "43'5700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 18509824, 'num_objects': 4519, 'num_object_clones': 0, 'num_object_copies': 9038, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4519, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12482, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 569}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:52.339 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:19:52.339 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'15794", 'reported_seq': 15934, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:47.093297+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:19:47.093297+0000', 'last_peered': '2024-09-11T01:19:47.093297+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:19:47.093297+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "43'5700", 'ondisk_log_start': "43'5700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17948672, 'num_objects': 4382, 'num_object_clones': 0, 'num_object_copies': 8764, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4382, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12403, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 343}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:52.339 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:19:52.339 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'15615", 'reported_seq': 15753, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:47.115294+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:19:47.115294+0000', 'last_peered': '2024-09-11T01:19:47.115294+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:19:47.115294+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "43'5600", 'ondisk_log_start': "43'5600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17764969, 'num_objects': 4338, 'num_object_clones': 0, 'num_object_copies': 8676, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4338, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12685, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 421}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:52.340 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:19:53.406 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:19:53.406 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 37008/137034 objects degraded (27.006%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":529,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":68517,"data_bytes":280629908,"bytes_used":1847840768,"bytes_avail":288062451712,"bytes_total":289910292480,"degraded_objects":37008,"degraded_total":137034,"degraded_ratio":0.27006436358859848,"recovering_objects_per_sec":25,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":503},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:19:53.469 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:19:53.470 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:54.268 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:19:54.270 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:19:54.347 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":305,"stamp":"2024-09-11T01:19:52.721502+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":280629908,"num_objects":68517,"num_object_clones":0,"num_object_copies":137034,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37008,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68517,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":213774,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10632,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160825,"ondisk_log_size":160825,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1804532,"kb_used_data":740608,"kb_used_omap":12,"kb_used_meta":1063859,"kb_avail":281310988,"statfs":{"total":289910292480,"available":288062451712,"internally_reserved":0,"allocated":758382592,"data_stored":757517363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1089391620},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,4,6,4,12,8,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24739840,"num_objects":-6040,"num_object_clones":0,"num_object_copies":-12080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-309,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6040,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6040,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":309,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003742"},"pg_stats":[{"pgid":"2.f","version":"47'15959","reported_seq":16042,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:52.152859+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:19:52.152859+0000","last_peered":"2024-09-11T01:19:52.152859+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:19:52.152859+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17641472,"num_objects":4307,"num_object_clones":0,"num_object_copies":8614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15959,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":235}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16008","reported_seq":16115,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:49.611940+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:19:49.611940+0000","last_peered":"2024-09-11T01:19:49.611940+0000","last_clean":"2024-09-11T01:19:49.611940+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:19:49.611940+0000","last_undegraded":"2024-09-11T01:19:49.611940+0000","last_fullsized":"2024-09-11T01:19:49.611940+0000","mapping_epoch":35,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17793024,"num_objects":4344,"num_object_clones":0,"num_object_copies":8688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14247,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15743","reported_seq":15877,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:49.836985+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:19:49.836985+0000","last_peered":"2024-09-11T01:19:49.836985+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:19:49.836985+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18022424,"num_objects":4401,"num_object_clones":0,"num_object_copies":8802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4401,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14044,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":505}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15791","reported_seq":15866,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:49.763081+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:19:49.763081+0000","last_peered":"2024-09-11T01:19:49.763081+0000","last_clean":"2024-09-11T01:19:49.763081+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:19:49.763081+0000","last_undegraded":"2024-09-11T01:19:49.763081+0000","last_fullsized":"2024-09-11T01:19:49.763081+0000","mapping_epoch":35,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17788928,"num_objects":4343,"num_object_clones":0,"num_object_copies":8686,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4343,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15791,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15820","reported_seq":15953,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:52.153334+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:19:52.153334+0000","last_peered":"2024-09-11T01:19:52.153334+0000","last_clean":"2024-09-11T01:19:52.153334+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:19:52.153334+0000","last_undegraded":"2024-09-11T01:19:52.153334+0000","last_fullsized":"2024-09-11T01:19:52.153334+0000","mapping_epoch":34,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17158144,"num_objects":4189,"num_object_clones":0,"num_object_copies":8378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12825,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15880","reported_seq":16056,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:52.148636+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:19:52.148636+0000","last_peered":"2024-09-11T01:19:52.148636+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:19:52.148636+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17805312,"num_objects":4347,"num_object_clones":0,"num_object_copies":8694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12221,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":396}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15667","reported_seq":15803,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:51.969123+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:19:51.969123+0000","last_peered":"2024-09-11T01:19:51.969123+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:19:51.969123+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17461248,"num_objects":4263,"num_object_clones":0,"num_object_copies":8526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12524,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":277}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16013","reported_seq":16148,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:52.074391+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:19:52.074391+0000","last_peered":"2024-09-11T01:19:52.074391+0000","last_clean":"2024-09-11T01:19:52.074391+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:19:52.074391+0000","last_undegraded":"2024-09-11T01:19:52.074391+0000","last_fullsized":"2024-09-11T01:19:52.074391+0000","mapping_epoch":34,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17371136,"num_objects":4241,"num_object_clones":0,"num_object_copies":8482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12889,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934737+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934737+0000","last_peered":"2024-09-11T01:17:58.934737+0000","last_clean":"2024-09-11T01:17:58.934737+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934737+0000","last_undegraded":"2024-09-11T01:17:58.934737+0000","last_fullsized":"2024-09-11T01:17:58.934737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15705","reported_seq":21361,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:49.833006+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:19:49.833006+0000","last_peered":"2024-09-11T01:19:49.833006+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:19:49.833006+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17481728,"num_objects":4268,"num_object_clones":0,"num_object_copies":8536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4268,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13676,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2761,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3094}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15689","reported_seq":15823,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:52.148479+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:19:52.148479+0000","last_peered":"2024-09-11T01:19:52.148479+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:19:52.148479+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17117184,"num_objects":4179,"num_object_clones":0,"num_object_copies":8358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12276,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":195}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'15876","reported_seq":16000,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:49.765146+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:19:49.765146+0000","last_peered":"2024-09-11T01:19:49.765146+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:19:49.765146+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17739776,"num_objects":4331,"num_object_clones":0,"num_object_copies":8662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13693,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":307}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935141+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935141+0000","last_peered":"2024-09-11T01:17:58.935141+0000","last_clean":"2024-09-11T01:17:58.935141+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935141+0000","last_undegraded":"2024-09-11T01:17:58.935141+0000","last_fullsized":"2024-09-11T01:17:58.935141+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15516","reported_seq":23502,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:49.812557+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:19:49.812557+0000","last_peered":"2024-09-11T01:19:49.812557+0000","last_clean":"2024-09-11T01:19:49.812557+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:19:49.812557+0000","last_undegraded":"2024-09-11T01:19:49.812557+0000","last_fullsized":"2024-09-11T01:19:49.812557+0000","mapping_epoch":44,"log_start":"43'5500","ondisk_log_start":"43'5500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17530880,"num_objects":4280,"num_object_clones":0,"num_object_copies":8560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4280,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13301,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15900","reported_seq":16037,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:52.129785+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:19:52.129785+0000","last_peered":"2024-09-11T01:19:52.129785+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:19:52.129785+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17874944,"num_objects":4364,"num_object_clones":0,"num_object_copies":8728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12637,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":414}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938599+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938599+0000","last_peered":"2024-09-11T01:17:58.938599+0000","last_clean":"2024-09-11T01:17:58.938599+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938599+0000","last_undegraded":"2024-09-11T01:17:58.938599+0000","last_fullsized":"2024-09-11T01:17:58.938599+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15663","reported_seq":23699,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:51.576963+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:19:51.576963+0000","last_peered":"2024-09-11T01:19:51.576963+0000","last_clean":"2024-09-11T01:19:51.576963+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:19:51.576963+0000","last_undegraded":"2024-09-11T01:19:51.576963+0000","last_fullsized":"2024-09-11T01:19:51.576963+0000","mapping_epoch":46,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17289216,"num_objects":4221,"num_object_clones":0,"num_object_copies":8442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12318,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15946","reported_seq":16086,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:52.150310+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:19:52.150310+0000","last_peered":"2024-09-11T01:19:52.150310+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:19:52.150310+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17326080,"num_objects":4230,"num_object_clones":0,"num_object_copies":8460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12555,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":191}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971371+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971371+0000","last_peered":"2024-09-11T01:17:58.971371+0000","last_clean":"2024-09-11T01:17:58.971371+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971371+0000","last_undegraded":"2024-09-11T01:17:58.971371+0000","last_fullsized":"2024-09-11T01:17:58.971371+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938303+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938303+0000","last_peered":"2024-09-11T01:17:58.938303+0000","last_clean":"2024-09-11T01:17:58.938303+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938303+0000","last_undegraded":"2024-09-11T01:17:58.938303+0000","last_fullsized":"2024-09-11T01:17:58.938303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15746","reported_seq":15885,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:52.148818+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:19:52.148818+0000","last_peered":"2024-09-11T01:19:52.148818+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:19:52.148818+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17228393,"num_objects":4207,"num_object_clones":0,"num_object_copies":8414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12816,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":290}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":280629889,"num_objects":68515,"num_object_clones":0,"num_object_copies":137030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":37008,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68515,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":213772,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10631,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":755888128,"data_stored":755817362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160822,"ondisk_log_size":160822,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182500,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":669136,"kb_used_data":268224,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93702704,"statfs":{"total":96636764160,"available":95951568896,"internally_reserved":0,"allocated":274661376,"data_stored":274391717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":410513067},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,3,5,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986272,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591176,"kb_used_data":266612,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93780664,"statfs":{"total":96636764160,"available":96031399936,"internally_reserved":0,"allocated":273010688,"data_stored":272741034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332328618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,1,1,2,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659217,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":544220,"kb_used_data":205772,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93827620,"statfs":{"total":96636764160,"available":96079482880,"internally_reserved":0,"allocated":210710528,"data_stored":210384612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":346549935},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,0,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":273780736,"data_stored":273777257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":210210816,"data_stored":210151080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":271896576,"data_stored":271889025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:57.349 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:58.139 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:19:58.141 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:19:58.177 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-09-11T01:19:56.724556+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":277234324,"num_objects":67688,"num_object_clones":0,"num_object_copies":135376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36833,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67688,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":214603,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10807,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160854,"ondisk_log_size":160854,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1804216,"kb_used_data":736136,"kb_used_omap":12,"kb_used_meta":1068019,"kb_avail":281311304,"statfs":{"total":289910292480,"available":288062775296,"internally_reserved":0,"allocated":753803264,"data_stored":752938035,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1093651460},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,6,4,11,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22331392,"num_objects":-5452,"num_object_clones":0,"num_object_copies":-10904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-484,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5452,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5452,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":484,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005772"},"pg_stats":[{"pgid":"2.f","version":"47'15959","reported_seq":16042,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:52.152859+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:19:52.152859+0000","last_peered":"2024-09-11T01:19:52.152859+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:19:52.152859+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17641472,"num_objects":4307,"num_object_clones":0,"num_object_copies":8614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4307,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15959,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":235}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16147","reported_seq":16254,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:54.795350+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:19:54.795350+0000","last_peered":"2024-09-11T01:19:54.795350+0000","last_clean":"2024-09-11T01:19:54.795350+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:19:54.795350+0000","last_undegraded":"2024-09-11T01:19:54.795350+0000","last_fullsized":"2024-09-11T01:19:54.795350+0000","mapping_epoch":35,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17223680,"num_objects":4205,"num_object_clones":0,"num_object_copies":8410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14386,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15874","reported_seq":16008,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:54.860097+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:19:54.860097+0000","last_peered":"2024-09-11T01:19:54.860097+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:19:54.860097+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17485848,"num_objects":4270,"num_object_clones":0,"num_object_copies":8540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4270,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14175,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":374}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15939","reported_seq":16014,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:54.795351+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:19:54.795351+0000","last_peered":"2024-09-11T01:19:54.795351+0000","last_clean":"2024-09-11T01:19:54.795351+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:19:54.795351+0000","last_undegraded":"2024-09-11T01:19:54.795351+0000","last_fullsized":"2024-09-11T01:19:54.795351+0000","mapping_epoch":35,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17182720,"num_objects":4195,"num_object_clones":0,"num_object_copies":8390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15939,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15820","reported_seq":15953,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:52.153334+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:19:52.153334+0000","last_peered":"2024-09-11T01:19:52.153334+0000","last_clean":"2024-09-11T01:19:52.153334+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:19:52.153334+0000","last_undegraded":"2024-09-11T01:19:52.153334+0000","last_fullsized":"2024-09-11T01:19:52.153334+0000","mapping_epoch":34,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17158144,"num_objects":4189,"num_object_clones":0,"num_object_copies":8378,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4189,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12825,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'15880","reported_seq":16056,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:52.148636+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:19:52.148636+0000","last_peered":"2024-09-11T01:19:52.148636+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:19:52.148636+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17805312,"num_objects":4347,"num_object_clones":0,"num_object_copies":8694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4347,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12221,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":396}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15667","reported_seq":15803,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:51.969123+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:19:51.969123+0000","last_peered":"2024-09-11T01:19:51.969123+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:19:51.969123+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17461248,"num_objects":4263,"num_object_clones":0,"num_object_copies":8526,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4263,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12524,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":277}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16013","reported_seq":16148,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:52.074391+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:19:52.074391+0000","last_peered":"2024-09-11T01:19:52.074391+0000","last_clean":"2024-09-11T01:19:52.074391+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:19:52.074391+0000","last_undegraded":"2024-09-11T01:19:52.074391+0000","last_fullsized":"2024-09-11T01:19:52.074391+0000","mapping_epoch":34,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17371136,"num_objects":4241,"num_object_clones":0,"num_object_copies":8482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12889,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934737+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934737+0000","last_peered":"2024-09-11T01:17:58.934737+0000","last_clean":"2024-09-11T01:17:58.934737+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934737+0000","last_undegraded":"2024-09-11T01:17:58.934737+0000","last_fullsized":"2024-09-11T01:17:58.934737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15839","reported_seq":21845,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:54.882737+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:19:54.882737+0000","last_peered":"2024-09-11T01:19:54.882737+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:19:54.882737+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16932864,"num_objects":4134,"num_object_clones":0,"num_object_copies":8268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":999,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13810,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2936,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3135}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15689","reported_seq":15823,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:52.148479+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:19:52.148479+0000","last_peered":"2024-09-11T01:19:52.148479+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:19:52.148479+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17117184,"num_objects":4179,"num_object_clones":0,"num_object_copies":8358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4179,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12276,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":195}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16008","reported_seq":16132,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:54.848539+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:19:54.848539+0000","last_peered":"2024-09-11T01:19:54.848539+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:19:54.848539+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17199104,"num_objects":4199,"num_object_clones":0,"num_object_copies":8398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13825,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":175}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935141+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935141+0000","last_peered":"2024-09-11T01:17:58.935141+0000","last_clean":"2024-09-11T01:17:58.935141+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935141+0000","last_undegraded":"2024-09-11T01:17:58.935141+0000","last_fullsized":"2024-09-11T01:17:58.935141+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15661","reported_seq":23647,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:54.866277+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:19:54.866277+0000","last_peered":"2024-09-11T01:19:54.866277+0000","last_clean":"2024-09-11T01:19:54.866277+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:19:54.866277+0000","last_undegraded":"2024-09-11T01:19:54.866277+0000","last_fullsized":"2024-09-11T01:19:54.866277+0000","mapping_epoch":44,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16936960,"num_objects":4135,"num_object_clones":0,"num_object_copies":8270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13446,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'15900","reported_seq":16037,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:52.129785+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:19:52.129785+0000","last_peered":"2024-09-11T01:19:52.129785+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:19:52.129785+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17874944,"num_objects":4364,"num_object_clones":0,"num_object_copies":8728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12637,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":414}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938599+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938599+0000","last_peered":"2024-09-11T01:17:58.938599+0000","last_clean":"2024-09-11T01:17:58.938599+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938599+0000","last_undegraded":"2024-09-11T01:17:58.938599+0000","last_fullsized":"2024-09-11T01:17:58.938599+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15663","reported_seq":23699,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:51.576963+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:19:51.576963+0000","last_peered":"2024-09-11T01:19:51.576963+0000","last_clean":"2024-09-11T01:19:51.576963+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:19:51.576963+0000","last_undegraded":"2024-09-11T01:19:51.576963+0000","last_fullsized":"2024-09-11T01:19:51.576963+0000","mapping_epoch":46,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17289216,"num_objects":4221,"num_object_clones":0,"num_object_copies":8442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12318,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'15946","reported_seq":16086,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:52.150310+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:19:52.150310+0000","last_peered":"2024-09-11T01:19:52.150310+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:19:52.150310+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17326080,"num_objects":4230,"num_object_clones":0,"num_object_copies":8460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12555,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":191}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971371+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971371+0000","last_peered":"2024-09-11T01:17:58.971371+0000","last_clean":"2024-09-11T01:17:58.971371+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971371+0000","last_undegraded":"2024-09-11T01:17:58.971371+0000","last_fullsized":"2024-09-11T01:17:58.971371+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938303+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938303+0000","last_peered":"2024-09-11T01:17:58.938303+0000","last_clean":"2024-09-11T01:17:58.938303+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938303+0000","last_undegraded":"2024-09-11T01:17:58.938303+0000","last_fullsized":"2024-09-11T01:17:58.938303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15746","reported_seq":15885,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:52.148818+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:19:52.148818+0000","last_peered":"2024-09-11T01:19:52.148818+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:19:52.148818+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17228393,"num_objects":4207,"num_object_clones":0,"num_object_copies":8414,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4207,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12816,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":290}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":277234305,"num_objects":67686,"num_object_clones":0,"num_object_copies":135372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36833,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67686,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":214601,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10806,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":751280128,"data_stored":751209362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160851,"ondisk_log_size":160851,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182500,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":669136,"kb_used_data":268224,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93702704,"statfs":{"total":96636764160,"available":95951568896,"internally_reserved":0,"allocated":274661376,"data_stored":274391717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":410513067},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,3,5,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986273,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":590860,"kb_used_data":262140,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93780980,"statfs":{"total":96636764160,"available":96031723520,"internally_reserved":0,"allocated":268431360,"data_stored":268161706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336588458},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659217,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":544220,"kb_used_data":205772,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93827620,"statfs":{"total":96636764160,"available":96079482880,"internally_reserved":0,"allocated":210710528,"data_stored":210384612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":346549935},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,0,5,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":273780736,"data_stored":273777257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":210210816,"data_stored":210151080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":267288576,"data_stored":267281025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:58.179 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:19:58.980 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:19:58.983 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:19:59.036 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":308,"stamp":"2024-09-11T01:19:58.724999+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":271159956,"num_objects":66205,"num_object_clones":0,"num_object_copies":132410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36833,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66205,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":216086,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10807,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160637,"ondisk_log_size":160637,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1786072,"kb_used_data":722148,"kb_used_omap":12,"kb_used_meta":1063859,"kb_avail":281329448,"statfs":{"total":289910292480,"available":288081354752,"internally_reserved":0,"allocated":739479552,"data_stored":738614323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1089391620},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,2,5,9,8,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24244224,"num_objects":-5919,"num_object_clones":0,"num_object_copies":-11838,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-320,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5919,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5919,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":320,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005799"},"pg_stats":[{"pgid":"2.f","version":"47'16107","reported_seq":16190,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:57.097595+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:19:57.097595+0000","last_peered":"2024-09-11T01:19:57.097595+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:19:57.097595+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17035264,"num_objects":4159,"num_object_clones":0,"num_object_copies":8318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16107,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":87}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16147","reported_seq":16254,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:54.795350+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:19:54.795350+0000","last_peered":"2024-09-11T01:19:54.795350+0000","last_clean":"2024-09-11T01:19:54.795350+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:19:54.795350+0000","last_undegraded":"2024-09-11T01:19:54.795350+0000","last_fullsized":"2024-09-11T01:19:54.795350+0000","mapping_epoch":35,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17223680,"num_objects":4205,"num_object_clones":0,"num_object_copies":8410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4205,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14386,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'15874","reported_seq":16008,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:54.860097+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:19:54.860097+0000","last_peered":"2024-09-11T01:19:54.860097+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:19:54.860097+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17485848,"num_objects":4270,"num_object_clones":0,"num_object_copies":8540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4270,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14175,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":374}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'15939","reported_seq":16014,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:54.795351+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:19:54.795351+0000","last_peered":"2024-09-11T01:19:54.795351+0000","last_clean":"2024-09-11T01:19:54.795351+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:19:54.795351+0000","last_undegraded":"2024-09-11T01:19:54.795351+0000","last_fullsized":"2024-09-11T01:19:54.795351+0000","mapping_epoch":35,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17182720,"num_objects":4195,"num_object_clones":0,"num_object_copies":8390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15939,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15961","reported_seq":16094,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:57.122896+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:19:57.122896+0000","last_peered":"2024-09-11T01:19:57.122896+0000","last_clean":"2024-09-11T01:19:57.122896+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:19:57.122896+0000","last_undegraded":"2024-09-11T01:19:57.122896+0000","last_fullsized":"2024-09-11T01:19:57.122896+0000","mapping_epoch":34,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16580608,"num_objects":4048,"num_object_clones":0,"num_object_copies":8096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12966,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16036","reported_seq":16212,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:57.178154+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:19:57.178154+0000","last_peered":"2024-09-11T01:19:57.178154+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:19:57.178154+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17166336,"num_objects":4191,"num_object_clones":0,"num_object_copies":8382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12377,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":240}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15794","reported_seq":15930,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:56.966269+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:19:56.966269+0000","last_peered":"2024-09-11T01:19:56.966269+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:19:56.966269+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16941056,"num_objects":4136,"num_object_clones":0,"num_object_copies":8272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12651,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":150}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16144","reported_seq":16279,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:57.137381+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:19:57.137381+0000","last_peered":"2024-09-11T01:19:57.137381+0000","last_clean":"2024-09-11T01:19:57.137381+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:19:57.137381+0000","last_undegraded":"2024-09-11T01:19:57.137381+0000","last_fullsized":"2024-09-11T01:19:57.137381+0000","mapping_epoch":34,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16834560,"num_objects":4110,"num_object_clones":0,"num_object_copies":8220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13020,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934737+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934737+0000","last_peered":"2024-09-11T01:17:58.934737+0000","last_clean":"2024-09-11T01:17:58.934737+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934737+0000","last_undegraded":"2024-09-11T01:17:58.934737+0000","last_fullsized":"2024-09-11T01:17:58.934737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15839","reported_seq":21845,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:54.882737+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:19:54.882737+0000","last_peered":"2024-09-11T01:19:54.882737+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:19:54.882737+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16932864,"num_objects":4134,"num_object_clones":0,"num_object_copies":8268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":999,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4134,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13810,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2936,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3135}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15824","reported_seq":15958,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:57.177075+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:19:57.177075+0000","last_peered":"2024-09-11T01:19:57.177075+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:19:57.177075+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16564224,"num_objects":4044,"num_object_clones":0,"num_object_copies":8088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12411,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":60}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16008","reported_seq":16132,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:54.848539+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:19:54.848539+0000","last_peered":"2024-09-11T01:19:54.848539+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:19:54.848539+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17199104,"num_objects":4199,"num_object_clones":0,"num_object_copies":8398,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4199,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13825,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":175}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935141+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935141+0000","last_peered":"2024-09-11T01:17:58.935141+0000","last_clean":"2024-09-11T01:17:58.935141+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935141+0000","last_undegraded":"2024-09-11T01:17:58.935141+0000","last_fullsized":"2024-09-11T01:17:58.935141+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15661","reported_seq":23647,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:54.866277+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:19:54.866277+0000","last_peered":"2024-09-11T01:19:54.866277+0000","last_clean":"2024-09-11T01:19:54.866277+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:19:54.866277+0000","last_undegraded":"2024-09-11T01:19:54.866277+0000","last_fullsized":"2024-09-11T01:19:54.866277+0000","mapping_epoch":44,"log_start":"43'5600","ondisk_log_start":"43'5600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16936960,"num_objects":4135,"num_object_clones":0,"num_object_copies":8270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13446,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16064","reported_seq":16201,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:57.196830+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:19:57.196830+0000","last_peered":"2024-09-11T01:19:57.196830+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:19:57.196830+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17203200,"num_objects":4200,"num_object_clones":0,"num_object_copies":8400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12801,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":250}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938599+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938599+0000","last_peered":"2024-09-11T01:17:58.938599+0000","last_clean":"2024-09-11T01:17:58.938599+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938599+0000","last_undegraded":"2024-09-11T01:17:58.938599+0000","last_fullsized":"2024-09-11T01:17:58.938599+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15819","reported_seq":23855,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:56.659940+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:19:56.659940+0000","last_peered":"2024-09-11T01:19:56.659940+0000","last_clean":"2024-09-11T01:19:56.659940+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:19:56.659940+0000","last_undegraded":"2024-09-11T01:19:56.659940+0000","last_fullsized":"2024-09-11T01:19:56.659940+0000","mapping_epoch":46,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16650240,"num_objects":4065,"num_object_clones":0,"num_object_copies":8130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12474,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16111","reported_seq":16247,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:57.214389+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:19:57.214389+0000","last_peered":"2024-09-11T01:19:57.214389+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:19:57.214389+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16650240,"num_objects":4065,"num_object_clones":0,"num_object_copies":8130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12720,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":26}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971371+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971371+0000","last_peered":"2024-09-11T01:17:58.971371+0000","last_clean":"2024-09-11T01:17:58.971371+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971371+0000","last_undegraded":"2024-09-11T01:17:58.971371+0000","last_fullsized":"2024-09-11T01:17:58.971371+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938303+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938303+0000","last_peered":"2024-09-11T01:17:58.938303+0000","last_clean":"2024-09-11T01:17:58.938303+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938303+0000","last_undegraded":"2024-09-11T01:17:58.938303+0000","last_fullsized":"2024-09-11T01:17:58.938303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15906","reported_seq":16045,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:57.172399+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:19:57.172399+0000","last_peered":"2024-09-11T01:19:57.172399+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:19:57.172399+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16573033,"num_objects":4047,"num_object_clones":0,"num_object_copies":8094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12976,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":271159937,"num_objects":66203,"num_object_clones":0,"num_object_copies":132406,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36833,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66203,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":216084,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10806,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":736825344,"data_stored":736754578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160634,"ondisk_log_size":160634,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182501,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":649704,"kb_used_data":261268,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93722136,"statfs":{"total":96636764160,"available":95971467264,"internally_reserved":0,"allocated":267538432,"data_stored":267268773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":397733547},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,2,3,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986273,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":590860,"kb_used_data":262140,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93780980,"statfs":{"total":96636764160,"available":96031723520,"internally_reserved":0,"allocated":268431360,"data_stored":268161706,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336588458},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659218,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545508,"kb_used_data":198740,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93826332,"statfs":{"total":96636764160,"available":96078163968,"internally_reserved":0,"allocated":203509760,"data_stored":203183844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":355069615},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,5,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":266539008,"data_stored":266535529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":202997760,"data_stored":202938024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":267288576,"data_stored":267281025,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:19:59.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:19:59.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'16107", 'reported_seq': 16190, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:57.097595+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:19:57.097595+0000', 'last_peered': '2024-09-11T01:19:57.097595+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:19:57.097595+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "43'6100", 'ondisk_log_start': "43'6100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17035264, 'num_objects': 4159, 'num_object_clones': 0, 'num_object_copies': 8318, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4159, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16107, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 87}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:59.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:19:59.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'15874", 'reported_seq': 16008, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:54.860097+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:19:54.860097+0000', 'last_peered': '2024-09-11T01:19:54.860097+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:19:54.860097+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "43'5800", 'ondisk_log_start': "43'5800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17485848, 'num_objects': 4270, 'num_object_clones': 0, 'num_object_copies': 8540, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4270, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14175, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 374}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:59.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:19:59.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'16036", 'reported_seq': 16212, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:57.178154+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:19:57.178154+0000', 'last_peered': '2024-09-11T01:19:57.178154+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:19:57.178154+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "43'6000", 'ondisk_log_start': "43'6000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17166336, 'num_objects': 4191, 'num_object_clones': 0, 'num_object_copies': 8382, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4191, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12377, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 240}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:59.038 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:19:59.038 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'15794", 'reported_seq': 15930, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:56.966269+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:19:56.966269+0000', 'last_peered': '2024-09-11T01:19:56.966269+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:19:56.966269+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "43'5700", 'ondisk_log_start': "43'5700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16941056, 'num_objects': 4136, 'num_object_clones': 0, 'num_object_copies': 8272, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4136, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12651, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 150}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:59.039 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:19:59.040 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'15839", 'reported_seq': 21845, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:54.882737+0000', 'last_change': '2024-09-11T01:17:57.355893+0000', 'last_active': '2024-09-11T01:19:54.882737+0000', 'last_peered': '2024-09-11T01:19:54.882737+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:19:54.882737+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "43'5800", 'ondisk_log_start': "43'5800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16932864, 'num_objects': 4134, 'num_object_clones': 0, 'num_object_copies': 8268, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 999, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4134, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13810, 'num_write_kb': 31844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2936, 'num_bytes_recovered': 4505600, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3135}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:59.040 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:19:59.040 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'15824", 'reported_seq': 15958, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:57.177075+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:19:57.177075+0000', 'last_peered': '2024-09-11T01:19:57.177075+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:19:57.177075+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "43'5800", 'ondisk_log_start': "43'5800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10024, 'log_dups_size': 0, 'ondisk_log_size': 10024, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16564224, 'num_objects': 4044, 'num_object_clones': 0, 'num_object_copies': 8088, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4044, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12411, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 60}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:59.040 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:19:59.040 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'16008", 'reported_seq': 16132, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:54.848539+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:19:54.848539+0000', 'last_peered': '2024-09-11T01:19:54.848539+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:19:54.848539+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "43'6000", 'ondisk_log_start': "43'6000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10008, 'log_dups_size': 0, 'ondisk_log_size': 10008, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17199104, 'num_objects': 4199, 'num_object_clones': 0, 'num_object_copies': 8398, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4199, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13825, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 175}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:19:59.040 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:19:59.040 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'16064", 'reported_seq': 16201, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:57.196830+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:19:57.196830+0000', 'last_peered': '2024-09-11T01:19:57.196830+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:19:57.196830+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "43'6000", 'ondisk_log_start': "43'6000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 17203200, 'num_objects': 4200, 'num_object_clones': 0, 'num_object_copies': 8400, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4200, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12801, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 250}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:59.040 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:19:59.040 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'16111", 'reported_seq': 16247, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:57.214389+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:19:57.214389+0000', 'last_peered': '2024-09-11T01:19:57.214389+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:19:57.214389+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "43'6100", 'ondisk_log_start': "43'6100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16650240, 'num_objects': 4065, 'num_object_clones': 0, 'num_object_copies': 8130, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4065, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12720, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 26}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:59.040 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:19:59.041 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'15906", 'reported_seq': 16045, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:57.172399+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:19:57.172399+0000', 'last_peered': '2024-09-11T01:19:57.172399+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:19:57.172399+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "43'5900", 'ondisk_log_start': "43'5900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16573033, 'num_objects': 4047, 'num_object_clones': 0, 'num_object_copies': 8094, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4047, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12976, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 130}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:19:59.041 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:20:00.022 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:20:00.022 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36833/132410 objects degraded (27.817%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":536,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":66205,"data_bytes":271159956,"bytes_used":1828937728,"bytes_avail":288081354752,"bytes_total":289910292480,"degraded_objects":36833,"degraded_total":132410,"degraded_ratio":0.27817385393852428,"recovering_objects_per_sec":26,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":320,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":493},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:20:00.081 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:20:00.082 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:00.846 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:20:00.847 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:20:00.900 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":309,"stamp":"2024-09-11T01:20:00.725470+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":267559572,"num_objects":65326,"num_object_clones":0,"num_object_copies":130652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36659,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65326,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":216965,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10981,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160816,"ondisk_log_size":160816,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1768956,"kb_used_data":717508,"kb_used_omap":12,"kb_used_meta":1051379,"kb_avail":281346564,"statfs":{"total":289910292480,"available":288098881536,"internally_reserved":0,"allocated":734728192,"data_stored":733862963,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1076612100},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,3,1,5,12,9,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22708224,"num_objects":-5544,"num_object_clones":0,"num_object_copies":-11088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-494,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5544,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5544,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":494,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005812"},"pg_stats":[{"pgid":"2.f","version":"47'16107","reported_seq":16190,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:57.097595+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:19:57.097595+0000","last_peered":"2024-09-11T01:19:57.097595+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:19:57.097595+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17035264,"num_objects":4159,"num_object_clones":0,"num_object_copies":8318,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4159,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16107,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":87}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16299","reported_seq":16406,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:59.909597+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:19:59.909597+0000","last_peered":"2024-09-11T01:19:59.909597+0000","last_clean":"2024-09-11T01:19:59.909597+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:19:59.909597+0000","last_undegraded":"2024-09-11T01:19:59.909597+0000","last_fullsized":"2024-09-11T01:19:59.909597+0000","mapping_epoch":35,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16601088,"num_objects":4053,"num_object_clones":0,"num_object_copies":8106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4053,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14538,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16033","reported_seq":16167,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:59.835270+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:19:59.835270+0000","last_peered":"2024-09-11T01:19:59.835270+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:19:59.835270+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16834584,"num_objects":4111,"num_object_clones":0,"num_object_copies":8222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4111,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14334,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":215}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16084","reported_seq":16159,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:59.944473+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:19:59.944473+0000","last_peered":"2024-09-11T01:19:59.944473+0000","last_clean":"2024-09-11T01:19:59.944473+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:19:59.944473+0000","last_undegraded":"2024-09-11T01:19:59.944473+0000","last_fullsized":"2024-09-11T01:19:59.944473+0000","mapping_epoch":35,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16588800,"num_objects":4050,"num_object_clones":0,"num_object_copies":8100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16084,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'15961","reported_seq":16094,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:57.122896+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:19:57.122896+0000","last_peered":"2024-09-11T01:19:57.122896+0000","last_clean":"2024-09-11T01:19:57.122896+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:19:57.122896+0000","last_undegraded":"2024-09-11T01:19:57.122896+0000","last_fullsized":"2024-09-11T01:19:57.122896+0000","mapping_epoch":34,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16580608,"num_objects":4048,"num_object_clones":0,"num_object_copies":8096,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4048,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12966,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16036","reported_seq":16212,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:57.178154+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:19:57.178154+0000","last_peered":"2024-09-11T01:19:57.178154+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:19:57.178154+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17166336,"num_objects":4191,"num_object_clones":0,"num_object_copies":8382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4191,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12377,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":240}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15794","reported_seq":15930,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:56.966269+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:19:56.966269+0000","last_peered":"2024-09-11T01:19:56.966269+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:19:56.966269+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16941056,"num_objects":4136,"num_object_clones":0,"num_object_copies":8272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4136,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12651,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":150}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16144","reported_seq":16279,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:57.137381+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:19:57.137381+0000","last_peered":"2024-09-11T01:19:57.137381+0000","last_clean":"2024-09-11T01:19:57.137381+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:19:57.137381+0000","last_undegraded":"2024-09-11T01:19:57.137381+0000","last_fullsized":"2024-09-11T01:19:57.137381+0000","mapping_epoch":34,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16834560,"num_objects":4110,"num_object_clones":0,"num_object_copies":8220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13020,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934737+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934737+0000","last_peered":"2024-09-11T01:17:58.934737+0000","last_clean":"2024-09-11T01:17:58.934737+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934737+0000","last_undegraded":"2024-09-11T01:17:58.934737+0000","last_fullsized":"2024-09-11T01:17:58.934737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15991","reported_seq":22344,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:59.873337+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:19:59.873337+0000","last_peered":"2024-09-11T01:19:59.873337+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:19:59.873337+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16310272,"num_objects":3982,"num_object_clones":0,"num_object_copies":7964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":825,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13962,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3110,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3157}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15824","reported_seq":15958,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:57.177075+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:19:57.177075+0000","last_peered":"2024-09-11T01:19:57.177075+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:19:57.177075+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16564224,"num_objects":4044,"num_object_clones":0,"num_object_copies":8088,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4044,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12411,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":60}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16146","reported_seq":16270,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:59.847840+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:19:59.847840+0000","last_peered":"2024-09-11T01:19:59.847840+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:19:59.847840+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16633856,"num_objects":4061,"num_object_clones":0,"num_object_copies":8122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4061,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13963,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":37}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935141+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935141+0000","last_peered":"2024-09-11T01:17:58.935141+0000","last_clean":"2024-09-11T01:17:58.935141+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935141+0000","last_undegraded":"2024-09-11T01:17:58.935141+0000","last_fullsized":"2024-09-11T01:17:58.935141+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15794","reported_seq":23780,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:59.904185+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:19:59.904185+0000","last_peered":"2024-09-11T01:19:59.904185+0000","last_clean":"2024-09-11T01:19:59.904185+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:19:59.904185+0000","last_undegraded":"2024-09-11T01:19:59.904185+0000","last_fullsized":"2024-09-11T01:19:59.904185+0000","mapping_epoch":44,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16392192,"num_objects":4002,"num_object_clones":0,"num_object_copies":8004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13579,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16064","reported_seq":16201,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:57.196830+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:19:57.196830+0000","last_peered":"2024-09-11T01:19:57.196830+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:19:57.196830+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":17203200,"num_objects":4200,"num_object_clones":0,"num_object_copies":8400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12801,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":250}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938599+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938599+0000","last_peered":"2024-09-11T01:17:58.938599+0000","last_clean":"2024-09-11T01:17:58.938599+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938599+0000","last_undegraded":"2024-09-11T01:17:58.938599+0000","last_fullsized":"2024-09-11T01:17:58.938599+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15819","reported_seq":23855,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:56.659940+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:19:56.659940+0000","last_peered":"2024-09-11T01:19:56.659940+0000","last_clean":"2024-09-11T01:19:56.659940+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:19:56.659940+0000","last_undegraded":"2024-09-11T01:19:56.659940+0000","last_fullsized":"2024-09-11T01:19:56.659940+0000","mapping_epoch":46,"log_start":"43'5800","ondisk_log_start":"43'5800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16650240,"num_objects":4065,"num_object_clones":0,"num_object_copies":8130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12474,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16111","reported_seq":16247,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:57.214389+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:19:57.214389+0000","last_peered":"2024-09-11T01:19:57.214389+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:19:57.214389+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16650240,"num_objects":4065,"num_object_clones":0,"num_object_copies":8130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12720,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":26}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971371+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971371+0000","last_peered":"2024-09-11T01:17:58.971371+0000","last_clean":"2024-09-11T01:17:58.971371+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971371+0000","last_undegraded":"2024-09-11T01:17:58.971371+0000","last_fullsized":"2024-09-11T01:17:58.971371+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938303+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938303+0000","last_peered":"2024-09-11T01:17:58.938303+0000","last_clean":"2024-09-11T01:17:58.938303+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938303+0000","last_undegraded":"2024-09-11T01:17:58.938303+0000","last_fullsized":"2024-09-11T01:17:58.938303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'15906","reported_seq":16045,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:57.172399+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:19:57.172399+0000","last_peered":"2024-09-11T01:19:57.172399+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:19:57.172399+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16573033,"num_objects":4047,"num_object_clones":0,"num_object_copies":8094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12976,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":130}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":267559553,"num_objects":65324,"num_object_clones":0,"num_object_copies":130648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36659,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65324,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":216963,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10980,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":732123136,"data_stored":732052370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160813,"ondisk_log_size":160813,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182501,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":649704,"kb_used_data":261268,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93722136,"statfs":{"total":96636764160,"available":95971467264,"internally_reserved":0,"allocated":267538432,"data_stored":267268773,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":397733547},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,0,2,3,6,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986274,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":573744,"kb_used_data":257500,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93798096,"statfs":{"total":96636764160,"available":96049250304,"internally_reserved":0,"allocated":263680000,"data_stored":263410346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323808938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,1,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659218,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545508,"kb_used_data":198740,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93826332,"statfs":{"total":96636764160,"available":96078163968,"internally_reserved":0,"allocated":203509760,"data_stored":203183844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":355069615},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,5,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":266539008,"data_stored":266535529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":202997760,"data_stored":202938024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":262586368,"data_stored":262578817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:03.902 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:04.673 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:20:04.673 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:20:04.746 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":310,"stamp":"2024-09-11T01:20:02.725970+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":261894804,"num_objects":63943,"num_object_clones":0,"num_object_copies":127886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36659,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63943,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":218348,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10981,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160999,"ondisk_log_size":160999,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1787468,"kb_used_data":704084,"kb_used_omap":12,"kb_used_meta":1083315,"kb_avail":281328052,"statfs":{"total":289910292480,"available":288079925248,"internally_reserved":0,"allocated":720982016,"data_stored":720116787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1109314564},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,5,12,7,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24768512,"num_objects":-6047,"num_object_clones":0,"num_object_copies":-12094,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-349,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6047,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6047,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005859"},"pg_stats":[{"pgid":"2.f","version":"47'16262","reported_seq":16345,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:02.112774+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:20:02.112774+0000","last_peered":"2024-09-11T01:20:02.112774+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:20:02.112774+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16400384,"num_objects":4004,"num_object_clones":0,"num_object_copies":8008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16262,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-68}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16299","reported_seq":16406,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:59.909597+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:19:59.909597+0000","last_peered":"2024-09-11T01:19:59.909597+0000","last_clean":"2024-09-11T01:19:59.909597+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:19:59.909597+0000","last_undegraded":"2024-09-11T01:19:59.909597+0000","last_fullsized":"2024-09-11T01:19:59.909597+0000","mapping_epoch":35,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16601088,"num_objects":4053,"num_object_clones":0,"num_object_copies":8106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4053,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14538,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16033","reported_seq":16167,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:59.835270+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:19:59.835270+0000","last_peered":"2024-09-11T01:19:59.835270+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:19:59.835270+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16834584,"num_objects":4111,"num_object_clones":0,"num_object_copies":8222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4111,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14334,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":215}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16084","reported_seq":16159,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:59.944473+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:19:59.944473+0000","last_peered":"2024-09-11T01:19:59.944473+0000","last_clean":"2024-09-11T01:19:59.944473+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:19:59.944473+0000","last_undegraded":"2024-09-11T01:19:59.944473+0000","last_fullsized":"2024-09-11T01:19:59.944473+0000","mapping_epoch":35,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16588800,"num_objects":4050,"num_object_clones":0,"num_object_copies":8100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16084,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16095","reported_seq":16228,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:02.195523+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:20:02.195523+0000","last_peered":"2024-09-11T01:20:02.195523+0000","last_clean":"2024-09-11T01:20:02.195523+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:20:02.195523+0000","last_undegraded":"2024-09-11T01:20:02.195523+0000","last_fullsized":"2024-09-11T01:20:02.195523+0000","mapping_epoch":34,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16031744,"num_objects":3914,"num_object_clones":0,"num_object_copies":7828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13100,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16184","reported_seq":16359,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:02.206818+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:20:02.206818+0000","last_peered":"2024-09-11T01:20:02.206818+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:20:02.206818+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16560128,"num_objects":4043,"num_object_clones":0,"num_object_copies":8086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12525,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":92}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15933","reported_seq":16069,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:02.181405+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:20:02.181405+0000","last_peered":"2024-09-11T01:20:02.181405+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:20:02.181405+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16371712,"num_objects":3997,"num_object_clones":0,"num_object_copies":7994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3997,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12790,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":11}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16282","reported_seq":16417,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:02.086636+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:20:02.086636+0000","last_peered":"2024-09-11T01:20:02.086636+0000","last_clean":"2024-09-11T01:20:02.086636+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:20:02.086636+0000","last_undegraded":"2024-09-11T01:20:02.086636+0000","last_fullsized":"2024-09-11T01:20:02.086636+0000","mapping_epoch":34,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16269312,"num_objects":3972,"num_object_clones":0,"num_object_copies":7944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13158,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934737+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934737+0000","last_peered":"2024-09-11T01:17:58.934737+0000","last_clean":"2024-09-11T01:17:58.934737+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934737+0000","last_undegraded":"2024-09-11T01:17:58.934737+0000","last_fullsized":"2024-09-11T01:17:58.934737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15991","reported_seq":22344,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:59.873337+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:19:59.873337+0000","last_peered":"2024-09-11T01:19:59.873337+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:19:59.873337+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16310272,"num_objects":3982,"num_object_clones":0,"num_object_copies":7964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":825,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13962,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3110,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3157}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15953","reported_seq":16087,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:02.182590+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:20:02.182590+0000","last_peered":"2024-09-11T01:20:02.182590+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:20:02.182590+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16035840,"num_objects":3915,"num_object_clones":0,"num_object_copies":7830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12540,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-69}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16146","reported_seq":16270,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:59.847840+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:19:59.847840+0000","last_peered":"2024-09-11T01:19:59.847840+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:19:59.847840+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16633856,"num_objects":4061,"num_object_clones":0,"num_object_copies":8122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4061,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13963,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":37}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935141+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935141+0000","last_peered":"2024-09-11T01:17:58.935141+0000","last_clean":"2024-09-11T01:17:58.935141+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935141+0000","last_undegraded":"2024-09-11T01:17:58.935141+0000","last_fullsized":"2024-09-11T01:17:58.935141+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15794","reported_seq":23780,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:59.904185+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:19:59.904185+0000","last_peered":"2024-09-11T01:19:59.904185+0000","last_clean":"2024-09-11T01:19:59.904185+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:19:59.904185+0000","last_undegraded":"2024-09-11T01:19:59.904185+0000","last_fullsized":"2024-09-11T01:19:59.904185+0000","mapping_epoch":44,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16392192,"num_objects":4002,"num_object_clones":0,"num_object_copies":8004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13579,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16207","reported_seq":16343,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:02.202780+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:20:02.202780+0000","last_peered":"2024-09-11T01:20:02.202780+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:20:02.202780+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16617472,"num_objects":4057,"num_object_clones":0,"num_object_copies":8114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12944,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":107}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938599+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938599+0000","last_peered":"2024-09-11T01:17:58.938599+0000","last_clean":"2024-09-11T01:17:58.938599+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938599+0000","last_undegraded":"2024-09-11T01:17:58.938599+0000","last_fullsized":"2024-09-11T01:17:58.938599+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15942","reported_seq":23978,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:01.787832+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:20:01.787832+0000","last_peered":"2024-09-11T01:20:01.787832+0000","last_clean":"2024-09-11T01:20:01.787832+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:20:01.787832+0000","last_undegraded":"2024-09-11T01:20:01.787832+0000","last_fullsized":"2024-09-11T01:20:01.787832+0000","mapping_epoch":46,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16146432,"num_objects":3942,"num_object_clones":0,"num_object_copies":7884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12597,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16236","reported_seq":16376,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:02.084169+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:20:02.084169+0000","last_peered":"2024-09-11T01:20:02.084169+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:20:02.084169+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16138240,"num_objects":3940,"num_object_clones":0,"num_object_copies":7880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12845,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-99}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971371+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971371+0000","last_peered":"2024-09-11T01:17:58.971371+0000","last_clean":"2024-09-11T01:17:58.971371+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971371+0000","last_undegraded":"2024-09-11T01:17:58.971371+0000","last_fullsized":"2024-09-11T01:17:58.971371+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938303+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938303+0000","last_peered":"2024-09-11T01:17:58.938303+0000","last_clean":"2024-09-11T01:17:58.938303+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938303+0000","last_undegraded":"2024-09-11T01:17:58.938303+0000","last_fullsized":"2024-09-11T01:17:58.938303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16055","reported_seq":16194,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:02.074207+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:20:02.074207+0000","last_peered":"2024-09-11T01:20:02.074207+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:20:02.074207+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15962729,"num_objects":3898,"num_object_clones":0,"num_object_copies":7796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13125,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-19}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":261894785,"num_objects":63941,"num_object_clones":0,"num_object_copies":127882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36659,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63941,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":218346,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10980,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":718315520,"data_stored":718244754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160996,"ondisk_log_size":160996,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182502,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":669432,"kb_used_data":254632,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93702408,"statfs":{"total":96636764160,"available":95951265792,"internally_reserved":0,"allocated":260743168,"data_stored":260473509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":424734379},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,3,0,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986274,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":573744,"kb_used_data":257500,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93798096,"statfs":{"total":96636764160,"available":96049250304,"internally_reserved":0,"allocated":263680000,"data_stored":263410346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323808938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,1,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659219,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":544292,"kb_used_data":191952,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93827548,"statfs":{"total":96636764160,"available":96079409152,"internally_reserved":0,"allocated":196558848,"data_stored":196232932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":360771247},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,5,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":259702784,"data_stored":259699305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":196026368,"data_stored":195966632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":262586368,"data_stored":262578817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:04.748 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:05.596 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:20:05.596 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:20:05.644 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":311,"stamp":"2024-09-11T01:20:04.728405+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":261894804,"num_objects":63943,"num_object_clones":0,"num_object_copies":127886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36659,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63943,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":218348,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10981,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160999,"ondisk_log_size":160999,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1787468,"kb_used_data":704084,"kb_used_omap":12,"kb_used_meta":1083315,"kb_avail":281328052,"statfs":{"total":289910292480,"available":288079925248,"internally_reserved":0,"allocated":720982016,"data_stored":720116787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1109314564},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,5,12,7,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18735104,"num_objects":-4574,"num_object_clones":0,"num_object_copies":-9148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-349,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4574,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006903"},"pg_stats":[{"pgid":"2.f","version":"47'16262","reported_seq":16345,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:02.112774+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:20:02.112774+0000","last_peered":"2024-09-11T01:20:02.112774+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:20:02.112774+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16400384,"num_objects":4004,"num_object_clones":0,"num_object_copies":8008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16262,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-68}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16299","reported_seq":16406,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:59.909597+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:19:59.909597+0000","last_peered":"2024-09-11T01:19:59.909597+0000","last_clean":"2024-09-11T01:19:59.909597+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:19:59.909597+0000","last_undegraded":"2024-09-11T01:19:59.909597+0000","last_fullsized":"2024-09-11T01:19:59.909597+0000","mapping_epoch":35,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16601088,"num_objects":4053,"num_object_clones":0,"num_object_copies":8106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4053,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14538,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16033","reported_seq":16167,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:59.835270+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:19:59.835270+0000","last_peered":"2024-09-11T01:19:59.835270+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:19:59.835270+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16834584,"num_objects":4111,"num_object_clones":0,"num_object_copies":8222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4111,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14334,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":215}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16084","reported_seq":16159,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:59.944473+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:19:59.944473+0000","last_peered":"2024-09-11T01:19:59.944473+0000","last_clean":"2024-09-11T01:19:59.944473+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:19:59.944473+0000","last_undegraded":"2024-09-11T01:19:59.944473+0000","last_fullsized":"2024-09-11T01:19:59.944473+0000","mapping_epoch":35,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16588800,"num_objects":4050,"num_object_clones":0,"num_object_copies":8100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4050,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16084,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16095","reported_seq":16228,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:02.195523+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:20:02.195523+0000","last_peered":"2024-09-11T01:20:02.195523+0000","last_clean":"2024-09-11T01:20:02.195523+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:20:02.195523+0000","last_undegraded":"2024-09-11T01:20:02.195523+0000","last_fullsized":"2024-09-11T01:20:02.195523+0000","mapping_epoch":34,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16031744,"num_objects":3914,"num_object_clones":0,"num_object_copies":7828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13100,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16184","reported_seq":16359,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:02.206818+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:20:02.206818+0000","last_peered":"2024-09-11T01:20:02.206818+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:20:02.206818+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16560128,"num_objects":4043,"num_object_clones":0,"num_object_copies":8086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12525,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":92}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15933","reported_seq":16069,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:02.181405+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:20:02.181405+0000","last_peered":"2024-09-11T01:20:02.181405+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:20:02.181405+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16371712,"num_objects":3997,"num_object_clones":0,"num_object_copies":7994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3997,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12790,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":11}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16282","reported_seq":16417,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:02.086636+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:20:02.086636+0000","last_peered":"2024-09-11T01:20:02.086636+0000","last_clean":"2024-09-11T01:20:02.086636+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:20:02.086636+0000","last_undegraded":"2024-09-11T01:20:02.086636+0000","last_fullsized":"2024-09-11T01:20:02.086636+0000","mapping_epoch":34,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16269312,"num_objects":3972,"num_object_clones":0,"num_object_copies":7944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13158,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934737+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934737+0000","last_peered":"2024-09-11T01:17:58.934737+0000","last_clean":"2024-09-11T01:17:58.934737+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934737+0000","last_undegraded":"2024-09-11T01:17:58.934737+0000","last_fullsized":"2024-09-11T01:17:58.934737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'15991","reported_seq":22344,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:59.873337+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:19:59.873337+0000","last_peered":"2024-09-11T01:19:59.873337+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:19:59.873337+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16310272,"num_objects":3982,"num_object_clones":0,"num_object_copies":7964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":825,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13962,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3110,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3157}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15953","reported_seq":16087,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:02.182590+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:20:02.182590+0000","last_peered":"2024-09-11T01:20:02.182590+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:20:02.182590+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16035840,"num_objects":3915,"num_object_clones":0,"num_object_copies":7830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12540,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-69}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16146","reported_seq":16270,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:19:59.847840+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:19:59.847840+0000","last_peered":"2024-09-11T01:19:59.847840+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:19:59.847840+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16633856,"num_objects":4061,"num_object_clones":0,"num_object_copies":8122,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4061,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13963,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":37}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935141+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935141+0000","last_peered":"2024-09-11T01:17:58.935141+0000","last_clean":"2024-09-11T01:17:58.935141+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935141+0000","last_undegraded":"2024-09-11T01:17:58.935141+0000","last_fullsized":"2024-09-11T01:17:58.935141+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15794","reported_seq":23780,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:19:59.904185+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:19:59.904185+0000","last_peered":"2024-09-11T01:19:59.904185+0000","last_clean":"2024-09-11T01:19:59.904185+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:19:59.904185+0000","last_undegraded":"2024-09-11T01:19:59.904185+0000","last_fullsized":"2024-09-11T01:19:59.904185+0000","mapping_epoch":44,"log_start":"43'5700","ondisk_log_start":"43'5700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16392192,"num_objects":4002,"num_object_clones":0,"num_object_copies":8004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4002,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13579,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16207","reported_seq":16343,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:02.202780+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:20:02.202780+0000","last_peered":"2024-09-11T01:20:02.202780+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:20:02.202780+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16617472,"num_objects":4057,"num_object_clones":0,"num_object_copies":8114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12944,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":107}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938599+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938599+0000","last_peered":"2024-09-11T01:17:58.938599+0000","last_clean":"2024-09-11T01:17:58.938599+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938599+0000","last_undegraded":"2024-09-11T01:17:58.938599+0000","last_fullsized":"2024-09-11T01:17:58.938599+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15942","reported_seq":23978,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:01.787832+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:20:01.787832+0000","last_peered":"2024-09-11T01:20:01.787832+0000","last_clean":"2024-09-11T01:20:01.787832+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:20:01.787832+0000","last_undegraded":"2024-09-11T01:20:01.787832+0000","last_fullsized":"2024-09-11T01:20:01.787832+0000","mapping_epoch":46,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16146432,"num_objects":3942,"num_object_clones":0,"num_object_copies":7884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12597,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16236","reported_seq":16376,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:02.084169+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:20:02.084169+0000","last_peered":"2024-09-11T01:20:02.084169+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:20:02.084169+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16138240,"num_objects":3940,"num_object_clones":0,"num_object_copies":7880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12845,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-99}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971371+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971371+0000","last_peered":"2024-09-11T01:17:58.971371+0000","last_clean":"2024-09-11T01:17:58.971371+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971371+0000","last_undegraded":"2024-09-11T01:17:58.971371+0000","last_fullsized":"2024-09-11T01:17:58.971371+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938303+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938303+0000","last_peered":"2024-09-11T01:17:58.938303+0000","last_clean":"2024-09-11T01:17:58.938303+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938303+0000","last_undegraded":"2024-09-11T01:17:58.938303+0000","last_fullsized":"2024-09-11T01:17:58.938303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16055","reported_seq":16194,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:02.074207+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:20:02.074207+0000","last_peered":"2024-09-11T01:20:02.074207+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:20:02.074207+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15962729,"num_objects":3898,"num_object_clones":0,"num_object_copies":7796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13125,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-19}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":261894785,"num_objects":63941,"num_object_clones":0,"num_object_copies":127882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36659,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63941,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":218346,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10980,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":718315520,"data_stored":718244754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160996,"ondisk_log_size":160996,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182502,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":669432,"kb_used_data":254632,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93702408,"statfs":{"total":96636764160,"available":95951265792,"internally_reserved":0,"allocated":260743168,"data_stored":260473509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":424734379},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,3,0,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986274,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":573744,"kb_used_data":257500,"kb_used_omap":4,"kb_used_meta":316219,"kb_avail":93798096,"statfs":{"total":96636764160,"available":96049250304,"internally_reserved":0,"allocated":263680000,"data_stored":263410346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":323808938},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,1,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659219,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":544292,"kb_used_data":191952,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93827548,"statfs":{"total":96636764160,"available":96079409152,"internally_reserved":0,"allocated":196558848,"data_stored":196232932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":360771247},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,5,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":259702784,"data_stored":259699305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":196026368,"data_stored":195966632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":262586368,"data_stored":262578817,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:05.645 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:20:05.645 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'16262", 'reported_seq': 16345, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:02.112774+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:20:02.112774+0000', 'last_peered': '2024-09-11T01:20:02.112774+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:20:02.112774+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "43'6200", 'ondisk_log_start': "43'6200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16400384, 'num_objects': 4004, 'num_object_clones': 0, 'num_object_copies': 8008, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4004, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16262, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -68}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:05.645 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:20:05.646 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'16033", 'reported_seq': 16167, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:59.835270+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:19:59.835270+0000', 'last_peered': '2024-09-11T01:19:59.835270+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:19:59.835270+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "43'6000", 'ondisk_log_start': "43'6000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16834584, 'num_objects': 4111, 'num_object_clones': 0, 'num_object_copies': 8222, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4111, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14334, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 215}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:05.646 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:20:05.646 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'16184", 'reported_seq': 16359, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:02.206818+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:20:02.206818+0000', 'last_peered': '2024-09-11T01:20:02.206818+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:20:02.206818+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "43'6100", 'ondisk_log_start': "43'6100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16560128, 'num_objects': 4043, 'num_object_clones': 0, 'num_object_copies': 8086, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4043, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12525, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 92}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:05.646 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:20:05.646 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'15933", 'reported_seq': 16069, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:02.181405+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:20:02.181405+0000', 'last_peered': '2024-09-11T01:20:02.181405+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:20:02.181405+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "43'5900", 'ondisk_log_start': "43'5900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16371712, 'num_objects': 3997, 'num_object_clones': 0, 'num_object_copies': 7994, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3997, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12790, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 11}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:05.646 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:20:05.646 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'15991", 'reported_seq': 22344, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:59.873337+0000', 'last_change': '2024-09-11T01:17:57.355893+0000', 'last_active': '2024-09-11T01:19:59.873337+0000', 'last_peered': '2024-09-11T01:19:59.873337+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:19:59.873337+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "43'5900", 'ondisk_log_start': "43'5900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 10091, 'log_dups_size': 0, 'ondisk_log_size': 10091, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16310272, 'num_objects': 3982, 'num_object_clones': 0, 'num_object_copies': 7964, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 825, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3982, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13962, 'num_write_kb': 31844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3110, 'num_bytes_recovered': 4505600, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3157}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:05.646 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:20:05.646 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'15953", 'reported_seq': 16087, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:02.182590+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:20:02.182590+0000', 'last_peered': '2024-09-11T01:20:02.182590+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:20:02.182590+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "43'5900", 'ondisk_log_start': "43'5900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16035840, 'num_objects': 3915, 'num_object_clones': 0, 'num_object_copies': 7830, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3915, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12540, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -69}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:05.646 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:20:05.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'16146", 'reported_seq': 16270, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:19:59.847840+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:19:59.847840+0000', 'last_peered': '2024-09-11T01:19:59.847840+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:19:59.847840+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "43'6100", 'ondisk_log_start': "43'6100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16633856, 'num_objects': 4061, 'num_object_clones': 0, 'num_object_copies': 8122, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4061, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13963, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 37}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:05.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:20:05.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'16207", 'reported_seq': 16343, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:02.202780+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:20:02.202780+0000', 'last_peered': '2024-09-11T01:20:02.202780+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:20:02.202780+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "43'6200", 'ondisk_log_start': "43'6200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16617472, 'num_objects': 4057, 'num_object_clones': 0, 'num_object_copies': 8114, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 4057, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12944, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': 107}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:05.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:20:05.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'16236", 'reported_seq': 16376, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:02.084169+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:20:02.084169+0000', 'last_peered': '2024-09-11T01:20:02.084169+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:20:02.084169+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "43'6200", 'ondisk_log_start': "43'6200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 16138240, 'num_objects': 3940, 'num_object_clones': 0, 'num_object_copies': 7880, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3940, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12845, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -99}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:05.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:20:05.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'16055", 'reported_seq': 16194, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:02.074207+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:20:02.074207+0000', 'last_peered': '2024-09-11T01:20:02.074207+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:20:02.074207+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "43'6000", 'ondisk_log_start': "43'6000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15962729, 'num_objects': 3898, 'num_object_clones': 0, 'num_object_copies': 7796, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3898, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13125, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -19}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:05.647 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:20:06.658 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:20:06.659 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36659/127886 objects degraded (28.665%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":543,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":63943,"data_bytes":261894804,"bytes_used":1830367232,"bytes_avail":288079925248,"bytes_total":289910292480,"degraded_objects":36659,"degraded_total":127886,"degraded_ratio":0.28665373848583897,"recovering_objects_per_sec":29,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":349,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":380},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:20:06.687 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:20:06.687 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:07.395 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:20:07.396 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:20:07.449 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":312,"stamp":"2024-09-11T01:20:06.728843+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":258429588,"num_objects":63097,"num_object_clones":0,"num_object_copies":126194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36481,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63097,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":219194,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11159,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160745,"ondisk_log_size":160745,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1791176,"kb_used_data":699468,"kb_used_omap":12,"kb_used_meta":1091635,"kb_avail":281324344,"statfs":{"total":289910292480,"available":288076128256,"internally_reserved":0,"allocated":716255232,"data_stored":715390003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1117834244},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,7,6,10,6,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22200320,"num_objects":-5420,"num_object_clones":0,"num_object_copies":-10840,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-527,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5420,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5420,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":527,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006883"},"pg_stats":[{"pgid":"2.f","version":"47'16262","reported_seq":16345,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:02.112774+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:20:02.112774+0000","last_peered":"2024-09-11T01:20:02.112774+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:20:02.112774+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16400384,"num_objects":4004,"num_object_clones":0,"num_object_copies":8008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4004,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16262,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-68}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16441","reported_seq":16548,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:04.901823+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:20:04.901823+0000","last_peered":"2024-09-11T01:20:04.901823+0000","last_clean":"2024-09-11T01:20:04.901823+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:20:04.901823+0000","last_undegraded":"2024-09-11T01:20:04.901823+0000","last_fullsized":"2024-09-11T01:20:04.901823+0000","mapping_epoch":35,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16019456,"num_objects":3911,"num_object_clones":0,"num_object_copies":7822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3911,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14680,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16161","reported_seq":16295,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:04.808557+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:20:04.808557+0000","last_peered":"2024-09-11T01:20:04.808557+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:20:04.808557+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16310296,"num_objects":3983,"num_object_clones":0,"num_object_copies":7966,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3983,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14462,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":87}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16224","reported_seq":16299,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:04.823732+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:20:04.823732+0000","last_peered":"2024-09-11T01:20:04.823732+0000","last_clean":"2024-09-11T01:20:04.823732+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:20:04.823732+0000","last_undegraded":"2024-09-11T01:20:04.823732+0000","last_fullsized":"2024-09-11T01:20:04.823732+0000","mapping_epoch":35,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16015360,"num_objects":3910,"num_object_clones":0,"num_object_copies":7820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16224,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16095","reported_seq":16228,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:02.195523+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:20:02.195523+0000","last_peered":"2024-09-11T01:20:02.195523+0000","last_clean":"2024-09-11T01:20:02.195523+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:20:02.195523+0000","last_undegraded":"2024-09-11T01:20:02.195523+0000","last_fullsized":"2024-09-11T01:20:02.195523+0000","mapping_epoch":34,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16031744,"num_objects":3914,"num_object_clones":0,"num_object_copies":7828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3914,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13100,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16184","reported_seq":16359,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:02.206818+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:20:02.206818+0000","last_peered":"2024-09-11T01:20:02.206818+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:20:02.206818+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16560128,"num_objects":4043,"num_object_clones":0,"num_object_copies":8086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4043,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12525,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":92}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'15933","reported_seq":16069,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:02.181405+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:20:02.181405+0000","last_peered":"2024-09-11T01:20:02.181405+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:20:02.181405+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16371712,"num_objects":3997,"num_object_clones":0,"num_object_copies":7994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3997,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12790,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":11}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16282","reported_seq":16417,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:02.086636+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:20:02.086636+0000","last_peered":"2024-09-11T01:20:02.086636+0000","last_clean":"2024-09-11T01:20:02.086636+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:20:02.086636+0000","last_undegraded":"2024-09-11T01:20:02.086636+0000","last_fullsized":"2024-09-11T01:20:02.086636+0000","mapping_epoch":34,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16269312,"num_objects":3972,"num_object_clones":0,"num_object_copies":7944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3972,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13158,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934737+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934737+0000","last_peered":"2024-09-11T01:17:58.934737+0000","last_clean":"2024-09-11T01:17:58.934737+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934737+0000","last_undegraded":"2024-09-11T01:17:58.934737+0000","last_fullsized":"2024-09-11T01:17:58.934737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16123","reported_seq":22833,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:04.934655+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:20:04.934655+0000","last_peered":"2024-09-11T01:20:04.934655+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:20:04.934655+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15769600,"num_objects":3850,"num_object_clones":0,"num_object_copies":7700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":647,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3850,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14094,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3288,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3203}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'15953","reported_seq":16087,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:02.182590+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:20:02.182590+0000","last_peered":"2024-09-11T01:20:02.182590+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:20:02.182590+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16035840,"num_objects":3915,"num_object_clones":0,"num_object_copies":7830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12540,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-69}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16317","reported_seq":16441,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:04.864006+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:20:04.864006+0000","last_peered":"2024-09-11T01:20:04.864006+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:20:04.864006+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15933440,"num_objects":3890,"num_object_clones":0,"num_object_copies":7780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14134,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-134}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935141+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935141+0000","last_peered":"2024-09-11T01:17:58.935141+0000","last_clean":"2024-09-11T01:17:58.935141+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935141+0000","last_undegraded":"2024-09-11T01:17:58.935141+0000","last_fullsized":"2024-09-11T01:17:58.935141+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'15927","reported_seq":23913,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:04.924507+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:20:04.924507+0000","last_peered":"2024-09-11T01:20:04.924507+0000","last_clean":"2024-09-11T01:20:04.924507+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:20:04.924507+0000","last_undegraded":"2024-09-11T01:20:04.924507+0000","last_fullsized":"2024-09-11T01:20:04.924507+0000","mapping_epoch":44,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15847424,"num_objects":3869,"num_object_clones":0,"num_object_copies":7738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3869,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13712,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16207","reported_seq":16343,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:02.202780+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:20:02.202780+0000","last_peered":"2024-09-11T01:20:02.202780+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:20:02.202780+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16617472,"num_objects":4057,"num_object_clones":0,"num_object_copies":8114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4057,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12944,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":107}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938599+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938599+0000","last_peered":"2024-09-11T01:17:58.938599+0000","last_clean":"2024-09-11T01:17:58.938599+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938599+0000","last_undegraded":"2024-09-11T01:17:58.938599+0000","last_fullsized":"2024-09-11T01:17:58.938599+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'15942","reported_seq":23978,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:01.787832+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:20:01.787832+0000","last_peered":"2024-09-11T01:20:01.787832+0000","last_clean":"2024-09-11T01:20:01.787832+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:20:01.787832+0000","last_undegraded":"2024-09-11T01:20:01.787832+0000","last_fullsized":"2024-09-11T01:20:01.787832+0000","mapping_epoch":46,"log_start":"43'5900","ondisk_log_start":"43'5900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16146432,"num_objects":3942,"num_object_clones":0,"num_object_copies":7884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12597,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16236","reported_seq":16376,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:02.084169+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:20:02.084169+0000","last_peered":"2024-09-11T01:20:02.084169+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:20:02.084169+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":16138240,"num_objects":3940,"num_object_clones":0,"num_object_copies":7880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3940,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12845,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-99}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971371+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971371+0000","last_peered":"2024-09-11T01:17:58.971371+0000","last_clean":"2024-09-11T01:17:58.971371+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971371+0000","last_undegraded":"2024-09-11T01:17:58.971371+0000","last_fullsized":"2024-09-11T01:17:58.971371+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938303+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938303+0000","last_peered":"2024-09-11T01:17:58.938303+0000","last_clean":"2024-09-11T01:17:58.938303+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938303+0000","last_undegraded":"2024-09-11T01:17:58.938303+0000","last_fullsized":"2024-09-11T01:17:58.938303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16055","reported_seq":16194,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:02.074207+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:20:02.074207+0000","last_peered":"2024-09-11T01:20:02.074207+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:20:02.074207+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15962729,"num_objects":3898,"num_object_clones":0,"num_object_copies":7796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3898,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13125,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-19}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":258429569,"num_objects":63095,"num_object_clones":0,"num_object_copies":126190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36481,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63095,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":219192,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11158,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":713691136,"data_stored":713620370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160742,"ondisk_log_size":160742,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182502,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":669432,"kb_used_data":254632,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93702408,"statfs":{"total":96636764160,"available":95951265792,"internally_reserved":0,"allocated":260743168,"data_stored":260473509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":424734379},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,3,0,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986275,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":577452,"kb_used_data":252884,"kb_used_omap":4,"kb_used_meta":324539,"kb_avail":93794388,"statfs":{"total":96636764160,"available":96045453312,"internally_reserved":0,"allocated":258953216,"data_stored":258683562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":332328618},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,6,2,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659219,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":544292,"kb_used_data":191952,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93827548,"statfs":{"total":96636764160,"available":96079409152,"internally_reserved":0,"allocated":196558848,"data_stored":196232932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":360771247},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,3,5,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":259702784,"data_stored":259699305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":196026368,"data_stored":195966632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":257961984,"data_stored":257954433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:10.452 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:11.290 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:20:11.291 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:20:11.347 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-09-11T01:20:10.729878+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":248988308,"num_objects":60792,"num_object_clones":0,"num_object_copies":121584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36294,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60792,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":221499,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11346,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160850,"ondisk_log_size":160850,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1789640,"kb_used_data":681304,"kb_used_omap":12,"kb_used_meta":1108275,"kb_avail":281325880,"statfs":{"total":289910292480,"available":288077701120,"internally_reserved":0,"allocated":697655296,"data_stored":696790067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1134873604},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,4,3,10,12,8,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22171648,"num_objects":-5413,"num_object_clones":0,"num_object_copies":-10826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-539,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5413,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5413,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":539,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004878"},"pg_stats":[{"pgid":"2.f","version":"47'16413","reported_seq":16496,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:07.278036+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:20:07.278036+0000","last_peered":"2024-09-11T01:20:07.278036+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:20:07.278036+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15781888,"num_objects":3853,"num_object_clones":0,"num_object_copies":7706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16413,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-219}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16586","reported_seq":16693,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:09.908380+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:20:09.908380+0000","last_peered":"2024-09-11T01:20:09.908380+0000","last_clean":"2024-09-11T01:20:09.908380+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:20:09.908380+0000","last_undegraded":"2024-09-11T01:20:09.908380+0000","last_fullsized":"2024-09-11T01:20:09.908380+0000","mapping_epoch":35,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15425536,"num_objects":3766,"num_object_clones":0,"num_object_copies":7532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14825,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16321","reported_seq":16455,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:09.960161+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:20:09.960161+0000","last_peered":"2024-09-11T01:20:09.960161+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:20:09.960161+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15654936,"num_objects":3823,"num_object_clones":0,"num_object_copies":7646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3823,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14622,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-73}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16360","reported_seq":16435,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:09.836285+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:20:09.836285+0000","last_peered":"2024-09-11T01:20:09.836285+0000","last_clean":"2024-09-11T01:20:09.836285+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:20:09.836285+0000","last_undegraded":"2024-09-11T01:20:09.836285+0000","last_fullsized":"2024-09-11T01:20:09.836285+0000","mapping_epoch":35,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15458304,"num_objects":3774,"num_object_clones":0,"num_object_copies":7548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16360,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16243","reported_seq":16376,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:07.308111+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:20:07.308111+0000","last_peered":"2024-09-11T01:20:07.308111+0000","last_clean":"2024-09-11T01:20:07.308111+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:20:07.308111+0000","last_undegraded":"2024-09-11T01:20:07.308111+0000","last_fullsized":"2024-09-11T01:20:07.308111+0000","mapping_epoch":34,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15425536,"num_objects":3766,"num_object_clones":0,"num_object_copies":7532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13248,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16331","reported_seq":16506,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:07.136460+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:20:07.136460+0000","last_peered":"2024-09-11T01:20:07.136460+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:20:07.136460+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15958016,"num_objects":3896,"num_object_clones":0,"num_object_copies":7792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12672,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-55}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16083","reported_seq":16217,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:07.277915+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:20:07.277915+0000","last_peered":"2024-09-11T01:20:07.277915+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:20:07.277915+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15757312,"num_objects":3847,"num_object_clones":0,"num_object_copies":7694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3847,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12940,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16417","reported_seq":16552,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:07.277719+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:20:07.277719+0000","last_peered":"2024-09-11T01:20:07.277719+0000","last_clean":"2024-09-11T01:20:07.277719+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:20:07.277719+0000","last_undegraded":"2024-09-11T01:20:07.277719+0000","last_fullsized":"2024-09-11T01:20:07.277719+0000","mapping_epoch":34,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15716352,"num_objects":3837,"num_object_clones":0,"num_object_copies":7674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13293,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934737+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934737+0000","last_peered":"2024-09-11T01:17:58.934737+0000","last_clean":"2024-09-11T01:17:58.934737+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934737+0000","last_undegraded":"2024-09-11T01:17:58.934737+0000","last_fullsized":"2024-09-11T01:17:58.934737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16276","reported_seq":23360,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:09.961894+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:20:09.961894+0000","last_peered":"2024-09-11T01:20:09.961894+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:20:09.961894+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15142912,"num_objects":3697,"num_object_clones":0,"num_object_copies":7394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":460,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14247,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3475,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3237}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16109","reported_seq":16243,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:07.229578+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:20:07.229578+0000","last_peered":"2024-09-11T01:20:07.229578+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:20:07.229578+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15396864,"num_objects":3759,"num_object_clones":0,"num_object_copies":7518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3759,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12696,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-225}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16443","reported_seq":16567,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:09.933837+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:20:09.933837+0000","last_peered":"2024-09-11T01:20:09.933837+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:20:09.933837+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15417344,"num_objects":3764,"num_object_clones":0,"num_object_copies":7528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14260,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-260}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935141+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935141+0000","last_peered":"2024-09-11T01:17:58.935141+0000","last_clean":"2024-09-11T01:17:58.935141+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935141+0000","last_undegraded":"2024-09-11T01:17:58.935141+0000","last_fullsized":"2024-09-11T01:17:58.935141+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16082","reported_seq":24069,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:09.963447+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:20:09.963447+0000","last_peered":"2024-09-11T01:20:09.963447+0000","last_clean":"2024-09-11T01:20:09.963447+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:20:09.963447+0000","last_undegraded":"2024-09-11T01:20:09.963447+0000","last_fullsized":"2024-09-11T01:20:09.963447+0000","mapping_epoch":44,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15212544,"num_objects":3714,"num_object_clones":0,"num_object_copies":7428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3714,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13867,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16361","reported_seq":16498,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:07.205513+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:20:07.205513+0000","last_peered":"2024-09-11T01:20:07.205513+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:20:07.205513+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15986688,"num_objects":3903,"num_object_clones":0,"num_object_copies":7806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13098,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-47}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938599+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938599+0000","last_peered":"2024-09-11T01:17:58.938599+0000","last_clean":"2024-09-11T01:17:58.938599+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938599+0000","last_undegraded":"2024-09-11T01:17:58.938599+0000","last_fullsized":"2024-09-11T01:17:58.938599+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16069","reported_seq":24106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:06.837744+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:20:06.837744+0000","last_peered":"2024-09-11T01:20:06.837744+0000","last_clean":"2024-09-11T01:20:06.837744+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:20:06.837744+0000","last_undegraded":"2024-09-11T01:20:06.837744+0000","last_fullsized":"2024-09-11T01:20:06.837744+0000","mapping_epoch":46,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15626240,"num_objects":3815,"num_object_clones":0,"num_object_copies":7630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12724,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16385","reported_seq":16525,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:07.307130+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:20:07.307130+0000","last_peered":"2024-09-11T01:20:07.307130+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:20:07.307130+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15527936,"num_objects":3791,"num_object_clones":0,"num_object_copies":7582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12994,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-248}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971371+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971371+0000","last_peered":"2024-09-11T01:17:58.971371+0000","last_clean":"2024-09-11T01:17:58.971371+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971371+0000","last_undegraded":"2024-09-11T01:17:58.971371+0000","last_fullsized":"2024-09-11T01:17:58.971371+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938303+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938303+0000","last_peered":"2024-09-11T01:17:58.938303+0000","last_clean":"2024-09-11T01:17:58.938303+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938303+0000","last_undegraded":"2024-09-11T01:17:58.938303+0000","last_fullsized":"2024-09-11T01:17:58.938303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16168","reported_seq":16307,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:07.170241+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:20:07.170241+0000","last_peered":"2024-09-11T01:20:07.170241+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:20:07.170241+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15499881,"num_objects":3785,"num_object_clones":0,"num_object_copies":7570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13238,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-132}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":248988289,"num_objects":60790,"num_object_clones":0,"num_object_copies":121580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36294,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60790,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":221497,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11345,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":694956032,"data_stored":694885266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160847,"ondisk_log_size":160847,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182503,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":666844,"kb_used_data":247884,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93704996,"statfs":{"total":96636764160,"available":95953915904,"internally_reserved":0,"allocated":253833216,"data_stored":253563557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":428994219},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,4,4,6,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986276,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":577116,"kb_used_data":248396,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93794724,"statfs":{"total":96636764160,"available":96045797376,"internally_reserved":0,"allocated":254357504,"data_stored":254087850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336588458},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,3,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659220,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545680,"kb_used_data":185024,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93826160,"statfs":{"total":96636764160,"available":96077987840,"internally_reserved":0,"allocated":189464576,"data_stored":189138660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":369290927},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,0,0,3,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":252735488,"data_stored":252732009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":188952576,"data_stored":188892840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":253267968,"data_stored":253260417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:11.348 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:12.284 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:20:12.285 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:20:12.325 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-09-11T01:20:10.729878+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":248988308,"num_objects":60792,"num_object_clones":0,"num_object_copies":121584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36294,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60792,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":221499,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11346,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160850,"ondisk_log_size":160850,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1789640,"kb_used_data":681304,"kb_used_omap":12,"kb_used_meta":1108275,"kb_avail":281325880,"statfs":{"total":289910292480,"available":288077701120,"internally_reserved":0,"allocated":697655296,"data_stored":696790067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1134873604},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,2,4,3,10,12,8,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22171648,"num_objects":-5413,"num_object_clones":0,"num_object_copies":-10826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-539,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5413,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5413,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":539,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004878"},"pg_stats":[{"pgid":"2.f","version":"47'16413","reported_seq":16496,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:07.278036+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:20:07.278036+0000","last_peered":"2024-09-11T01:20:07.278036+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:20:07.278036+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15781888,"num_objects":3853,"num_object_clones":0,"num_object_copies":7706,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3853,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16413,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-219}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16586","reported_seq":16693,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:09.908380+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:20:09.908380+0000","last_peered":"2024-09-11T01:20:09.908380+0000","last_clean":"2024-09-11T01:20:09.908380+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:20:09.908380+0000","last_undegraded":"2024-09-11T01:20:09.908380+0000","last_fullsized":"2024-09-11T01:20:09.908380+0000","mapping_epoch":35,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15425536,"num_objects":3766,"num_object_clones":0,"num_object_copies":7532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14825,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16321","reported_seq":16455,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:09.960161+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:20:09.960161+0000","last_peered":"2024-09-11T01:20:09.960161+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:20:09.960161+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15654936,"num_objects":3823,"num_object_clones":0,"num_object_copies":7646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3823,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14622,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-73}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16360","reported_seq":16435,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:09.836285+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:20:09.836285+0000","last_peered":"2024-09-11T01:20:09.836285+0000","last_clean":"2024-09-11T01:20:09.836285+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:20:09.836285+0000","last_undegraded":"2024-09-11T01:20:09.836285+0000","last_fullsized":"2024-09-11T01:20:09.836285+0000","mapping_epoch":35,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15458304,"num_objects":3774,"num_object_clones":0,"num_object_copies":7548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16360,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16243","reported_seq":16376,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:07.308111+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:20:07.308111+0000","last_peered":"2024-09-11T01:20:07.308111+0000","last_clean":"2024-09-11T01:20:07.308111+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:20:07.308111+0000","last_undegraded":"2024-09-11T01:20:07.308111+0000","last_fullsized":"2024-09-11T01:20:07.308111+0000","mapping_epoch":34,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15425536,"num_objects":3766,"num_object_clones":0,"num_object_copies":7532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13248,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16331","reported_seq":16506,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:07.136460+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:20:07.136460+0000","last_peered":"2024-09-11T01:20:07.136460+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:20:07.136460+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15958016,"num_objects":3896,"num_object_clones":0,"num_object_copies":7792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12672,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-55}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16083","reported_seq":16217,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:07.277915+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:20:07.277915+0000","last_peered":"2024-09-11T01:20:07.277915+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:20:07.277915+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15757312,"num_objects":3847,"num_object_clones":0,"num_object_copies":7694,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3847,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12940,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-139}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16417","reported_seq":16552,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:07.277719+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:20:07.277719+0000","last_peered":"2024-09-11T01:20:07.277719+0000","last_clean":"2024-09-11T01:20:07.277719+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:20:07.277719+0000","last_undegraded":"2024-09-11T01:20:07.277719+0000","last_fullsized":"2024-09-11T01:20:07.277719+0000","mapping_epoch":34,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15716352,"num_objects":3837,"num_object_clones":0,"num_object_copies":7674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3837,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13293,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934737+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934737+0000","last_peered":"2024-09-11T01:17:58.934737+0000","last_clean":"2024-09-11T01:17:58.934737+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934737+0000","last_undegraded":"2024-09-11T01:17:58.934737+0000","last_fullsized":"2024-09-11T01:17:58.934737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16276","reported_seq":23360,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:09.961894+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:20:09.961894+0000","last_peered":"2024-09-11T01:20:09.961894+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:20:09.961894+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15142912,"num_objects":3697,"num_object_clones":0,"num_object_copies":7394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":460,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14247,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3475,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3237}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16109","reported_seq":16243,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:07.229578+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:20:07.229578+0000","last_peered":"2024-09-11T01:20:07.229578+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:20:07.229578+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15396864,"num_objects":3759,"num_object_clones":0,"num_object_copies":7518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3759,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12696,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-225}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16443","reported_seq":16567,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:09.933837+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:20:09.933837+0000","last_peered":"2024-09-11T01:20:09.933837+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:20:09.933837+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15417344,"num_objects":3764,"num_object_clones":0,"num_object_copies":7528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14260,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-260}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935141+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935141+0000","last_peered":"2024-09-11T01:17:58.935141+0000","last_clean":"2024-09-11T01:17:58.935141+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935141+0000","last_undegraded":"2024-09-11T01:17:58.935141+0000","last_fullsized":"2024-09-11T01:17:58.935141+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16082","reported_seq":24069,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:09.963447+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:20:09.963447+0000","last_peered":"2024-09-11T01:20:09.963447+0000","last_clean":"2024-09-11T01:20:09.963447+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:20:09.963447+0000","last_undegraded":"2024-09-11T01:20:09.963447+0000","last_fullsized":"2024-09-11T01:20:09.963447+0000","mapping_epoch":44,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15212544,"num_objects":3714,"num_object_clones":0,"num_object_copies":7428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3714,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13867,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16361","reported_seq":16498,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:07.205513+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:20:07.205513+0000","last_peered":"2024-09-11T01:20:07.205513+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:20:07.205513+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15986688,"num_objects":3903,"num_object_clones":0,"num_object_copies":7806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13098,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-47}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938599+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938599+0000","last_peered":"2024-09-11T01:17:58.938599+0000","last_clean":"2024-09-11T01:17:58.938599+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938599+0000","last_undegraded":"2024-09-11T01:17:58.938599+0000","last_fullsized":"2024-09-11T01:17:58.938599+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16069","reported_seq":24106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:06.837744+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:20:06.837744+0000","last_peered":"2024-09-11T01:20:06.837744+0000","last_clean":"2024-09-11T01:20:06.837744+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:20:06.837744+0000","last_undegraded":"2024-09-11T01:20:06.837744+0000","last_fullsized":"2024-09-11T01:20:06.837744+0000","mapping_epoch":46,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15626240,"num_objects":3815,"num_object_clones":0,"num_object_copies":7630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3815,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12724,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16385","reported_seq":16525,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:07.307130+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:20:07.307130+0000","last_peered":"2024-09-11T01:20:07.307130+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:20:07.307130+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15527936,"num_objects":3791,"num_object_clones":0,"num_object_copies":7582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3791,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12994,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-248}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971371+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971371+0000","last_peered":"2024-09-11T01:17:58.971371+0000","last_clean":"2024-09-11T01:17:58.971371+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971371+0000","last_undegraded":"2024-09-11T01:17:58.971371+0000","last_fullsized":"2024-09-11T01:17:58.971371+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938303+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938303+0000","last_peered":"2024-09-11T01:17:58.938303+0000","last_clean":"2024-09-11T01:17:58.938303+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938303+0000","last_undegraded":"2024-09-11T01:17:58.938303+0000","last_fullsized":"2024-09-11T01:17:58.938303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16168","reported_seq":16307,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:07.170241+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:20:07.170241+0000","last_peered":"2024-09-11T01:20:07.170241+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:20:07.170241+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'6100","ondisk_log_start":"43'6100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15499881,"num_objects":3785,"num_object_clones":0,"num_object_copies":7570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3785,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13238,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-132}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":248988289,"num_objects":60790,"num_object_clones":0,"num_object_copies":121580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36294,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60790,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":221497,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11345,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":694956032,"data_stored":694885266,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160847,"ondisk_log_size":160847,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182503,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":666844,"kb_used_data":247884,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93704996,"statfs":{"total":96636764160,"available":95953915904,"internally_reserved":0,"allocated":253833216,"data_stored":253563557,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":428994219},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,1,4,4,6,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986276,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":577116,"kb_used_data":248396,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93794724,"statfs":{"total":96636764160,"available":96045797376,"internally_reserved":0,"allocated":254357504,"data_stored":254087850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336588458},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,3,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659220,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":545680,"kb_used_data":185024,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93826160,"statfs":{"total":96636764160,"available":96077987840,"internally_reserved":0,"allocated":189464576,"data_stored":189138660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":369290927},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,1,0,0,3,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":252735488,"data_stored":252732009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":188952576,"data_stored":188892840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":253267968,"data_stored":253260417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:12.326 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:20:12.326 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'16413", 'reported_seq': 16496, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:07.278036+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:20:07.278036+0000', 'last_peered': '2024-09-11T01:20:07.278036+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:20:07.278036+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "43'6400", 'ondisk_log_start': "43'6400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15781888, 'num_objects': 3853, 'num_object_clones': 0, 'num_object_copies': 7706, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3853, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16413, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -219}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:12.327 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:20:12.327 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'16321", 'reported_seq': 16455, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:09.960161+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:20:09.960161+0000', 'last_peered': '2024-09-11T01:20:09.960161+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:20:09.960161+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "43'6300", 'ondisk_log_start': "43'6300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15654936, 'num_objects': 3823, 'num_object_clones': 0, 'num_object_copies': 7646, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3823, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14622, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -73}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:12.327 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:20:12.327 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'16331", 'reported_seq': 16506, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:07.136460+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:20:07.136460+0000', 'last_peered': '2024-09-11T01:20:07.136460+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:20:07.136460+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "43'6300", 'ondisk_log_start': "43'6300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15958016, 'num_objects': 3896, 'num_object_clones': 0, 'num_object_copies': 7792, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3896, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12672, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -55}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:12.327 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:20:12.327 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'16083", 'reported_seq': 16217, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:07.277915+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:20:07.277915+0000', 'last_peered': '2024-09-11T01:20:07.277915+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:20:07.277915+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "43'6000", 'ondisk_log_start': "43'6000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10083, 'log_dups_size': 0, 'ondisk_log_size': 10083, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15757312, 'num_objects': 3847, 'num_object_clones': 0, 'num_object_copies': 7694, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3847, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12940, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -139}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:12.327 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:20:12.327 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'16276", 'reported_seq': 23360, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:09.961894+0000', 'last_change': '2024-09-11T01:17:57.355893+0000', 'last_active': '2024-09-11T01:20:09.961894+0000', 'last_peered': '2024-09-11T01:20:09.961894+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:20:09.961894+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "43'6200", 'ondisk_log_start': "43'6200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15142912, 'num_objects': 3697, 'num_object_clones': 0, 'num_object_copies': 7394, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 460, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3697, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14247, 'num_write_kb': 31844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3475, 'num_bytes_recovered': 4505600, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3237}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:12.327 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:20:12.327 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'16109", 'reported_seq': 16243, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:07.229578+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:20:07.229578+0000', 'last_peered': '2024-09-11T01:20:07.229578+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:20:07.229578+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "43'6100", 'ondisk_log_start': "43'6100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10009, 'log_dups_size': 0, 'ondisk_log_size': 10009, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15396864, 'num_objects': 3759, 'num_object_clones': 0, 'num_object_copies': 7518, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3759, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12696, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -225}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:12.327 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:20:12.328 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'16443", 'reported_seq': 16567, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:09.933837+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:20:09.933837+0000', 'last_peered': '2024-09-11T01:20:09.933837+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:20:09.933837+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "43'6400", 'ondisk_log_start': "43'6400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10043, 'log_dups_size': 0, 'ondisk_log_size': 10043, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15417344, 'num_objects': 3764, 'num_object_clones': 0, 'num_object_copies': 7528, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3764, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14260, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -260}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:12.328 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:20:12.328 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'16361", 'reported_seq': 16498, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:07.205513+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:20:07.205513+0000', 'last_peered': '2024-09-11T01:20:07.205513+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:20:07.205513+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "43'6300", 'ondisk_log_start': "43'6300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15986688, 'num_objects': 3903, 'num_object_clones': 0, 'num_object_copies': 7806, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3903, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13098, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -47}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:12.328 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:20:12.328 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'16385", 'reported_seq': 16525, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:07.307130+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:20:07.307130+0000', 'last_peered': '2024-09-11T01:20:07.307130+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:20:07.307130+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "43'6300", 'ondisk_log_start': "43'6300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15527936, 'num_objects': 3791, 'num_object_clones': 0, 'num_object_copies': 7582, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3791, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12994, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -248}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:12.328 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:20:12.328 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'16168", 'reported_seq': 16307, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:07.170241+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:20:07.170241+0000', 'last_peered': '2024-09-11T01:20:07.170241+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:20:07.170241+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "43'6100", 'ondisk_log_start': "43'6100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15499881, 'num_objects': 3785, 'num_object_clones': 0, 'num_object_copies': 7570, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3785, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13238, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -132}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:12.328 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:20:13.313 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:20:13.313 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36294/118634 objects degraded (30.593%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":549,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":59317,"data_bytes":242946708,"bytes_used":1800908800,"bytes_avail":288109383680,"bytes_total":289910292480,"degraded_objects":36294,"degraded_total":118634,"degraded_ratio":0.30593253198914305,"recovering_objects_per_sec":30,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":365,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":500},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:20:13.360 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:20:13.360 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:14.152 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:20:14.153 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:20:14.194 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":315,"stamp":"2024-09-11T01:20:12.730361+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":242946708,"num_objects":59317,"num_object_clones":0,"num_object_copies":118634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36294,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59317,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":222974,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11346,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160725,"ondisk_log_size":160725,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1758700,"kb_used_data":666996,"kb_used_omap":12,"kb_used_meta":1091635,"kb_avail":281356820,"statfs":{"total":289910292480,"available":288109383680,"internally_reserved":0,"allocated":683003904,"data_stored":682138675,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1117834244},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,5,9,17,9,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24612864,"num_objects":-6009,"num_object_clones":0,"num_object_copies":-12018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-365,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6009,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6009,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":365,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004891"},"pg_stats":[{"pgid":"2.f","version":"47'16574","reported_seq":16653,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:12.315636+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:20:12.315636+0000","last_peered":"2024-09-11T01:20:12.315636+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:20:12.315636+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15122432,"num_objects":3692,"num_object_clones":0,"num_object_copies":7384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16574,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-380}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16586","reported_seq":16693,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:09.908380+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:20:09.908380+0000","last_peered":"2024-09-11T01:20:09.908380+0000","last_clean":"2024-09-11T01:20:09.908380+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:20:09.908380+0000","last_undegraded":"2024-09-11T01:20:09.908380+0000","last_fullsized":"2024-09-11T01:20:09.908380+0000","mapping_epoch":35,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15425536,"num_objects":3766,"num_object_clones":0,"num_object_copies":7532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3766,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14825,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16321","reported_seq":16455,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:09.960161+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:20:09.960161+0000","last_peered":"2024-09-11T01:20:09.960161+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:20:09.960161+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15654936,"num_objects":3823,"num_object_clones":0,"num_object_copies":7646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3823,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14622,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-73}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16360","reported_seq":16435,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:09.836285+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:20:09.836285+0000","last_peered":"2024-09-11T01:20:09.836285+0000","last_clean":"2024-09-11T01:20:09.836285+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:20:09.836285+0000","last_undegraded":"2024-09-11T01:20:09.836285+0000","last_fullsized":"2024-09-11T01:20:09.836285+0000","mapping_epoch":35,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15458304,"num_objects":3774,"num_object_clones":0,"num_object_copies":7548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3774,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16360,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16402","reported_seq":16535,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:12.247367+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:20:12.247367+0000","last_peered":"2024-09-11T01:20:12.247367+0000","last_clean":"2024-09-11T01:20:12.247367+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:20:12.247367+0000","last_undegraded":"2024-09-11T01:20:12.247367+0000","last_fullsized":"2024-09-11T01:20:12.247367+0000","mapping_epoch":34,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14774272,"num_objects":3607,"num_object_clones":0,"num_object_copies":7214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13407,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16485","reported_seq":16661,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:12.337453+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:20:12.337453+0000","last_peered":"2024-09-11T01:20:12.337453+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:20:12.337453+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15327232,"num_objects":3742,"num_object_clones":0,"num_object_copies":7484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12826,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-209}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16225","reported_seq":16359,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:12.285682+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:20:12.285682+0000","last_peered":"2024-09-11T01:20:12.285682+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:20:12.285682+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15175680,"num_objects":3705,"num_object_clones":0,"num_object_copies":7410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13082,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-281}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16538","reported_seq":16673,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:12.247392+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:20:12.247392+0000","last_peered":"2024-09-11T01:20:12.247392+0000","last_clean":"2024-09-11T01:20:12.247392+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:20:12.247392+0000","last_undegraded":"2024-09-11T01:20:12.247392+0000","last_fullsized":"2024-09-11T01:20:12.247392+0000","mapping_epoch":34,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15220736,"num_objects":3716,"num_object_clones":0,"num_object_copies":7432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13414,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934737+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934737+0000","last_peered":"2024-09-11T01:17:58.934737+0000","last_clean":"2024-09-11T01:17:58.934737+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934737+0000","last_undegraded":"2024-09-11T01:17:58.934737+0000","last_fullsized":"2024-09-11T01:17:58.934737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16276","reported_seq":23360,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:09.961894+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:20:09.961894+0000","last_peered":"2024-09-11T01:20:09.961894+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:20:09.961894+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15142912,"num_objects":3697,"num_object_clones":0,"num_object_copies":7394,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":460,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3697,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14247,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3475,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3237}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16242","reported_seq":16376,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:12.167487+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:20:12.167487+0000","last_peered":"2024-09-11T01:20:12.167487+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:20:12.167487+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14852096,"num_objects":3626,"num_object_clones":0,"num_object_copies":7252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12829,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-358}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16443","reported_seq":16567,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:09.933837+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:20:09.933837+0000","last_peered":"2024-09-11T01:20:09.933837+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:20:09.933837+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15417344,"num_objects":3764,"num_object_clones":0,"num_object_copies":7528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14260,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-260}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935141+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935141+0000","last_peered":"2024-09-11T01:17:58.935141+0000","last_clean":"2024-09-11T01:17:58.935141+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935141+0000","last_undegraded":"2024-09-11T01:17:58.935141+0000","last_fullsized":"2024-09-11T01:17:58.935141+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16082","reported_seq":24069,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:09.963447+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:20:09.963447+0000","last_peered":"2024-09-11T01:20:09.963447+0000","last_clean":"2024-09-11T01:20:09.963447+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:20:09.963447+0000","last_undegraded":"2024-09-11T01:20:09.963447+0000","last_fullsized":"2024-09-11T01:20:09.963447+0000","mapping_epoch":44,"log_start":"43'6000","ondisk_log_start":"43'6000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15212544,"num_objects":3714,"num_object_clones":0,"num_object_copies":7428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3714,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13867,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16514","reported_seq":16651,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:12.241139+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:20:12.241139+0000","last_peered":"2024-09-11T01:20:12.241139+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:20:12.241139+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15360000,"num_objects":3750,"num_object_clones":0,"num_object_copies":7500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13251,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-200}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938599+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938599+0000","last_peered":"2024-09-11T01:17:58.938599+0000","last_clean":"2024-09-11T01:17:58.938599+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938599+0000","last_undegraded":"2024-09-11T01:17:58.938599+0000","last_fullsized":"2024-09-11T01:17:58.938599+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16233","reported_seq":24270,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:11.818050+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:20:11.818050+0000","last_peered":"2024-09-11T01:20:11.818050+0000","last_clean":"2024-09-11T01:20:11.818050+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:20:11.818050+0000","last_undegraded":"2024-09-11T01:20:11.818050+0000","last_fullsized":"2024-09-11T01:20:11.818050+0000","mapping_epoch":46,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14954496,"num_objects":3651,"num_object_clones":0,"num_object_copies":7302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12888,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16521","reported_seq":16661,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:12.274913+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:20:12.274913+0000","last_peered":"2024-09-11T01:20:12.274913+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:20:12.274913+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14970880,"num_objects":3655,"num_object_clones":0,"num_object_copies":7310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13130,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-384}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971371+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971371+0000","last_peered":"2024-09-11T01:17:58.971371+0000","last_clean":"2024-09-11T01:17:58.971371+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971371+0000","last_undegraded":"2024-09-11T01:17:58.971371+0000","last_fullsized":"2024-09-11T01:17:58.971371+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938303+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938303+0000","last_peered":"2024-09-11T01:17:58.938303+0000","last_clean":"2024-09-11T01:17:58.938303+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938303+0000","last_undegraded":"2024-09-11T01:17:58.938303+0000","last_fullsized":"2024-09-11T01:17:58.938303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16320","reported_seq":16459,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:12.270233+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:20:12.270233+0000","last_peered":"2024-09-11T01:20:12.270233+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:20:12.270233+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14877289,"num_objects":3633,"num_object_clones":0,"num_object_copies":7266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13390,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-284}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":242946689,"num_objects":59315,"num_object_clones":0,"num_object_copies":118630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36294,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59315,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":222972,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11345,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":680337408,"data_stored":680266642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160722,"ondisk_log_size":160722,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182504,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":651464,"kb_used_data":240824,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93720376,"statfs":{"total":96636764160,"available":95969665024,"internally_reserved":0,"allocated":246603776,"data_stored":246334117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":420474539},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986276,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":577116,"kb_used_data":248396,"kb_used_omap":4,"kb_used_meta":328699,"kb_avail":93794724,"statfs":{"total":96636764160,"available":96045797376,"internally_reserved":0,"allocated":254357504,"data_stored":254087850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":336588458},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,2,3,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659221,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":530120,"kb_used_data":177776,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93841720,"statfs":{"total":96636764160,"available":96093921280,"internally_reserved":0,"allocated":182042624,"data_stored":181716708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":360771247},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,4,8,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":245514240,"data_stored":245510761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":181555200,"data_stored":181495464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":253267968,"data_stored":253260417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:17.197 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:17.943 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:20:17.952 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:20:18.007 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2024-09-11T01:20:16.731257+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":239399572,"num_objects":58451,"num_object_clones":0,"num_object_copies":116902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58451,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":223840,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11542,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160591,"ondisk_log_size":160591,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1758236,"kb_used_data":662372,"kb_used_omap":12,"kb_used_meta":1095795,"kb_avail":281357284,"statfs":{"total":289910292480,"available":288109858816,"internally_reserved":0,"allocated":678268928,"data_stored":677403699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1122094084},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,4,8,15,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22495232,"num_objects":-5492,"num_object_clones":0,"num_object_copies":-10984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-561,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5492,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":561,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002852"},"pg_stats":[{"pgid":"2.f","version":"47'16574","reported_seq":16653,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:12.315636+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:20:12.315636+0000","last_peered":"2024-09-11T01:20:12.315636+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:20:12.315636+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15122432,"num_objects":3692,"num_object_clones":0,"num_object_copies":7384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16574,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-380}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16727","reported_seq":16834,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:14.893101+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:20:14.893101+0000","last_peered":"2024-09-11T01:20:14.893101+0000","last_clean":"2024-09-11T01:20:14.893101+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:20:14.893101+0000","last_undegraded":"2024-09-11T01:20:14.893101+0000","last_fullsized":"2024-09-11T01:20:14.893101+0000","mapping_epoch":35,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14848000,"num_objects":3625,"num_object_clones":0,"num_object_copies":7250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3625,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14966,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16464","reported_seq":16598,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:14.929093+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:20:14.929093+0000","last_peered":"2024-09-11T01:20:14.929093+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:20:14.929093+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15069208,"num_objects":3680,"num_object_clones":0,"num_object_copies":7360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3680,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14765,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-216}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16501","reported_seq":16572,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:14.935024+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:20:14.935024+0000","last_peered":"2024-09-11T01:20:14.935024+0000","last_clean":"2024-09-11T01:20:14.935024+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:20:14.935024+0000","last_undegraded":"2024-09-11T01:20:14.935024+0000","last_fullsized":"2024-09-11T01:20:14.935024+0000","mapping_epoch":35,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14880768,"num_objects":3633,"num_object_clones":0,"num_object_copies":7266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16501,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16402","reported_seq":16535,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:12.247367+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:20:12.247367+0000","last_peered":"2024-09-11T01:20:12.247367+0000","last_clean":"2024-09-11T01:20:12.247367+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:20:12.247367+0000","last_undegraded":"2024-09-11T01:20:12.247367+0000","last_fullsized":"2024-09-11T01:20:12.247367+0000","mapping_epoch":34,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14774272,"num_objects":3607,"num_object_clones":0,"num_object_copies":7214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13407,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16485","reported_seq":16661,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:12.337453+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:20:12.337453+0000","last_peered":"2024-09-11T01:20:12.337453+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:20:12.337453+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15327232,"num_objects":3742,"num_object_clones":0,"num_object_copies":7484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12826,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-209}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16225","reported_seq":16359,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:12.285682+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:20:12.285682+0000","last_peered":"2024-09-11T01:20:12.285682+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:20:12.285682+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15175680,"num_objects":3705,"num_object_clones":0,"num_object_copies":7410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13082,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-281}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16538","reported_seq":16673,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:12.247392+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:20:12.247392+0000","last_peered":"2024-09-11T01:20:12.247392+0000","last_clean":"2024-09-11T01:20:12.247392+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:20:12.247392+0000","last_undegraded":"2024-09-11T01:20:12.247392+0000","last_fullsized":"2024-09-11T01:20:12.247392+0000","mapping_epoch":34,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15220736,"num_objects":3716,"num_object_clones":0,"num_object_copies":7432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13414,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934737+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934737+0000","last_peered":"2024-09-11T01:17:58.934737+0000","last_clean":"2024-09-11T01:17:58.934737+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934737+0000","last_undegraded":"2024-09-11T01:17:58.934737+0000","last_fullsized":"2024-09-11T01:17:58.934737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16427","reported_seq":23902,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:14.936233+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:20:14.936233+0000","last_peered":"2024-09-11T01:20:14.936233+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:20:14.936233+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14524416,"num_objects":3546,"num_object_clones":0,"num_object_copies":7092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":264,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14398,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3671,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3282}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16242","reported_seq":16376,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:12.167487+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:20:12.167487+0000","last_peered":"2024-09-11T01:20:12.167487+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:20:12.167487+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14852096,"num_objects":3626,"num_object_clones":0,"num_object_copies":7252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12829,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-358}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16576","reported_seq":16700,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:14.881838+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:20:14.881838+0000","last_peered":"2024-09-11T01:20:14.881838+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:20:14.881838+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14872576,"num_objects":3631,"num_object_clones":0,"num_object_copies":7262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14393,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-393}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935141+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935141+0000","last_peered":"2024-09-11T01:17:58.935141+0000","last_clean":"2024-09-11T01:17:58.935141+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935141+0000","last_undegraded":"2024-09-11T01:17:58.935141+0000","last_fullsized":"2024-09-11T01:17:58.935141+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16239","reported_seq":24226,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:14.908429+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:20:14.908429+0000","last_peered":"2024-09-11T01:20:14.908429+0000","last_clean":"2024-09-11T01:20:14.908429+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:20:14.908429+0000","last_undegraded":"2024-09-11T01:20:14.908429+0000","last_fullsized":"2024-09-11T01:20:14.908429+0000","mapping_epoch":44,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14569472,"num_objects":3557,"num_object_clones":0,"num_object_copies":7114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3557,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14024,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16514","reported_seq":16651,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:12.241139+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:20:12.241139+0000","last_peered":"2024-09-11T01:20:12.241139+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:20:12.241139+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15360000,"num_objects":3750,"num_object_clones":0,"num_object_copies":7500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13251,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-200}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938599+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938599+0000","last_peered":"2024-09-11T01:17:58.938599+0000","last_clean":"2024-09-11T01:17:58.938599+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938599+0000","last_undegraded":"2024-09-11T01:17:58.938599+0000","last_fullsized":"2024-09-11T01:17:58.938599+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16233","reported_seq":24270,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:11.818050+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:20:11.818050+0000","last_peered":"2024-09-11T01:20:11.818050+0000","last_clean":"2024-09-11T01:20:11.818050+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:20:11.818050+0000","last_undegraded":"2024-09-11T01:20:11.818050+0000","last_fullsized":"2024-09-11T01:20:11.818050+0000","mapping_epoch":46,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14954496,"num_objects":3651,"num_object_clones":0,"num_object_copies":7302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12888,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16521","reported_seq":16661,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:12.274913+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:20:12.274913+0000","last_peered":"2024-09-11T01:20:12.274913+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:20:12.274913+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14970880,"num_objects":3655,"num_object_clones":0,"num_object_copies":7310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13130,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-384}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971371+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971371+0000","last_peered":"2024-09-11T01:17:58.971371+0000","last_clean":"2024-09-11T01:17:58.971371+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971371+0000","last_undegraded":"2024-09-11T01:17:58.971371+0000","last_fullsized":"2024-09-11T01:17:58.971371+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938303+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938303+0000","last_peered":"2024-09-11T01:17:58.938303+0000","last_clean":"2024-09-11T01:17:58.938303+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938303+0000","last_undegraded":"2024-09-11T01:17:58.938303+0000","last_fullsized":"2024-09-11T01:17:58.938303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16320","reported_seq":16459,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:12.270233+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:20:12.270233+0000","last_peered":"2024-09-11T01:20:12.270233+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:20:12.270233+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14877289,"num_objects":3633,"num_object_clones":0,"num_object_copies":7266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13390,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-284}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":239399553,"num_objects":58449,"num_object_clones":0,"num_object_copies":116898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58449,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":223838,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11541,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":675659776,"data_stored":675589010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160588,"ondisk_log_size":160588,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182504,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":651464,"kb_used_data":240824,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93720376,"statfs":{"total":96636764160,"available":95969665024,"internally_reserved":0,"allocated":246603776,"data_stored":246334117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":420474539},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986277,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576652,"kb_used_data":243772,"kb_used_omap":4,"kb_used_meta":332859,"kb_avail":93795188,"statfs":{"total":96636764160,"available":96046272512,"internally_reserved":0,"allocated":249622528,"data_stored":249352874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":340848298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,0,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659221,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":530120,"kb_used_data":177776,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93841720,"statfs":{"total":96636764160,"available":96093921280,"internally_reserved":0,"allocated":182042624,"data_stored":181716708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":360771247},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,4,8,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":245514240,"data_stored":245510761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":181555200,"data_stored":181495464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":248590336,"data_stored":248582785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:18.009 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:18.732 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:20:18.733 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:20:18.796 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2024-09-11T01:20:16.731257+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":239399572,"num_objects":58451,"num_object_clones":0,"num_object_copies":116902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58451,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":223840,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11542,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160591,"ondisk_log_size":160591,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1758236,"kb_used_data":662372,"kb_used_omap":12,"kb_used_meta":1095795,"kb_avail":281357284,"statfs":{"total":289910292480,"available":288109858816,"internally_reserved":0,"allocated":678268928,"data_stored":677403699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1122094084},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,4,8,15,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22495232,"num_objects":-5492,"num_object_clones":0,"num_object_copies":-10984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-561,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5492,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5492,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":561,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002852"},"pg_stats":[{"pgid":"2.f","version":"47'16574","reported_seq":16653,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:12.315636+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:20:12.315636+0000","last_peered":"2024-09-11T01:20:12.315636+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:20:12.315636+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15122432,"num_objects":3692,"num_object_clones":0,"num_object_copies":7384,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3692,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16574,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-380}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16727","reported_seq":16834,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:14.893101+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:20:14.893101+0000","last_peered":"2024-09-11T01:20:14.893101+0000","last_clean":"2024-09-11T01:20:14.893101+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:20:14.893101+0000","last_undegraded":"2024-09-11T01:20:14.893101+0000","last_fullsized":"2024-09-11T01:20:14.893101+0000","mapping_epoch":35,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14848000,"num_objects":3625,"num_object_clones":0,"num_object_copies":7250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3625,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14966,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16464","reported_seq":16598,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:14.929093+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:20:14.929093+0000","last_peered":"2024-09-11T01:20:14.929093+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:20:14.929093+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15069208,"num_objects":3680,"num_object_clones":0,"num_object_copies":7360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3680,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14765,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-216}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16501","reported_seq":16572,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:14.935024+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:20:14.935024+0000","last_peered":"2024-09-11T01:20:14.935024+0000","last_clean":"2024-09-11T01:20:14.935024+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:20:14.935024+0000","last_undegraded":"2024-09-11T01:20:14.935024+0000","last_fullsized":"2024-09-11T01:20:14.935024+0000","mapping_epoch":35,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14880768,"num_objects":3633,"num_object_clones":0,"num_object_copies":7266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16501,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16402","reported_seq":16535,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:12.247367+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:20:12.247367+0000","last_peered":"2024-09-11T01:20:12.247367+0000","last_clean":"2024-09-11T01:20:12.247367+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:20:12.247367+0000","last_undegraded":"2024-09-11T01:20:12.247367+0000","last_fullsized":"2024-09-11T01:20:12.247367+0000","mapping_epoch":34,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14774272,"num_objects":3607,"num_object_clones":0,"num_object_copies":7214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3607,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13407,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16485","reported_seq":16661,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:12.337453+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:20:12.337453+0000","last_peered":"2024-09-11T01:20:12.337453+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:20:12.337453+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15327232,"num_objects":3742,"num_object_clones":0,"num_object_copies":7484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3742,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12826,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-209}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16225","reported_seq":16359,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:12.285682+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:20:12.285682+0000","last_peered":"2024-09-11T01:20:12.285682+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:20:12.285682+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15175680,"num_objects":3705,"num_object_clones":0,"num_object_copies":7410,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3705,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13082,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-281}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16538","reported_seq":16673,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:12.247392+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:20:12.247392+0000","last_peered":"2024-09-11T01:20:12.247392+0000","last_clean":"2024-09-11T01:20:12.247392+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:20:12.247392+0000","last_undegraded":"2024-09-11T01:20:12.247392+0000","last_fullsized":"2024-09-11T01:20:12.247392+0000","mapping_epoch":34,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15220736,"num_objects":3716,"num_object_clones":0,"num_object_copies":7432,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3716,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13414,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934737+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934737+0000","last_peered":"2024-09-11T01:17:58.934737+0000","last_clean":"2024-09-11T01:17:58.934737+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934737+0000","last_undegraded":"2024-09-11T01:17:58.934737+0000","last_fullsized":"2024-09-11T01:17:58.934737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16427","reported_seq":23902,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:14.936233+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:20:14.936233+0000","last_peered":"2024-09-11T01:20:14.936233+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:20:14.936233+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14524416,"num_objects":3546,"num_object_clones":0,"num_object_copies":7092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":264,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14398,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3671,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3282}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16242","reported_seq":16376,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:12.167487+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:20:12.167487+0000","last_peered":"2024-09-11T01:20:12.167487+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:20:12.167487+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14852096,"num_objects":3626,"num_object_clones":0,"num_object_copies":7252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3626,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12829,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-358}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":106,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.928087+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:17:58.928087+0000","last_peered":"2024-09-11T01:17:58.928087+0000","last_clean":"2024-09-11T01:17:58.928087+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:17:58.928087+0000","last_undegraded":"2024-09-11T01:17:58.928087+0000","last_fullsized":"2024-09-11T01:17:58.928087+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16576","reported_seq":16700,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:14.881838+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:20:14.881838+0000","last_peered":"2024-09-11T01:20:14.881838+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:20:14.881838+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14872576,"num_objects":3631,"num_object_clones":0,"num_object_copies":7262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14393,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-393}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935141+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935141+0000","last_peered":"2024-09-11T01:17:58.935141+0000","last_clean":"2024-09-11T01:17:58.935141+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935141+0000","last_undegraded":"2024-09-11T01:17:58.935141+0000","last_fullsized":"2024-09-11T01:17:58.935141+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16239","reported_seq":24226,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:14.908429+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:20:14.908429+0000","last_peered":"2024-09-11T01:20:14.908429+0000","last_clean":"2024-09-11T01:20:14.908429+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:20:14.908429+0000","last_undegraded":"2024-09-11T01:20:14.908429+0000","last_fullsized":"2024-09-11T01:20:14.908429+0000","mapping_epoch":44,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14569472,"num_objects":3557,"num_object_clones":0,"num_object_copies":7114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3557,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14024,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16514","reported_seq":16651,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:12.241139+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:20:12.241139+0000","last_peered":"2024-09-11T01:20:12.241139+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:20:12.241139+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15360000,"num_objects":3750,"num_object_clones":0,"num_object_copies":7500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3750,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13251,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-200}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938599+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938599+0000","last_peered":"2024-09-11T01:17:58.938599+0000","last_clean":"2024-09-11T01:17:58.938599+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938599+0000","last_undegraded":"2024-09-11T01:17:58.938599+0000","last_fullsized":"2024-09-11T01:17:58.938599+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16233","reported_seq":24270,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:11.818050+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:20:11.818050+0000","last_peered":"2024-09-11T01:20:11.818050+0000","last_clean":"2024-09-11T01:20:11.818050+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:20:11.818050+0000","last_undegraded":"2024-09-11T01:20:11.818050+0000","last_fullsized":"2024-09-11T01:20:11.818050+0000","mapping_epoch":46,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14954496,"num_objects":3651,"num_object_clones":0,"num_object_copies":7302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12888,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16521","reported_seq":16661,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:12.274913+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:20:12.274913+0000","last_peered":"2024-09-11T01:20:12.274913+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:20:12.274913+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14970880,"num_objects":3655,"num_object_clones":0,"num_object_copies":7310,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3655,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13130,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-384}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971371+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971371+0000","last_peered":"2024-09-11T01:17:58.971371+0000","last_clean":"2024-09-11T01:17:58.971371+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971371+0000","last_undegraded":"2024-09-11T01:17:58.971371+0000","last_fullsized":"2024-09-11T01:17:58.971371+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938303+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938303+0000","last_peered":"2024-09-11T01:17:58.938303+0000","last_clean":"2024-09-11T01:17:58.938303+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938303+0000","last_undegraded":"2024-09-11T01:17:58.938303+0000","last_fullsized":"2024-09-11T01:17:58.938303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16320","reported_seq":16459,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:12.270233+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:20:12.270233+0000","last_peered":"2024-09-11T01:20:12.270233+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:20:12.270233+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14877289,"num_objects":3633,"num_object_clones":0,"num_object_copies":7266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13390,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-284}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":239399553,"num_objects":58449,"num_object_clones":0,"num_object_copies":116898,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58449,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":223838,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11541,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":675659776,"data_stored":675589010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160588,"ondisk_log_size":160588,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182504,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":651464,"kb_used_data":240824,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93720376,"statfs":{"total":96636764160,"available":95969665024,"internally_reserved":0,"allocated":246603776,"data_stored":246334117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":420474539},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986277,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576652,"kb_used_data":243772,"kb_used_omap":4,"kb_used_meta":332859,"kb_avail":93795188,"statfs":{"total":96636764160,"available":96046272512,"internally_reserved":0,"allocated":249622528,"data_stored":249352874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":340848298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,0,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659221,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":530120,"kb_used_data":177776,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93841720,"statfs":{"total":96636764160,"available":96093921280,"internally_reserved":0,"allocated":182042624,"data_stored":181716708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":360771247},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,4,8,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":245514240,"data_stored":245510761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":181555200,"data_stored":181495464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":248590336,"data_stored":248582785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:18.798 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:20:18.798 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "47'16574", 'reported_seq': 16653, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:12.315636+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:20:12.315636+0000', 'last_peered': '2024-09-11T01:20:12.315636+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:20:12.315636+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "43'6500", 'ondisk_log_start': "43'6500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15122432, 'num_objects': 3692, 'num_object_clones': 0, 'num_object_copies': 7384, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3692, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16574, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -380}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:18.798 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:20:18.798 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'16464", 'reported_seq': 16598, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:14.929093+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:20:14.929093+0000', 'last_peered': '2024-09-11T01:20:14.929093+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:20:14.929093+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "43'6400", 'ondisk_log_start': "43'6400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15069208, 'num_objects': 3680, 'num_object_clones': 0, 'num_object_copies': 7360, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3680, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14765, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -216}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:18.798 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:20:18.798 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "47'16485", 'reported_seq': 16661, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:12.337453+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:20:12.337453+0000', 'last_peered': '2024-09-11T01:20:12.337453+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:20:12.337453+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "43'6400", 'ondisk_log_start': "43'6400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15327232, 'num_objects': 3742, 'num_object_clones': 0, 'num_object_copies': 7484, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3742, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12826, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -209}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:18.798 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:20:18.798 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "47'16225", 'reported_seq': 16359, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:12.285682+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:20:12.285682+0000', 'last_peered': '2024-09-11T01:20:12.285682+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:20:12.285682+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "43'6200", 'ondisk_log_start': "43'6200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15175680, 'num_objects': 3705, 'num_object_clones': 0, 'num_object_copies': 7410, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3705, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13082, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -281}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:18.799 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:20:18.799 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'16427", 'reported_seq': 23902, 'reported_epoch': 47, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:14.936233+0000', 'last_change': '2024-09-11T01:17:57.355893+0000', 'last_active': '2024-09-11T01:20:14.936233+0000', 'last_peered': '2024-09-11T01:20:14.936233+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:20:14.936233+0000', 'last_undegraded': '2024-09-11T01:13:09.744222+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 42, 'log_start': "43'6400", 'ondisk_log_start': "43'6400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14524416, 'num_objects': 3546, 'num_object_clones': 0, 'num_object_copies': 7092, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 264, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3546, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14398, 'num_write_kb': 31844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3671, 'num_bytes_recovered': 4505600, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': 3282}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:18.799 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:20:18.799 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "47'16242", 'reported_seq': 16376, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:12.167487+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:20:12.167487+0000', 'last_peered': '2024-09-11T01:20:12.167487+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:20:12.167487+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "43'6200", 'ondisk_log_start': "43'6200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14852096, 'num_objects': 3626, 'num_object_clones': 0, 'num_object_copies': 7252, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3626, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 12829, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -358}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:18.799 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:20:18.799 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'16576", 'reported_seq': 16700, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:14.881838+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:20:14.881838+0000', 'last_peered': '2024-09-11T01:20:14.881838+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:20:14.881838+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "43'6500", 'ondisk_log_start': "43'6500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14872576, 'num_objects': 3631, 'num_object_clones': 0, 'num_object_copies': 7262, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3631, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14393, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -393}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:18.799 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:20:18.799 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "47'16514", 'reported_seq': 16651, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:12.241139+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:20:12.241139+0000', 'last_peered': '2024-09-11T01:20:12.241139+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:20:12.241139+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "43'6500", 'ondisk_log_start': "43'6500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 15360000, 'num_objects': 3750, 'num_object_clones': 0, 'num_object_copies': 7500, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3750, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13251, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -200}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:18.799 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:20:18.799 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "47'16521", 'reported_seq': 16661, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:12.274913+0000', 'last_change': '2024-09-11T01:13:10.227777+0000', 'last_active': '2024-09-11T01:20:12.274913+0000', 'last_peered': '2024-09-11T01:20:12.274913+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:20:12.274913+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "43'6500", 'ondisk_log_start': "43'6500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10021, 'log_dups_size': 0, 'ondisk_log_size': 10021, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14970880, 'num_objects': 3655, 'num_object_clones': 0, 'num_object_copies': 7310, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4041, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3655, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13130, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -384}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:18.800 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:20:18.800 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "47'16320", 'reported_seq': 16459, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:12.270233+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:20:12.270233+0000', 'last_peered': '2024-09-11T01:20:12.270233+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:20:12.270233+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "43'6300", 'ondisk_log_start': "43'6300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10020, 'log_dups_size': 0, 'ondisk_log_size': 10020, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14877289, 'num_objects': 3633, 'num_object_clones': 0, 'num_object_copies': 7266, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3633, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13390, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -284}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:18.800 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:20:19.701 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:20:19.701 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 36098/113916 objects degraded (31.688%), 10 pgs degraded, 10 pgs undersized","count":20},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":556,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":47,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":10},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":9},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":56958,"data_bytes":233284244,"bytes_used":1798807552,"bytes_avail":288111484928,"bytes_total":289910292480,"degraded_objects":36098,"degraded_total":113916,"degraded_ratio":0.31688261526036726,"recovering_objects_per_sec":31,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":383,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":511},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:20:19.762 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:20:19.762 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:20.454 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:20:20.455 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:20:20.508 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2024-09-11T01:20:18.735656+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":233284244,"num_objects":56958,"num_object_clones":0,"num_object_copies":113916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56958,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":225333,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11542,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160884,"ondisk_log_size":160884,"up":48,"acting":38,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1756648,"kb_used_data":648312,"kb_used_omap":12,"kb_used_meta":1108275,"kb_avail":281358872,"statfs":{"total":289910292480,"available":288111484928,"internally_reserved":0,"allocated":663871488,"data_stored":663006259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1134873604},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,2,4,10,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25145344,"num_objects":-6139,"num_object_clones":0,"num_object_copies":-12278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-383,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6139,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6139,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":383,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006813"},"pg_stats":[{"pgid":"2.f","version":"47'16723","reported_seq":16806,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:17.300868+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:20:17.300868+0000","last_peered":"2024-09-11T01:20:17.300868+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:20:17.300868+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14512128,"num_objects":3543,"num_object_clones":0,"num_object_copies":7086,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3543,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16723,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-529}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16727","reported_seq":16834,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:14.893101+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:20:14.893101+0000","last_peered":"2024-09-11T01:20:14.893101+0000","last_clean":"2024-09-11T01:20:14.893101+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:20:14.893101+0000","last_undegraded":"2024-09-11T01:20:14.893101+0000","last_fullsized":"2024-09-11T01:20:14.893101+0000","mapping_epoch":35,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14848000,"num_objects":3625,"num_object_clones":0,"num_object_copies":7250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3625,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14966,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16464","reported_seq":16598,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:14.929093+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:20:14.929093+0000","last_peered":"2024-09-11T01:20:14.929093+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:20:14.929093+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":15069208,"num_objects":3680,"num_object_clones":0,"num_object_copies":7360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3680,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14765,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-216}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16501","reported_seq":16572,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:14.935024+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:20:14.935024+0000","last_peered":"2024-09-11T01:20:14.935024+0000","last_clean":"2024-09-11T01:20:14.935024+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:20:14.935024+0000","last_undegraded":"2024-09-11T01:20:14.935024+0000","last_fullsized":"2024-09-11T01:20:14.935024+0000","mapping_epoch":35,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14880768,"num_objects":3633,"num_object_clones":0,"num_object_copies":7266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3633,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16501,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"47'16545","reported_seq":16678,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:17.349276+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:20:17.349276+0000","last_peered":"2024-09-11T01:20:17.349276+0000","last_clean":"2024-09-11T01:20:17.349276+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:20:17.349276+0000","last_undegraded":"2024-09-11T01:20:17.349276+0000","last_fullsized":"2024-09-11T01:20:17.349276+0000","mapping_epoch":34,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14188544,"num_objects":3464,"num_object_clones":0,"num_object_copies":6928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13550,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"47'16633","reported_seq":16809,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:17.197353+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:20:17.197353+0000","last_peered":"2024-09-11T01:20:17.197353+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:20:17.197353+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14721024,"num_objects":3594,"num_object_clones":0,"num_object_copies":7188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3594,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12974,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-357}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"47'16394","reported_seq":16530,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:17.355916+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:20:17.355916+0000","last_peered":"2024-09-11T01:20:17.355916+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:20:17.355916+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14483456,"num_objects":3536,"num_object_clones":0,"num_object_copies":7072,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3536,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13251,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-450}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"47'16682","reported_seq":16817,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:17.322973+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:20:17.322973+0000","last_peered":"2024-09-11T01:20:17.322973+0000","last_clean":"2024-09-11T01:20:17.322973+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:20:17.322973+0000","last_undegraded":"2024-09-11T01:20:17.322973+0000","last_fullsized":"2024-09-11T01:20:17.322973+0000","mapping_epoch":34,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14630912,"num_objects":3572,"num_object_clones":0,"num_object_copies":7144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3572,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13558,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.934737+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:17:58.934737+0000","last_peered":"2024-09-11T01:17:58.934737+0000","last_clean":"2024-09-11T01:17:58.934737+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:17:58.934737+0000","last_undegraded":"2024-09-11T01:17:58.934737+0000","last_fullsized":"2024-09-11T01:17:58.934737+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16427","reported_seq":23902,"reported_epoch":47,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:14.936233+0000","last_change":"2024-09-11T01:17:57.355893+0000","last_active":"2024-09-11T01:20:14.936233+0000","last_peered":"2024-09-11T01:20:14.936233+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:20:14.936233+0000","last_undegraded":"2024-09-11T01:13:09.744222+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14524416,"num_objects":3546,"num_object_clones":0,"num_object_copies":7092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":264,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14398,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3671,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":3282}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"47'16390","reported_seq":16523,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:17.357375+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:20:17.357375+0000","last_peered":"2024-09-11T01:20:17.357375+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:20:17.357375+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14245888,"num_objects":3478,"num_object_clones":0,"num_object_copies":6956,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3478,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12977,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-506}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":107,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:18:11.473009+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:18:11.473009+0000","last_peered":"2024-09-11T01:18:11.473009+0000","last_clean":"2024-09-11T01:18:11.473009+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:18:11.473009+0000","last_undegraded":"2024-09-11T01:18:11.473009+0000","last_fullsized":"2024-09-11T01:18:11.473009+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16576","reported_seq":16700,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:14.881838+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:20:14.881838+0000","last_peered":"2024-09-11T01:20:14.881838+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:20:14.881838+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14872576,"num_objects":3631,"num_object_clones":0,"num_object_copies":7262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3631,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14393,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-393}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":85,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.935141+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:17:58.935141+0000","last_peered":"2024-09-11T01:17:58.935141+0000","last_clean":"2024-09-11T01:17:58.935141+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:17:58.935141+0000","last_undegraded":"2024-09-11T01:17:58.935141+0000","last_fullsized":"2024-09-11T01:17:58.935141+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16239","reported_seq":24226,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:14.908429+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:20:14.908429+0000","last_peered":"2024-09-11T01:20:14.908429+0000","last_clean":"2024-09-11T01:20:14.908429+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:20:14.908429+0000","last_undegraded":"2024-09-11T01:20:14.908429+0000","last_fullsized":"2024-09-11T01:20:14.908429+0000","mapping_epoch":44,"log_start":"43'6200","ondisk_log_start":"43'6200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14569472,"num_objects":3557,"num_object_clones":0,"num_object_copies":7114,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3557,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14024,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"47'16685","reported_seq":16821,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:17.344580+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:20:17.344580+0000","last_peered":"2024-09-11T01:20:17.344580+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:20:17.344580+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14659584,"num_objects":3579,"num_object_clones":0,"num_object_copies":7158,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3579,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13422,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938599+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:17:58.938599+0000","last_peered":"2024-09-11T01:17:58.938599+0000","last_clean":"2024-09-11T01:17:58.938599+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:17:58.938599+0000","last_undegraded":"2024-09-11T01:17:58.938599+0000","last_fullsized":"2024-09-11T01:17:58.938599+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"47'16374","reported_seq":24411,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:16.911134+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:20:16.911134+0000","last_peered":"2024-09-11T01:20:16.911134+0000","last_clean":"2024-09-11T01:20:16.911134+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:20:16.911134+0000","last_undegraded":"2024-09-11T01:20:16.911134+0000","last_fullsized":"2024-09-11T01:20:16.911134+0000","mapping_epoch":46,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14376960,"num_objects":3510,"num_object_clones":0,"num_object_copies":7020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3510,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13029,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"47'16663","reported_seq":16802,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:17.341598+0000","last_change":"2024-09-11T01:13:10.227777+0000","last_active":"2024-09-11T01:20:17.341598+0000","last_peered":"2024-09-11T01:20:17.341598+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:20:17.341598+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14389248,"num_objects":3513,"num_object_clones":0,"num_object_copies":7026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4041,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3513,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13272,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-526}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.971371+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:17:58.971371+0000","last_peered":"2024-09-11T01:17:58.971371+0000","last_clean":"2024-09-11T01:17:58.971371+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:17:58.971371+0000","last_undegraded":"2024-09-11T01:17:58.971371+0000","last_fullsized":"2024-09-11T01:17:58.971371+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":80,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938303+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:17:58.938303+0000","last_peered":"2024-09-11T01:17:58.938303+0000","last_clean":"2024-09-11T01:17:58.938303+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:17:58.938303+0000","last_undegraded":"2024-09-11T01:17:58.938303+0000","last_fullsized":"2024-09-11T01:17:58.938303+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"47'16458","reported_seq":16597,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:17.329498+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:20:17.329498+0000","last_peered":"2024-09-11T01:20:17.329498+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:20:17.329498+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14312041,"num_objects":3495,"num_object_clones":0,"num_object_copies":6990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13528,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-422}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":233284225,"num_objects":56956,"num_object_clones":0,"num_object_copies":113912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":36098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56956,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":225331,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11541,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":661188608,"data_stored":661117842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160881,"ondisk_log_size":160881,"up":32,"acting":22,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182505,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":648600,"kb_used_data":233800,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93723240,"statfs":{"total":96636764160,"available":95972597760,"internally_reserved":0,"allocated":239411200,"data_stored":239141541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":424734379},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,6,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986277,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576652,"kb_used_data":243772,"kb_used_omap":4,"kb_used_meta":332859,"kb_avail":93795188,"statfs":{"total":96636764160,"available":96046272512,"internally_reserved":0,"allocated":249622528,"data_stored":249352874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":340848298},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,0,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659222,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":531396,"kb_used_data":170740,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93840444,"statfs":{"total":96636764160,"available":96092614656,"internally_reserved":0,"allocated":174837760,"data_stored":174511844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":369290927},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,0,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":238256128,"data_stored":238252649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":174342144,"data_stored":174282408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":248590336,"data_stored":248582785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:23.510 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:24.403 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:20:24.406 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:20:24.448 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":322,"stamp":"2024-09-11T01:20:22.736662+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":224359060,"num_objects":54779,"num_object_clones":0,"num_object_copies":109558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35768,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54779,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":227512,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11872,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160863,"ondisk_log_size":160863,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1740452,"kb_used_data":630704,"kb_used_omap":12,"kb_used_meta":1109683,"kb_avail":281375068,"statfs":{"total":289910292480,"available":288128069632,"internally_reserved":0,"allocated":645840896,"data_stored":644972151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1136315396},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,4,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18587648,"num_objects":-4538,"num_object_clones":0,"num_object_copies":-9076,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-526,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4538,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4538,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":526,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.005844"},"pg_stats":[{"pgid":"2.f","version":"49'16862","reported_seq":16949,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:22.398897+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:20:22.398897+0000","last_peered":"2024-09-11T01:20:22.398897+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:20:22.398897+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13942784,"num_objects":3404,"num_object_clones":0,"num_object_copies":6808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16862,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-668}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16859","reported_seq":16966,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:19.805164+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:20:19.805164+0000","last_peered":"2024-09-11T01:20:19.805164+0000","last_clean":"2024-09-11T01:20:19.805164+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:20:19.805164+0000","last_undegraded":"2024-09-11T01:20:19.805164+0000","last_fullsized":"2024-09-11T01:20:19.805164+0000","mapping_epoch":35,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14307328,"num_objects":3493,"num_object_clones":0,"num_object_copies":6986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15098,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16611","reported_seq":16745,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:19.904736+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:20:19.904736+0000","last_peered":"2024-09-11T01:20:19.904736+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:20:19.904736+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14467096,"num_objects":3533,"num_object_clones":0,"num_object_copies":7066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3533,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14912,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-363}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16644","reported_seq":16719,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:19.895649+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:20:19.895649+0000","last_peered":"2024-09-11T01:20:19.895649+0000","last_clean":"2024-09-11T01:20:19.895649+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:20:19.895649+0000","last_undegraded":"2024-09-11T01:20:19.895649+0000","last_fullsized":"2024-09-11T01:20:19.895649+0000","mapping_epoch":35,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14295040,"num_objects":3490,"num_object_clones":0,"num_object_copies":6980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16644,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'16677","reported_seq":16811,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.404832+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:20:22.404832+0000","last_peered":"2024-09-11T01:20:22.404832+0000","last_clean":"2024-09-11T01:20:22.404832+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:20:22.404832+0000","last_undegraded":"2024-09-11T01:20:22.404832+0000","last_fullsized":"2024-09-11T01:20:22.404832+0000","mapping_epoch":34,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13647872,"num_objects":3332,"num_object_clones":0,"num_object_copies":6664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13682,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'16771","reported_seq":16950,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:22.363903+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:20:22.363903+0000","last_peered":"2024-09-11T01:20:22.363903+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:20:22.363903+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14155776,"num_objects":3456,"num_object_clones":0,"num_object_copies":6912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13112,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'16519","reported_seq":16659,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:22.344480+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:20:22.344480+0000","last_peered":"2024-09-11T01:20:22.344480+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:20:22.344480+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13971456,"num_objects":3411,"num_object_clones":0,"num_object_copies":6822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13376,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-575}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'16827","reported_seq":16966,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.228484+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:20:22.228484+0000","last_peered":"2024-09-11T01:20:22.228484+0000","last_clean":"2024-09-11T01:20:22.228484+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:20:22.228484+0000","last_undegraded":"2024-09-11T01:20:22.228484+0000","last_fullsized":"2024-09-11T01:20:22.228484+0000","mapping_epoch":34,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14036992,"num_objects":3427,"num_object_clones":0,"num_object_copies":6854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13703,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T01:20:21.009682+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:21.009682+0000","last_peered":"2024-09-11T01:20:21.009682+0000","last_clean":"2024-09-11T01:20:21.009682+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:21.009682+0000","last_undegraded":"2024-09-11T01:20:21.009682+0000","last_fullsized":"2024-09-11T01:20:21.009682+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16575","reported_seq":24584,"reported_epoch":48,"state":"peering","last_fresh":"2024-09-11T01:20:21.009263+0000","last_change":"2024-09-11T01:20:21.937200+0000","last_active":"2024-09-11T01:20:21.937200+0000","last_peered":"2024-09-11T01:20:21.009263+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:20:21.009263+0000","last_undegraded":"2024-09-11T01:20:21.009263+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":48,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13918208,"num_objects":3398,"num_object_clones":0,"num_object_copies":6796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3398,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14546,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'16526","reported_seq":16662,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:22.366151+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:20:22.366151+0000","last_peered":"2024-09-11T01:20:22.366151+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:20:22.366151+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13688832,"num_objects":3342,"num_object_clones":0,"num_object_copies":6684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13113,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-642}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16702","reported_seq":16826,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:19.967736+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:20:19.967736+0000","last_peered":"2024-09-11T01:20:19.967736+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:20:19.967736+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14356480,"num_objects":3505,"num_object_clones":0,"num_object_copies":7010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14519,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-519}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":87,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T01:20:21.010191+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:21.010191+0000","last_peered":"2024-09-11T01:20:21.010191+0000","last_clean":"2024-09-11T01:20:21.010191+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:21.010191+0000","last_undegraded":"2024-09-11T01:20:21.010191+0000","last_fullsized":"2024-09-11T01:20:21.010191+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16383","reported_seq":24370,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:19.938454+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:20:19.938454+0000","last_peered":"2024-09-11T01:20:19.938454+0000","last_clean":"2024-09-11T01:20:19.938454+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:20:19.938454+0000","last_undegraded":"2024-09-11T01:20:19.938454+0000","last_fullsized":"2024-09-11T01:20:19.938454+0000","mapping_epoch":44,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13979648,"num_objects":3413,"num_object_clones":0,"num_object_copies":6826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3413,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14168,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'16834","reported_seq":16975,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:22.347830+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:20:22.347830+0000","last_peered":"2024-09-11T01:20:22.347830+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:20:22.347830+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14049280,"num_objects":3430,"num_object_clones":0,"num_object_copies":6860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13571,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-520}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T01:20:21.010085+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:21.010085+0000","last_peered":"2024-09-11T01:20:21.010085+0000","last_clean":"2024-09-11T01:20:21.010085+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:21.010085+0000","last_undegraded":"2024-09-11T01:20:21.010085+0000","last_fullsized":"2024-09-11T01:20:21.010085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'16489","reported_seq":24528,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T01:20:21.611854+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:20:21.611854+0000","last_peered":"2024-09-11T01:20:21.611854+0000","last_clean":"2024-09-11T01:20:21.611854+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:20:21.611854+0000","last_undegraded":"2024-09-11T01:20:21.611854+0000","last_fullsized":"2024-09-11T01:20:21.611854+0000","mapping_epoch":46,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13905920,"num_objects":3395,"num_object_clones":0,"num_object_copies":6790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13144,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'16795","reported_seq":17072,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:22.405979+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:20:22.405979+0000","last_peered":"2024-09-11T01:20:22.405979+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:20:22.405979+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13848576,"num_objects":3381,"num_object_clones":0,"num_object_copies":6762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3975,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13404,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-594}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T01:20:21.010018+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:21.010018+0000","last_peered":"2024-09-11T01:20:21.010018+0000","last_clean":"2024-09-11T01:20:21.010018+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:21.010018+0000","last_undegraded":"2024-09-11T01:20:21.010018+0000","last_fullsized":"2024-09-11T01:20:21.010018+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T01:20:21.023463+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:21.023463+0000","last_peered":"2024-09-11T01:20:21.023463+0000","last_clean":"2024-09-11T01:20:21.023463+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:21.023463+0000","last_undegraded":"2024-09-11T01:20:21.023463+0000","last_fullsized":"2024-09-11T01:20:21.023463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'16586","reported_seq":16729,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:22.306509+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:20:22.306509+0000","last_peered":"2024-09-11T01:20:22.306509+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:20:22.306509+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13787753,"num_objects":3367,"num_object_clones":0,"num_object_copies":6734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13656,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-550}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":224359041,"num_objects":54777,"num_object_clones":0,"num_object_copies":109554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35768,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54777,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":227510,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11871,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":643502080,"data_stored":643438272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160860,"ondisk_log_size":160860,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182506,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":646120,"kb_used_data":227160,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93725720,"statfs":{"total":96636764160,"available":95975137280,"internally_reserved":0,"allocated":232611840,"data_stored":232336944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":428994219},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986278,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":577708,"kb_used_data":239256,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93794132,"statfs":{"total":96636764160,"available":96045191168,"internally_reserved":0,"allocated":244998144,"data_stored":244728490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,3,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659223,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":516624,"kb_used_data":164288,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93855216,"statfs":{"total":96636764160,"available":96107741184,"internally_reserved":0,"allocated":168230912,"data_stored":167906717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":360771247},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,1,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":231727104,"data_stored":231723625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":167776256,"data_stored":167723478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":243998720,"data_stored":243991169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:24.453 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:25.402 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:20:25.403 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:20:25.455 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2024-09-11T01:20:24.737160+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":224359060,"num_objects":54779,"num_object_clones":0,"num_object_copies":109558,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35768,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54779,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":227512,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11872,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160863,"ondisk_log_size":160863,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1740452,"kb_used_data":630704,"kb_used_omap":12,"kb_used_meta":1109683,"kb_avail":281375068,"statfs":{"total":289910292480,"available":288128069632,"internally_reserved":0,"allocated":645840896,"data_stored":644972151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1136315396},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,4,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-15040512,"num_objects":-3672,"num_object_clones":0,"num_object_copies":-7344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-330,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-3672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3672,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":330,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.005902"},"pg_stats":[{"pgid":"2.f","version":"49'16862","reported_seq":16949,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:22.398897+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:20:22.398897+0000","last_peered":"2024-09-11T01:20:22.398897+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:20:22.398897+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13942784,"num_objects":3404,"num_object_clones":0,"num_object_copies":6808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16862,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-668}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"47'16859","reported_seq":16966,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:19.805164+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:20:19.805164+0000","last_peered":"2024-09-11T01:20:19.805164+0000","last_clean":"2024-09-11T01:20:19.805164+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:20:19.805164+0000","last_undegraded":"2024-09-11T01:20:19.805164+0000","last_fullsized":"2024-09-11T01:20:19.805164+0000","mapping_epoch":35,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14307328,"num_objects":3493,"num_object_clones":0,"num_object_copies":6986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15098,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"47'16611","reported_seq":16745,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:19.904736+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:20:19.904736+0000","last_peered":"2024-09-11T01:20:19.904736+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:20:19.904736+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14467096,"num_objects":3533,"num_object_clones":0,"num_object_copies":7066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3533,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":14912,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-363}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"47'16644","reported_seq":16719,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:19.895649+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:20:19.895649+0000","last_peered":"2024-09-11T01:20:19.895649+0000","last_clean":"2024-09-11T01:20:19.895649+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:20:19.895649+0000","last_undegraded":"2024-09-11T01:20:19.895649+0000","last_fullsized":"2024-09-11T01:20:19.895649+0000","mapping_epoch":35,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14295040,"num_objects":3490,"num_object_clones":0,"num_object_copies":6980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3490,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16644,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'16677","reported_seq":16811,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.404832+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:20:22.404832+0000","last_peered":"2024-09-11T01:20:22.404832+0000","last_clean":"2024-09-11T01:20:22.404832+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:20:22.404832+0000","last_undegraded":"2024-09-11T01:20:22.404832+0000","last_fullsized":"2024-09-11T01:20:22.404832+0000","mapping_epoch":34,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13647872,"num_objects":3332,"num_object_clones":0,"num_object_copies":6664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13682,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'16771","reported_seq":16950,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:22.363903+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:20:22.363903+0000","last_peered":"2024-09-11T01:20:22.363903+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:20:22.363903+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14155776,"num_objects":3456,"num_object_clones":0,"num_object_copies":6912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13112,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'16519","reported_seq":16659,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:22.344480+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:20:22.344480+0000","last_peered":"2024-09-11T01:20:22.344480+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:20:22.344480+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13971456,"num_objects":3411,"num_object_clones":0,"num_object_copies":6822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13376,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-575}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'16827","reported_seq":16966,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.228484+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:20:22.228484+0000","last_peered":"2024-09-11T01:20:22.228484+0000","last_clean":"2024-09-11T01:20:22.228484+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:20:22.228484+0000","last_undegraded":"2024-09-11T01:20:22.228484+0000","last_fullsized":"2024-09-11T01:20:22.228484+0000","mapping_epoch":34,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14036992,"num_objects":3427,"num_object_clones":0,"num_object_copies":6854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13703,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T01:20:21.009682+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:21.009682+0000","last_peered":"2024-09-11T01:20:21.009682+0000","last_clean":"2024-09-11T01:20:21.009682+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:21.009682+0000","last_undegraded":"2024-09-11T01:20:21.009682+0000","last_fullsized":"2024-09-11T01:20:21.009682+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16575","reported_seq":24584,"reported_epoch":48,"state":"peering","last_fresh":"2024-09-11T01:20:21.009263+0000","last_change":"2024-09-11T01:20:21.937200+0000","last_active":"2024-09-11T01:20:21.937200+0000","last_peered":"2024-09-11T01:20:21.009263+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:20:21.009263+0000","last_undegraded":"2024-09-11T01:20:21.009263+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":48,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13918208,"num_objects":3398,"num_object_clones":0,"num_object_copies":6796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3398,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14546,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.938341+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:17:58.938341+0000","last_peered":"2024-09-11T01:17:58.938341+0000","last_clean":"2024-09-11T01:17:58.938341+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:17:58.938341+0000","last_undegraded":"2024-09-11T01:17:58.938341+0000","last_fullsized":"2024-09-11T01:17:58.938341+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'16526","reported_seq":16662,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:22.366151+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:20:22.366151+0000","last_peered":"2024-09-11T01:20:22.366151+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:20:22.366151+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13688832,"num_objects":3342,"num_object_clones":0,"num_object_copies":6684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13113,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-642}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"47'16702","reported_seq":16826,"reported_epoch":47,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:19.967736+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:20:19.967736+0000","last_peered":"2024-09-11T01:20:19.967736+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:20:19.967736+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14356480,"num_objects":3505,"num_object_clones":0,"num_object_copies":7010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14519,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-519}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":87,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T01:20:21.010191+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:21.010191+0000","last_peered":"2024-09-11T01:20:21.010191+0000","last_clean":"2024-09-11T01:20:21.010191+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:21.010191+0000","last_undegraded":"2024-09-11T01:20:21.010191+0000","last_fullsized":"2024-09-11T01:20:21.010191+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"47'16383","reported_seq":24370,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:20:19.938454+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:20:19.938454+0000","last_peered":"2024-09-11T01:20:19.938454+0000","last_clean":"2024-09-11T01:20:19.938454+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:20:19.938454+0000","last_undegraded":"2024-09-11T01:20:19.938454+0000","last_fullsized":"2024-09-11T01:20:19.938454+0000","mapping_epoch":44,"log_start":"43'6300","ondisk_log_start":"43'6300","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13979648,"num_objects":3413,"num_object_clones":0,"num_object_copies":6826,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3413,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14168,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'16834","reported_seq":16975,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:22.347830+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:20:22.347830+0000","last_peered":"2024-09-11T01:20:22.347830+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:20:22.347830+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14049280,"num_objects":3430,"num_object_clones":0,"num_object_copies":6860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13571,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-520}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T01:20:21.010085+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:21.010085+0000","last_peered":"2024-09-11T01:20:21.010085+0000","last_clean":"2024-09-11T01:20:21.010085+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:21.010085+0000","last_undegraded":"2024-09-11T01:20:21.010085+0000","last_fullsized":"2024-09-11T01:20:21.010085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'16489","reported_seq":24528,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T01:20:21.611854+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:20:21.611854+0000","last_peered":"2024-09-11T01:20:21.611854+0000","last_clean":"2024-09-11T01:20:21.611854+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:20:21.611854+0000","last_undegraded":"2024-09-11T01:20:21.611854+0000","last_fullsized":"2024-09-11T01:20:21.611854+0000","mapping_epoch":46,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13905920,"num_objects":3395,"num_object_clones":0,"num_object_copies":6790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13144,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":75,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-11T01:17:58.941410+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:17:58.941410+0000","last_peered":"2024-09-11T01:17:58.941410+0000","last_clean":"2024-09-11T01:17:58.941410+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:17:58.941410+0000","last_undegraded":"2024-09-11T01:17:58.941410+0000","last_fullsized":"2024-09-11T01:17:58.941410+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'16795","reported_seq":17072,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:22.405979+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:20:22.405979+0000","last_peered":"2024-09-11T01:20:22.405979+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:20:22.405979+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13848576,"num_objects":3381,"num_object_clones":0,"num_object_copies":6762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3975,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13404,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-594}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T01:20:21.010018+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:21.010018+0000","last_peered":"2024-09-11T01:20:21.010018+0000","last_clean":"2024-09-11T01:20:21.010018+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:21.010018+0000","last_undegraded":"2024-09-11T01:20:21.010018+0000","last_fullsized":"2024-09-11T01:20:21.010018+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T01:20:21.023463+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:21.023463+0000","last_peered":"2024-09-11T01:20:21.023463+0000","last_clean":"2024-09-11T01:20:21.023463+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:21.023463+0000","last_undegraded":"2024-09-11T01:20:21.023463+0000","last_fullsized":"2024-09-11T01:20:21.023463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'16586","reported_seq":16729,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:22.306509+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:20:22.306509+0000","last_peered":"2024-09-11T01:20:22.306509+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:20:22.306509+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13787753,"num_objects":3367,"num_object_clones":0,"num_object_copies":6734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13656,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-550}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":224359041,"num_objects":54777,"num_object_clones":0,"num_object_copies":109554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35768,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54777,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":227510,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11871,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":643502080,"data_stored":643438272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160860,"ondisk_log_size":160860,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182506,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":646120,"kb_used_data":227160,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93725720,"statfs":{"total":96636764160,"available":95975137280,"internally_reserved":0,"allocated":232611840,"data_stored":232336944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":428994219},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986278,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":577708,"kb_used_data":239256,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93794132,"statfs":{"total":96636764160,"available":96045191168,"internally_reserved":0,"allocated":244998144,"data_stored":244728490,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,3,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659223,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":516624,"kb_used_data":164288,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93855216,"statfs":{"total":96636764160,"available":96107741184,"internally_reserved":0,"allocated":168230912,"data_stored":167906717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":360771247},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,1,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":231727104,"data_stored":231723625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":167776256,"data_stored":167723478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":243998720,"data_stored":243991169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:25.456 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:20:25.456 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'16862", 'reported_seq': 16949, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:22.398897+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:20:22.398897+0000', 'last_peered': '2024-09-11T01:20:22.398897+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:20:22.398897+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13942784, 'num_objects': 3404, 'num_object_clones': 0, 'num_object_copies': 6808, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3404, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16862, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -668}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:25.456 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:20:25.456 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "47'16611", 'reported_seq': 16745, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:19.904736+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:20:19.904736+0000', 'last_peered': '2024-09-11T01:20:19.904736+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:20:19.904736+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "43'6600", 'ondisk_log_start': "43'6600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14467096, 'num_objects': 3533, 'num_object_clones': 0, 'num_object_copies': 7066, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3533, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 14912, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -363}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:25.456 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:20:25.456 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'16771", 'reported_seq': 16950, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:22.363903+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:20:22.363903+0000', 'last_peered': '2024-09-11T01:20:22.363903+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:20:22.363903+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "43'6700", 'ondisk_log_start': "43'6700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10071, 'log_dups_size': 0, 'ondisk_log_size': 10071, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14155776, 'num_objects': 3456, 'num_object_clones': 0, 'num_object_copies': 6912, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3456, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13112, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -495}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:25.457 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:20:25.457 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'16519", 'reported_seq': 16659, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:22.344480+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:20:22.344480+0000', 'last_peered': '2024-09-11T01:20:22.344480+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:20:22.344480+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "43'6500", 'ondisk_log_start': "43'6500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13971456, 'num_objects': 3411, 'num_object_clones': 0, 'num_object_copies': 6822, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3411, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13376, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -575}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:25.457 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.3 is not active+clean 2024-09-11T01:20:25.457 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.3', 'version': "47'16575", 'reported_seq': 24584, 'reported_epoch': 48, 'state': 'peering', 'last_fresh': '2024-09-11T01:20:21.009263+0000', 'last_change': '2024-09-11T01:20:21.937200+0000', 'last_active': '2024-09-11T01:20:21.937200+0000', 'last_peered': '2024-09-11T01:20:21.009263+0000', 'last_clean': '2024-09-11T01:13:07.618645+0000', 'last_became_active': '2024-09-11T01:13:10.152092+0000', 'last_became_peered': '2024-09-11T01:13:10.152092+0000', 'last_unstale': '2024-09-11T01:20:21.009263+0000', 'last_undegraded': '2024-09-11T01:20:21.009263+0000', 'last_fullsized': '2024-09-11T01:13:09.658458+0000', 'mapping_epoch': 48, 'log_start': "43'6500", 'ondisk_log_start': "43'6500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2071", 'last_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:40.856413+0000', 'objects_scrubbed': 2022, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 959, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13918208, 'num_objects': 3398, 'num_object_clones': 0, 'num_object_copies': 6796, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3398, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14546, 'num_write_kb': 31844, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3935, 'num_bytes_recovered': 4505600, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-11T01:20:25.457 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:20:25.457 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'16526", 'reported_seq': 16662, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:22.366151+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:20:22.366151+0000', 'last_peered': '2024-09-11T01:20:22.366151+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:20:22.366151+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "43'6500", 'ondisk_log_start': "43'6500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13688832, 'num_objects': 3342, 'num_object_clones': 0, 'num_object_copies': 6684, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3342, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13113, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -642}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:25.457 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:20:25.457 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "47'16702", 'reported_seq': 16826, 'reported_epoch': 47, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:19.967736+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:20:19.967736+0000', 'last_peered': '2024-09-11T01:20:19.967736+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:20:19.967736+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "43'6700", 'ondisk_log_start': "43'6700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14356480, 'num_objects': 3505, 'num_object_clones': 0, 'num_object_copies': 7010, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3505, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14519, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -519}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:25.457 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:20:25.457 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'16834", 'reported_seq': 16975, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:22.347830+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:20:22.347830+0000', 'last_peered': '2024-09-11T01:20:22.347830+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:20:22.347830+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10034, 'log_dups_size': 0, 'ondisk_log_size': 10034, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 14049280, 'num_objects': 3430, 'num_object_clones': 0, 'num_object_copies': 6860, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3430, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13571, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -520}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:25.457 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:20:25.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'16795", 'reported_seq': 17072, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:22.405979+0000', 'last_change': '2024-09-11T01:20:20.985862+0000', 'last_active': '2024-09-11T01:20:22.405979+0000', 'last_peered': '2024-09-11T01:20:22.405979+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:20:22.405979+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "43'6700", 'ondisk_log_start': "43'6700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13848576, 'num_objects': 3381, 'num_object_clones': 0, 'num_object_copies': 6762, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3975, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3381, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13404, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 66, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -594}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:25.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:20:25.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'16586", 'reported_seq': 16729, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:22.306509+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:20:22.306509+0000', 'last_peered': '2024-09-11T01:20:22.306509+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:20:22.306509+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "43'6500", 'ondisk_log_start': "43'6500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13787753, 'num_objects': 3367, 'num_object_clones': 0, 'num_object_copies': 6734, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3367, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13656, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -550}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:25.458 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:20:26.445 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:20:26.445 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35768/109558 objects degraded (32.648%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":562,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":14},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"peering","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":54779,"data_bytes":224359060,"bytes_used":1782222848,"bytes_avail":288128069632,"bytes_total":289910292480,"inactive_pgs_ratio":0.041666667908430099,"degraded_objects":35768,"degraded_total":109558,"degraded_ratio":0.32647547417806094,"recovering_objects_per_sec":41,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":330,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":458},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:20:26.485 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:20:26.485 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:27.229 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:20:27.230 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:20:27.343 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2024-09-11T01:20:26.737592+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":221672084,"num_objects":54123,"num_object_clones":0,"num_object_copies":108246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35768,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54123,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":228168,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11872,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160919,"ondisk_log_size":160919,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1740412,"kb_used_data":626504,"kb_used_omap":12,"kb_used_meta":1113843,"kb_avail":281375108,"statfs":{"total":289910292480,"available":288128110592,"internally_reserved":0,"allocated":641540096,"data_stored":640660856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1140575236},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,3,1,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11612160,"num_objects":-2835,"num_object_clones":0,"num_object_copies":-5670,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-330,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2835,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2835,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":330,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001936"},"pg_stats":[{"pgid":"2.f","version":"49'16862","reported_seq":16949,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:22.398897+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:20:22.398897+0000","last_peered":"2024-09-11T01:20:22.398897+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:20:22.398897+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13942784,"num_objects":3404,"num_object_clones":0,"num_object_copies":6808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3404,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16862,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-668}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'16985","reported_seq":17096,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:24.886749+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:20:24.886749+0000","last_peered":"2024-09-11T01:20:24.886749+0000","last_clean":"2024-09-11T01:20:24.886749+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:20:24.886749+0000","last_undegraded":"2024-09-11T01:20:24.886749+0000","last_fullsized":"2024-09-11T01:20:24.886749+0000","mapping_epoch":35,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13791232,"num_objects":3367,"num_object_clones":0,"num_object_copies":6734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15224,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'16746","reported_seq":16884,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:24.996463+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:20:24.996463+0000","last_peered":"2024-09-11T01:20:24.996463+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:20:24.996463+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13914136,"num_objects":3398,"num_object_clones":0,"num_object_copies":6796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3398,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15047,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-498}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'16788","reported_seq":16867,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:24.944572+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:20:24.944572+0000","last_peered":"2024-09-11T01:20:24.944572+0000","last_clean":"2024-09-11T01:20:24.944572+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:20:24.944572+0000","last_undegraded":"2024-09-11T01:20:24.944572+0000","last_fullsized":"2024-09-11T01:20:24.944572+0000","mapping_epoch":35,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13705216,"num_objects":3346,"num_object_clones":0,"num_object_copies":6692,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3346,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16788,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'16677","reported_seq":16811,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.404832+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:20:22.404832+0000","last_peered":"2024-09-11T01:20:22.404832+0000","last_clean":"2024-09-11T01:20:22.404832+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:20:22.404832+0000","last_undegraded":"2024-09-11T01:20:22.404832+0000","last_fullsized":"2024-09-11T01:20:22.404832+0000","mapping_epoch":34,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13647872,"num_objects":3332,"num_object_clones":0,"num_object_copies":6664,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3332,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13682,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'16771","reported_seq":16950,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:22.363903+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:20:22.363903+0000","last_peered":"2024-09-11T01:20:22.363903+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:20:22.363903+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14155776,"num_objects":3456,"num_object_clones":0,"num_object_copies":6912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3456,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13112,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-495}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'16519","reported_seq":16659,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:22.344480+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:20:22.344480+0000","last_peered":"2024-09-11T01:20:22.344480+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:20:22.344480+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13971456,"num_objects":3411,"num_object_clones":0,"num_object_copies":6822,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3411,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13376,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-575}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'16827","reported_seq":16966,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.228484+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:20:22.228484+0000","last_peered":"2024-09-11T01:20:22.228484+0000","last_clean":"2024-09-11T01:20:22.228484+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:20:22.228484+0000","last_undegraded":"2024-09-11T01:20:22.228484+0000","last_fullsized":"2024-09-11T01:20:22.228484+0000","mapping_epoch":34,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14036992,"num_objects":3427,"num_object_clones":0,"num_object_copies":6854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3427,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13703,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":77,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T01:20:21.009682+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:21.009682+0000","last_peered":"2024-09-11T01:20:21.009682+0000","last_clean":"2024-09-11T01:20:21.009682+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:21.009682+0000","last_undegraded":"2024-09-11T01:20:21.009682+0000","last_fullsized":"2024-09-11T01:20:21.009682+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"47'16575","reported_seq":24584,"reported_epoch":48,"state":"peering","last_fresh":"2024-09-11T01:20:21.009263+0000","last_change":"2024-09-11T01:20:21.937200+0000","last_active":"2024-09-11T01:20:21.937200+0000","last_peered":"2024-09-11T01:20:21.009263+0000","last_clean":"2024-09-11T01:13:07.618645+0000","last_became_active":"2024-09-11T01:13:10.152092+0000","last_became_peered":"2024-09-11T01:13:10.152092+0000","last_unstale":"2024-09-11T01:20:21.009263+0000","last_undegraded":"2024-09-11T01:20:21.009263+0000","last_fullsized":"2024-09-11T01:13:09.658458+0000","mapping_epoch":48,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13918208,"num_objects":3398,"num_object_clones":0,"num_object_copies":6796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3398,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14546,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017961+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017961+0000","last_peered":"2024-09-11T01:20:22.017961+0000","last_clean":"2024-09-11T01:20:22.017961+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017961+0000","last_undegraded":"2024-09-11T01:20:22.017961+0000","last_fullsized":"2024-09-11T01:20:22.017961+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'16526","reported_seq":16662,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:22.366151+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:20:22.366151+0000","last_peered":"2024-09-11T01:20:22.366151+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:20:22.366151+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13688832,"num_objects":3342,"num_object_clones":0,"num_object_copies":6684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3342,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13113,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-642}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'16826","reported_seq":16954,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:24.767918+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:20:24.767918+0000","last_peered":"2024-09-11T01:20:24.767918+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:20:24.767918+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13848576,"num_objects":3381,"num_object_clones":0,"num_object_copies":6762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14643,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-643}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":87,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T01:20:21.010191+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:21.010191+0000","last_peered":"2024-09-11T01:20:21.010191+0000","last_clean":"2024-09-11T01:20:21.010191+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:21.010191+0000","last_undegraded":"2024-09-11T01:20:21.010191+0000","last_fullsized":"2024-09-11T01:20:21.010191+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'16510","reported_seq":24501,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:24.943785+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:20:24.943785+0000","last_peered":"2024-09-11T01:20:24.943785+0000","last_clean":"2024-09-11T01:20:24.943785+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:20:24.943785+0000","last_undegraded":"2024-09-11T01:20:24.943785+0000","last_fullsized":"2024-09-11T01:20:24.943785+0000","mapping_epoch":44,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13459456,"num_objects":3286,"num_object_clones":0,"num_object_copies":6572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14295,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'16834","reported_seq":16975,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:22.347830+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:20:22.347830+0000","last_peered":"2024-09-11T01:20:22.347830+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:20:22.347830+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":14049280,"num_objects":3430,"num_object_clones":0,"num_object_copies":6860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13571,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-520}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":77,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T01:20:21.010085+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:21.010085+0000","last_peered":"2024-09-11T01:20:21.010085+0000","last_clean":"2024-09-11T01:20:21.010085+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:21.010085+0000","last_undegraded":"2024-09-11T01:20:21.010085+0000","last_fullsized":"2024-09-11T01:20:21.010085+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"48'16489","reported_seq":24528,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T01:20:21.611854+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:20:21.611854+0000","last_peered":"2024-09-11T01:20:21.611854+0000","last_clean":"2024-09-11T01:20:21.611854+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:20:21.611854+0000","last_undegraded":"2024-09-11T01:20:21.611854+0000","last_fullsized":"2024-09-11T01:20:21.611854+0000","mapping_epoch":46,"log_start":"43'6400","ondisk_log_start":"43'6400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13905920,"num_objects":3395,"num_object_clones":0,"num_object_copies":6790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13144,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017690+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017690+0000","last_peered":"2024-09-11T01:20:22.017690+0000","last_clean":"2024-09-11T01:20:22.017690+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017690+0000","last_undegraded":"2024-09-11T01:20:22.017690+0000","last_fullsized":"2024-09-11T01:20:22.017690+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'16795","reported_seq":17072,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:22.405979+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:20:22.405979+0000","last_peered":"2024-09-11T01:20:22.405979+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:20:22.405979+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13848576,"num_objects":3381,"num_object_clones":0,"num_object_copies":6762,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3975,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3381,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13404,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":66,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-594}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":77,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T01:20:21.010018+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:21.010018+0000","last_peered":"2024-09-11T01:20:21.010018+0000","last_clean":"2024-09-11T01:20:21.010018+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:21.010018+0000","last_undegraded":"2024-09-11T01:20:21.010018+0000","last_fullsized":"2024-09-11T01:20:21.010018+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":82,"reported_epoch":48,"state":"active+clean","last_fresh":"2024-09-11T01:20:21.023463+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:21.023463+0000","last_peered":"2024-09-11T01:20:21.023463+0000","last_clean":"2024-09-11T01:20:21.023463+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:21.023463+0000","last_undegraded":"2024-09-11T01:20:21.023463+0000","last_fullsized":"2024-09-11T01:20:21.023463+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'16586","reported_seq":16729,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:22.306509+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:20:22.306509+0000","last_peered":"2024-09-11T01:20:22.306509+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:20:22.306509+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'6500","ondisk_log_start":"43'6500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13787753,"num_objects":3367,"num_object_clones":0,"num_object_copies":6734,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3367,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13656,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-550}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":221672065,"num_objects":54121,"num_object_clones":0,"num_object_copies":108242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35768,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54121,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":228166,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11871,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":639152128,"data_stored":639088320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160916,"ondisk_log_size":160916,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182506,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":646120,"kb_used_data":227160,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93725720,"statfs":{"total":96636764160,"available":95975137280,"internally_reserved":0,"allocated":232611840,"data_stored":232336944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":428994219},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986279,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":577668,"kb_used_data":235056,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93794172,"statfs":{"total":96636764160,"available":96045232128,"internally_reserved":0,"allocated":240697344,"data_stored":240417195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659223,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":516624,"kb_used_data":164288,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93855216,"statfs":{"total":96636764160,"available":96107741184,"internally_reserved":0,"allocated":168230912,"data_stored":167906717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":360771247},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,1,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":231727104,"data_stored":231723625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":167776256,"data_stored":167723478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":239648768,"data_stored":239641217,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:30.345 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:31.145 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:20:31.146 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:20:31.213 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2024-09-11T01:20:30.738508+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":212099732,"num_objects":51786,"num_object_clones":0,"num_object_copies":103572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35682,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51786,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":230505,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11958,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160856,"ondisk_log_size":160856,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1722428,"kb_used_data":608512,"kb_used_omap":12,"kb_used_meta":1113843,"kb_avail":281393092,"statfs":{"total":289910292480,"available":288146526208,"internally_reserved":0,"allocated":623116288,"data_stored":622226532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1140575236},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,7,2,13,6,10,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17809408,"num_objects":-4348,"num_object_clones":0,"num_object_copies":-8696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-201,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4348,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.729917"},"pg_stats":[{"pgid":"2.f","version":"49'17017","reported_seq":17104,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:27.309917+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:20:27.309917+0000","last_peered":"2024-09-11T01:20:27.309917+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:20:27.309917+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13307904,"num_objects":3249,"num_object_clones":0,"num_object_copies":6498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17017,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-823}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'17131","reported_seq":17242,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:29.998616+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:20:29.998616+0000","last_peered":"2024-09-11T01:20:29.998616+0000","last_clean":"2024-09-11T01:20:29.998616+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:20:29.998616+0000","last_undegraded":"2024-09-11T01:20:29.998616+0000","last_fullsized":"2024-09-11T01:20:29.998616+0000","mapping_epoch":35,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13193216,"num_objects":3221,"num_object_clones":0,"num_object_copies":6442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15370,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'16884","reported_seq":17022,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:29.951901+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:20:29.951901+0000","last_peered":"2024-09-11T01:20:29.951901+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:20:29.951901+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13348888,"num_objects":3260,"num_object_clones":0,"num_object_copies":6520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3260,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15185,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-636}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'16911","reported_seq":16989,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:29.942194+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:20:29.942194+0000","last_peered":"2024-09-11T01:20:29.942194+0000","last_clean":"2024-09-11T01:20:29.942194+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:20:29.942194+0000","last_undegraded":"2024-09-11T01:20:29.942194+0000","last_fullsized":"2024-09-11T01:20:29.942194+0000","mapping_epoch":35,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13201408,"num_objects":3223,"num_object_clones":0,"num_object_copies":6446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3223,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16911,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'16833","reported_seq":16970,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:27.406298+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:20:27.406298+0000","last_peered":"2024-09-11T01:20:27.406298+0000","last_clean":"2024-09-11T01:20:27.406298+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:20:27.406298+0000","last_undegraded":"2024-09-11T01:20:27.406298+0000","last_fullsized":"2024-09-11T01:20:27.406298+0000","mapping_epoch":34,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13008896,"num_objects":3176,"num_object_clones":0,"num_object_copies":6352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13838,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'16899","reported_seq":17079,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:27.314880+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:20:27.314880+0000","last_peered":"2024-09-11T01:20:27.314880+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:20:27.314880+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13631488,"num_objects":3328,"num_object_clones":0,"num_object_copies":6656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13240,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'16675","reported_seq":16815,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:27.448895+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:20:27.448895+0000","last_peered":"2024-09-11T01:20:27.448895+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:20:27.448895+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13332480,"num_objects":3255,"num_object_clones":0,"num_object_copies":6510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13532,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-731}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'16977","reported_seq":17116,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:27.400751+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:20:27.400751+0000","last_peered":"2024-09-11T01:20:27.400751+0000","last_clean":"2024-09-11T01:20:27.400751+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:20:27.400751+0000","last_undegraded":"2024-09-11T01:20:27.400751+0000","last_fullsized":"2024-09-11T01:20:27.400751+0000","mapping_epoch":34,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13422592,"num_objects":3277,"num_object_clones":0,"num_object_copies":6554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13853,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003131+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003131+0000","last_peered":"2024-09-11T01:20:22.003131+0000","last_clean":"2024-09-11T01:20:22.003131+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003131+0000","last_undegraded":"2024-09-11T01:20:22.003131+0000","last_fullsized":"2024-09-11T01:20:22.003131+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'16732","reported_seq":24750,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:26.892870+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:20:26.892870+0000","last_peered":"2024-09-11T01:20:26.892870+0000","last_clean":"2024-09-11T01:20:26.892870+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:20:26.892870+0000","last_undegraded":"2024-09-11T01:20:26.892870+0000","last_fullsized":"2024-09-11T01:20:26.892870+0000","mapping_epoch":48,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13275136,"num_objects":3241,"num_object_clones":0,"num_object_copies":6482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14703,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'16678","reported_seq":16815,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:27.363092+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:20:27.363092+0000","last_peered":"2024-09-11T01:20:27.363092+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:20:27.363092+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13066240,"num_objects":3190,"num_object_clones":0,"num_object_copies":6380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13265,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-794}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'16974","reported_seq":17103,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:29.996664+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:20:29.996664+0000","last_peered":"2024-09-11T01:20:29.996664+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:20:29.996664+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13242368,"num_objects":3233,"num_object_clones":0,"num_object_copies":6466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14791,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-791}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003225+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003225+0000","last_peered":"2024-09-11T01:20:22.003225+0000","last_clean":"2024-09-11T01:20:22.003225+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003225+0000","last_undegraded":"2024-09-11T01:20:22.003225+0000","last_fullsized":"2024-09-11T01:20:22.003225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'16658","reported_seq":24649,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:29.981115+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:20:29.981115+0000","last_peered":"2024-09-11T01:20:29.981115+0000","last_clean":"2024-09-11T01:20:29.981115+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:20:29.981115+0000","last_undegraded":"2024-09-11T01:20:29.981115+0000","last_fullsized":"2024-09-11T01:20:29.981115+0000","mapping_epoch":44,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12853248,"num_objects":3138,"num_object_clones":0,"num_object_copies":6276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14443,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'16987","reported_seq":17127,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:27.326105+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:20:27.326105+0000","last_peered":"2024-09-11T01:20:27.326105+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:20:27.326105+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13422592,"num_objects":3277,"num_object_clones":0,"num_object_copies":6554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13724,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-673}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007532+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007532+0000","last_peered":"2024-09-11T01:20:22.007532+0000","last_clean":"2024-09-11T01:20:22.007532+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007532+0000","last_undegraded":"2024-09-11T01:20:22.007532+0000","last_fullsized":"2024-09-11T01:20:22.007532+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'16631","reported_seq":24672,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:26.982831+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:20:26.982831+0000","last_peered":"2024-09-11T01:20:26.982831+0000","last_clean":"2024-09-11T01:20:26.982831+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:20:26.982831+0000","last_undegraded":"2024-09-11T01:20:26.982831+0000","last_fullsized":"2024-09-11T01:20:26.982831+0000","mapping_epoch":46,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13324288,"num_objects":3253,"num_object_clones":0,"num_object_copies":6506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13286,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'16935","reported_seq":17385,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:27.447519+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:20:27.447519+0000","last_peered":"2024-09-11T01:20:27.447519+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:20:27.447519+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13275136,"num_objects":3241,"num_object_clones":0,"num_object_copies":6482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3889,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13544,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007560+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007560+0000","last_peered":"2024-09-11T01:20:22.007560+0000","last_clean":"2024-09-11T01:20:22.007560+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007560+0000","last_undegraded":"2024-09-11T01:20:22.007560+0000","last_fullsized":"2024-09-11T01:20:22.007560+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008171+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008171+0000","last_peered":"2024-09-11T01:20:22.008171+0000","last_clean":"2024-09-11T01:20:22.008171+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008171+0000","last_undegraded":"2024-09-11T01:20:22.008171+0000","last_fullsized":"2024-09-11T01:20:22.008171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'16731","reported_seq":16875,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:27.395305+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:20:27.395305+0000","last_peered":"2024-09-11T01:20:27.395305+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:20:27.395305+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13193833,"num_objects":3222,"num_object_clones":0,"num_object_copies":6444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13801,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-695}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":212099713,"num_objects":51784,"num_object_clones":0,"num_object_copies":103568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35682,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51784,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":230503,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11957,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":620146688,"data_stored":620082880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160853,"ondisk_log_size":160853,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182507,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":631116,"kb_used_data":220464,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93740724,"statfs":{"total":96636764160,"available":95990501376,"internally_reserved":0,"allocated":225755136,"data_stored":225474982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":420474539},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,0,2,2,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986280,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":577180,"kb_used_data":230412,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93794660,"statfs":{"total":96636764160,"available":96045731840,"internally_reserved":0,"allocated":235941888,"data_stored":235661739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,4,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659224,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":514132,"kb_used_data":157636,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93857708,"statfs":{"total":96636764160,"available":96110292992,"internally_reserved":0,"allocated":161419264,"data_stored":161089811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":365031087},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,7,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":224481280,"data_stored":224477801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":160772096,"data_stored":160719318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":234893312,"data_stored":234885761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:31.215 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:31.903 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:20:31.903 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:20:31.983 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":326,"stamp":"2024-09-11T01:20:30.738508+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":212099732,"num_objects":51786,"num_object_clones":0,"num_object_copies":103572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35682,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51786,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":230505,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11958,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160856,"ondisk_log_size":160856,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1722428,"kb_used_data":608512,"kb_used_omap":12,"kb_used_meta":1113843,"kb_avail":281393092,"statfs":{"total":289910292480,"available":288146526208,"internally_reserved":0,"allocated":623116288,"data_stored":622226532,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1140575236},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,7,2,13,6,10,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17809408,"num_objects":-4348,"num_object_clones":0,"num_object_copies":-8696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-201,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4348,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":201,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.729917"},"pg_stats":[{"pgid":"2.f","version":"49'17017","reported_seq":17104,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:27.309917+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:20:27.309917+0000","last_peered":"2024-09-11T01:20:27.309917+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:20:27.309917+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13307904,"num_objects":3249,"num_object_clones":0,"num_object_copies":6498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3249,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17017,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-823}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'17131","reported_seq":17242,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:29.998616+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:20:29.998616+0000","last_peered":"2024-09-11T01:20:29.998616+0000","last_clean":"2024-09-11T01:20:29.998616+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:20:29.998616+0000","last_undegraded":"2024-09-11T01:20:29.998616+0000","last_fullsized":"2024-09-11T01:20:29.998616+0000","mapping_epoch":35,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13193216,"num_objects":3221,"num_object_clones":0,"num_object_copies":6442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15370,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'16884","reported_seq":17022,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:29.951901+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:20:29.951901+0000","last_peered":"2024-09-11T01:20:29.951901+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:20:29.951901+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13348888,"num_objects":3260,"num_object_clones":0,"num_object_copies":6520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3260,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15185,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-636}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'16911","reported_seq":16989,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:29.942194+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:20:29.942194+0000","last_peered":"2024-09-11T01:20:29.942194+0000","last_clean":"2024-09-11T01:20:29.942194+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:20:29.942194+0000","last_undegraded":"2024-09-11T01:20:29.942194+0000","last_fullsized":"2024-09-11T01:20:29.942194+0000","mapping_epoch":35,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13201408,"num_objects":3223,"num_object_clones":0,"num_object_copies":6446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3223,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16911,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'16833","reported_seq":16970,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:27.406298+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:20:27.406298+0000","last_peered":"2024-09-11T01:20:27.406298+0000","last_clean":"2024-09-11T01:20:27.406298+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:20:27.406298+0000","last_undegraded":"2024-09-11T01:20:27.406298+0000","last_fullsized":"2024-09-11T01:20:27.406298+0000","mapping_epoch":34,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13008896,"num_objects":3176,"num_object_clones":0,"num_object_copies":6352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13838,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'16899","reported_seq":17079,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:27.314880+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:20:27.314880+0000","last_peered":"2024-09-11T01:20:27.314880+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:20:27.314880+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13631488,"num_objects":3328,"num_object_clones":0,"num_object_copies":6656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13240,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-623}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'16675","reported_seq":16815,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:27.448895+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:20:27.448895+0000","last_peered":"2024-09-11T01:20:27.448895+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:20:27.448895+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13332480,"num_objects":3255,"num_object_clones":0,"num_object_copies":6510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3255,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13532,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-731}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'16977","reported_seq":17116,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:27.400751+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:20:27.400751+0000","last_peered":"2024-09-11T01:20:27.400751+0000","last_clean":"2024-09-11T01:20:27.400751+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:20:27.400751+0000","last_undegraded":"2024-09-11T01:20:27.400751+0000","last_fullsized":"2024-09-11T01:20:27.400751+0000","mapping_epoch":34,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13422592,"num_objects":3277,"num_object_clones":0,"num_object_copies":6554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13853,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003131+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003131+0000","last_peered":"2024-09-11T01:20:22.003131+0000","last_clean":"2024-09-11T01:20:22.003131+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003131+0000","last_undegraded":"2024-09-11T01:20:22.003131+0000","last_fullsized":"2024-09-11T01:20:22.003131+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'16732","reported_seq":24750,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:26.892870+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:20:26.892870+0000","last_peered":"2024-09-11T01:20:26.892870+0000","last_clean":"2024-09-11T01:20:26.892870+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:20:26.892870+0000","last_undegraded":"2024-09-11T01:20:26.892870+0000","last_fullsized":"2024-09-11T01:20:26.892870+0000","mapping_epoch":48,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10032,"log_dups_size":0,"ondisk_log_size":10032,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13275136,"num_objects":3241,"num_object_clones":0,"num_object_copies":6482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14703,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'16678","reported_seq":16815,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:27.363092+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:20:27.363092+0000","last_peered":"2024-09-11T01:20:27.363092+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:20:27.363092+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13066240,"num_objects":3190,"num_object_clones":0,"num_object_copies":6380,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3190,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13265,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-794}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'16974","reported_seq":17103,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:29.996664+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:20:29.996664+0000","last_peered":"2024-09-11T01:20:29.996664+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:20:29.996664+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13242368,"num_objects":3233,"num_object_clones":0,"num_object_copies":6466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14791,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-791}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003225+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003225+0000","last_peered":"2024-09-11T01:20:22.003225+0000","last_clean":"2024-09-11T01:20:22.003225+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003225+0000","last_undegraded":"2024-09-11T01:20:22.003225+0000","last_fullsized":"2024-09-11T01:20:22.003225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'16658","reported_seq":24649,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:29.981115+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:20:29.981115+0000","last_peered":"2024-09-11T01:20:29.981115+0000","last_clean":"2024-09-11T01:20:29.981115+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:20:29.981115+0000","last_undegraded":"2024-09-11T01:20:29.981115+0000","last_fullsized":"2024-09-11T01:20:29.981115+0000","mapping_epoch":44,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12853248,"num_objects":3138,"num_object_clones":0,"num_object_copies":6276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14443,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'16987","reported_seq":17127,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:27.326105+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:20:27.326105+0000","last_peered":"2024-09-11T01:20:27.326105+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:20:27.326105+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13422592,"num_objects":3277,"num_object_clones":0,"num_object_copies":6554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13724,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-673}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007532+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007532+0000","last_peered":"2024-09-11T01:20:22.007532+0000","last_clean":"2024-09-11T01:20:22.007532+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007532+0000","last_undegraded":"2024-09-11T01:20:22.007532+0000","last_fullsized":"2024-09-11T01:20:22.007532+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'16631","reported_seq":24672,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:26.982831+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:20:26.982831+0000","last_peered":"2024-09-11T01:20:26.982831+0000","last_clean":"2024-09-11T01:20:26.982831+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:20:26.982831+0000","last_undegraded":"2024-09-11T01:20:26.982831+0000","last_fullsized":"2024-09-11T01:20:26.982831+0000","mapping_epoch":46,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13324288,"num_objects":3253,"num_object_clones":0,"num_object_copies":6506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3253,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13286,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'16935","reported_seq":17385,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:27.447519+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:20:27.447519+0000","last_peered":"2024-09-11T01:20:27.447519+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:20:27.447519+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13275136,"num_objects":3241,"num_object_clones":0,"num_object_copies":6482,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3889,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3241,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13544,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":152,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-648}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007560+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007560+0000","last_peered":"2024-09-11T01:20:22.007560+0000","last_clean":"2024-09-11T01:20:22.007560+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007560+0000","last_undegraded":"2024-09-11T01:20:22.007560+0000","last_fullsized":"2024-09-11T01:20:22.007560+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008171+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008171+0000","last_peered":"2024-09-11T01:20:22.008171+0000","last_clean":"2024-09-11T01:20:22.008171+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008171+0000","last_undegraded":"2024-09-11T01:20:22.008171+0000","last_fullsized":"2024-09-11T01:20:22.008171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'16731","reported_seq":16875,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:27.395305+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:20:27.395305+0000","last_peered":"2024-09-11T01:20:27.395305+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:20:27.395305+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13193833,"num_objects":3222,"num_object_clones":0,"num_object_copies":6444,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3222,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13801,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-695}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":212099713,"num_objects":51784,"num_object_clones":0,"num_object_copies":103568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35682,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51784,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":230503,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11957,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":620146688,"data_stored":620082880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160853,"ondisk_log_size":160853,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182507,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":631116,"kb_used_data":220464,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93740724,"statfs":{"total":96636764160,"available":95990501376,"internally_reserved":0,"allocated":225755136,"data_stored":225474982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":420474539},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,0,2,2,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986280,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":577180,"kb_used_data":230412,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93794660,"statfs":{"total":96636764160,"available":96045731840,"internally_reserved":0,"allocated":235941888,"data_stored":235661739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,4,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659224,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":514132,"kb_used_data":157636,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93857708,"statfs":{"total":96636764160,"available":96110292992,"internally_reserved":0,"allocated":161419264,"data_stored":161089811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":365031087},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,7,3,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":224481280,"data_stored":224477801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":160772096,"data_stored":160719318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":234893312,"data_stored":234885761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:31.985 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:20:31.985 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'17017", 'reported_seq': 17104, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:27.309917+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:20:27.309917+0000', 'last_peered': '2024-09-11T01:20:27.309917+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:20:27.309917+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "43'7000", 'ondisk_log_start': "43'7000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13307904, 'num_objects': 3249, 'num_object_clones': 0, 'num_object_copies': 6498, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3249, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17017, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -823}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:31.985 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:20:31.985 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'16884", 'reported_seq': 17022, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:29.951901+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:20:29.951901+0000', 'last_peered': '2024-09-11T01:20:29.951901+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:20:29.951901+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10084, 'log_dups_size': 0, 'ondisk_log_size': 10084, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13348888, 'num_objects': 3260, 'num_object_clones': 0, 'num_object_copies': 6520, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3260, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15185, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -636}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:31.986 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:20:31.986 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'16899", 'reported_seq': 17079, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:27.314880+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:20:27.314880+0000', 'last_peered': '2024-09-11T01:20:27.314880+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:20:27.314880+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13631488, 'num_objects': 3328, 'num_object_clones': 0, 'num_object_copies': 6656, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3328, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13240, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -623}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:31.986 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:20:31.986 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'16675", 'reported_seq': 16815, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:27.448895+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:20:27.448895+0000', 'last_peered': '2024-09-11T01:20:27.448895+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:20:27.448895+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "43'6600", 'ondisk_log_start': "43'6600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13332480, 'num_objects': 3255, 'num_object_clones': 0, 'num_object_copies': 6510, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3255, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13532, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -731}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:31.986 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:20:31.986 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'16678", 'reported_seq': 16815, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:27.363092+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:20:27.363092+0000', 'last_peered': '2024-09-11T01:20:27.363092+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:20:27.363092+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "43'6600", 'ondisk_log_start': "43'6600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13066240, 'num_objects': 3190, 'num_object_clones': 0, 'num_object_copies': 6380, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3190, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13265, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -794}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:31.986 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:20:31.986 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'16974", 'reported_seq': 17103, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:29.996664+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:20:29.996664+0000', 'last_peered': '2024-09-11T01:20:29.996664+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:20:29.996664+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "43'6900", 'ondisk_log_start': "43'6900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13242368, 'num_objects': 3233, 'num_object_clones': 0, 'num_object_copies': 6466, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3233, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14791, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -791}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:31.986 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:20:31.986 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'16987", 'reported_seq': 17127, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:27.326105+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:20:27.326105+0000', 'last_peered': '2024-09-11T01:20:27.326105+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:20:27.326105+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "43'6900", 'ondisk_log_start': "43'6900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13422592, 'num_objects': 3277, 'num_object_clones': 0, 'num_object_copies': 6554, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3277, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13724, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -673}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:31.986 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:20:31.987 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'16935", 'reported_seq': 17385, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:27.447519+0000', 'last_change': '2024-09-11T01:20:20.985862+0000', 'last_active': '2024-09-11T01:20:27.447519+0000', 'last_peered': '2024-09-11T01:20:27.447519+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:20:27.447519+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "43'6900", 'ondisk_log_start': "43'6900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13275136, 'num_objects': 3241, 'num_object_clones': 0, 'num_object_copies': 6482, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3889, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3241, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13544, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 152, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -648}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:31.987 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:20:31.987 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'16731", 'reported_seq': 16875, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:27.395305+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:20:27.395305+0000', 'last_peered': '2024-09-11T01:20:27.395305+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:20:27.395305+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "43'6700", 'ondisk_log_start': "43'6700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10031, 'log_dups_size': 0, 'ondisk_log_size': 10031, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 13193833, 'num_objects': 3222, 'num_object_clones': 0, 'num_object_copies': 6444, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3222, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13801, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -695}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:31.987 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:20:32.956 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:20:32.956 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35682/103572 objects degraded (34.451%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":569,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":51786,"data_bytes":212099732,"bytes_used":1763766272,"bytes_avail":288146526208,"bytes_total":289910292480,"degraded_objects":35682,"degraded_total":103572,"degraded_ratio":0.34451396130228246,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":201,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":446},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:20:33.045 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:20:33.046 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:33.820 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:20:33.822 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:20:33.883 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-09-11T01:20:32.742924+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":205329044,"num_objects":50133,"num_object_clones":0,"num_object_copies":100266,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35604,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50133,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":232158,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12036,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160909,"ondisk_log_size":160909,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1720920,"kb_used_data":594536,"kb_used_omap":12,"kb_used_meta":1126323,"kb_avail":281394600,"statfs":{"total":289910292480,"available":288148070400,"internally_reserved":0,"allocated":608804864,"data_stored":607915108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1153354756},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,6,11,14,6],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24043520,"num_objects":-5870,"num_object_clones":0,"num_object_copies":-11740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-230,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5870,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.740639"},"pg_stats":[{"pgid":"2.f","version":"49'17155","reported_seq":17242,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.417165+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:20:32.417165+0000","last_peered":"2024-09-11T01:20:32.417165+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:20:32.417165+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12742656,"num_objects":3111,"num_object_clones":0,"num_object_copies":6222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-961}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'17131","reported_seq":17242,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:29.998616+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:20:29.998616+0000","last_peered":"2024-09-11T01:20:29.998616+0000","last_clean":"2024-09-11T01:20:29.998616+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:20:29.998616+0000","last_undegraded":"2024-09-11T01:20:29.998616+0000","last_fullsized":"2024-09-11T01:20:29.998616+0000","mapping_epoch":35,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13193216,"num_objects":3221,"num_object_clones":0,"num_object_copies":6442,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3221,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15370,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'16884","reported_seq":17022,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:29.951901+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:20:29.951901+0000","last_peered":"2024-09-11T01:20:29.951901+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:20:29.951901+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13348888,"num_objects":3260,"num_object_clones":0,"num_object_copies":6520,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3260,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15185,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-636}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'16911","reported_seq":16989,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:29.942194+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:20:29.942194+0000","last_peered":"2024-09-11T01:20:29.942194+0000","last_clean":"2024-09-11T01:20:29.942194+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:20:29.942194+0000","last_undegraded":"2024-09-11T01:20:29.942194+0000","last_fullsized":"2024-09-11T01:20:29.942194+0000","mapping_epoch":35,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13201408,"num_objects":3223,"num_object_clones":0,"num_object_copies":6446,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3223,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16911,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'16980","reported_seq":17117,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:32.387127+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:20:32.387127+0000","last_peered":"2024-09-11T01:20:32.387127+0000","last_clean":"2024-09-11T01:20:32.387127+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:20:32.387127+0000","last_undegraded":"2024-09-11T01:20:32.387127+0000","last_fullsized":"2024-09-11T01:20:32.387127+0000","mapping_epoch":34,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12406784,"num_objects":3029,"num_object_clones":0,"num_object_copies":6058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13985,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'17058","reported_seq":17238,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.443104+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:20:32.443104+0000","last_peered":"2024-09-11T01:20:32.443104+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:20:32.443104+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12980224,"num_objects":3169,"num_object_clones":0,"num_object_copies":6338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13399,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-782}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'16823","reported_seq":16963,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.493007+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:20:32.493007+0000","last_peered":"2024-09-11T01:20:32.493007+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:20:32.493007+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12726272,"num_objects":3107,"num_object_clones":0,"num_object_copies":6214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13680,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-879}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'17134","reported_seq":17273,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:32.498104+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:20:32.498104+0000","last_peered":"2024-09-11T01:20:32.498104+0000","last_clean":"2024-09-11T01:20:32.498104+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:20:32.498104+0000","last_undegraded":"2024-09-11T01:20:32.498104+0000","last_fullsized":"2024-09-11T01:20:32.498104+0000","mapping_epoch":34,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12779520,"num_objects":3120,"num_object_clones":0,"num_object_copies":6240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14010,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003131+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003131+0000","last_peered":"2024-09-11T01:20:22.003131+0000","last_clean":"2024-09-11T01:20:22.003131+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003131+0000","last_undegraded":"2024-09-11T01:20:22.003131+0000","last_fullsized":"2024-09-11T01:20:22.003131+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'16868","reported_seq":24886,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:32.023072+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:20:32.023072+0000","last_peered":"2024-09-11T01:20:32.023072+0000","last_clean":"2024-09-11T01:20:32.023072+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:20:32.023072+0000","last_undegraded":"2024-09-11T01:20:32.023072+0000","last_fullsized":"2024-09-11T01:20:32.023072+0000","mapping_epoch":48,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12718080,"num_objects":3105,"num_object_clones":0,"num_object_copies":6210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14839,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'16826","reported_seq":16964,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.472502+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:20:32.472502+0000","last_peered":"2024-09-11T01:20:32.472502+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:20:32.472502+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12460032,"num_objects":3042,"num_object_clones":0,"num_object_copies":6084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13413,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-942}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'16974","reported_seq":17103,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:29.996664+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:20:29.996664+0000","last_peered":"2024-09-11T01:20:29.996664+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:20:29.996664+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13242368,"num_objects":3233,"num_object_clones":0,"num_object_copies":6466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14791,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-791}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003225+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003225+0000","last_peered":"2024-09-11T01:20:22.003225+0000","last_clean":"2024-09-11T01:20:22.003225+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003225+0000","last_undegraded":"2024-09-11T01:20:22.003225+0000","last_fullsized":"2024-09-11T01:20:22.003225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'16658","reported_seq":24649,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:29.981115+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:20:29.981115+0000","last_peered":"2024-09-11T01:20:29.981115+0000","last_clean":"2024-09-11T01:20:29.981115+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:20:29.981115+0000","last_undegraded":"2024-09-11T01:20:29.981115+0000","last_fullsized":"2024-09-11T01:20:29.981115+0000","mapping_epoch":44,"log_start":"43'6600","ondisk_log_start":"43'6600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12853248,"num_objects":3138,"num_object_clones":0,"num_object_copies":6276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3138,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14443,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'17138","reported_seq":17276,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.481136+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:20:32.481136+0000","last_peered":"2024-09-11T01:20:32.481136+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:20:32.481136+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12804096,"num_objects":3126,"num_object_clones":0,"num_object_copies":6252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13875,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-824}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007532+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007532+0000","last_peered":"2024-09-11T01:20:22.007532+0000","last_clean":"2024-09-11T01:20:22.007532+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007532+0000","last_undegraded":"2024-09-11T01:20:22.007532+0000","last_fullsized":"2024-09-11T01:20:22.007532+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'16784","reported_seq":24825,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:31.940390+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:20:31.940390+0000","last_peered":"2024-09-11T01:20:31.940390+0000","last_clean":"2024-09-11T01:20:31.940390+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:20:31.940390+0000","last_undegraded":"2024-09-11T01:20:31.940390+0000","last_fullsized":"2024-09-11T01:20:31.940390+0000","mapping_epoch":46,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12697600,"num_objects":3100,"num_object_clones":0,"num_object_copies":6200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13439,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'17085","reported_seq":17691,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.436830+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:20:32.436830+0000","last_peered":"2024-09-11T01:20:32.436830+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:20:32.436830+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12660736,"num_objects":3091,"num_object_clones":0,"num_object_copies":6182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3811,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3091,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13694,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-720}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007560+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007560+0000","last_peered":"2024-09-11T01:20:22.007560+0000","last_clean":"2024-09-11T01:20:22.007560+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007560+0000","last_undegraded":"2024-09-11T01:20:22.007560+0000","last_fullsized":"2024-09-11T01:20:22.007560+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008171+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008171+0000","last_peered":"2024-09-11T01:20:22.008171+0000","last_clean":"2024-09-11T01:20:22.008171+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008171+0000","last_undegraded":"2024-09-11T01:20:22.008171+0000","last_fullsized":"2024-09-11T01:20:22.008171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'16897","reported_seq":17041,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.504892+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:20:32.504892+0000","last_peered":"2024-09-11T01:20:32.504892+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:20:32.504892+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12513897,"num_objects":3056,"num_object_clones":0,"num_object_copies":6112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3056,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13967,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-861}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":205329025,"num_objects":50131,"num_object_clones":0,"num_object_copies":100262,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35604,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50131,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":232156,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12035,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":605454336,"data_stored":605390528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160906,"ondisk_log_size":160906,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182508,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":628168,"kb_used_data":213368,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93743672,"statfs":{"total":96636764160,"available":95993520128,"internally_reserved":0,"allocated":218488832,"data_stored":218208678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":424734379},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,8,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986280,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":577180,"kb_used_data":230412,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93794660,"statfs":{"total":96636764160,"available":96045731840,"internally_reserved":0,"allocated":235941888,"data_stored":235661739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,0,4,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659225,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515572,"kb_used_data":150756,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93856268,"statfs":{"total":96636764160,"available":96108818432,"internally_reserved":0,"allocated":154374144,"data_stored":154044691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":373550767},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,5,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":217063424,"data_stored":217059945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":153497600,"data_stored":153444822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":234893312,"data_stored":234885761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:36.886 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:37.536 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:20:37.537 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:20:37.593 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2024-09-11T01:20:36.743792+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":202285716,"num_objects":49390,"num_object_clones":0,"num_object_copies":98780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35604,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49390,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":232901,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12036,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160952,"ondisk_log_size":160952,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1720468,"kb_used_data":589924,"kb_used_omap":12,"kb_used_meta":1130483,"kb_avail":281395052,"statfs":{"total":289910292480,"available":288148533248,"internally_reserved":0,"allocated":604082176,"data_stored":603192420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1157614596},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,1,8,8,15,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22073344,"num_objects":-5389,"num_object_clones":0,"num_object_copies":-10778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5389,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006632"},"pg_stats":[{"pgid":"2.f","version":"49'17155","reported_seq":17242,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.417165+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:20:32.417165+0000","last_peered":"2024-09-11T01:20:32.417165+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:20:32.417165+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12742656,"num_objects":3111,"num_object_clones":0,"num_object_copies":6222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-961}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'17283","reported_seq":17394,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:35.043091+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:20:35.043091+0000","last_peered":"2024-09-11T01:20:35.043091+0000","last_clean":"2024-09-11T01:20:35.043091+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:20:35.043091+0000","last_undegraded":"2024-09-11T01:20:35.043091+0000","last_fullsized":"2024-09-11T01:20:35.043091+0000","mapping_epoch":35,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12570624,"num_objects":3069,"num_object_clones":0,"num_object_copies":6138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15522,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'17040","reported_seq":17177,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:35.017385+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:20:35.017385+0000","last_peered":"2024-09-11T01:20:35.017385+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:20:35.017385+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12709912,"num_objects":3104,"num_object_clones":0,"num_object_copies":6208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3104,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15341,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-792}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'17061","reported_seq":17140,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:34.981512+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:20:34.981512+0000","last_peered":"2024-09-11T01:20:34.981512+0000","last_clean":"2024-09-11T01:20:34.981512+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:20:34.981512+0000","last_undegraded":"2024-09-11T01:20:34.981512+0000","last_fullsized":"2024-09-11T01:20:34.981512+0000","mapping_epoch":35,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12587008,"num_objects":3073,"num_object_clones":0,"num_object_copies":6146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17061,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'16980","reported_seq":17117,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:32.387127+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:20:32.387127+0000","last_peered":"2024-09-11T01:20:32.387127+0000","last_clean":"2024-09-11T01:20:32.387127+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:20:32.387127+0000","last_undegraded":"2024-09-11T01:20:32.387127+0000","last_fullsized":"2024-09-11T01:20:32.387127+0000","mapping_epoch":34,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12406784,"num_objects":3029,"num_object_clones":0,"num_object_copies":6058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13985,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'17058","reported_seq":17238,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.443104+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:20:32.443104+0000","last_peered":"2024-09-11T01:20:32.443104+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:20:32.443104+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12980224,"num_objects":3169,"num_object_clones":0,"num_object_copies":6338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13399,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-782}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'16823","reported_seq":16963,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.493007+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:20:32.493007+0000","last_peered":"2024-09-11T01:20:32.493007+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:20:32.493007+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12726272,"num_objects":3107,"num_object_clones":0,"num_object_copies":6214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13680,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-879}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'17134","reported_seq":17273,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:32.498104+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:20:32.498104+0000","last_peered":"2024-09-11T01:20:32.498104+0000","last_clean":"2024-09-11T01:20:32.498104+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:20:32.498104+0000","last_undegraded":"2024-09-11T01:20:32.498104+0000","last_fullsized":"2024-09-11T01:20:32.498104+0000","mapping_epoch":34,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12779520,"num_objects":3120,"num_object_clones":0,"num_object_copies":6240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14010,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003131+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003131+0000","last_peered":"2024-09-11T01:20:22.003131+0000","last_clean":"2024-09-11T01:20:22.003131+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003131+0000","last_undegraded":"2024-09-11T01:20:22.003131+0000","last_fullsized":"2024-09-11T01:20:22.003131+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'16868","reported_seq":24886,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:32.023072+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:20:32.023072+0000","last_peered":"2024-09-11T01:20:32.023072+0000","last_clean":"2024-09-11T01:20:32.023072+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:20:32.023072+0000","last_undegraded":"2024-09-11T01:20:32.023072+0000","last_fullsized":"2024-09-11T01:20:32.023072+0000","mapping_epoch":48,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12718080,"num_objects":3105,"num_object_clones":0,"num_object_copies":6210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14839,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'16826","reported_seq":16964,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.472502+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:20:32.472502+0000","last_peered":"2024-09-11T01:20:32.472502+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:20:32.472502+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12460032,"num_objects":3042,"num_object_clones":0,"num_object_copies":6084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13413,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-942}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'17122","reported_seq":17249,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:35.016408+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:20:35.016408+0000","last_peered":"2024-09-11T01:20:35.016408+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:20:35.016408+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12636160,"num_objects":3085,"num_object_clones":0,"num_object_copies":6170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14939,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-939}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003225+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003225+0000","last_peered":"2024-09-11T01:20:22.003225+0000","last_clean":"2024-09-11T01:20:22.003225+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003225+0000","last_undegraded":"2024-09-11T01:20:22.003225+0000","last_fullsized":"2024-09-11T01:20:22.003225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'16795","reported_seq":24786,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:34.934963+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:20:34.934963+0000","last_peered":"2024-09-11T01:20:34.934963+0000","last_clean":"2024-09-11T01:20:34.934963+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:20:34.934963+0000","last_undegraded":"2024-09-11T01:20:34.934963+0000","last_fullsized":"2024-09-11T01:20:34.934963+0000","mapping_epoch":44,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12292096,"num_objects":3001,"num_object_clones":0,"num_object_copies":6002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14580,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'17138","reported_seq":17276,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.481136+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:20:32.481136+0000","last_peered":"2024-09-11T01:20:32.481136+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:20:32.481136+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12804096,"num_objects":3126,"num_object_clones":0,"num_object_copies":6252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13875,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-824}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007532+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007532+0000","last_peered":"2024-09-11T01:20:22.007532+0000","last_clean":"2024-09-11T01:20:22.007532+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007532+0000","last_undegraded":"2024-09-11T01:20:22.007532+0000","last_fullsized":"2024-09-11T01:20:22.007532+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'16784","reported_seq":24825,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:31.940390+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:20:31.940390+0000","last_peered":"2024-09-11T01:20:31.940390+0000","last_clean":"2024-09-11T01:20:31.940390+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:20:31.940390+0000","last_undegraded":"2024-09-11T01:20:31.940390+0000","last_fullsized":"2024-09-11T01:20:31.940390+0000","mapping_epoch":46,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12697600,"num_objects":3100,"num_object_clones":0,"num_object_copies":6200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13439,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'17085","reported_seq":17691,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.436830+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:20:32.436830+0000","last_peered":"2024-09-11T01:20:32.436830+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:20:32.436830+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12660736,"num_objects":3091,"num_object_clones":0,"num_object_copies":6182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3811,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3091,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13694,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-720}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007560+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007560+0000","last_peered":"2024-09-11T01:20:22.007560+0000","last_clean":"2024-09-11T01:20:22.007560+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007560+0000","last_undegraded":"2024-09-11T01:20:22.007560+0000","last_fullsized":"2024-09-11T01:20:22.007560+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008171+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008171+0000","last_peered":"2024-09-11T01:20:22.008171+0000","last_clean":"2024-09-11T01:20:22.008171+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008171+0000","last_undegraded":"2024-09-11T01:20:22.008171+0000","last_fullsized":"2024-09-11T01:20:22.008171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'16897","reported_seq":17041,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.504892+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:20:32.504892+0000","last_peered":"2024-09-11T01:20:32.504892+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:20:32.504892+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12513897,"num_objects":3056,"num_object_clones":0,"num_object_copies":6112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3056,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13967,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-861}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":202285697,"num_objects":49388,"num_object_clones":0,"num_object_copies":98776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35604,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49388,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":232899,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12035,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":600768512,"data_stored":600704704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160949,"ondisk_log_size":160949,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182508,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":628168,"kb_used_data":213368,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93743672,"statfs":{"total":96636764160,"available":95993520128,"internally_reserved":0,"allocated":218488832,"data_stored":218208678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":424734379},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,8,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986281,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576728,"kb_used_data":225800,"kb_used_omap":4,"kb_used_meta":350907,"kb_avail":93795112,"statfs":{"total":96636764160,"available":96046194688,"internally_reserved":0,"allocated":231219200,"data_stored":230939051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":359329450},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,2,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659225,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515572,"kb_used_data":150756,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93856268,"statfs":{"total":96636764160,"available":96108818432,"internally_reserved":0,"allocated":154374144,"data_stored":154044691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":373550767},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,5,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":217063424,"data_stored":217059945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":153497600,"data_stored":153444822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":230207488,"data_stored":230199937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:37.595 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:38.463 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:20:38.464 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:20:38.521 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2024-09-11T01:20:36.743792+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":202285716,"num_objects":49390,"num_object_clones":0,"num_object_copies":98780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35604,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49390,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":232901,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12036,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160952,"ondisk_log_size":160952,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1720468,"kb_used_data":589924,"kb_used_omap":12,"kb_used_meta":1130483,"kb_avail":281395052,"statfs":{"total":289910292480,"available":288148533248,"internally_reserved":0,"allocated":604082176,"data_stored":603192420,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1157614596},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,1,8,8,15,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22073344,"num_objects":-5389,"num_object_clones":0,"num_object_copies":-10778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5389,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006632"},"pg_stats":[{"pgid":"2.f","version":"49'17155","reported_seq":17242,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.417165+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:20:32.417165+0000","last_peered":"2024-09-11T01:20:32.417165+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:20:32.417165+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12742656,"num_objects":3111,"num_object_clones":0,"num_object_copies":6222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17155,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-961}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'17283","reported_seq":17394,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:35.043091+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:20:35.043091+0000","last_peered":"2024-09-11T01:20:35.043091+0000","last_clean":"2024-09-11T01:20:35.043091+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:20:35.043091+0000","last_undegraded":"2024-09-11T01:20:35.043091+0000","last_fullsized":"2024-09-11T01:20:35.043091+0000","mapping_epoch":35,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12570624,"num_objects":3069,"num_object_clones":0,"num_object_copies":6138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15522,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'17040","reported_seq":17177,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:35.017385+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:20:35.017385+0000","last_peered":"2024-09-11T01:20:35.017385+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:20:35.017385+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12709912,"num_objects":3104,"num_object_clones":0,"num_object_copies":6208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3104,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15341,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-792}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'17061","reported_seq":17140,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:34.981512+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:20:34.981512+0000","last_peered":"2024-09-11T01:20:34.981512+0000","last_clean":"2024-09-11T01:20:34.981512+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:20:34.981512+0000","last_undegraded":"2024-09-11T01:20:34.981512+0000","last_fullsized":"2024-09-11T01:20:34.981512+0000","mapping_epoch":35,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12587008,"num_objects":3073,"num_object_clones":0,"num_object_copies":6146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17061,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'16980","reported_seq":17117,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:32.387127+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:20:32.387127+0000","last_peered":"2024-09-11T01:20:32.387127+0000","last_clean":"2024-09-11T01:20:32.387127+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:20:32.387127+0000","last_undegraded":"2024-09-11T01:20:32.387127+0000","last_fullsized":"2024-09-11T01:20:32.387127+0000","mapping_epoch":34,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12406784,"num_objects":3029,"num_object_clones":0,"num_object_copies":6058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3029,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13985,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'17058","reported_seq":17238,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.443104+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:20:32.443104+0000","last_peered":"2024-09-11T01:20:32.443104+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:20:32.443104+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12980224,"num_objects":3169,"num_object_clones":0,"num_object_copies":6338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13399,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-782}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'16823","reported_seq":16963,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.493007+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:20:32.493007+0000","last_peered":"2024-09-11T01:20:32.493007+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:20:32.493007+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12726272,"num_objects":3107,"num_object_clones":0,"num_object_copies":6214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3107,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13680,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-879}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'17134","reported_seq":17273,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:32.498104+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:20:32.498104+0000","last_peered":"2024-09-11T01:20:32.498104+0000","last_clean":"2024-09-11T01:20:32.498104+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:20:32.498104+0000","last_undegraded":"2024-09-11T01:20:32.498104+0000","last_fullsized":"2024-09-11T01:20:32.498104+0000","mapping_epoch":34,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12779520,"num_objects":3120,"num_object_clones":0,"num_object_copies":6240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3120,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14010,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003131+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003131+0000","last_peered":"2024-09-11T01:20:22.003131+0000","last_clean":"2024-09-11T01:20:22.003131+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003131+0000","last_undegraded":"2024-09-11T01:20:22.003131+0000","last_fullsized":"2024-09-11T01:20:22.003131+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'16868","reported_seq":24886,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:32.023072+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:20:32.023072+0000","last_peered":"2024-09-11T01:20:32.023072+0000","last_clean":"2024-09-11T01:20:32.023072+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:20:32.023072+0000","last_undegraded":"2024-09-11T01:20:32.023072+0000","last_fullsized":"2024-09-11T01:20:32.023072+0000","mapping_epoch":48,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12718080,"num_objects":3105,"num_object_clones":0,"num_object_copies":6210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3105,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14839,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'16826","reported_seq":16964,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.472502+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:20:32.472502+0000","last_peered":"2024-09-11T01:20:32.472502+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:20:32.472502+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12460032,"num_objects":3042,"num_object_clones":0,"num_object_copies":6084,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3042,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13413,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-942}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'17122","reported_seq":17249,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:35.016408+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:20:35.016408+0000","last_peered":"2024-09-11T01:20:35.016408+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:20:35.016408+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12636160,"num_objects":3085,"num_object_clones":0,"num_object_copies":6170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14939,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-939}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003225+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003225+0000","last_peered":"2024-09-11T01:20:22.003225+0000","last_clean":"2024-09-11T01:20:22.003225+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003225+0000","last_undegraded":"2024-09-11T01:20:22.003225+0000","last_fullsized":"2024-09-11T01:20:22.003225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'16795","reported_seq":24786,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:34.934963+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:20:34.934963+0000","last_peered":"2024-09-11T01:20:34.934963+0000","last_clean":"2024-09-11T01:20:34.934963+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:20:34.934963+0000","last_undegraded":"2024-09-11T01:20:34.934963+0000","last_fullsized":"2024-09-11T01:20:34.934963+0000","mapping_epoch":44,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12292096,"num_objects":3001,"num_object_clones":0,"num_object_copies":6002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14580,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'17138","reported_seq":17276,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.481136+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:20:32.481136+0000","last_peered":"2024-09-11T01:20:32.481136+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:20:32.481136+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12804096,"num_objects":3126,"num_object_clones":0,"num_object_copies":6252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3126,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13875,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-824}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007532+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007532+0000","last_peered":"2024-09-11T01:20:22.007532+0000","last_clean":"2024-09-11T01:20:22.007532+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007532+0000","last_undegraded":"2024-09-11T01:20:22.007532+0000","last_fullsized":"2024-09-11T01:20:22.007532+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'16784","reported_seq":24825,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:31.940390+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:20:31.940390+0000","last_peered":"2024-09-11T01:20:31.940390+0000","last_clean":"2024-09-11T01:20:31.940390+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:20:31.940390+0000","last_undegraded":"2024-09-11T01:20:31.940390+0000","last_fullsized":"2024-09-11T01:20:31.940390+0000","mapping_epoch":46,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12697600,"num_objects":3100,"num_object_clones":0,"num_object_copies":6200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13439,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'17085","reported_seq":17691,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.436830+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:20:32.436830+0000","last_peered":"2024-09-11T01:20:32.436830+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:20:32.436830+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12660736,"num_objects":3091,"num_object_clones":0,"num_object_copies":6182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3811,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3091,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13694,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-720}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007560+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007560+0000","last_peered":"2024-09-11T01:20:22.007560+0000","last_clean":"2024-09-11T01:20:22.007560+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007560+0000","last_undegraded":"2024-09-11T01:20:22.007560+0000","last_fullsized":"2024-09-11T01:20:22.007560+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008171+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008171+0000","last_peered":"2024-09-11T01:20:22.008171+0000","last_clean":"2024-09-11T01:20:22.008171+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008171+0000","last_undegraded":"2024-09-11T01:20:22.008171+0000","last_fullsized":"2024-09-11T01:20:22.008171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'16897","reported_seq":17041,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:32.504892+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:20:32.504892+0000","last_peered":"2024-09-11T01:20:32.504892+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:20:32.504892+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'6800","ondisk_log_start":"43'6800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12513897,"num_objects":3056,"num_object_clones":0,"num_object_copies":6112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3056,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13967,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-861}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":202285697,"num_objects":49388,"num_object_clones":0,"num_object_copies":98776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35604,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49388,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":232899,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12035,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":600768512,"data_stored":600704704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160949,"ondisk_log_size":160949,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182508,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":628168,"kb_used_data":213368,"kb_used_omap":4,"kb_used_meta":414779,"kb_avail":93743672,"statfs":{"total":96636764160,"available":95993520128,"internally_reserved":0,"allocated":218488832,"data_stored":218208678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":424734379},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,8,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986281,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576728,"kb_used_data":225800,"kb_used_omap":4,"kb_used_meta":350907,"kb_avail":93795112,"statfs":{"total":96636764160,"available":96046194688,"internally_reserved":0,"allocated":231219200,"data_stored":230939051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":359329450},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,2,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659225,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515572,"kb_used_data":150756,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93856268,"statfs":{"total":96636764160,"available":96108818432,"internally_reserved":0,"allocated":154374144,"data_stored":154044691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":373550767},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,3,5,5],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":217063424,"data_stored":217059945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":153497600,"data_stored":153444822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":230207488,"data_stored":230199937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:38.522 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:20:38.522 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'17155", 'reported_seq': 17242, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:32.417165+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:20:32.417165+0000', 'last_peered': '2024-09-11T01:20:32.417165+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:20:32.417165+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "43'7100", 'ondisk_log_start': "43'7100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12742656, 'num_objects': 3111, 'num_object_clones': 0, 'num_object_copies': 6222, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3111, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17155, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -961}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:38.523 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:20:38.523 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'17040", 'reported_seq': 17177, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:35.017385+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:20:35.017385+0000', 'last_peered': '2024-09-11T01:20:35.017385+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:20:35.017385+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "43'7000", 'ondisk_log_start': "43'7000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10040, 'log_dups_size': 0, 'ondisk_log_size': 10040, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12709912, 'num_objects': 3104, 'num_object_clones': 0, 'num_object_copies': 6208, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3104, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15341, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -792}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:38.523 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:20:38.523 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'17058", 'reported_seq': 17238, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:32.443104+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:20:32.443104+0000', 'last_peered': '2024-09-11T01:20:32.443104+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:20:32.443104+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "43'7000", 'ondisk_log_start': "43'7000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12980224, 'num_objects': 3169, 'num_object_clones': 0, 'num_object_copies': 6338, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3169, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13399, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -782}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:38.523 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:20:38.523 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'16823", 'reported_seq': 16963, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:32.493007+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:20:32.493007+0000', 'last_peered': '2024-09-11T01:20:32.493007+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:20:32.493007+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12726272, 'num_objects': 3107, 'num_object_clones': 0, 'num_object_copies': 6214, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3107, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13680, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -879}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:38.523 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:20:38.523 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'16826", 'reported_seq': 16964, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:32.472502+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:20:32.472502+0000', 'last_peered': '2024-09-11T01:20:32.472502+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:20:32.472502+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12460032, 'num_objects': 3042, 'num_object_clones': 0, 'num_object_copies': 6084, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3042, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13413, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -942}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:38.523 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:20:38.523 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'17122", 'reported_seq': 17249, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:35.016408+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:20:35.016408+0000', 'last_peered': '2024-09-11T01:20:35.016408+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:20:35.016408+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "43'7100", 'ondisk_log_start': "43'7100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10022, 'log_dups_size': 0, 'ondisk_log_size': 10022, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12636160, 'num_objects': 3085, 'num_object_clones': 0, 'num_object_copies': 6170, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3085, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14939, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -939}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:38.524 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:20:38.524 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'17138", 'reported_seq': 17276, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:32.481136+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:20:32.481136+0000', 'last_peered': '2024-09-11T01:20:32.481136+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:20:32.481136+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "43'7100", 'ondisk_log_start': "43'7100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12804096, 'num_objects': 3126, 'num_object_clones': 0, 'num_object_copies': 6252, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3126, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13875, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -824}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:38.524 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:20:38.524 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'17085", 'reported_seq': 17691, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:32.436830+0000', 'last_change': '2024-09-11T01:20:20.985862+0000', 'last_active': '2024-09-11T01:20:32.436830+0000', 'last_peered': '2024-09-11T01:20:32.436830+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:20:32.436830+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "43'7000", 'ondisk_log_start': "43'7000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12660736, 'num_objects': 3091, 'num_object_clones': 0, 'num_object_copies': 6182, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3811, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3091, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13694, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 230, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -720}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:38.524 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:20:38.524 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'16897", 'reported_seq': 17041, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:32.504892+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:20:32.504892+0000', 'last_peered': '2024-09-11T01:20:32.504892+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:20:32.504892+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "43'6800", 'ondisk_log_start': "43'6800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10097, 'log_dups_size': 0, 'ondisk_log_size': 10097, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12513897, 'num_objects': 3056, 'num_object_clones': 0, 'num_object_copies': 6112, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 3056, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13967, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -861}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:38.524 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:20:39.467 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:20:39.468 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35523/95550 objects degraded (37.177%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":575,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":47775,"data_bytes":195670676,"bytes_used":1730048000,"bytes_avail":288180244480,"bytes_total":289910292480,"degraded_objects":35523,"degraded_total":95550,"degraded_ratio":0.37177394034536898,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":245,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":528},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:20:39.538 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:20:39.539 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:40.362 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:20:40.365 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:20:40.425 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":330,"stamp":"2024-09-11T01:20:38.744238+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":195670676,"num_objects":47775,"num_object_clones":0,"num_object_copies":95550,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35523,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47775,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":234516,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12117,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160767,"ondisk_log_size":160767,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1689500,"kb_used_data":575596,"kb_used_omap":12,"kb_used_meta":1113843,"kb_avail":281426020,"statfs":{"total":289910292480,"available":288180244480,"internally_reserved":0,"allocated":589410304,"data_stored":588520548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1140575236},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,6,2,3,6,7,9,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26001408,"num_objects":-6348,"num_object_clones":0,"num_object_copies":-12696,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-245,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6348,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6348,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":245,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006645"},"pg_stats":[{"pgid":"2.f","version":"49'17302","reported_seq":17385,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:37.487208+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:20:37.487208+0000","last_peered":"2024-09-11T01:20:37.487208+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:20:37.487208+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12140544,"num_objects":2964,"num_object_clones":0,"num_object_copies":5928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2964,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17302,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1108}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'17283","reported_seq":17394,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:35.043091+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:20:35.043091+0000","last_peered":"2024-09-11T01:20:35.043091+0000","last_clean":"2024-09-11T01:20:35.043091+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:20:35.043091+0000","last_undegraded":"2024-09-11T01:20:35.043091+0000","last_fullsized":"2024-09-11T01:20:35.043091+0000","mapping_epoch":35,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12570624,"num_objects":3069,"num_object_clones":0,"num_object_copies":6138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3069,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15522,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'17040","reported_seq":17177,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:35.017385+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:20:35.017385+0000","last_peered":"2024-09-11T01:20:35.017385+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:20:35.017385+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12709912,"num_objects":3104,"num_object_clones":0,"num_object_copies":6208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3104,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15341,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-792}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'17061","reported_seq":17140,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:34.981512+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:20:34.981512+0000","last_peered":"2024-09-11T01:20:34.981512+0000","last_clean":"2024-09-11T01:20:34.981512+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:20:34.981512+0000","last_undegraded":"2024-09-11T01:20:34.981512+0000","last_fullsized":"2024-09-11T01:20:34.981512+0000","mapping_epoch":35,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12587008,"num_objects":3073,"num_object_clones":0,"num_object_copies":6146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3073,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17061,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'17108","reported_seq":17245,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:37.540562+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:20:37.540562+0000","last_peered":"2024-09-11T01:20:37.540562+0000","last_clean":"2024-09-11T01:20:37.540562+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:20:37.540562+0000","last_undegraded":"2024-09-11T01:20:37.540562+0000","last_fullsized":"2024-09-11T01:20:37.540562+0000","mapping_epoch":34,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11882496,"num_objects":2901,"num_object_clones":0,"num_object_copies":5802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14113,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'17194","reported_seq":17374,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:37.461930+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:20:37.461930+0000","last_peered":"2024-09-11T01:20:37.461930+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:20:37.461930+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12423168,"num_objects":3033,"num_object_clones":0,"num_object_copies":6066,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3033,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13535,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-918}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'16968","reported_seq":17108,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:37.441100+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:20:37.441100+0000","last_peered":"2024-09-11T01:20:37.441100+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:20:37.441100+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12132352,"num_objects":2962,"num_object_clones":0,"num_object_copies":5924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2962,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13825,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1024}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'17281","reported_seq":17421,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:37.515233+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:20:37.515233+0000","last_peered":"2024-09-11T01:20:37.515233+0000","last_clean":"2024-09-11T01:20:37.515233+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:20:37.515233+0000","last_undegraded":"2024-09-11T01:20:37.515233+0000","last_fullsized":"2024-09-11T01:20:37.515233+0000","mapping_epoch":34,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12177408,"num_objects":2973,"num_object_clones":0,"num_object_copies":5946,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2973,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14157,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003131+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003131+0000","last_peered":"2024-09-11T01:20:22.003131+0000","last_clean":"2024-09-11T01:20:22.003131+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003131+0000","last_undegraded":"2024-09-11T01:20:22.003131+0000","last_fullsized":"2024-09-11T01:20:22.003131+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'17007","reported_seq":25025,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:37.075528+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:20:37.075528+0000","last_peered":"2024-09-11T01:20:37.075528+0000","last_clean":"2024-09-11T01:20:37.075528+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:20:37.075528+0000","last_undegraded":"2024-09-11T01:20:37.075528+0000","last_fullsized":"2024-09-11T01:20:37.075528+0000","mapping_epoch":48,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12148736,"num_objects":2966,"num_object_clones":0,"num_object_copies":5932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14978,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'16964","reported_seq":17102,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:37.424920+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:20:37.424920+0000","last_peered":"2024-09-11T01:20:37.424920+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:20:37.424920+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11894784,"num_objects":2904,"num_object_clones":0,"num_object_copies":5808,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2904,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13551,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1080}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'17122","reported_seq":17249,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:35.016408+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:20:35.016408+0000","last_peered":"2024-09-11T01:20:35.016408+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:20:35.016408+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12636160,"num_objects":3085,"num_object_clones":0,"num_object_copies":6170,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3085,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14939,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-939}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003225+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003225+0000","last_peered":"2024-09-11T01:20:22.003225+0000","last_clean":"2024-09-11T01:20:22.003225+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003225+0000","last_undegraded":"2024-09-11T01:20:22.003225+0000","last_fullsized":"2024-09-11T01:20:22.003225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'16795","reported_seq":24786,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:34.934963+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:20:34.934963+0000","last_peered":"2024-09-11T01:20:34.934963+0000","last_clean":"2024-09-11T01:20:34.934963+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:20:34.934963+0000","last_undegraded":"2024-09-11T01:20:34.934963+0000","last_fullsized":"2024-09-11T01:20:34.934963+0000","mapping_epoch":44,"log_start":"43'6700","ondisk_log_start":"43'6700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12292096,"num_objects":3001,"num_object_clones":0,"num_object_copies":6002,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3001,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14580,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'17313","reported_seq":17451,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:37.492826+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:20:37.492826+0000","last_peered":"2024-09-11T01:20:37.492826+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:20:37.492826+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12087296,"num_objects":2951,"num_object_clones":0,"num_object_copies":5902,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2951,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14050,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-999}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007532+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007532+0000","last_peered":"2024-09-11T01:20:22.007532+0000","last_clean":"2024-09-11T01:20:22.007532+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007532+0000","last_undegraded":"2024-09-11T01:20:22.007532+0000","last_fullsized":"2024-09-11T01:20:22.007532+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'16951","reported_seq":24992,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:37.048051+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:20:37.048051+0000","last_peered":"2024-09-11T01:20:37.048051+0000","last_clean":"2024-09-11T01:20:37.048051+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:20:37.048051+0000","last_undegraded":"2024-09-11T01:20:37.048051+0000","last_fullsized":"2024-09-11T01:20:37.048051+0000","mapping_epoch":46,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12013568,"num_objects":2933,"num_object_clones":0,"num_object_copies":5866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2933,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13606,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'17234","reported_seq":18001,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:37.485027+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:20:37.485027+0000","last_peered":"2024-09-11T01:20:37.485027+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:20:37.485027+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12050432,"num_objects":2942,"num_object_clones":0,"num_object_copies":5884,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3730,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2942,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13843,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":311,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-788}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007560+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007560+0000","last_peered":"2024-09-11T01:20:22.007560+0000","last_clean":"2024-09-11T01:20:22.007560+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007560+0000","last_undegraded":"2024-09-11T01:20:22.007560+0000","last_fullsized":"2024-09-11T01:20:22.007560+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008171+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008171+0000","last_peered":"2024-09-11T01:20:22.008171+0000","last_clean":"2024-09-11T01:20:22.008171+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008171+0000","last_undegraded":"2024-09-11T01:20:22.008171+0000","last_fullsized":"2024-09-11T01:20:22.008171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'17041","reported_seq":17185,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:37.481448+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:20:37.481448+0000","last_peered":"2024-09-11T01:20:37.481448+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:20:37.481448+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11924073,"num_objects":2912,"num_object_clones":0,"num_object_copies":5824,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2912,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14111,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1005}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":195670657,"num_objects":47773,"num_object_clones":0,"num_object_copies":95546,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35523,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47773,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":234514,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12116,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":586145792,"data_stored":586081984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160764,"ondisk_log_size":160764,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182509,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":612664,"kb_used_data":206184,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93759176,"statfs":{"total":96636764160,"available":96009396224,"internally_reserved":0,"allocated":211132416,"data_stored":210852262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":416214699},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,2,1,1,2,2,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986281,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":576728,"kb_used_data":225800,"kb_used_omap":4,"kb_used_meta":350907,"kb_avail":93795112,"statfs":{"total":96636764160,"available":96046194688,"internally_reserved":0,"allocated":231219200,"data_stored":230939051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":359329450},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,0,2,1,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659226,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":500108,"kb_used_data":143612,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93871732,"statfs":{"total":96636764160,"available":96124653568,"internally_reserved":0,"allocated":147058688,"data_stored":146729235,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":365031087},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,3,3,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":209780736,"data_stored":209777257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":146157568,"data_stored":146104790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":230207488,"data_stored":230199937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:43.428 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:44.289 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:20:44.291 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:20:44.396 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-09-11T01:20:42.745139+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":186204820,"num_objects":45464,"num_object_clones":0,"num_object_copies":90928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35432,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45464,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":236827,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12208,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160878,"ondisk_log_size":160878,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1685176,"kb_used_data":557368,"kb_used_omap":12,"kb_used_meta":1127731,"kb_avail":281430344,"statfs":{"total":289910292480,"available":288184672256,"internally_reserved":0,"allocated":570744832,"data_stored":569855076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1154796548},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,4,7,8,9,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25894912,"num_objects":-6322,"num_object_clones":0,"num_object_copies":-12644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-250,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6322,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":250,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006631"},"pg_stats":[{"pgid":"2.f","version":"49'17453","reported_seq":17537,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.582069+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:20:42.582069+0000","last_peered":"2024-09-11T01:20:42.582069+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:20:42.582069+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11522048,"num_objects":2813,"num_object_clones":0,"num_object_copies":5626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17453,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'17419","reported_seq":17530,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:40.026399+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:20:40.026399+0000","last_peered":"2024-09-11T01:20:40.026399+0000","last_clean":"2024-09-11T01:20:40.026399+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:20:40.026399+0000","last_undegraded":"2024-09-11T01:20:40.026399+0000","last_fullsized":"2024-09-11T01:20:40.026399+0000","mapping_epoch":35,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12013568,"num_objects":2933,"num_object_clones":0,"num_object_copies":5866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2933,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15658,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'17186","reported_seq":17324,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:39.988755+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:20:39.988755+0000","last_peered":"2024-09-11T01:20:39.988755+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:20:39.988755+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12111896,"num_objects":2958,"num_object_clones":0,"num_object_copies":5916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2958,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15487,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-938}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'17207","reported_seq":17286,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:39.919763+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:20:39.919763+0000","last_peered":"2024-09-11T01:20:39.919763+0000","last_clean":"2024-09-11T01:20:39.919763+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:20:39.919763+0000","last_undegraded":"2024-09-11T01:20:39.919763+0000","last_fullsized":"2024-09-11T01:20:39.919763+0000","mapping_epoch":35,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11988992,"num_objects":2927,"num_object_clones":0,"num_object_copies":5854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17207,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'17263","reported_seq":17400,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:42.456320+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:20:42.456320+0000","last_peered":"2024-09-11T01:20:42.456320+0000","last_clean":"2024-09-11T01:20:42.456320+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:20:42.456320+0000","last_undegraded":"2024-09-11T01:20:42.456320+0000","last_fullsized":"2024-09-11T01:20:42.456320+0000","mapping_epoch":34,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11247616,"num_objects":2746,"num_object_clones":0,"num_object_copies":5492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14268,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'17345","reported_seq":17525,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.552072+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:20:42.552072+0000","last_peered":"2024-09-11T01:20:42.552072+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:20:42.552072+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11804672,"num_objects":2882,"num_object_clones":0,"num_object_copies":5764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13686,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1069}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'17126","reported_seq":17266,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.515070+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:20:42.515070+0000","last_peered":"2024-09-11T01:20:42.515070+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:20:42.515070+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11485184,"num_objects":2804,"num_object_clones":0,"num_object_copies":5608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13983,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1182}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'17430","reported_seq":17570,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:42.484476+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:20:42.484476+0000","last_peered":"2024-09-11T01:20:42.484476+0000","last_clean":"2024-09-11T01:20:42.484476+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:20:42.484476+0000","last_undegraded":"2024-09-11T01:20:42.484476+0000","last_fullsized":"2024-09-11T01:20:42.484476+0000","mapping_epoch":34,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11567104,"num_objects":2824,"num_object_clones":0,"num_object_copies":5648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14306,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003131+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003131+0000","last_peered":"2024-09-11T01:20:22.003131+0000","last_clean":"2024-09-11T01:20:22.003131+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003131+0000","last_undegraded":"2024-09-11T01:20:22.003131+0000","last_fullsized":"2024-09-11T01:20:22.003131+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'17155","reported_seq":25169,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:42.048631+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:20:42.048631+0000","last_peered":"2024-09-11T01:20:42.048631+0000","last_clean":"2024-09-11T01:20:42.048631+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:20:42.048631+0000","last_undegraded":"2024-09-11T01:20:42.048631+0000","last_fullsized":"2024-09-11T01:20:42.048631+0000","mapping_epoch":48,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11542528,"num_objects":2818,"num_object_clones":0,"num_object_copies":5636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2818,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15126,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'17099","reported_seq":17237,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.569530+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:20:42.569530+0000","last_peered":"2024-09-11T01:20:42.569530+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:20:42.569530+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11341824,"num_objects":2769,"num_object_clones":0,"num_object_copies":5538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13686,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1215}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'17255","reported_seq":17382,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:39.952516+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:20:39.952516+0000","last_peered":"2024-09-11T01:20:39.952516+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:20:39.952516+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12091392,"num_objects":2952,"num_object_clones":0,"num_object_copies":5904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15072,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1072}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003225+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003225+0000","last_peered":"2024-09-11T01:20:22.003225+0000","last_clean":"2024-09-11T01:20:22.003225+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003225+0000","last_undegraded":"2024-09-11T01:20:22.003225+0000","last_fullsized":"2024-09-11T01:20:22.003225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'16954","reported_seq":24945,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:40.034838+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:20:40.034838+0000","last_peered":"2024-09-11T01:20:40.034838+0000","last_clean":"2024-09-11T01:20:40.034838+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:20:40.034838+0000","last_undegraded":"2024-09-11T01:20:40.034838+0000","last_fullsized":"2024-09-11T01:20:40.034838+0000","mapping_epoch":44,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11640832,"num_objects":2842,"num_object_clones":0,"num_object_copies":5684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14739,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'17447","reported_seq":17587,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.542570+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:20:42.542570+0000","last_peered":"2024-09-11T01:20:42.542570+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:20:42.542570+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11538432,"num_objects":2817,"num_object_clones":0,"num_object_copies":5634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14184,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1133}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007532+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007532+0000","last_peered":"2024-09-11T01:20:22.007532+0000","last_clean":"2024-09-11T01:20:22.007532+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007532+0000","last_undegraded":"2024-09-11T01:20:22.007532+0000","last_fullsized":"2024-09-11T01:20:22.007532+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'17090","reported_seq":25131,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:42.068545+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:20:42.068545+0000","last_peered":"2024-09-11T01:20:42.068545+0000","last_clean":"2024-09-11T01:20:42.068545+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:20:42.068545+0000","last_undegraded":"2024-09-11T01:20:42.068545+0000","last_fullsized":"2024-09-11T01:20:42.068545+0000","mapping_epoch":46,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11444224,"num_objects":2794,"num_object_clones":0,"num_object_copies":5588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2794,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13745,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'17358","reported_seq":18308,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.590829+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:20:42.590829+0000","last_peered":"2024-09-11T01:20:42.590829+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:20:42.590829+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11542528,"num_objects":2818,"num_object_clones":0,"num_object_copies":5636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3639,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2818,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13967,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":402,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-821}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007560+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007560+0000","last_peered":"2024-09-11T01:20:22.007560+0000","last_clean":"2024-09-11T01:20:22.007560+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007560+0000","last_undegraded":"2024-09-11T01:20:22.007560+0000","last_fullsized":"2024-09-11T01:20:22.007560+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008171+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008171+0000","last_peered":"2024-09-11T01:20:22.008171+0000","last_clean":"2024-09-11T01:20:22.008171+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008171+0000","last_undegraded":"2024-09-11T01:20:22.008171+0000","last_fullsized":"2024-09-11T01:20:22.008171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'17188","reported_seq":17332,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.475825+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:20:42.475825+0000","last_peered":"2024-09-11T01:20:42.475825+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:20:42.475825+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11321961,"num_objects":2765,"num_object_clones":0,"num_object_copies":5530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2765,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14258,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1152}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":186204801,"num_objects":45462,"num_object_clones":0,"num_object_copies":90924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35432,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45462,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":236825,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12207,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":567193600,"data_stored":567129792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160875,"ondisk_log_size":160875,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182510,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610104,"kb_used_data":199460,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93761736,"statfs":{"total":96636764160,"available":96012017664,"internally_reserved":0,"allocated":204247040,"data_stored":203966886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":420474539},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,4,2,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986282,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":559616,"kb_used_data":221160,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93812224,"statfs":{"total":96636764160,"available":96063717376,"internally_reserved":0,"allocated":226467840,"data_stored":226187691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659227,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515456,"kb_used_data":136748,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93856384,"statfs":{"total":96636764160,"available":96108937216,"internally_reserved":0,"allocated":140029952,"data_stored":139700499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":387772079},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,4,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":202661888,"data_stored":202658409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139083776,"data_stored":139030998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":225447936,"data_stored":225440385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:44.398 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:45.153 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:20:45.154 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:20:45.220 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":333,"stamp":"2024-09-11T01:20:44.745614+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":186204820,"num_objects":45464,"num_object_clones":0,"num_object_copies":90928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35432,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45464,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":236827,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12208,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160878,"ondisk_log_size":160878,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1685176,"kb_used_data":557368,"kb_used_omap":12,"kb_used_meta":1127731,"kb_avail":281430344,"statfs":{"total":289910292480,"available":288184672256,"internally_reserved":0,"allocated":570744832,"data_stored":569855076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1154796548},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,2,4,7,8,9,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19124224,"num_objects":-4669,"num_object_clones":0,"num_object_copies":-9338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-172,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4669,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002690"},"pg_stats":[{"pgid":"2.f","version":"49'17453","reported_seq":17537,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.582069+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:20:42.582069+0000","last_peered":"2024-09-11T01:20:42.582069+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:20:42.582069+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11522048,"num_objects":2813,"num_object_clones":0,"num_object_copies":5626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17453,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'17419","reported_seq":17530,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:40.026399+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:20:40.026399+0000","last_peered":"2024-09-11T01:20:40.026399+0000","last_clean":"2024-09-11T01:20:40.026399+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:20:40.026399+0000","last_undegraded":"2024-09-11T01:20:40.026399+0000","last_fullsized":"2024-09-11T01:20:40.026399+0000","mapping_epoch":35,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12013568,"num_objects":2933,"num_object_clones":0,"num_object_copies":5866,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2933,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15658,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'17186","reported_seq":17324,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:39.988755+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:20:39.988755+0000","last_peered":"2024-09-11T01:20:39.988755+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:20:39.988755+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12111896,"num_objects":2958,"num_object_clones":0,"num_object_copies":5916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2958,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15487,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-938}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'17207","reported_seq":17286,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:39.919763+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:20:39.919763+0000","last_peered":"2024-09-11T01:20:39.919763+0000","last_clean":"2024-09-11T01:20:39.919763+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:20:39.919763+0000","last_undegraded":"2024-09-11T01:20:39.919763+0000","last_fullsized":"2024-09-11T01:20:39.919763+0000","mapping_epoch":35,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11988992,"num_objects":2927,"num_object_clones":0,"num_object_copies":5854,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2927,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17207,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'17263","reported_seq":17400,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:42.456320+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:20:42.456320+0000","last_peered":"2024-09-11T01:20:42.456320+0000","last_clean":"2024-09-11T01:20:42.456320+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:20:42.456320+0000","last_undegraded":"2024-09-11T01:20:42.456320+0000","last_fullsized":"2024-09-11T01:20:42.456320+0000","mapping_epoch":34,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11247616,"num_objects":2746,"num_object_clones":0,"num_object_copies":5492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14268,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'17345","reported_seq":17525,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.552072+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:20:42.552072+0000","last_peered":"2024-09-11T01:20:42.552072+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:20:42.552072+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11804672,"num_objects":2882,"num_object_clones":0,"num_object_copies":5764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13686,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1069}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'17126","reported_seq":17266,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.515070+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:20:42.515070+0000","last_peered":"2024-09-11T01:20:42.515070+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:20:42.515070+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11485184,"num_objects":2804,"num_object_clones":0,"num_object_copies":5608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13983,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1182}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'17430","reported_seq":17570,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:42.484476+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:20:42.484476+0000","last_peered":"2024-09-11T01:20:42.484476+0000","last_clean":"2024-09-11T01:20:42.484476+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:20:42.484476+0000","last_undegraded":"2024-09-11T01:20:42.484476+0000","last_fullsized":"2024-09-11T01:20:42.484476+0000","mapping_epoch":34,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11567104,"num_objects":2824,"num_object_clones":0,"num_object_copies":5648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14306,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003131+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003131+0000","last_peered":"2024-09-11T01:20:22.003131+0000","last_clean":"2024-09-11T01:20:22.003131+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003131+0000","last_undegraded":"2024-09-11T01:20:22.003131+0000","last_fullsized":"2024-09-11T01:20:22.003131+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'17155","reported_seq":25169,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:42.048631+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:20:42.048631+0000","last_peered":"2024-09-11T01:20:42.048631+0000","last_clean":"2024-09-11T01:20:42.048631+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:20:42.048631+0000","last_undegraded":"2024-09-11T01:20:42.048631+0000","last_fullsized":"2024-09-11T01:20:42.048631+0000","mapping_epoch":48,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11542528,"num_objects":2818,"num_object_clones":0,"num_object_copies":5636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2818,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15126,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'17099","reported_seq":17237,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.569530+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:20:42.569530+0000","last_peered":"2024-09-11T01:20:42.569530+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:20:42.569530+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11341824,"num_objects":2769,"num_object_clones":0,"num_object_copies":5538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13686,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1215}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'17255","reported_seq":17382,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:39.952516+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:20:39.952516+0000","last_peered":"2024-09-11T01:20:39.952516+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:20:39.952516+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":12091392,"num_objects":2952,"num_object_clones":0,"num_object_copies":5904,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2952,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15072,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1072}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003225+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003225+0000","last_peered":"2024-09-11T01:20:22.003225+0000","last_clean":"2024-09-11T01:20:22.003225+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003225+0000","last_undegraded":"2024-09-11T01:20:22.003225+0000","last_fullsized":"2024-09-11T01:20:22.003225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'16954","reported_seq":24945,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:40.034838+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:20:40.034838+0000","last_peered":"2024-09-11T01:20:40.034838+0000","last_clean":"2024-09-11T01:20:40.034838+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:20:40.034838+0000","last_undegraded":"2024-09-11T01:20:40.034838+0000","last_fullsized":"2024-09-11T01:20:40.034838+0000","mapping_epoch":44,"log_start":"43'6900","ondisk_log_start":"43'6900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11640832,"num_objects":2842,"num_object_clones":0,"num_object_copies":5684,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2842,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14739,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'17447","reported_seq":17587,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.542570+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:20:42.542570+0000","last_peered":"2024-09-11T01:20:42.542570+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:20:42.542570+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11538432,"num_objects":2817,"num_object_clones":0,"num_object_copies":5634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14184,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1133}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007532+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007532+0000","last_peered":"2024-09-11T01:20:22.007532+0000","last_clean":"2024-09-11T01:20:22.007532+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007532+0000","last_undegraded":"2024-09-11T01:20:22.007532+0000","last_fullsized":"2024-09-11T01:20:22.007532+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'17090","reported_seq":25131,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:42.068545+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:20:42.068545+0000","last_peered":"2024-09-11T01:20:42.068545+0000","last_clean":"2024-09-11T01:20:42.068545+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:20:42.068545+0000","last_undegraded":"2024-09-11T01:20:42.068545+0000","last_fullsized":"2024-09-11T01:20:42.068545+0000","mapping_epoch":46,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11444224,"num_objects":2794,"num_object_clones":0,"num_object_copies":5588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2794,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13745,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'17358","reported_seq":18308,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.590829+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:20:42.590829+0000","last_peered":"2024-09-11T01:20:42.590829+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:20:42.590829+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11542528,"num_objects":2818,"num_object_clones":0,"num_object_copies":5636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3639,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2818,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13967,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":402,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-821}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007560+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007560+0000","last_peered":"2024-09-11T01:20:22.007560+0000","last_clean":"2024-09-11T01:20:22.007560+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007560+0000","last_undegraded":"2024-09-11T01:20:22.007560+0000","last_fullsized":"2024-09-11T01:20:22.007560+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008171+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008171+0000","last_peered":"2024-09-11T01:20:22.008171+0000","last_clean":"2024-09-11T01:20:22.008171+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008171+0000","last_undegraded":"2024-09-11T01:20:22.008171+0000","last_fullsized":"2024-09-11T01:20:22.008171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'17188","reported_seq":17332,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.475825+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:20:42.475825+0000","last_peered":"2024-09-11T01:20:42.475825+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:20:42.475825+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11321961,"num_objects":2765,"num_object_clones":0,"num_object_copies":5530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2765,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14258,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1152}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":186204801,"num_objects":45462,"num_object_clones":0,"num_object_copies":90924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35432,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45462,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":236825,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12207,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":567193600,"data_stored":567129792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160875,"ondisk_log_size":160875,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182510,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610104,"kb_used_data":199460,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93761736,"statfs":{"total":96636764160,"available":96012017664,"internally_reserved":0,"allocated":204247040,"data_stored":203966886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":420474539},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,4,2,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986282,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":559616,"kb_used_data":221160,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93812224,"statfs":{"total":96636764160,"available":96063717376,"internally_reserved":0,"allocated":226467840,"data_stored":226187691,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,1,2,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659227,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515456,"kb_used_data":136748,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93856384,"statfs":{"total":96636764160,"available":96108937216,"internally_reserved":0,"allocated":140029952,"data_stored":139700499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":387772079},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,4,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":202661888,"data_stored":202658409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139083776,"data_stored":139030998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":225447936,"data_stored":225440385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:45.222 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:20:45.222 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'17453", 'reported_seq': 17537, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:42.582069+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:20:42.582069+0000', 'last_peered': '2024-09-11T01:20:42.582069+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:20:42.582069+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "43'7400", 'ondisk_log_start': "43'7400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10053, 'log_dups_size': 0, 'ondisk_log_size': 10053, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11522048, 'num_objects': 2813, 'num_object_clones': 0, 'num_object_copies': 5626, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2813, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17453, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1259}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:45.222 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:20:45.222 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'17186", 'reported_seq': 17324, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:39.988755+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:20:39.988755+0000', 'last_peered': '2024-09-11T01:20:39.988755+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:20:39.988755+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "43'7100", 'ondisk_log_start': "43'7100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12111896, 'num_objects': 2958, 'num_object_clones': 0, 'num_object_copies': 5916, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2958, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15487, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -938}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:45.223 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:20:45.223 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'17345", 'reported_seq': 17525, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:42.552072+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:20:42.552072+0000', 'last_peered': '2024-09-11T01:20:42.552072+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:20:42.552072+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "43'7300", 'ondisk_log_start': "43'7300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11804672, 'num_objects': 2882, 'num_object_clones': 0, 'num_object_copies': 5764, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2882, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13686, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1069}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:45.223 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:20:45.224 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'17126", 'reported_seq': 17266, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:42.515070+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:20:42.515070+0000', 'last_peered': '2024-09-11T01:20:42.515070+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:20:42.515070+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "43'7100", 'ondisk_log_start': "43'7100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10026, 'log_dups_size': 0, 'ondisk_log_size': 10026, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11485184, 'num_objects': 2804, 'num_object_clones': 0, 'num_object_copies': 5608, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2804, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13983, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1182}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:45.224 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:20:45.224 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'17099", 'reported_seq': 17237, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:42.569530+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:20:42.569530+0000', 'last_peered': '2024-09-11T01:20:42.569530+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:20:42.569530+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "43'7000", 'ondisk_log_start': "43'7000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11341824, 'num_objects': 2769, 'num_object_clones': 0, 'num_object_copies': 5538, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2769, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13686, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1215}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:45.225 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:20:45.225 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'17255", 'reported_seq': 17382, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:39.952516+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:20:39.952516+0000', 'last_peered': '2024-09-11T01:20:39.952516+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:20:39.952516+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "43'7200", 'ondisk_log_start': "43'7200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 12091392, 'num_objects': 2952, 'num_object_clones': 0, 'num_object_copies': 5904, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2952, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15072, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1072}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:45.225 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:20:45.225 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'17447", 'reported_seq': 17587, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:42.542570+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:20:42.542570+0000', 'last_peered': '2024-09-11T01:20:42.542570+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:20:42.542570+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "43'7400", 'ondisk_log_start': "43'7400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11538432, 'num_objects': 2817, 'num_object_clones': 0, 'num_object_copies': 5634, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2817, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14184, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1133}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:45.225 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:20:45.226 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'17358", 'reported_seq': 18308, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:42.590829+0000', 'last_change': '2024-09-11T01:20:20.985862+0000', 'last_active': '2024-09-11T01:20:42.590829+0000', 'last_peered': '2024-09-11T01:20:42.590829+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:20:42.590829+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "43'7300", 'ondisk_log_start': "43'7300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11542528, 'num_objects': 2818, 'num_object_clones': 0, 'num_object_copies': 5636, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3639, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2818, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13967, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 402, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -821}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:45.226 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:20:45.226 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'17188", 'reported_seq': 17332, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:42.475825+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:20:42.475825+0000', 'last_peered': '2024-09-11T01:20:42.475825+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:20:42.475825+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "43'7100", 'ondisk_log_start': "43'7100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10088, 'log_dups_size': 0, 'ondisk_log_size': 10088, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11321961, 'num_objects': 2765, 'num_object_clones': 0, 'num_object_copies': 5530, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2765, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14258, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1152}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:45.227 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:20:46.250 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:20:46.250 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35432/90928 objects degraded (38.967%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":582,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":45464,"data_bytes":186204820,"bytes_used":1725620224,"bytes_avail":288184672256,"bytes_total":289910292480,"degraded_objects":35432,"degraded_total":90928,"degraded_ratio":0.38967094844272393,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":172,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":388},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:20:46.313 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:20:46.313 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:47.197 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:20:47.197 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:20:47.257 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":334,"stamp":"2024-09-11T01:20:46.746021+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":183231124,"num_objects":44738,"num_object_clones":0,"num_object_copies":89476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35432,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44738,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":237553,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12208,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160904,"ondisk_log_size":160904,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1684524,"kb_used_data":552556,"kb_used_omap":12,"kb_used_meta":1131891,"kb_avail":281430996,"statfs":{"total":289910292480,"available":288185339904,"internally_reserved":0,"allocated":565817344,"data_stored":564927588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1159056388},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,5,7,7,8,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22097920,"num_objects":-5395,"num_object_clones":0,"num_object_copies":-10790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-172,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5395,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002625"},"pg_stats":[{"pgid":"2.f","version":"49'17453","reported_seq":17537,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.582069+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:20:42.582069+0000","last_peered":"2024-09-11T01:20:42.582069+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:20:42.582069+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11522048,"num_objects":2813,"num_object_clones":0,"num_object_copies":5626,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2813,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17453,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1259}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'17562","reported_seq":17673,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:44.954521+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:20:44.954521+0000","last_peered":"2024-09-11T01:20:44.954521+0000","last_clean":"2024-09-11T01:20:44.954521+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:20:44.954521+0000","last_undegraded":"2024-09-11T01:20:44.954521+0000","last_fullsized":"2024-09-11T01:20:44.954521+0000","mapping_epoch":35,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11427840,"num_objects":2790,"num_object_clones":0,"num_object_copies":5580,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2790,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15801,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'17321","reported_seq":17459,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:45.001228+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:20:45.001228+0000","last_peered":"2024-09-11T01:20:45.001228+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:20:45.001228+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11558936,"num_objects":2823,"num_object_clones":0,"num_object_copies":5646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2823,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15622,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1073}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'17367","reported_seq":17446,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.080432+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:20:45.080432+0000","last_peered":"2024-09-11T01:20:45.080432+0000","last_clean":"2024-09-11T01:20:45.080432+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:20:45.080432+0000","last_undegraded":"2024-09-11T01:20:45.080432+0000","last_fullsized":"2024-09-11T01:20:45.080432+0000","mapping_epoch":35,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11333632,"num_objects":2767,"num_object_clones":0,"num_object_copies":5534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2767,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17367,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'17263","reported_seq":17400,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:42.456320+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:20:42.456320+0000","last_peered":"2024-09-11T01:20:42.456320+0000","last_clean":"2024-09-11T01:20:42.456320+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:20:42.456320+0000","last_undegraded":"2024-09-11T01:20:42.456320+0000","last_fullsized":"2024-09-11T01:20:42.456320+0000","mapping_epoch":34,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11247616,"num_objects":2746,"num_object_clones":0,"num_object_copies":5492,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2746,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14268,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'17345","reported_seq":17525,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.552072+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:20:42.552072+0000","last_peered":"2024-09-11T01:20:42.552072+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:20:42.552072+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11804672,"num_objects":2882,"num_object_clones":0,"num_object_copies":5764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2882,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13686,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1069}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'17126","reported_seq":17266,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.515070+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:20:42.515070+0000","last_peered":"2024-09-11T01:20:42.515070+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:20:42.515070+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11485184,"num_objects":2804,"num_object_clones":0,"num_object_copies":5608,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2804,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13983,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1182}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'17430","reported_seq":17570,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:42.484476+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:20:42.484476+0000","last_peered":"2024-09-11T01:20:42.484476+0000","last_clean":"2024-09-11T01:20:42.484476+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:20:42.484476+0000","last_undegraded":"2024-09-11T01:20:42.484476+0000","last_fullsized":"2024-09-11T01:20:42.484476+0000","mapping_epoch":34,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11567104,"num_objects":2824,"num_object_clones":0,"num_object_copies":5648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2824,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14306,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003131+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003131+0000","last_peered":"2024-09-11T01:20:22.003131+0000","last_clean":"2024-09-11T01:20:22.003131+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003131+0000","last_undegraded":"2024-09-11T01:20:22.003131+0000","last_fullsized":"2024-09-11T01:20:22.003131+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'17155","reported_seq":25169,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:42.048631+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:20:42.048631+0000","last_peered":"2024-09-11T01:20:42.048631+0000","last_clean":"2024-09-11T01:20:42.048631+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:20:42.048631+0000","last_undegraded":"2024-09-11T01:20:42.048631+0000","last_fullsized":"2024-09-11T01:20:42.048631+0000","mapping_epoch":48,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11542528,"num_objects":2818,"num_object_clones":0,"num_object_copies":5636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2818,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15126,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'17099","reported_seq":17237,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.569530+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:20:42.569530+0000","last_peered":"2024-09-11T01:20:42.569530+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:20:42.569530+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11341824,"num_objects":2769,"num_object_clones":0,"num_object_copies":5538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13686,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1215}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'17401","reported_seq":17530,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:45.003960+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:20:45.003960+0000","last_peered":"2024-09-11T01:20:45.003960+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:20:45.003960+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11493376,"num_objects":2806,"num_object_clones":0,"num_object_copies":5612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2806,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15218,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1218}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003225+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003225+0000","last_peered":"2024-09-11T01:20:22.003225+0000","last_clean":"2024-09-11T01:20:22.003225+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003225+0000","last_undegraded":"2024-09-11T01:20:22.003225+0000","last_fullsized":"2024-09-11T01:20:22.003225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'17096","reported_seq":25087,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:45.015917+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:20:45.015917+0000","last_peered":"2024-09-11T01:20:45.015917+0000","last_clean":"2024-09-11T01:20:45.015917+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:20:45.015917+0000","last_undegraded":"2024-09-11T01:20:45.015917+0000","last_fullsized":"2024-09-11T01:20:45.015917+0000","mapping_epoch":44,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11059200,"num_objects":2700,"num_object_clones":0,"num_object_copies":5400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14881,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'17447","reported_seq":17587,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.542570+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:20:42.542570+0000","last_peered":"2024-09-11T01:20:42.542570+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:20:42.542570+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11538432,"num_objects":2817,"num_object_clones":0,"num_object_copies":5634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14184,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1133}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007532+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007532+0000","last_peered":"2024-09-11T01:20:22.007532+0000","last_clean":"2024-09-11T01:20:22.007532+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007532+0000","last_undegraded":"2024-09-11T01:20:22.007532+0000","last_fullsized":"2024-09-11T01:20:22.007532+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'17090","reported_seq":25131,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:42.068545+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:20:42.068545+0000","last_peered":"2024-09-11T01:20:42.068545+0000","last_clean":"2024-09-11T01:20:42.068545+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:20:42.068545+0000","last_undegraded":"2024-09-11T01:20:42.068545+0000","last_fullsized":"2024-09-11T01:20:42.068545+0000","mapping_epoch":46,"log_start":"43'7000","ondisk_log_start":"43'7000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11444224,"num_objects":2794,"num_object_clones":0,"num_object_copies":5588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2794,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13745,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'17358","reported_seq":18308,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.590829+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:20:42.590829+0000","last_peered":"2024-09-11T01:20:42.590829+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:20:42.590829+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11542528,"num_objects":2818,"num_object_clones":0,"num_object_copies":5636,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3639,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2818,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13967,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":402,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-821}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007560+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007560+0000","last_peered":"2024-09-11T01:20:22.007560+0000","last_clean":"2024-09-11T01:20:22.007560+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007560+0000","last_undegraded":"2024-09-11T01:20:22.007560+0000","last_fullsized":"2024-09-11T01:20:22.007560+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008171+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008171+0000","last_peered":"2024-09-11T01:20:22.008171+0000","last_clean":"2024-09-11T01:20:22.008171+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008171+0000","last_undegraded":"2024-09-11T01:20:22.008171+0000","last_fullsized":"2024-09-11T01:20:22.008171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'17188","reported_seq":17332,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:42.475825+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:20:42.475825+0000","last_peered":"2024-09-11T01:20:42.475825+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:20:42.475825+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'7100","ondisk_log_start":"43'7100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10088,"log_dups_size":0,"ondisk_log_size":10088,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11321961,"num_objects":2765,"num_object_clones":0,"num_object_copies":5530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2765,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14258,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1152}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":183231105,"num_objects":44736,"num_object_clones":0,"num_object_copies":89472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35432,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44736,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":237551,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12207,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":562388992,"data_stored":562325184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160901,"ondisk_log_size":160901,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182510,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":610104,"kb_used_data":199460,"kb_used_omap":4,"kb_used_meta":410619,"kb_avail":93761736,"statfs":{"total":96636764160,"available":96012017664,"internally_reserved":0,"allocated":204247040,"data_stored":203966886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":420474539},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,4,2,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986283,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":558964,"kb_used_data":216348,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93812876,"statfs":{"total":96636764160,"available":96064385024,"internally_reserved":0,"allocated":221540352,"data_stored":221260203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659227,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515456,"kb_used_data":136748,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93856384,"statfs":{"total":96636764160,"available":96108937216,"internally_reserved":0,"allocated":140029952,"data_stored":139700499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":387772079},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,2,2,4,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":202661888,"data_stored":202658409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":139083776,"data_stored":139030998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":220643328,"data_stored":220635777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:50.260 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:51.013 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:20:51.013 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:20:51.052 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2024-09-11T01:20:50.747045+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":173597332,"num_objects":42386,"num_object_clones":0,"num_object_copies":84772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35359,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42386,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":239905,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12281,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160756,"ondisk_log_size":160756,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1669924,"kb_used_data":533812,"kb_used_omap":12,"kb_used_meta":1136051,"kb_avail":281445596,"statfs":{"total":289910292480,"available":288200290304,"internally_reserved":0,"allocated":546623488,"data_stored":545733732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1163316228},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,6,5,10,12,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22073344,"num_objects":-5389,"num_object_clones":0,"num_object_copies":-10778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5389,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002806"},"pg_stats":[{"pgid":"2.f","version":"49'17589","reported_seq":17676,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:47.599273+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:20:47.599273+0000","last_peered":"2024-09-11T01:20:47.599273+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:20:47.599273+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10964992,"num_objects":2677,"num_object_clones":0,"num_object_copies":5354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17589,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1395}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'17708","reported_seq":17818,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:50.054436+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:20:50.054436+0000","last_peered":"2024-09-11T01:20:50.054436+0000","last_clean":"2024-09-11T01:20:50.054436+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:20:50.054436+0000","last_undegraded":"2024-09-11T01:20:50.054436+0000","last_fullsized":"2024-09-11T01:20:50.054436+0000","mapping_epoch":35,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10829824,"num_objects":2644,"num_object_clones":0,"num_object_copies":5288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15947,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'17478","reported_seq":17616,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:50.060051+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:20:50.060051+0000","last_peered":"2024-09-11T01:20:50.060051+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:20:50.060051+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10915864,"num_objects":2666,"num_object_clones":0,"num_object_copies":5332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2666,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15779,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1230}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'17498","reported_seq":17577,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:49.966892+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:20:49.966892+0000","last_peered":"2024-09-11T01:20:49.966892+0000","last_clean":"2024-09-11T01:20:49.966892+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:20:49.966892+0000","last_undegraded":"2024-09-11T01:20:49.966892+0000","last_fullsized":"2024-09-11T01:20:49.966892+0000","mapping_epoch":35,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10797056,"num_objects":2636,"num_object_clones":0,"num_object_copies":5272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17498,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'17406","reported_seq":17544,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:47.640824+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:20:47.640824+0000","last_peered":"2024-09-11T01:20:47.640824+0000","last_clean":"2024-09-11T01:20:47.640824+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:20:47.640824+0000","last_undegraded":"2024-09-11T01:20:47.640824+0000","last_fullsized":"2024-09-11T01:20:47.640824+0000","mapping_epoch":34,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10661888,"num_objects":2603,"num_object_clones":0,"num_object_copies":5206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2603,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14411,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'17502","reported_seq":17680,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:47.574317+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:20:47.574317+0000","last_peered":"2024-09-11T01:20:47.574317+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:20:47.574317+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11161600,"num_objects":2725,"num_object_clones":0,"num_object_copies":5450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13843,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1226}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'17260","reported_seq":17400,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:47.620068+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:20:47.620068+0000","last_peered":"2024-09-11T01:20:47.620068+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:20:47.620068+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10936320,"num_objects":2670,"num_object_clones":0,"num_object_copies":5340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14117,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1316}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'17559","reported_seq":17699,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:47.614330+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:20:47.614330+0000","last_peered":"2024-09-11T01:20:47.614330+0000","last_clean":"2024-09-11T01:20:47.614330+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:20:47.614330+0000","last_undegraded":"2024-09-11T01:20:47.614330+0000","last_fullsized":"2024-09-11T01:20:47.614330+0000","mapping_epoch":34,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11038720,"num_objects":2695,"num_object_clones":0,"num_object_copies":5390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14435,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003131+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003131+0000","last_peered":"2024-09-11T01:20:22.003131+0000","last_clean":"2024-09-11T01:20:22.003131+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003131+0000","last_undegraded":"2024-09-11T01:20:22.003131+0000","last_fullsized":"2024-09-11T01:20:22.003131+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'17305","reported_seq":25323,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:47.219990+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:20:47.219990+0000","last_peered":"2024-09-11T01:20:47.219990+0000","last_clean":"2024-09-11T01:20:47.219990+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:20:47.219990+0000","last_undegraded":"2024-09-11T01:20:47.219990+0000","last_fullsized":"2024-09-11T01:20:47.219990+0000","mapping_epoch":48,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10928128,"num_objects":2668,"num_object_clones":0,"num_object_copies":5336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2668,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15276,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'17252","reported_seq":17387,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:47.590911+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:20:47.590911+0000","last_peered":"2024-09-11T01:20:47.590911+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:20:47.590911+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10715136,"num_objects":2616,"num_object_clones":0,"num_object_copies":5232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13839,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1368}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'17546","reported_seq":17673,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:49.989977+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:20:49.989977+0000","last_peered":"2024-09-11T01:20:49.989977+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:20:49.989977+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10899456,"num_objects":2661,"num_object_clones":0,"num_object_copies":5322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15363,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1363}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003225+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003225+0000","last_peered":"2024-09-11T01:20:22.003225+0000","last_clean":"2024-09-11T01:20:22.003225+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003225+0000","last_undegraded":"2024-09-11T01:20:22.003225+0000","last_fullsized":"2024-09-11T01:20:22.003225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'17291","reported_seq":25282,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:50.038829+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:20:50.038829+0000","last_peered":"2024-09-11T01:20:50.038829+0000","last_clean":"2024-09-11T01:20:50.038829+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:20:50.038829+0000","last_undegraded":"2024-09-11T01:20:50.038829+0000","last_fullsized":"2024-09-11T01:20:50.038829+0000","mapping_epoch":44,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10260480,"num_objects":2505,"num_object_clones":0,"num_object_copies":5010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15076,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'17578","reported_seq":17720,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:47.520065+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:20:47.520065+0000","last_peered":"2024-09-11T01:20:47.520065+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:20:47.520065+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11001856,"num_objects":2686,"num_object_clones":0,"num_object_copies":5372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14315,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1264}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007532+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007532+0000","last_peered":"2024-09-11T01:20:22.007532+0000","last_clean":"2024-09-11T01:20:22.007532+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007532+0000","last_undegraded":"2024-09-11T01:20:22.007532+0000","last_fullsized":"2024-09-11T01:20:22.007532+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'17227","reported_seq":25268,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:47.200729+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:20:47.200729+0000","last_peered":"2024-09-11T01:20:47.200729+0000","last_clean":"2024-09-11T01:20:47.200729+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:20:47.200729+0000","last_undegraded":"2024-09-11T01:20:47.200729+0000","last_fullsized":"2024-09-11T01:20:47.200729+0000","mapping_epoch":46,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10883072,"num_objects":2657,"num_object_clones":0,"num_object_copies":5314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13882,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'17513","reported_seq":18609,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:47.633396+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:20:47.633396+0000","last_peered":"2024-09-11T01:20:47.633396+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:20:47.633396+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10907648,"num_objects":2663,"num_object_clones":0,"num_object_copies":5326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3566,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14122,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":475,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-903}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007560+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007560+0000","last_peered":"2024-09-11T01:20:22.007560+0000","last_clean":"2024-09-11T01:20:22.007560+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007560+0000","last_undegraded":"2024-09-11T01:20:22.007560+0000","last_fullsized":"2024-09-11T01:20:22.007560+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008171+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008171+0000","last_peered":"2024-09-11T01:20:22.008171+0000","last_clean":"2024-09-11T01:20:22.008171+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008171+0000","last_undegraded":"2024-09-11T01:20:22.008171+0000","last_fullsized":"2024-09-11T01:20:22.008171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'17341","reported_seq":17485,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:47.574830+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:20:47.574830+0000","last_peered":"2024-09-11T01:20:47.574830+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:20:47.574830+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10695273,"num_objects":2612,"num_object_clones":0,"num_object_copies":5224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14411,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1305}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":173597313,"num_objects":42384,"num_object_clones":0,"num_object_copies":84768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35359,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42384,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":239903,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12280,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":543154176,"data_stored":543090368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160753,"ondisk_log_size":160753,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182511,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":611516,"kb_used_data":192556,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93760324,"statfs":{"total":96636764160,"available":96010571776,"internally_reserved":0,"allocated":197177344,"data_stored":196897190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":428994219},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,5,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986284,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":558364,"kb_used_data":211596,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93813476,"statfs":{"total":96636764160,"available":96064999424,"internally_reserved":0,"allocated":216674304,"data_stored":216394155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659228,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":500044,"kb_used_data":129660,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93871796,"statfs":{"total":96636764160,"available":96124719104,"internally_reserved":0,"allocated":132771840,"data_stored":132442387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":379252399},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,2,2,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195690496,"data_stored":195687017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":131747840,"data_stored":131695062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":215715840,"data_stored":215708289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:51.053 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:51.773 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:20:51.774 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:20:51.820 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":336,"stamp":"2024-09-11T01:20:50.747045+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":173597332,"num_objects":42386,"num_object_clones":0,"num_object_copies":84772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35359,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42386,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":239905,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12281,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160756,"ondisk_log_size":160756,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1669924,"kb_used_data":533812,"kb_used_omap":12,"kb_used_meta":1136051,"kb_avail":281445596,"statfs":{"total":289910292480,"available":288200290304,"internally_reserved":0,"allocated":546623488,"data_stored":545733732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1163316228},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,6,5,10,12,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22073344,"num_objects":-5389,"num_object_clones":0,"num_object_copies":-10778,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5389,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5389,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":164,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002806"},"pg_stats":[{"pgid":"2.f","version":"49'17589","reported_seq":17676,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:47.599273+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:20:47.599273+0000","last_peered":"2024-09-11T01:20:47.599273+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:20:47.599273+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10964992,"num_objects":2677,"num_object_clones":0,"num_object_copies":5354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2677,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17589,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1395}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'17708","reported_seq":17818,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:50.054436+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:20:50.054436+0000","last_peered":"2024-09-11T01:20:50.054436+0000","last_clean":"2024-09-11T01:20:50.054436+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:20:50.054436+0000","last_undegraded":"2024-09-11T01:20:50.054436+0000","last_fullsized":"2024-09-11T01:20:50.054436+0000","mapping_epoch":35,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10829824,"num_objects":2644,"num_object_clones":0,"num_object_copies":5288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15947,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'17478","reported_seq":17616,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:50.060051+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:20:50.060051+0000","last_peered":"2024-09-11T01:20:50.060051+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:20:50.060051+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10915864,"num_objects":2666,"num_object_clones":0,"num_object_copies":5332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2666,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15779,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1230}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'17498","reported_seq":17577,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:49.966892+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:20:49.966892+0000","last_peered":"2024-09-11T01:20:49.966892+0000","last_clean":"2024-09-11T01:20:49.966892+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:20:49.966892+0000","last_undegraded":"2024-09-11T01:20:49.966892+0000","last_fullsized":"2024-09-11T01:20:49.966892+0000","mapping_epoch":35,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10797056,"num_objects":2636,"num_object_clones":0,"num_object_copies":5272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17498,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'17406","reported_seq":17544,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:47.640824+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:20:47.640824+0000","last_peered":"2024-09-11T01:20:47.640824+0000","last_clean":"2024-09-11T01:20:47.640824+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:20:47.640824+0000","last_undegraded":"2024-09-11T01:20:47.640824+0000","last_fullsized":"2024-09-11T01:20:47.640824+0000","mapping_epoch":34,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10661888,"num_objects":2603,"num_object_clones":0,"num_object_copies":5206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2603,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14411,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'17502","reported_seq":17680,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:47.574317+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:20:47.574317+0000","last_peered":"2024-09-11T01:20:47.574317+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:20:47.574317+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11161600,"num_objects":2725,"num_object_clones":0,"num_object_copies":5450,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2725,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13843,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1226}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'17260","reported_seq":17400,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:47.620068+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:20:47.620068+0000","last_peered":"2024-09-11T01:20:47.620068+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:20:47.620068+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10936320,"num_objects":2670,"num_object_clones":0,"num_object_copies":5340,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2670,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14117,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1316}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'17559","reported_seq":17699,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:47.614330+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:20:47.614330+0000","last_peered":"2024-09-11T01:20:47.614330+0000","last_clean":"2024-09-11T01:20:47.614330+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:20:47.614330+0000","last_undegraded":"2024-09-11T01:20:47.614330+0000","last_fullsized":"2024-09-11T01:20:47.614330+0000","mapping_epoch":34,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11038720,"num_objects":2695,"num_object_clones":0,"num_object_copies":5390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14435,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003131+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003131+0000","last_peered":"2024-09-11T01:20:22.003131+0000","last_clean":"2024-09-11T01:20:22.003131+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003131+0000","last_undegraded":"2024-09-11T01:20:22.003131+0000","last_fullsized":"2024-09-11T01:20:22.003131+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'17305","reported_seq":25323,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:47.219990+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:20:47.219990+0000","last_peered":"2024-09-11T01:20:47.219990+0000","last_clean":"2024-09-11T01:20:47.219990+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:20:47.219990+0000","last_undegraded":"2024-09-11T01:20:47.219990+0000","last_fullsized":"2024-09-11T01:20:47.219990+0000","mapping_epoch":48,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10928128,"num_objects":2668,"num_object_clones":0,"num_object_copies":5336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2668,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15276,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'17252","reported_seq":17387,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:47.590911+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:20:47.590911+0000","last_peered":"2024-09-11T01:20:47.590911+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:20:47.590911+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10715136,"num_objects":2616,"num_object_clones":0,"num_object_copies":5232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2616,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13839,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1368}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'17546","reported_seq":17673,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:49.989977+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:20:49.989977+0000","last_peered":"2024-09-11T01:20:49.989977+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:20:49.989977+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10899456,"num_objects":2661,"num_object_clones":0,"num_object_copies":5322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15363,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1363}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003225+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003225+0000","last_peered":"2024-09-11T01:20:22.003225+0000","last_clean":"2024-09-11T01:20:22.003225+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003225+0000","last_undegraded":"2024-09-11T01:20:22.003225+0000","last_fullsized":"2024-09-11T01:20:22.003225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'17291","reported_seq":25282,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:50.038829+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:20:50.038829+0000","last_peered":"2024-09-11T01:20:50.038829+0000","last_clean":"2024-09-11T01:20:50.038829+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:20:50.038829+0000","last_undegraded":"2024-09-11T01:20:50.038829+0000","last_fullsized":"2024-09-11T01:20:50.038829+0000","mapping_epoch":44,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10260480,"num_objects":2505,"num_object_clones":0,"num_object_copies":5010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15076,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'17578","reported_seq":17720,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:47.520065+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:20:47.520065+0000","last_peered":"2024-09-11T01:20:47.520065+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:20:47.520065+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":11001856,"num_objects":2686,"num_object_clones":0,"num_object_copies":5372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2686,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14315,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1264}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007532+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007532+0000","last_peered":"2024-09-11T01:20:22.007532+0000","last_clean":"2024-09-11T01:20:22.007532+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007532+0000","last_undegraded":"2024-09-11T01:20:22.007532+0000","last_fullsized":"2024-09-11T01:20:22.007532+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'17227","reported_seq":25268,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:47.200729+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:20:47.200729+0000","last_peered":"2024-09-11T01:20:47.200729+0000","last_clean":"2024-09-11T01:20:47.200729+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:20:47.200729+0000","last_undegraded":"2024-09-11T01:20:47.200729+0000","last_fullsized":"2024-09-11T01:20:47.200729+0000","mapping_epoch":46,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10883072,"num_objects":2657,"num_object_clones":0,"num_object_copies":5314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2657,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13882,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'17513","reported_seq":18609,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:47.633396+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:20:47.633396+0000","last_peered":"2024-09-11T01:20:47.633396+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:20:47.633396+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10907648,"num_objects":2663,"num_object_clones":0,"num_object_copies":5326,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3566,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2663,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14122,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":475,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-903}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007560+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007560+0000","last_peered":"2024-09-11T01:20:22.007560+0000","last_clean":"2024-09-11T01:20:22.007560+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007560+0000","last_undegraded":"2024-09-11T01:20:22.007560+0000","last_fullsized":"2024-09-11T01:20:22.007560+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008171+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008171+0000","last_peered":"2024-09-11T01:20:22.008171+0000","last_clean":"2024-09-11T01:20:22.008171+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008171+0000","last_undegraded":"2024-09-11T01:20:22.008171+0000","last_fullsized":"2024-09-11T01:20:22.008171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'17341","reported_seq":17485,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:47.574830+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:20:47.574830+0000","last_peered":"2024-09-11T01:20:47.574830+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:20:47.574830+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10695273,"num_objects":2612,"num_object_clones":0,"num_object_copies":5224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14411,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1305}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":173597313,"num_objects":42384,"num_object_clones":0,"num_object_copies":84768,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35359,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":42384,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":239903,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12280,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":543154176,"data_stored":543090368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160753,"ondisk_log_size":160753,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182511,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":611516,"kb_used_data":192556,"kb_used_omap":4,"kb_used_meta":418939,"kb_avail":93760324,"statfs":{"total":96636764160,"available":96010571776,"internally_reserved":0,"allocated":197177344,"data_stored":196897190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":428994219},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,1,1,5,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986284,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":558364,"kb_used_data":211596,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93813476,"statfs":{"total":96636764160,"available":96064999424,"internally_reserved":0,"allocated":216674304,"data_stored":216394155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659228,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":500044,"kb_used_data":129660,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93871796,"statfs":{"total":96636764160,"available":96124719104,"internally_reserved":0,"allocated":132771840,"data_stored":132442387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":379252399},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,2,2,7,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":195690496,"data_stored":195687017,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":131747840,"data_stored":131695062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":215715840,"data_stored":215708289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:51.821 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:20:51.821 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'17589", 'reported_seq': 17676, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:47.599273+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:20:47.599273+0000', 'last_peered': '2024-09-11T01:20:47.599273+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:20:47.599273+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "43'7500", 'ondisk_log_start': "43'7500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10964992, 'num_objects': 2677, 'num_object_clones': 0, 'num_object_copies': 5354, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2677, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17589, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1395}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:51.821 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:20:51.822 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'17478", 'reported_seq': 17616, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:50.060051+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:20:50.060051+0000', 'last_peered': '2024-09-11T01:20:50.060051+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:20:50.060051+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "43'7400", 'ondisk_log_start': "43'7400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10915864, 'num_objects': 2666, 'num_object_clones': 0, 'num_object_copies': 5332, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2666, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15779, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1230}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:51.822 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:20:51.822 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'17502", 'reported_seq': 17680, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:47.574317+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:20:47.574317+0000', 'last_peered': '2024-09-11T01:20:47.574317+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:20:47.574317+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "43'7500", 'ondisk_log_start': "43'7500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11161600, 'num_objects': 2725, 'num_object_clones': 0, 'num_object_copies': 5450, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2725, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13843, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1226}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:51.822 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:20:51.822 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'17260", 'reported_seq': 17400, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:47.620068+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:20:47.620068+0000', 'last_peered': '2024-09-11T01:20:47.620068+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:20:47.620068+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "43'7200", 'ondisk_log_start': "43'7200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10936320, 'num_objects': 2670, 'num_object_clones': 0, 'num_object_copies': 5340, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2670, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14117, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1316}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:51.822 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:20:51.822 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'17252", 'reported_seq': 17387, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:47.590911+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:20:47.590911+0000', 'last_peered': '2024-09-11T01:20:47.590911+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:20:47.590911+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "43'7200", 'ondisk_log_start': "43'7200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10715136, 'num_objects': 2616, 'num_object_clones': 0, 'num_object_copies': 5232, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2616, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13839, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1368}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:51.822 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:20:51.822 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'17546", 'reported_seq': 17673, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:49.989977+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:20:49.989977+0000', 'last_peered': '2024-09-11T01:20:49.989977+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:20:49.989977+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "43'7500", 'ondisk_log_start': "43'7500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10899456, 'num_objects': 2661, 'num_object_clones': 0, 'num_object_copies': 5322, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2661, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15363, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1363}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:51.822 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:20:51.823 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'17578", 'reported_seq': 17720, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:47.520065+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:20:47.520065+0000', 'last_peered': '2024-09-11T01:20:47.520065+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:20:47.520065+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "43'7500", 'ondisk_log_start': "43'7500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 11001856, 'num_objects': 2686, 'num_object_clones': 0, 'num_object_copies': 5372, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2686, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14315, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1264}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:51.823 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:20:51.823 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'17513", 'reported_seq': 18609, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:47.633396+0000', 'last_change': '2024-09-11T01:20:20.985862+0000', 'last_active': '2024-09-11T01:20:47.633396+0000', 'last_peered': '2024-09-11T01:20:47.633396+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:20:47.633396+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "43'7500", 'ondisk_log_start': "43'7500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10013, 'log_dups_size': 0, 'ondisk_log_size': 10013, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10907648, 'num_objects': 2663, 'num_object_clones': 0, 'num_object_copies': 5326, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3566, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2663, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14122, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 475, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -903}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:51.823 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:20:51.823 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'17341", 'reported_seq': 17485, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:47.574830+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:20:47.574830+0000', 'last_peered': '2024-09-11T01:20:47.574830+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:20:47.574830+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "43'7300", 'ondisk_log_start': "43'7300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10695273, 'num_objects': 2612, 'num_object_clones': 0, 'num_object_copies': 5224, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2612, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14411, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1305}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:51.823 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:20:52.778 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:20:52.778 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35359/84772 objects degraded (41.711%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":589,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":42386,"data_bytes":173597332,"bytes_used":1710002176,"bytes_avail":288200290304,"bytes_total":289910292480,"degraded_objects":35359,"degraded_total":84772,"degraded_ratio":0.41710706365309297,"recovering_objects_per_sec":13,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":164,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":448},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:20:52.849 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:20:52.849 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:53.712 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:20:53.715 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:20:53.823 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":337,"stamp":"2024-09-11T01:20:52.747490+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":166965908,"num_objects":40767,"num_object_clones":0,"num_object_copies":81534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35287,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40767,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":241524,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12353,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160975,"ondisk_log_size":160975,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1651588,"kb_used_data":519632,"kb_used_omap":12,"kb_used_meta":1131891,"kb_avail":281463932,"statfs":{"total":289910292480,"available":288219066368,"internally_reserved":0,"allocated":532103168,"data_stored":531213412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1159056388},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,5,3,12,7,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25755648,"num_objects":-6288,"num_object_clones":0,"num_object_copies":-12576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-236,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6288,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6288,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":236,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002784"},"pg_stats":[{"pgid":"2.f","version":"49'17742","reported_seq":17829,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:52.642612+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:20:52.642612+0000","last_peered":"2024-09-11T01:20:52.642612+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:20:52.642612+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10338304,"num_objects":2524,"num_object_clones":0,"num_object_copies":5048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17742,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1548}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'17708","reported_seq":17818,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:50.054436+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:20:50.054436+0000","last_peered":"2024-09-11T01:20:50.054436+0000","last_clean":"2024-09-11T01:20:50.054436+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:20:50.054436+0000","last_undegraded":"2024-09-11T01:20:50.054436+0000","last_fullsized":"2024-09-11T01:20:50.054436+0000","mapping_epoch":35,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10829824,"num_objects":2644,"num_object_clones":0,"num_object_copies":5288,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2644,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15947,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'17478","reported_seq":17616,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:50.060051+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:20:50.060051+0000","last_peered":"2024-09-11T01:20:50.060051+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:20:50.060051+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10915864,"num_objects":2666,"num_object_clones":0,"num_object_copies":5332,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2666,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15779,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1230}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'17498","reported_seq":17577,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:49.966892+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:20:49.966892+0000","last_peered":"2024-09-11T01:20:49.966892+0000","last_clean":"2024-09-11T01:20:49.966892+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:20:49.966892+0000","last_undegraded":"2024-09-11T01:20:49.966892+0000","last_fullsized":"2024-09-11T01:20:49.966892+0000","mapping_epoch":35,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10797056,"num_objects":2636,"num_object_clones":0,"num_object_copies":5272,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2636,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17498,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'17549","reported_seq":17687,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:52.563965+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:20:52.563965+0000","last_peered":"2024-09-11T01:20:52.563965+0000","last_clean":"2024-09-11T01:20:52.563965+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:20:52.563965+0000","last_undegraded":"2024-09-11T01:20:52.563965+0000","last_fullsized":"2024-09-11T01:20:52.563965+0000","mapping_epoch":34,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10076160,"num_objects":2460,"num_object_clones":0,"num_object_copies":4920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14554,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'17652","reported_seq":17832,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:52.644746+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:20:52.644746+0000","last_peered":"2024-09-11T01:20:52.644746+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:20:52.644746+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10547200,"num_objects":2575,"num_object_clones":0,"num_object_copies":5150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13993,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1376}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'17407","reported_seq":17543,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:52.589049+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:20:52.589049+0000","last_peered":"2024-09-11T01:20:52.589049+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:20:52.589049+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10334208,"num_objects":2523,"num_object_clones":0,"num_object_copies":5046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14264,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1463}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'17692","reported_seq":17832,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:52.593975+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:20:52.593975+0000","last_peered":"2024-09-11T01:20:52.593975+0000","last_clean":"2024-09-11T01:20:52.593975+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:20:52.593975+0000","last_undegraded":"2024-09-11T01:20:52.593975+0000","last_fullsized":"2024-09-11T01:20:52.593975+0000","mapping_epoch":34,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10493952,"num_objects":2562,"num_object_clones":0,"num_object_copies":5124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14568,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003131+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003131+0000","last_peered":"2024-09-11T01:20:22.003131+0000","last_clean":"2024-09-11T01:20:22.003131+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003131+0000","last_undegraded":"2024-09-11T01:20:22.003131+0000","last_fullsized":"2024-09-11T01:20:22.003131+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'17467","reported_seq":25485,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:52.050919+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:20:52.050919+0000","last_peered":"2024-09-11T01:20:52.050919+0000","last_clean":"2024-09-11T01:20:52.050919+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:20:52.050919+0000","last_undegraded":"2024-09-11T01:20:52.050919+0000","last_fullsized":"2024-09-11T01:20:52.050919+0000","mapping_epoch":48,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10264576,"num_objects":2506,"num_object_clones":0,"num_object_copies":5012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2506,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15438,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'17386","reported_seq":17524,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:52.632628+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:20:52.632628+0000","last_peered":"2024-09-11T01:20:52.632628+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:20:52.632628+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10166272,"num_objects":2482,"num_object_clones":0,"num_object_copies":4964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13973,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1502}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'17546","reported_seq":17673,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:49.989977+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:20:49.989977+0000","last_peered":"2024-09-11T01:20:49.989977+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:20:49.989977+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10899456,"num_objects":2661,"num_object_clones":0,"num_object_copies":5322,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2661,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15363,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1363}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003225+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003225+0000","last_peered":"2024-09-11T01:20:22.003225+0000","last_clean":"2024-09-11T01:20:22.003225+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003225+0000","last_undegraded":"2024-09-11T01:20:22.003225+0000","last_fullsized":"2024-09-11T01:20:22.003225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'17291","reported_seq":25282,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:50.038829+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:20:50.038829+0000","last_peered":"2024-09-11T01:20:50.038829+0000","last_clean":"2024-09-11T01:20:50.038829+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:20:50.038829+0000","last_undegraded":"2024-09-11T01:20:50.038829+0000","last_fullsized":"2024-09-11T01:20:50.038829+0000","mapping_epoch":44,"log_start":"43'7200","ondisk_log_start":"43'7200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10091,"log_dups_size":0,"ondisk_log_size":10091,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10260480,"num_objects":2505,"num_object_clones":0,"num_object_copies":5010,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2505,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15076,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'17730","reported_seq":17871,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:52.599396+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:20:52.599396+0000","last_peered":"2024-09-11T01:20:52.599396+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:20:52.599396+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10379264,"num_objects":2534,"num_object_clones":0,"num_object_copies":5068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14467,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1416}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007532+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007532+0000","last_peered":"2024-09-11T01:20:22.007532+0000","last_clean":"2024-09-11T01:20:22.007532+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007532+0000","last_undegraded":"2024-09-11T01:20:22.007532+0000","last_fullsized":"2024-09-11T01:20:22.007532+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'17365","reported_seq":25406,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:52.199867+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:20:52.199867+0000","last_peered":"2024-09-11T01:20:52.199867+0000","last_clean":"2024-09-11T01:20:52.199867+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:20:52.199867+0000","last_undegraded":"2024-09-11T01:20:52.199867+0000","last_fullsized":"2024-09-11T01:20:52.199867+0000","mapping_epoch":46,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10317824,"num_objects":2519,"num_object_clones":0,"num_object_copies":5038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14020,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'17674","reported_seq":18914,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:52.639000+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:20:52.639000+0000","last_peered":"2024-09-11T01:20:52.639000+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:20:52.639000+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10248192,"num_objects":2502,"num_object_clones":0,"num_object_copies":5004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3494,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14283,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":547,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-992}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007560+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007560+0000","last_peered":"2024-09-11T01:20:22.007560+0000","last_clean":"2024-09-11T01:20:22.007560+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007560+0000","last_undegraded":"2024-09-11T01:20:22.007560+0000","last_fullsized":"2024-09-11T01:20:22.007560+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008171+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008171+0000","last_peered":"2024-09-11T01:20:22.008171+0000","last_clean":"2024-09-11T01:20:22.008171+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008171+0000","last_undegraded":"2024-09-11T01:20:22.008171+0000","last_fullsized":"2024-09-11T01:20:22.008171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'17487","reported_seq":17631,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:52.569466+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:20:52.569466+0000","last_peered":"2024-09-11T01:20:52.569466+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:20:52.569466+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10097257,"num_objects":2466,"num_object_clones":0,"num_object_copies":4932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14557,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":166965889,"num_objects":40765,"num_object_clones":0,"num_object_copies":81530,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35287,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40765,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":241522,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12352,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":528666624,"data_stored":528602816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160972,"ondisk_log_size":160972,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182512,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591972,"kb_used_data":185488,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93779868,"statfs":{"total":96636764160,"available":96030584832,"internally_reserved":0,"allocated":189939712,"data_stored":189659558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":416214699},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,0,7,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986284,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":558364,"kb_used_data":211596,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93813476,"statfs":{"total":96636764160,"available":96064999424,"internally_reserved":0,"allocated":216674304,"data_stored":216394155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,3,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659229,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501252,"kb_used_data":122548,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93870588,"statfs":{"total":96636764160,"available":96123482112,"internally_reserved":0,"allocated":125489152,"data_stored":125159699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":387772079},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":188555264,"data_stored":188551785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":124395520,"data_stored":124342742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":215715840,"data_stored":215708289,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:56.826 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:57.603 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:20:57.607 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:20:57.663 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":339,"stamp":"2024-09-11T01:20:56.750062+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":164127380,"num_objects":40074,"num_object_clones":0,"num_object_copies":80148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35287,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40074,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":242217,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12353,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160968,"ondisk_log_size":160968,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1634676,"kb_used_data":515200,"kb_used_omap":12,"kb_used_meta":1119411,"kb_avail":281480844,"statfs":{"total":289910292480,"available":288236384256,"internally_reserved":0,"allocated":527564800,"data_stored":526675044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1146276868},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,5,1,10,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22077440,"num_objects":-5390,"num_object_clones":0,"num_object_copies":-10780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-145,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5390,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004447"},"pg_stats":[{"pgid":"2.f","version":"49'17742","reported_seq":17829,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:52.642612+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:20:52.642612+0000","last_peered":"2024-09-11T01:20:52.642612+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:20:52.642612+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10338304,"num_objects":2524,"num_object_clones":0,"num_object_copies":5048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17742,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1548}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'17863","reported_seq":17974,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:54.992887+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:20:54.992887+0000","last_peered":"2024-09-11T01:20:54.992887+0000","last_clean":"2024-09-11T01:20:54.992887+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:20:54.992887+0000","last_undegraded":"2024-09-11T01:20:54.992887+0000","last_fullsized":"2024-09-11T01:20:54.992887+0000","mapping_epoch":35,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10194944,"num_objects":2489,"num_object_clones":0,"num_object_copies":4978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2489,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16102,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'17599","reported_seq":17737,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:54.958099+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:20:54.958099+0000","last_peered":"2024-09-11T01:20:54.958099+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:20:54.958099+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10420248,"num_objects":2545,"num_object_clones":0,"num_object_copies":5090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2545,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15900,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1351}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'17626","reported_seq":17705,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:55.044684+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:20:55.044684+0000","last_peered":"2024-09-11T01:20:55.044684+0000","last_clean":"2024-09-11T01:20:55.044684+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:20:55.044684+0000","last_undegraded":"2024-09-11T01:20:55.044684+0000","last_fullsized":"2024-09-11T01:20:55.044684+0000","mapping_epoch":35,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10272768,"num_objects":2508,"num_object_clones":0,"num_object_copies":5016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17626,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'17549","reported_seq":17687,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:52.563965+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:20:52.563965+0000","last_peered":"2024-09-11T01:20:52.563965+0000","last_clean":"2024-09-11T01:20:52.563965+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:20:52.563965+0000","last_undegraded":"2024-09-11T01:20:52.563965+0000","last_fullsized":"2024-09-11T01:20:52.563965+0000","mapping_epoch":34,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10076160,"num_objects":2460,"num_object_clones":0,"num_object_copies":4920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14554,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'17652","reported_seq":17832,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:52.644746+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:20:52.644746+0000","last_peered":"2024-09-11T01:20:52.644746+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:20:52.644746+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10547200,"num_objects":2575,"num_object_clones":0,"num_object_copies":5150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13993,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1376}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'17407","reported_seq":17543,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:52.589049+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:20:52.589049+0000","last_peered":"2024-09-11T01:20:52.589049+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:20:52.589049+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10334208,"num_objects":2523,"num_object_clones":0,"num_object_copies":5046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14264,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1463}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'17692","reported_seq":17832,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:52.593975+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:20:52.593975+0000","last_peered":"2024-09-11T01:20:52.593975+0000","last_clean":"2024-09-11T01:20:52.593975+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:20:52.593975+0000","last_undegraded":"2024-09-11T01:20:52.593975+0000","last_fullsized":"2024-09-11T01:20:52.593975+0000","mapping_epoch":34,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10493952,"num_objects":2562,"num_object_clones":0,"num_object_copies":5124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14568,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003131+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003131+0000","last_peered":"2024-09-11T01:20:22.003131+0000","last_clean":"2024-09-11T01:20:22.003131+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003131+0000","last_undegraded":"2024-09-11T01:20:22.003131+0000","last_fullsized":"2024-09-11T01:20:22.003131+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'17467","reported_seq":25485,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:52.050919+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:20:52.050919+0000","last_peered":"2024-09-11T01:20:52.050919+0000","last_clean":"2024-09-11T01:20:52.050919+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:20:52.050919+0000","last_undegraded":"2024-09-11T01:20:52.050919+0000","last_fullsized":"2024-09-11T01:20:52.050919+0000","mapping_epoch":48,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10264576,"num_objects":2506,"num_object_clones":0,"num_object_copies":5012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2506,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15438,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'17386","reported_seq":17524,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:52.632628+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:20:52.632628+0000","last_peered":"2024-09-11T01:20:52.632628+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:20:52.632628+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10166272,"num_objects":2482,"num_object_clones":0,"num_object_copies":4964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13973,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1502}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'17700","reported_seq":17829,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:55.065741+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:20:55.065741+0000","last_peered":"2024-09-11T01:20:55.065741+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:20:55.065741+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10268672,"num_objects":2507,"num_object_clones":0,"num_object_copies":5014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15517,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1517}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003225+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003225+0000","last_peered":"2024-09-11T01:20:22.003225+0000","last_clean":"2024-09-11T01:20:22.003225+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003225+0000","last_undegraded":"2024-09-11T01:20:22.003225+0000","last_fullsized":"2024-09-11T01:20:22.003225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'17426","reported_seq":25417,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:55.029445+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:20:55.029445+0000","last_peered":"2024-09-11T01:20:55.029445+0000","last_clean":"2024-09-11T01:20:55.029445+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:20:55.029445+0000","last_undegraded":"2024-09-11T01:20:55.029445+0000","last_fullsized":"2024-09-11T01:20:55.029445+0000","mapping_epoch":44,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9707520,"num_objects":2370,"num_object_clones":0,"num_object_copies":4740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15211,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'17730","reported_seq":17871,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:52.599396+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:20:52.599396+0000","last_peered":"2024-09-11T01:20:52.599396+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:20:52.599396+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10379264,"num_objects":2534,"num_object_clones":0,"num_object_copies":5068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14467,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1416}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007532+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007532+0000","last_peered":"2024-09-11T01:20:22.007532+0000","last_clean":"2024-09-11T01:20:22.007532+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007532+0000","last_undegraded":"2024-09-11T01:20:22.007532+0000","last_fullsized":"2024-09-11T01:20:22.007532+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'17365","reported_seq":25406,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:52.199867+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:20:52.199867+0000","last_peered":"2024-09-11T01:20:52.199867+0000","last_clean":"2024-09-11T01:20:52.199867+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:20:52.199867+0000","last_undegraded":"2024-09-11T01:20:52.199867+0000","last_fullsized":"2024-09-11T01:20:52.199867+0000","mapping_epoch":46,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10317824,"num_objects":2519,"num_object_clones":0,"num_object_copies":5038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14020,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'17674","reported_seq":18914,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:52.639000+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:20:52.639000+0000","last_peered":"2024-09-11T01:20:52.639000+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:20:52.639000+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10248192,"num_objects":2502,"num_object_clones":0,"num_object_copies":5004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3494,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14283,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":547,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-992}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007560+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007560+0000","last_peered":"2024-09-11T01:20:22.007560+0000","last_clean":"2024-09-11T01:20:22.007560+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007560+0000","last_undegraded":"2024-09-11T01:20:22.007560+0000","last_fullsized":"2024-09-11T01:20:22.007560+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008171+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008171+0000","last_peered":"2024-09-11T01:20:22.008171+0000","last_clean":"2024-09-11T01:20:22.008171+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008171+0000","last_undegraded":"2024-09-11T01:20:22.008171+0000","last_fullsized":"2024-09-11T01:20:22.008171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'17487","reported_seq":17631,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:52.569466+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:20:52.569466+0000","last_peered":"2024-09-11T01:20:52.569466+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:20:52.569466+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10097257,"num_objects":2466,"num_object_clones":0,"num_object_copies":4932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14557,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":164127361,"num_objects":40072,"num_object_clones":0,"num_object_copies":80144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35287,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40072,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":242215,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12352,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":524242944,"data_stored":524179136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160965,"ondisk_log_size":160965,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182512,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591972,"kb_used_data":185488,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93779868,"statfs":{"total":96636764160,"available":96030584832,"internally_reserved":0,"allocated":189939712,"data_stored":189659558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":416214699},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,0,7,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986285,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":541452,"kb_used_data":207164,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93830388,"statfs":{"total":96636764160,"available":96082317312,"internally_reserved":0,"allocated":212135936,"data_stored":211855787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342290090},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,0,1,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659229,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501252,"kb_used_data":122548,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93870588,"statfs":{"total":96636764160,"available":96123482112,"internally_reserved":0,"allocated":125489152,"data_stored":125159699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":387772079},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":188555264,"data_stored":188551785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":124395520,"data_stored":124342742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":211292160,"data_stored":211284609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:57.664 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:20:58.453 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:20:58.454 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:20:58.545 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":339,"stamp":"2024-09-11T01:20:56.750062+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":164127380,"num_objects":40074,"num_object_clones":0,"num_object_copies":80148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35287,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40074,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":242217,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12353,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160968,"ondisk_log_size":160968,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1634676,"kb_used_data":515200,"kb_used_omap":12,"kb_used_meta":1119411,"kb_avail":281480844,"statfs":{"total":289910292480,"available":288236384256,"internally_reserved":0,"allocated":527564800,"data_stored":526675044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1146276868},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,3,5,1,10,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22077440,"num_objects":-5390,"num_object_clones":0,"num_object_copies":-10780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-145,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5390,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5390,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":145,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004447"},"pg_stats":[{"pgid":"2.f","version":"49'17742","reported_seq":17829,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:52.642612+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:20:52.642612+0000","last_peered":"2024-09-11T01:20:52.642612+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:20:52.642612+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10338304,"num_objects":2524,"num_object_clones":0,"num_object_copies":5048,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2524,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17742,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1548}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'17863","reported_seq":17974,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:54.992887+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:20:54.992887+0000","last_peered":"2024-09-11T01:20:54.992887+0000","last_clean":"2024-09-11T01:20:54.992887+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:20:54.992887+0000","last_undegraded":"2024-09-11T01:20:54.992887+0000","last_fullsized":"2024-09-11T01:20:54.992887+0000","mapping_epoch":35,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10194944,"num_objects":2489,"num_object_clones":0,"num_object_copies":4978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2489,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16102,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'17599","reported_seq":17737,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:54.958099+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:20:54.958099+0000","last_peered":"2024-09-11T01:20:54.958099+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:20:54.958099+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10420248,"num_objects":2545,"num_object_clones":0,"num_object_copies":5090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2545,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15900,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1351}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'17626","reported_seq":17705,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:55.044684+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:20:55.044684+0000","last_peered":"2024-09-11T01:20:55.044684+0000","last_clean":"2024-09-11T01:20:55.044684+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:20:55.044684+0000","last_undegraded":"2024-09-11T01:20:55.044684+0000","last_fullsized":"2024-09-11T01:20:55.044684+0000","mapping_epoch":35,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10272768,"num_objects":2508,"num_object_clones":0,"num_object_copies":5016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17626,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'17549","reported_seq":17687,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:52.563965+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:20:52.563965+0000","last_peered":"2024-09-11T01:20:52.563965+0000","last_clean":"2024-09-11T01:20:52.563965+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:20:52.563965+0000","last_undegraded":"2024-09-11T01:20:52.563965+0000","last_fullsized":"2024-09-11T01:20:52.563965+0000","mapping_epoch":34,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10076160,"num_objects":2460,"num_object_clones":0,"num_object_copies":4920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2460,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14554,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'17652","reported_seq":17832,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:52.644746+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:20:52.644746+0000","last_peered":"2024-09-11T01:20:52.644746+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:20:52.644746+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10547200,"num_objects":2575,"num_object_clones":0,"num_object_copies":5150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2575,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13993,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1376}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'17407","reported_seq":17543,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:52.589049+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:20:52.589049+0000","last_peered":"2024-09-11T01:20:52.589049+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:20:52.589049+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10334208,"num_objects":2523,"num_object_clones":0,"num_object_copies":5046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14264,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1463}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'17692","reported_seq":17832,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:52.593975+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:20:52.593975+0000","last_peered":"2024-09-11T01:20:52.593975+0000","last_clean":"2024-09-11T01:20:52.593975+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:20:52.593975+0000","last_undegraded":"2024-09-11T01:20:52.593975+0000","last_fullsized":"2024-09-11T01:20:52.593975+0000","mapping_epoch":34,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10493952,"num_objects":2562,"num_object_clones":0,"num_object_copies":5124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2562,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14568,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003131+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003131+0000","last_peered":"2024-09-11T01:20:22.003131+0000","last_clean":"2024-09-11T01:20:22.003131+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003131+0000","last_undegraded":"2024-09-11T01:20:22.003131+0000","last_fullsized":"2024-09-11T01:20:22.003131+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'17467","reported_seq":25485,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:52.050919+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:20:52.050919+0000","last_peered":"2024-09-11T01:20:52.050919+0000","last_clean":"2024-09-11T01:20:52.050919+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:20:52.050919+0000","last_undegraded":"2024-09-11T01:20:52.050919+0000","last_fullsized":"2024-09-11T01:20:52.050919+0000","mapping_epoch":48,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10264576,"num_objects":2506,"num_object_clones":0,"num_object_copies":5012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2506,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15438,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'17386","reported_seq":17524,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:52.632628+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:20:52.632628+0000","last_peered":"2024-09-11T01:20:52.632628+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:20:52.632628+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10166272,"num_objects":2482,"num_object_clones":0,"num_object_copies":4964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2482,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":13973,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1502}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'17700","reported_seq":17829,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:55.065741+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:20:55.065741+0000","last_peered":"2024-09-11T01:20:55.065741+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:20:55.065741+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10268672,"num_objects":2507,"num_object_clones":0,"num_object_copies":5014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15517,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1517}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003225+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003225+0000","last_peered":"2024-09-11T01:20:22.003225+0000","last_clean":"2024-09-11T01:20:22.003225+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003225+0000","last_undegraded":"2024-09-11T01:20:22.003225+0000","last_fullsized":"2024-09-11T01:20:22.003225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'17426","reported_seq":25417,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:55.029445+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:20:55.029445+0000","last_peered":"2024-09-11T01:20:55.029445+0000","last_clean":"2024-09-11T01:20:55.029445+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:20:55.029445+0000","last_undegraded":"2024-09-11T01:20:55.029445+0000","last_fullsized":"2024-09-11T01:20:55.029445+0000","mapping_epoch":44,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9707520,"num_objects":2370,"num_object_clones":0,"num_object_copies":4740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15211,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'17730","reported_seq":17871,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:52.599396+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:20:52.599396+0000","last_peered":"2024-09-11T01:20:52.599396+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:20:52.599396+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10379264,"num_objects":2534,"num_object_clones":0,"num_object_copies":5068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14467,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1416}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007532+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007532+0000","last_peered":"2024-09-11T01:20:22.007532+0000","last_clean":"2024-09-11T01:20:22.007532+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007532+0000","last_undegraded":"2024-09-11T01:20:22.007532+0000","last_fullsized":"2024-09-11T01:20:22.007532+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'17365","reported_seq":25406,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:52.199867+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:20:52.199867+0000","last_peered":"2024-09-11T01:20:52.199867+0000","last_clean":"2024-09-11T01:20:52.199867+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:20:52.199867+0000","last_undegraded":"2024-09-11T01:20:52.199867+0000","last_fullsized":"2024-09-11T01:20:52.199867+0000","mapping_epoch":46,"log_start":"43'7300","ondisk_log_start":"43'7300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10065,"log_dups_size":0,"ondisk_log_size":10065,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10317824,"num_objects":2519,"num_object_clones":0,"num_object_copies":5038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2519,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14020,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'17674","reported_seq":18914,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:52.639000+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:20:52.639000+0000","last_peered":"2024-09-11T01:20:52.639000+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:20:52.639000+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10074,"log_dups_size":0,"ondisk_log_size":10074,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10248192,"num_objects":2502,"num_object_clones":0,"num_object_copies":5004,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3494,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2502,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14283,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":547,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-992}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007560+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007560+0000","last_peered":"2024-09-11T01:20:22.007560+0000","last_clean":"2024-09-11T01:20:22.007560+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007560+0000","last_undegraded":"2024-09-11T01:20:22.007560+0000","last_fullsized":"2024-09-11T01:20:22.007560+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008171+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008171+0000","last_peered":"2024-09-11T01:20:22.008171+0000","last_clean":"2024-09-11T01:20:22.008171+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008171+0000","last_undegraded":"2024-09-11T01:20:22.008171+0000","last_fullsized":"2024-09-11T01:20:22.008171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'17487","reported_seq":17631,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:52.569466+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:20:52.569466+0000","last_peered":"2024-09-11T01:20:52.569466+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:20:52.569466+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10097257,"num_objects":2466,"num_object_clones":0,"num_object_copies":4932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2466,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14557,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1451}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":164127361,"num_objects":40072,"num_object_clones":0,"num_object_copies":80144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35287,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":40072,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":242215,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12352,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":524242944,"data_stored":524179136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160965,"ondisk_log_size":160965,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182512,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":591972,"kb_used_data":185488,"kb_used_omap":4,"kb_used_meta":406459,"kb_avail":93779868,"statfs":{"total":96636764160,"available":96030584832,"internally_reserved":0,"allocated":189939712,"data_stored":189659558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":416214699},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,0,1,2,0,7,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986285,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":541452,"kb_used_data":207164,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93830388,"statfs":{"total":96636764160,"available":96082317312,"internally_reserved":0,"allocated":212135936,"data_stored":211855787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342290090},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,0,1,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659229,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501252,"kb_used_data":122548,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93870588,"statfs":{"total":96636764160,"available":96123482112,"internally_reserved":0,"allocated":125489152,"data_stored":125159699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":387772079},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,1,2,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":188555264,"data_stored":188551785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":124395520,"data_stored":124342742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":211292160,"data_stored":211284609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:20:58.546 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:20:58.546 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'17742", 'reported_seq': 17829, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:52.642612+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:20:52.642612+0000', 'last_peered': '2024-09-11T01:20:52.642612+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:20:52.642612+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "43'7700", 'ondisk_log_start': "43'7700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10338304, 'num_objects': 2524, 'num_object_clones': 0, 'num_object_copies': 5048, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2524, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17742, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1548}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:58.546 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:20:58.546 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'17599", 'reported_seq': 17737, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:54.958099+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:20:54.958099+0000', 'last_peered': '2024-09-11T01:20:54.958099+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:20:54.958099+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "43'7500", 'ondisk_log_start': "43'7500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10099, 'log_dups_size': 0, 'ondisk_log_size': 10099, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10420248, 'num_objects': 2545, 'num_object_clones': 0, 'num_object_copies': 5090, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2545, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 15900, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1351}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:58.546 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:20:58.547 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'17652", 'reported_seq': 17832, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:52.644746+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:20:52.644746+0000', 'last_peered': '2024-09-11T01:20:52.644746+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:20:52.644746+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "43'7600", 'ondisk_log_start': "43'7600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10547200, 'num_objects': 2575, 'num_object_clones': 0, 'num_object_copies': 5150, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2575, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13993, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1376}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:58.547 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:20:58.547 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'17407", 'reported_seq': 17543, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:52.589049+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:20:52.589049+0000', 'last_peered': '2024-09-11T01:20:52.589049+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:20:52.589049+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "43'7400", 'ondisk_log_start': "43'7400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10334208, 'num_objects': 2523, 'num_object_clones': 0, 'num_object_copies': 5046, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2523, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14264, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1463}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:58.547 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:20:58.547 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'17386", 'reported_seq': 17524, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:52.632628+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:20:52.632628+0000', 'last_peered': '2024-09-11T01:20:52.632628+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:20:52.632628+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "43'7300", 'ondisk_log_start': "43'7300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10166272, 'num_objects': 2482, 'num_object_clones': 0, 'num_object_copies': 4964, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2482, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 13973, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1502}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:58.547 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:20:58.547 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'17700", 'reported_seq': 17829, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:55.065741+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:20:55.065741+0000', 'last_peered': '2024-09-11T01:20:55.065741+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:20:55.065741+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "43'7600", 'ondisk_log_start': "43'7600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10268672, 'num_objects': 2507, 'num_object_clones': 0, 'num_object_copies': 5014, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2507, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15517, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1517}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:20:58.547 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:20:58.548 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'17730", 'reported_seq': 17871, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:52.599396+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:20:52.599396+0000', 'last_peered': '2024-09-11T01:20:52.599396+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:20:52.599396+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "43'7700", 'ondisk_log_start': "43'7700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10379264, 'num_objects': 2534, 'num_object_clones': 0, 'num_object_copies': 5068, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2534, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14467, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1416}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:58.548 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:20:58.548 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'17674", 'reported_seq': 18914, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:52.639000+0000', 'last_change': '2024-09-11T01:20:20.985862+0000', 'last_active': '2024-09-11T01:20:52.639000+0000', 'last_peered': '2024-09-11T01:20:52.639000+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:20:52.639000+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "43'7600", 'ondisk_log_start': "43'7600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10074, 'log_dups_size': 0, 'ondisk_log_size': 10074, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10248192, 'num_objects': 2502, 'num_object_clones': 0, 'num_object_copies': 5004, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3494, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2502, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14283, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 547, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -992}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:58.548 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:20:58.548 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'17487", 'reported_seq': 17631, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:20:52.569466+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:20:52.569466+0000', 'last_peered': '2024-09-11T01:20:52.569466+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:20:52.569466+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "43'7400", 'ondisk_log_start': "43'7400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 10097257, 'num_objects': 2466, 'num_object_clones': 0, 'num_object_copies': 4932, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2466, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14557, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1451}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:20:58.549 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:20:59.522 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:20:59.522 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35190/76814 objects degraded (45.812%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":596,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":38407,"data_bytes":157299348,"bytes_used":1690435584,"bytes_avail":288219856896,"bytes_total":289910292480,"degraded_objects":35190,"degraded_total":76814,"degraded_ratio":0.45811961361210196,"recovering_objects_per_sec":20,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":242,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":527},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:20:59.596 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:20:59.596 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:00.472 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:21:00.475 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:21:00.531 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":340,"stamp":"2024-09-11T01:20:58.754349+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":157299348,"num_objects":38407,"num_object_clones":0,"num_object_copies":76814,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35190,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38407,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":243884,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12450,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160935,"ondisk_log_size":160935,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1650816,"kb_used_data":500812,"kb_used_omap":12,"kb_used_meta":1149939,"kb_avail":281464704,"statfs":{"total":289910292480,"available":288219856896,"internally_reserved":0,"allocated":512831488,"data_stored":511941732,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1177537540},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,3,4,2,11,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25931776,"num_objects":-6331,"num_object_clones":0,"num_object_copies":-12662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-242,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6331,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6331,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":242,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008327"},"pg_stats":[{"pgid":"2.f","version":"49'17897","reported_seq":17985,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:57.676833+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:20:57.676833+0000","last_peered":"2024-09-11T01:20:57.676833+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:20:57.676833+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9703424,"num_objects":2369,"num_object_clones":0,"num_object_copies":4738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2369,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17897,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1703}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'17863","reported_seq":17974,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:54.992887+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:20:54.992887+0000","last_peered":"2024-09-11T01:20:54.992887+0000","last_clean":"2024-09-11T01:20:54.992887+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:20:54.992887+0000","last_undegraded":"2024-09-11T01:20:54.992887+0000","last_fullsized":"2024-09-11T01:20:54.992887+0000","mapping_epoch":35,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10194944,"num_objects":2489,"num_object_clones":0,"num_object_copies":4978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2489,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16102,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'17599","reported_seq":17737,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:54.958099+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:20:54.958099+0000","last_peered":"2024-09-11T01:20:54.958099+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:20:54.958099+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10420248,"num_objects":2545,"num_object_clones":0,"num_object_copies":5090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2545,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":15900,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1351}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'17626","reported_seq":17705,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:55.044684+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:20:55.044684+0000","last_peered":"2024-09-11T01:20:55.044684+0000","last_clean":"2024-09-11T01:20:55.044684+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:20:55.044684+0000","last_undegraded":"2024-09-11T01:20:55.044684+0000","last_fullsized":"2024-09-11T01:20:55.044684+0000","mapping_epoch":35,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10272768,"num_objects":2508,"num_object_clones":0,"num_object_copies":5016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2508,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17626,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'17685","reported_seq":17823,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:57.621286+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:20:57.621286+0000","last_peered":"2024-09-11T01:20:57.621286+0000","last_clean":"2024-09-11T01:20:57.621286+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:20:57.621286+0000","last_undegraded":"2024-09-11T01:20:57.621286+0000","last_fullsized":"2024-09-11T01:20:57.621286+0000","mapping_epoch":34,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9519104,"num_objects":2324,"num_object_clones":0,"num_object_copies":4648,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2324,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14690,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'17819","reported_seq":17996,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:57.640183+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:20:57.640183+0000","last_peered":"2024-09-11T01:20:57.640183+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:20:57.640183+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9863168,"num_objects":2408,"num_object_clones":0,"num_object_copies":4816,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2408,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14160,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1543}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'17557","reported_seq":17697,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:57.510794+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:20:57.510794+0000","last_peered":"2024-09-11T01:20:57.510794+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:20:57.510794+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9719808,"num_objects":2373,"num_object_clones":0,"num_object_copies":4746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2373,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14414,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1613}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'17851","reported_seq":17991,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:57.621374+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:20:57.621374+0000","last_peered":"2024-09-11T01:20:57.621374+0000","last_clean":"2024-09-11T01:20:57.621374+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:20:57.621374+0000","last_undegraded":"2024-09-11T01:20:57.621374+0000","last_fullsized":"2024-09-11T01:20:57.621374+0000","mapping_epoch":34,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9842688,"num_objects":2403,"num_object_clones":0,"num_object_copies":4806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2403,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14727,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003131+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003131+0000","last_peered":"2024-09-11T01:20:22.003131+0000","last_clean":"2024-09-11T01:20:22.003131+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003131+0000","last_undegraded":"2024-09-11T01:20:22.003131+0000","last_fullsized":"2024-09-11T01:20:22.003131+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'17599","reported_seq":25617,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:57.320211+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:20:57.320211+0000","last_peered":"2024-09-11T01:20:57.320211+0000","last_clean":"2024-09-11T01:20:57.320211+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:20:57.320211+0000","last_undegraded":"2024-09-11T01:20:57.320211+0000","last_fullsized":"2024-09-11T01:20:57.320211+0000","mapping_epoch":48,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9723904,"num_objects":2374,"num_object_clones":0,"num_object_copies":4748,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2374,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15570,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'17547","reported_seq":17685,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:57.609010+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:20:57.609010+0000","last_peered":"2024-09-11T01:20:57.609010+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:20:57.609010+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9506816,"num_objects":2321,"num_object_clones":0,"num_object_copies":4642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2321,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14134,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1663}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'17700","reported_seq":17829,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:55.065741+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:20:55.065741+0000","last_peered":"2024-09-11T01:20:55.065741+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:20:55.065741+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":10268672,"num_objects":2507,"num_object_clones":0,"num_object_copies":5014,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2507,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15517,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1517}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003225+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003225+0000","last_peered":"2024-09-11T01:20:22.003225+0000","last_clean":"2024-09-11T01:20:22.003225+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003225+0000","last_undegraded":"2024-09-11T01:20:22.003225+0000","last_fullsized":"2024-09-11T01:20:22.003225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'17426","reported_seq":25417,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:55.029445+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:20:55.029445+0000","last_peered":"2024-09-11T01:20:55.029445+0000","last_clean":"2024-09-11T01:20:55.029445+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:20:55.029445+0000","last_undegraded":"2024-09-11T01:20:55.029445+0000","last_fullsized":"2024-09-11T01:20:55.029445+0000","mapping_epoch":44,"log_start":"43'7400","ondisk_log_start":"43'7400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9707520,"num_objects":2370,"num_object_clones":0,"num_object_copies":4740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15211,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'17869","reported_seq":18010,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:57.610292+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:20:57.610292+0000","last_peered":"2024-09-11T01:20:57.610292+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:20:57.610292+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9809920,"num_objects":2395,"num_object_clones":0,"num_object_copies":4790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14606,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1555}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007532+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007532+0000","last_peered":"2024-09-11T01:20:22.007532+0000","last_clean":"2024-09-11T01:20:22.007532+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007532+0000","last_undegraded":"2024-09-11T01:20:22.007532+0000","last_fullsized":"2024-09-11T01:20:22.007532+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'17520","reported_seq":25561,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:57.301394+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:20:57.301394+0000","last_peered":"2024-09-11T01:20:57.301394+0000","last_clean":"2024-09-11T01:20:57.301394+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:20:57.301394+0000","last_undegraded":"2024-09-11T01:20:57.301394+0000","last_fullsized":"2024-09-11T01:20:57.301394+0000","mapping_epoch":46,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9682944,"num_objects":2364,"num_object_clones":0,"num_object_copies":4728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14175,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'17821","reported_seq":19252,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:57.661802+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:20:57.661802+0000","last_peered":"2024-09-11T01:20:57.661802+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:20:57.661802+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9646080,"num_objects":2355,"num_object_clones":0,"num_object_copies":4710,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3397,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2355,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14430,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":644,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1042}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007560+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007560+0000","last_peered":"2024-09-11T01:20:22.007560+0000","last_clean":"2024-09-11T01:20:22.007560+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007560+0000","last_undegraded":"2024-09-11T01:20:22.007560+0000","last_fullsized":"2024-09-11T01:20:22.007560+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008171+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008171+0000","last_peered":"2024-09-11T01:20:22.008171+0000","last_clean":"2024-09-11T01:20:22.008171+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008171+0000","last_undegraded":"2024-09-11T01:20:22.008171+0000","last_fullsized":"2024-09-11T01:20:22.008171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'17653","reported_seq":17797,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:20:57.644995+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:20:57.644995+0000","last_peered":"2024-09-11T01:20:57.644995+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:20:57.644995+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9417321,"num_objects":2300,"num_object_clones":0,"num_object_copies":4600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2300,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14723,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1617}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":157299329,"num_objects":38405,"num_object_clones":0,"num_object_copies":76810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35190,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":38405,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":243882,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12449,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":509394944,"data_stored":509331136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160932,"ondisk_log_size":160932,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182513,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":611076,"kb_used_data":178228,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93760764,"statfs":{"total":96636764160,"available":96011022336,"internally_reserved":0,"allocated":182505472,"data_stored":182225318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":443215531},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,1,6,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986285,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":541452,"kb_used_data":207164,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93830388,"statfs":{"total":96636764160,"available":96082317312,"internally_reserved":0,"allocated":212135936,"data_stored":211855787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342290090},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,0,1,1,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659230,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":498288,"kb_used_data":115420,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93873552,"statfs":{"total":96636764160,"available":96126517248,"internally_reserved":0,"allocated":118190080,"data_stored":117860627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":392031919},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,0,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":181071872,"data_stored":181068393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":117030912,"data_stored":116978134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":211292160,"data_stored":211284609,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:03.533 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:04.345 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:21:04.348 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:21:04.403 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":342,"stamp":"2024-09-11T01:21:02.760394+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":147735188,"num_objects":36072,"num_object_clones":0,"num_object_copies":72144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35102,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36072,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":246219,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12538,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160870,"ondisk_log_size":160870,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1631948,"kb_used_data":481948,"kb_used_omap":12,"kb_used_meta":1149939,"kb_avail":281483572,"statfs":{"total":289910292480,"available":288239177728,"internally_reserved":0,"allocated":493514752,"data_stored":492624996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1177537540},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,6,3,13,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25862144,"num_objects":-6314,"num_object_clones":0,"num_object_copies":-12628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6314,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6314,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.013349"},"pg_stats":[{"pgid":"2.f","version":"49'18042","reported_seq":18129,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:02.590122+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:21:02.590122+0000","last_peered":"2024-09-11T01:21:02.590122+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:21:02.590122+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9109504,"num_objects":2224,"num_object_clones":0,"num_object_copies":4448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18042,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1848}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'17992","reported_seq":18103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:59.992855+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:20:59.992855+0000","last_peered":"2024-09-11T01:20:59.992855+0000","last_clean":"2024-09-11T01:20:59.992855+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:20:59.992855+0000","last_undegraded":"2024-09-11T01:20:59.992855+0000","last_fullsized":"2024-09-11T01:20:59.992855+0000","mapping_epoch":35,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9666560,"num_objects":2360,"num_object_clones":0,"num_object_copies":4720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16231,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'17751","reported_seq":17889,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:00.066354+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:21:00.066354+0000","last_peered":"2024-09-11T01:21:00.066354+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:21:00.066354+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9797656,"num_objects":2393,"num_object_clones":0,"num_object_copies":4786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2393,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16052,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1503}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'17785","reported_seq":17864,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:00.016285+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:21:00.016285+0000","last_peered":"2024-09-11T01:21:00.016285+0000","last_clean":"2024-09-11T01:21:00.016285+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:21:00.016285+0000","last_undegraded":"2024-09-11T01:21:00.016285+0000","last_fullsized":"2024-09-11T01:21:00.016285+0000","mapping_epoch":35,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9621504,"num_objects":2349,"num_object_clones":0,"num_object_copies":4698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17785,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'17814","reported_seq":17952,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:02.584967+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:21:02.584967+0000","last_peered":"2024-09-11T01:21:02.584967+0000","last_clean":"2024-09-11T01:21:02.584967+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:21:02.584967+0000","last_undegraded":"2024-09-11T01:21:02.584967+0000","last_fullsized":"2024-09-11T01:21:02.584967+0000","mapping_epoch":34,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8990720,"num_objects":2195,"num_object_clones":0,"num_object_copies":4390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14819,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'17976","reported_seq":18156,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:02.630404+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:21:02.630404+0000","last_peered":"2024-09-11T01:21:02.630404+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:21:02.630404+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9220096,"num_objects":2251,"num_object_clones":0,"num_object_copies":4502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14317,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1700}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'17710","reported_seq":17846,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:02.532283+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:21:02.532283+0000","last_peered":"2024-09-11T01:21:02.532283+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:21:02.532283+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9093120,"num_objects":2220,"num_object_clones":0,"num_object_copies":4440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14567,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1766}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18022","reported_seq":18162,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:02.565989+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:21:02.565989+0000","last_peered":"2024-09-11T01:21:02.565989+0000","last_clean":"2024-09-11T01:21:02.565989+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:21:02.565989+0000","last_undegraded":"2024-09-11T01:21:02.565989+0000","last_fullsized":"2024-09-11T01:21:02.565989+0000","mapping_epoch":34,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9142272,"num_objects":2232,"num_object_clones":0,"num_object_copies":4464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2232,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14898,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003131+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003131+0000","last_peered":"2024-09-11T01:20:22.003131+0000","last_clean":"2024-09-11T01:20:22.003131+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003131+0000","last_undegraded":"2024-09-11T01:20:22.003131+0000","last_fullsized":"2024-09-11T01:20:22.003131+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'17740","reported_seq":25758,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:02.337917+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:21:02.337917+0000","last_peered":"2024-09-11T01:21:02.337917+0000","last_clean":"2024-09-11T01:21:02.337917+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:21:02.337917+0000","last_undegraded":"2024-09-11T01:21:02.337917+0000","last_fullsized":"2024-09-11T01:21:02.337917+0000","mapping_epoch":48,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9146368,"num_objects":2233,"num_object_clones":0,"num_object_copies":4466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15711,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'17685","reported_seq":17823,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:02.628739+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:21:02.628739+0000","last_peered":"2024-09-11T01:21:02.628739+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:21:02.628739+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8941568,"num_objects":2183,"num_object_clones":0,"num_object_copies":4366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14272,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1801}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'17837","reported_seq":17966,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:00.040174+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:21:00.040174+0000","last_peered":"2024-09-11T01:21:00.040174+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:21:00.040174+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9707520,"num_objects":2370,"num_object_clones":0,"num_object_copies":4740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15654,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1654}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003225+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003225+0000","last_peered":"2024-09-11T01:20:22.003225+0000","last_clean":"2024-09-11T01:20:22.003225+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003225+0000","last_undegraded":"2024-09-11T01:20:22.003225+0000","last_fullsized":"2024-09-11T01:20:22.003225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'17567","reported_seq":25558,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:00.039446+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:21:00.039446+0000","last_peered":"2024-09-11T01:21:00.039446+0000","last_clean":"2024-09-11T01:21:00.039446+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:21:00.039446+0000","last_undegraded":"2024-09-11T01:21:00.039446+0000","last_fullsized":"2024-09-11T01:21:00.039446+0000","mapping_epoch":44,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9129984,"num_objects":2229,"num_object_clones":0,"num_object_copies":4458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15352,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18030","reported_seq":18172,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:02.591252+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:21:02.591252+0000","last_peered":"2024-09-11T01:21:02.591252+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:21:02.591252+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9150464,"num_objects":2234,"num_object_clones":0,"num_object_copies":4468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14767,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1716}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007532+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007532+0000","last_peered":"2024-09-11T01:20:22.007532+0000","last_clean":"2024-09-11T01:20:22.007532+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007532+0000","last_undegraded":"2024-09-11T01:20:22.007532+0000","last_fullsized":"2024-09-11T01:20:22.007532+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'17660","reported_seq":25701,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:02.324848+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:21:02.324848+0000","last_peered":"2024-09-11T01:21:02.324848+0000","last_clean":"2024-09-11T01:21:02.324848+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:21:02.324848+0000","last_undegraded":"2024-09-11T01:21:02.324848+0000","last_fullsized":"2024-09-11T01:21:02.324848+0000","mapping_epoch":46,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9109504,"num_objects":2224,"num_object_clones":0,"num_object_copies":4448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14315,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'17961","reported_seq":19569,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:02.673133+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:21:02.673133+0000","last_peered":"2024-09-11T01:21:02.673133+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:21:02.673133+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9072640,"num_objects":2215,"num_object_clones":0,"num_object_copies":4430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3309,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2215,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14570,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":732,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007560+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007560+0000","last_peered":"2024-09-11T01:20:22.007560+0000","last_clean":"2024-09-11T01:20:22.007560+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007560+0000","last_undegraded":"2024-09-11T01:20:22.007560+0000","last_fullsized":"2024-09-11T01:20:22.007560+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008171+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008171+0000","last_peered":"2024-09-11T01:20:22.008171+0000","last_clean":"2024-09-11T01:20:22.008171+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008171+0000","last_undegraded":"2024-09-11T01:20:22.008171+0000","last_fullsized":"2024-09-11T01:20:22.008171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'17795","reported_seq":17938,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:02.622440+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:21:02.622440+0000","last_peered":"2024-09-11T01:21:02.622440+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:21:02.622440+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8835689,"num_objects":2158,"num_object_clones":0,"num_object_copies":4316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14865,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1759}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":147735169,"num_objects":36070,"num_object_clones":0,"num_object_copies":72140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35102,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36070,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":246217,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12537,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":490098688,"data_stored":490034880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160867,"ondisk_log_size":160867,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182514,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608188,"kb_used_data":171180,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93763652,"statfs":{"total":96636764160,"available":96013979648,"internally_reserved":0,"allocated":175288320,"data_stored":175008166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":447475371},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,3,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986286,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":540852,"kb_used_data":202404,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93830988,"statfs":{"total":96636764160,"available":96082931712,"internally_reserved":0,"allocated":207261696,"data_stored":206981547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659231,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482908,"kb_used_data":108364,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93888932,"statfs":{"total":96636764160,"available":96142266368,"internally_reserved":0,"allocated":110964736,"data_stored":110635283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":383512239},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,0,6,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":173809664,"data_stored":173806185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":109809664,"data_stored":109756886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":206479360,"data_stored":206471809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:04.405 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:05.214 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:21:05.216 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:21:05.273 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2024-09-11T01:21:04.760841+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":147735188,"num_objects":36072,"num_object_clones":0,"num_object_copies":72144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35102,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36072,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":246219,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12538,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160870,"ondisk_log_size":160870,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1631948,"kb_used_data":481948,"kb_used_omap":12,"kb_used_meta":1149939,"kb_avail":281483572,"statfs":{"total":289910292480,"available":288239177728,"internally_reserved":0,"allocated":493514752,"data_stored":492624996,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1177537540},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,2,6,3,13,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19230720,"num_objects":-4695,"num_object_clones":0,"num_object_copies":-9390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-185,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4695,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4695,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.013351"},"pg_stats":[{"pgid":"2.f","version":"49'18042","reported_seq":18129,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:02.590122+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:21:02.590122+0000","last_peered":"2024-09-11T01:21:02.590122+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:21:02.590122+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9109504,"num_objects":2224,"num_object_clones":0,"num_object_copies":4448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18042,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1848}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'17992","reported_seq":18103,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:59.992855+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:20:59.992855+0000","last_peered":"2024-09-11T01:20:59.992855+0000","last_clean":"2024-09-11T01:20:59.992855+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:20:59.992855+0000","last_undegraded":"2024-09-11T01:20:59.992855+0000","last_fullsized":"2024-09-11T01:20:59.992855+0000","mapping_epoch":35,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9666560,"num_objects":2360,"num_object_clones":0,"num_object_copies":4720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16231,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'17751","reported_seq":17889,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:00.066354+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:21:00.066354+0000","last_peered":"2024-09-11T01:21:00.066354+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:21:00.066354+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9797656,"num_objects":2393,"num_object_clones":0,"num_object_copies":4786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2393,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16052,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1503}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'17785","reported_seq":17864,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:00.016285+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:21:00.016285+0000","last_peered":"2024-09-11T01:21:00.016285+0000","last_clean":"2024-09-11T01:21:00.016285+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:21:00.016285+0000","last_undegraded":"2024-09-11T01:21:00.016285+0000","last_fullsized":"2024-09-11T01:21:00.016285+0000","mapping_epoch":35,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9621504,"num_objects":2349,"num_object_clones":0,"num_object_copies":4698,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2349,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17785,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'17814","reported_seq":17952,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:02.584967+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:21:02.584967+0000","last_peered":"2024-09-11T01:21:02.584967+0000","last_clean":"2024-09-11T01:21:02.584967+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:21:02.584967+0000","last_undegraded":"2024-09-11T01:21:02.584967+0000","last_fullsized":"2024-09-11T01:21:02.584967+0000","mapping_epoch":34,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8990720,"num_objects":2195,"num_object_clones":0,"num_object_copies":4390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14819,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'17976","reported_seq":18156,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:02.630404+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:21:02.630404+0000","last_peered":"2024-09-11T01:21:02.630404+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:21:02.630404+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9220096,"num_objects":2251,"num_object_clones":0,"num_object_copies":4502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14317,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1700}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'17710","reported_seq":17846,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:02.532283+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:21:02.532283+0000","last_peered":"2024-09-11T01:21:02.532283+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:21:02.532283+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9093120,"num_objects":2220,"num_object_clones":0,"num_object_copies":4440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14567,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1766}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18022","reported_seq":18162,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:02.565989+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:21:02.565989+0000","last_peered":"2024-09-11T01:21:02.565989+0000","last_clean":"2024-09-11T01:21:02.565989+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:21:02.565989+0000","last_undegraded":"2024-09-11T01:21:02.565989+0000","last_fullsized":"2024-09-11T01:21:02.565989+0000","mapping_epoch":34,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9142272,"num_objects":2232,"num_object_clones":0,"num_object_copies":4464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2232,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14898,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003131+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003131+0000","last_peered":"2024-09-11T01:20:22.003131+0000","last_clean":"2024-09-11T01:20:22.003131+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003131+0000","last_undegraded":"2024-09-11T01:20:22.003131+0000","last_fullsized":"2024-09-11T01:20:22.003131+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'17740","reported_seq":25758,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:02.337917+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:21:02.337917+0000","last_peered":"2024-09-11T01:21:02.337917+0000","last_clean":"2024-09-11T01:21:02.337917+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:21:02.337917+0000","last_undegraded":"2024-09-11T01:21:02.337917+0000","last_fullsized":"2024-09-11T01:21:02.337917+0000","mapping_epoch":48,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9146368,"num_objects":2233,"num_object_clones":0,"num_object_copies":4466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15711,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'17685","reported_seq":17823,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:02.628739+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:21:02.628739+0000","last_peered":"2024-09-11T01:21:02.628739+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:21:02.628739+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8941568,"num_objects":2183,"num_object_clones":0,"num_object_copies":4366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14272,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1801}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'17837","reported_seq":17966,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:00.040174+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:21:00.040174+0000","last_peered":"2024-09-11T01:21:00.040174+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:21:00.040174+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9707520,"num_objects":2370,"num_object_clones":0,"num_object_copies":4740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2370,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15654,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1654}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003225+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003225+0000","last_peered":"2024-09-11T01:20:22.003225+0000","last_clean":"2024-09-11T01:20:22.003225+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003225+0000","last_undegraded":"2024-09-11T01:20:22.003225+0000","last_fullsized":"2024-09-11T01:20:22.003225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'17567","reported_seq":25558,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:00.039446+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:21:00.039446+0000","last_peered":"2024-09-11T01:21:00.039446+0000","last_clean":"2024-09-11T01:21:00.039446+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:21:00.039446+0000","last_undegraded":"2024-09-11T01:21:00.039446+0000","last_fullsized":"2024-09-11T01:21:00.039446+0000","mapping_epoch":44,"log_start":"43'7500","ondisk_log_start":"43'7500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9129984,"num_objects":2229,"num_object_clones":0,"num_object_copies":4458,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2229,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15352,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18030","reported_seq":18172,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:02.591252+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:21:02.591252+0000","last_peered":"2024-09-11T01:21:02.591252+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:21:02.591252+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9150464,"num_objects":2234,"num_object_clones":0,"num_object_copies":4468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14767,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1716}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007532+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007532+0000","last_peered":"2024-09-11T01:20:22.007532+0000","last_clean":"2024-09-11T01:20:22.007532+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007532+0000","last_undegraded":"2024-09-11T01:20:22.007532+0000","last_fullsized":"2024-09-11T01:20:22.007532+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'17660","reported_seq":25701,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:02.324848+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:21:02.324848+0000","last_peered":"2024-09-11T01:21:02.324848+0000","last_clean":"2024-09-11T01:21:02.324848+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:21:02.324848+0000","last_undegraded":"2024-09-11T01:21:02.324848+0000","last_fullsized":"2024-09-11T01:21:02.324848+0000","mapping_epoch":46,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9109504,"num_objects":2224,"num_object_clones":0,"num_object_copies":4448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14315,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'17961","reported_seq":19569,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:02.673133+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:21:02.673133+0000","last_peered":"2024-09-11T01:21:02.673133+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:21:02.673133+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9072640,"num_objects":2215,"num_object_clones":0,"num_object_copies":4430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3309,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2215,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14570,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":732,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007560+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007560+0000","last_peered":"2024-09-11T01:20:22.007560+0000","last_clean":"2024-09-11T01:20:22.007560+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007560+0000","last_undegraded":"2024-09-11T01:20:22.007560+0000","last_fullsized":"2024-09-11T01:20:22.007560+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008171+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008171+0000","last_peered":"2024-09-11T01:20:22.008171+0000","last_clean":"2024-09-11T01:20:22.008171+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008171+0000","last_undegraded":"2024-09-11T01:20:22.008171+0000","last_fullsized":"2024-09-11T01:20:22.008171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'17795","reported_seq":17938,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:02.622440+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:21:02.622440+0000","last_peered":"2024-09-11T01:21:02.622440+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:21:02.622440+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8835689,"num_objects":2158,"num_object_clones":0,"num_object_copies":4316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14865,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1759}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":147735169,"num_objects":36070,"num_object_clones":0,"num_object_copies":72140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35102,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36070,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":246217,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12537,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":490098688,"data_stored":490034880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160867,"ondisk_log_size":160867,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182514,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608188,"kb_used_data":171180,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93763652,"statfs":{"total":96636764160,"available":96013979648,"internally_reserved":0,"allocated":175288320,"data_stored":175008166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":447475371},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,3,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986286,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":540852,"kb_used_data":202404,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93830988,"statfs":{"total":96636764160,"available":96082931712,"internally_reserved":0,"allocated":207261696,"data_stored":206981547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1],"upper_bound":16},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659231,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482908,"kb_used_data":108364,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93888932,"statfs":{"total":96636764160,"available":96142266368,"internally_reserved":0,"allocated":110964736,"data_stored":110635283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":383512239},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,0,6,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":173809664,"data_stored":173806185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":109809664,"data_stored":109756886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":206479360,"data_stored":206471809,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:05.274 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:21:05.275 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'18042", 'reported_seq': 18129, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:02.590122+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:21:02.590122+0000', 'last_peered': '2024-09-11T01:21:02.590122+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:21:02.590122+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "43'8000", 'ondisk_log_start': "43'8000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9109504, 'num_objects': 2224, 'num_object_clones': 0, 'num_object_copies': 4448, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2224, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18042, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1848}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:05.275 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:21:05.275 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'17751", 'reported_seq': 17889, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:00.066354+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:21:00.066354+0000', 'last_peered': '2024-09-11T01:21:00.066354+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:21:00.066354+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "43'7700", 'ondisk_log_start': "43'7700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10051, 'log_dups_size': 0, 'ondisk_log_size': 10051, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9797656, 'num_objects': 2393, 'num_object_clones': 0, 'num_object_copies': 4786, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2393, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16052, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1503}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:05.275 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:21:05.275 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'17976", 'reported_seq': 18156, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:02.630404+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:21:02.630404+0000', 'last_peered': '2024-09-11T01:21:02.630404+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:21:02.630404+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "43'7900", 'ondisk_log_start': "43'7900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9220096, 'num_objects': 2251, 'num_object_clones': 0, 'num_object_copies': 4502, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2251, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14317, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1700}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:05.275 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:21:05.275 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'17710", 'reported_seq': 17846, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:02.532283+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:21:02.532283+0000', 'last_peered': '2024-09-11T01:21:02.532283+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:21:02.532283+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "43'7700", 'ondisk_log_start': "43'7700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10010, 'log_dups_size': 0, 'ondisk_log_size': 10010, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9093120, 'num_objects': 2220, 'num_object_clones': 0, 'num_object_copies': 4440, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2220, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14567, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1766}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:05.275 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:21:05.275 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'17685", 'reported_seq': 17823, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:02.628739+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:21:02.628739+0000', 'last_peered': '2024-09-11T01:21:02.628739+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:21:02.628739+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "43'7600", 'ondisk_log_start': "43'7600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8941568, 'num_objects': 2183, 'num_object_clones': 0, 'num_object_copies': 4366, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2183, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14272, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1801}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:05.275 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:21:05.276 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'17837", 'reported_seq': 17966, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:00.040174+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:21:00.040174+0000', 'last_peered': '2024-09-11T01:21:00.040174+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:21:00.040174+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "43'7800", 'ondisk_log_start': "43'7800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9707520, 'num_objects': 2370, 'num_object_clones': 0, 'num_object_copies': 4740, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2370, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15654, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1654}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:05.276 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:21:05.276 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'18030", 'reported_seq': 18172, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:02.591252+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:21:02.591252+0000', 'last_peered': '2024-09-11T01:21:02.591252+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:21:02.591252+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "43'8000", 'ondisk_log_start': "43'8000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9150464, 'num_objects': 2234, 'num_object_clones': 0, 'num_object_copies': 4468, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2234, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14767, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1716}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:05.276 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:21:05.276 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'17961", 'reported_seq': 19569, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:02.673133+0000', 'last_change': '2024-09-11T01:20:20.985862+0000', 'last_active': '2024-09-11T01:21:02.673133+0000', 'last_peered': '2024-09-11T01:21:02.673133+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:21:02.673133+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "43'7900", 'ondisk_log_start': "43'7900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 9072640, 'num_objects': 2215, 'num_object_clones': 0, 'num_object_copies': 4430, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3309, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2215, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14570, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 732, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1094}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:05.276 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:21:05.276 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'17795", 'reported_seq': 17938, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:02.622440+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:21:02.622440+0000', 'last_peered': '2024-09-11T01:21:02.622440+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:21:02.622440+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "43'7700", 'ondisk_log_start': "43'7700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10095, 'log_dups_size': 0, 'ondisk_log_size': 10095, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8835689, 'num_objects': 2158, 'num_object_clones': 0, 'num_object_copies': 4316, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2158, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14865, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1759}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:05.277 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:21:06.283 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:21:06.284 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35102/72144 objects degraded (48.655%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":602,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":36072,"data_bytes":147735188,"bytes_used":1671114752,"bytes_avail":288239177728,"bytes_total":289910292480,"degraded_objects":35102,"degraded_total":72144,"degraded_ratio":0.48655466844089601,"recovering_objects_per_sec":15,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":185,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":390},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:21:06.335 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:21:06.336 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:07.117 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:21:07.118 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:21:07.206 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":344,"stamp":"2024-09-11T01:21:06.761248+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":144732820,"num_objects":35339,"num_object_clones":0,"num_object_copies":70678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35102,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35339,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":246952,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12538,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160903,"ondisk_log_size":160903,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1631476,"kb_used_data":477316,"kb_used_omap":12,"kb_used_meta":1154099,"kb_avail":281484044,"statfs":{"total":289910292480,"available":288239661056,"internally_reserved":0,"allocated":488771584,"data_stored":487881828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1181797380},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,3,2,7,4,14,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22233088,"num_objects":-5428,"num_object_clones":0,"num_object_copies":-10856,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-185,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5428,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5428,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":185,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.013283"},"pg_stats":[{"pgid":"2.f","version":"49'18042","reported_seq":18129,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:02.590122+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:21:02.590122+0000","last_peered":"2024-09-11T01:21:02.590122+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:21:02.590122+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9109504,"num_objects":2224,"num_object_clones":0,"num_object_copies":4448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18042,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1848}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18138","reported_seq":18249,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:05.049433+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:21:05.049433+0000","last_peered":"2024-09-11T01:21:05.049433+0000","last_clean":"2024-09-11T01:21:05.049433+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:21:05.049433+0000","last_undegraded":"2024-09-11T01:21:05.049433+0000","last_fullsized":"2024-09-11T01:21:05.049433+0000","mapping_epoch":35,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9068544,"num_objects":2214,"num_object_clones":0,"num_object_copies":4428,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2214,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16377,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'17885","reported_seq":18020,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:05.035095+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:21:05.035095+0000","last_peered":"2024-09-11T01:21:05.035095+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:21:05.035095+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9248792,"num_objects":2259,"num_object_clones":0,"num_object_copies":4518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2259,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16186,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1637}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'17948","reported_seq":18027,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:05.039295+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:21:05.039295+0000","last_peered":"2024-09-11T01:21:05.039295+0000","last_clean":"2024-09-11T01:21:05.039295+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:21:05.039295+0000","last_undegraded":"2024-09-11T01:21:05.039295+0000","last_fullsized":"2024-09-11T01:21:05.039295+0000","mapping_epoch":35,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8953856,"num_objects":2186,"num_object_clones":0,"num_object_copies":4372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17948,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'17814","reported_seq":17952,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:02.584967+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:21:02.584967+0000","last_peered":"2024-09-11T01:21:02.584967+0000","last_clean":"2024-09-11T01:21:02.584967+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:21:02.584967+0000","last_undegraded":"2024-09-11T01:21:02.584967+0000","last_fullsized":"2024-09-11T01:21:02.584967+0000","mapping_epoch":34,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8990720,"num_objects":2195,"num_object_clones":0,"num_object_copies":4390,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2195,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14819,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'17976","reported_seq":18156,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:02.630404+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:21:02.630404+0000","last_peered":"2024-09-11T01:21:02.630404+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:21:02.630404+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9220096,"num_objects":2251,"num_object_clones":0,"num_object_copies":4502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14317,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1700}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'17710","reported_seq":17846,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:02.532283+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:21:02.532283+0000","last_peered":"2024-09-11T01:21:02.532283+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:21:02.532283+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9093120,"num_objects":2220,"num_object_clones":0,"num_object_copies":4440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2220,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14567,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1766}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18022","reported_seq":18162,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:02.565989+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:21:02.565989+0000","last_peered":"2024-09-11T01:21:02.565989+0000","last_clean":"2024-09-11T01:21:02.565989+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:21:02.565989+0000","last_undegraded":"2024-09-11T01:21:02.565989+0000","last_fullsized":"2024-09-11T01:21:02.565989+0000","mapping_epoch":34,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9142272,"num_objects":2232,"num_object_clones":0,"num_object_copies":4464,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2232,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14898,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003131+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003131+0000","last_peered":"2024-09-11T01:20:22.003131+0000","last_clean":"2024-09-11T01:20:22.003131+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003131+0000","last_undegraded":"2024-09-11T01:20:22.003131+0000","last_fullsized":"2024-09-11T01:20:22.003131+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'17740","reported_seq":25758,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:02.337917+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:21:02.337917+0000","last_peered":"2024-09-11T01:21:02.337917+0000","last_clean":"2024-09-11T01:21:02.337917+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:21:02.337917+0000","last_undegraded":"2024-09-11T01:21:02.337917+0000","last_fullsized":"2024-09-11T01:21:02.337917+0000","mapping_epoch":48,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9146368,"num_objects":2233,"num_object_clones":0,"num_object_copies":4466,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2233,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15711,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'17685","reported_seq":17823,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:02.628739+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:21:02.628739+0000","last_peered":"2024-09-11T01:21:02.628739+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:21:02.628739+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8941568,"num_objects":2183,"num_object_clones":0,"num_object_copies":4366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14272,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1801}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'17998","reported_seq":18127,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:04.978671+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:21:04.978671+0000","last_peered":"2024-09-11T01:21:04.978671+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:21:04.978671+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9048064,"num_objects":2209,"num_object_clones":0,"num_object_copies":4418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2209,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15815,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1815}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003225+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003225+0000","last_peered":"2024-09-11T01:20:22.003225+0000","last_clean":"2024-09-11T01:20:22.003225+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003225+0000","last_undegraded":"2024-09-11T01:20:22.003225+0000","last_fullsized":"2024-09-11T01:20:22.003225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'17696","reported_seq":25687,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:04.913412+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:21:04.913412+0000","last_peered":"2024-09-11T01:21:04.913412+0000","last_clean":"2024-09-11T01:21:04.913412+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:21:04.913412+0000","last_undegraded":"2024-09-11T01:21:04.913412+0000","last_fullsized":"2024-09-11T01:21:04.913412+0000","mapping_epoch":44,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8601600,"num_objects":2100,"num_object_clones":0,"num_object_copies":4200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2100,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15481,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18030","reported_seq":18172,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:02.591252+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:21:02.591252+0000","last_peered":"2024-09-11T01:21:02.591252+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:21:02.591252+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9150464,"num_objects":2234,"num_object_clones":0,"num_object_copies":4468,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2234,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14767,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1716}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007532+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007532+0000","last_peered":"2024-09-11T01:20:22.007532+0000","last_clean":"2024-09-11T01:20:22.007532+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007532+0000","last_undegraded":"2024-09-11T01:20:22.007532+0000","last_fullsized":"2024-09-11T01:20:22.007532+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'17660","reported_seq":25701,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:02.324848+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:21:02.324848+0000","last_peered":"2024-09-11T01:21:02.324848+0000","last_clean":"2024-09-11T01:21:02.324848+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:21:02.324848+0000","last_undegraded":"2024-09-11T01:21:02.324848+0000","last_fullsized":"2024-09-11T01:21:02.324848+0000","mapping_epoch":46,"log_start":"43'7600","ondisk_log_start":"43'7600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9109504,"num_objects":2224,"num_object_clones":0,"num_object_copies":4448,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2224,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14315,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'17961","reported_seq":19569,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:02.673133+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:21:02.673133+0000","last_peered":"2024-09-11T01:21:02.673133+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:21:02.673133+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":9072640,"num_objects":2215,"num_object_clones":0,"num_object_copies":4430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3309,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2215,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14570,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":732,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1094}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007560+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007560+0000","last_peered":"2024-09-11T01:20:22.007560+0000","last_clean":"2024-09-11T01:20:22.007560+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007560+0000","last_undegraded":"2024-09-11T01:20:22.007560+0000","last_fullsized":"2024-09-11T01:20:22.007560+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008171+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008171+0000","last_peered":"2024-09-11T01:20:22.008171+0000","last_clean":"2024-09-11T01:20:22.008171+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008171+0000","last_undegraded":"2024-09-11T01:20:22.008171+0000","last_fullsized":"2024-09-11T01:20:22.008171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'17795","reported_seq":17938,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:02.622440+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:21:02.622440+0000","last_peered":"2024-09-11T01:21:02.622440+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:21:02.622440+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'7700","ondisk_log_start":"43'7700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8835689,"num_objects":2158,"num_object_clones":0,"num_object_copies":4316,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2158,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14865,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1759}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":144732801,"num_objects":35337,"num_object_clones":0,"num_object_copies":70674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35102,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35337,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":246950,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12537,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":485388288,"data_stored":485324480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160900,"ondisk_log_size":160900,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182514,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":608188,"kb_used_data":171180,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93763652,"statfs":{"total":96636764160,"available":96013979648,"internally_reserved":0,"allocated":175288320,"data_stored":175008166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":447475371},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,1,3,7,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986287,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":540380,"kb_used_data":197772,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93831460,"statfs":{"total":96636764160,"available":96083415040,"internally_reserved":0,"allocated":202518528,"data_stored":202238379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,1,1,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659231,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482908,"kb_used_data":108364,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93888932,"statfs":{"total":96636764160,"available":96142266368,"internally_reserved":0,"allocated":110964736,"data_stored":110635283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":383512239},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,5,0,6,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":173809664,"data_stored":173806185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":109809664,"data_stored":109756886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":201768960,"data_stored":201761409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:10.209 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:10.981 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:21:10.982 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:21:11.025 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-09-11T01:21:10.762126+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":135197332,"num_objects":33011,"num_object_clones":0,"num_object_copies":66022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35018,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33011,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":249280,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12622,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160831,"ondisk_log_size":160831,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1617108,"kb_used_data":458784,"kb_used_omap":12,"kb_used_meta":1158259,"kb_avail":281498412,"statfs":{"total":289910292480,"available":288254373888,"internally_reserved":0,"allocated":469794816,"data_stored":468905060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1186057220},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,6,6,6,5,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22102016,"num_objects":-5396,"num_object_clones":0,"num_object_copies":-10792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-172,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5396,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007777"},"pg_stats":[{"pgid":"2.f","version":"49'18200","reported_seq":18288,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:07.692111+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:21:07.692111+0000","last_peered":"2024-09-11T01:21:07.692111+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:21:07.692111+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8462336,"num_objects":2066,"num_object_clones":0,"num_object_copies":4132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18200,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2006}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18300","reported_seq":18412,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:10.055024+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:21:10.055024+0000","last_peered":"2024-09-11T01:21:10.055024+0000","last_clean":"2024-09-11T01:21:10.055024+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:21:10.055024+0000","last_undegraded":"2024-09-11T01:21:10.055024+0000","last_fullsized":"2024-09-11T01:21:10.055024+0000","mapping_epoch":35,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8404992,"num_objects":2052,"num_object_clones":0,"num_object_copies":4104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18035","reported_seq":18173,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:10.031517+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:21:10.031517+0000","last_peered":"2024-09-11T01:21:10.031517+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:21:10.031517+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8634392,"num_objects":2109,"num_object_clones":0,"num_object_copies":4218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2109,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16336,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1787}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'18099","reported_seq":18178,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:10.044932+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:21:10.044932+0000","last_peered":"2024-09-11T01:21:10.044932+0000","last_clean":"2024-09-11T01:21:10.044932+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:21:10.044932+0000","last_undegraded":"2024-09-11T01:21:10.044932+0000","last_fullsized":"2024-09-11T01:21:10.044932+0000","mapping_epoch":35,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8335360,"num_objects":2035,"num_object_clones":0,"num_object_copies":4070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18099,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'17951","reported_seq":18088,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:07.678951+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:21:07.678951+0000","last_peered":"2024-09-11T01:21:07.678951+0000","last_clean":"2024-09-11T01:21:07.678951+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:21:07.678951+0000","last_undegraded":"2024-09-11T01:21:07.678951+0000","last_fullsized":"2024-09-11T01:21:07.678951+0000","mapping_epoch":34,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8429568,"num_objects":2058,"num_object_clones":0,"num_object_copies":4116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14956,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18117","reported_seq":18297,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:07.632670+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:21:07.632670+0000","last_peered":"2024-09-11T01:21:07.632670+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:21:07.632670+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8642560,"num_objects":2110,"num_object_clones":0,"num_object_copies":4220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14458,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1841}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'17848","reported_seq":17988,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:07.694685+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:21:07.694685+0000","last_peered":"2024-09-11T01:21:07.694685+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:21:07.694685+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8527872,"num_objects":2082,"num_object_clones":0,"num_object_copies":4164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14705,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1904}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18159","reported_seq":18299,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:07.645085+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:21:07.645085+0000","last_peered":"2024-09-11T01:21:07.645085+0000","last_clean":"2024-09-11T01:21:07.645085+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:21:07.645085+0000","last_undegraded":"2024-09-11T01:21:07.645085+0000","last_fullsized":"2024-09-11T01:21:07.645085+0000","mapping_epoch":34,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8581120,"num_objects":2095,"num_object_clones":0,"num_object_copies":4190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2095,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15035,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003131+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003131+0000","last_peered":"2024-09-11T01:20:22.003131+0000","last_clean":"2024-09-11T01:20:22.003131+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003131+0000","last_undegraded":"2024-09-11T01:20:22.003131+0000","last_fullsized":"2024-09-11T01:20:22.003131+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'17877","reported_seq":25895,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:07.332777+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:21:07.332777+0000","last_peered":"2024-09-11T01:21:07.332777+0000","last_clean":"2024-09-11T01:21:07.332777+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:21:07.332777+0000","last_undegraded":"2024-09-11T01:21:07.332777+0000","last_fullsized":"2024-09-11T01:21:07.332777+0000","mapping_epoch":48,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8585216,"num_objects":2096,"num_object_clones":0,"num_object_copies":4192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15848,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'17823","reported_seq":17961,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:07.682180+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:21:07.682180+0000","last_peered":"2024-09-11T01:21:07.682180+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:21:07.682180+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8376320,"num_objects":2045,"num_object_clones":0,"num_object_copies":4090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14410,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1939}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'18158","reported_seq":18283,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:10.006717+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:21:10.006717+0000","last_peered":"2024-09-11T01:21:10.006717+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:21:10.006717+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8392704,"num_objects":2049,"num_object_clones":0,"num_object_copies":4098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15975,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1975}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003225+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003225+0000","last_peered":"2024-09-11T01:20:22.003225+0000","last_clean":"2024-09-11T01:20:22.003225+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003225+0000","last_undegraded":"2024-09-11T01:20:22.003225+0000","last_fullsized":"2024-09-11T01:20:22.003225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'17835","reported_seq":25826,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:10.065676+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:21:10.065676+0000","last_peered":"2024-09-11T01:21:10.065676+0000","last_clean":"2024-09-11T01:21:10.065676+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:21:10.065676+0000","last_undegraded":"2024-09-11T01:21:10.065676+0000","last_fullsized":"2024-09-11T01:21:10.065676+0000","mapping_epoch":44,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8032256,"num_objects":1961,"num_object_clones":0,"num_object_copies":3922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15620,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18182","reported_seq":18324,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:07.646002+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:21:07.646002+0000","last_peered":"2024-09-11T01:21:07.646002+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:21:07.646002+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8527872,"num_objects":2082,"num_object_clones":0,"num_object_copies":4164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14919,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007532+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007532+0000","last_peered":"2024-09-11T01:20:22.007532+0000","last_clean":"2024-09-11T01:20:22.007532+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007532+0000","last_undegraded":"2024-09-11T01:20:22.007532+0000","last_fullsized":"2024-09-11T01:20:22.007532+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'17804","reported_seq":25845,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:07.320235+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:21:07.320235+0000","last_peered":"2024-09-11T01:21:07.320235+0000","last_clean":"2024-09-11T01:21:07.320235+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:21:07.320235+0000","last_undegraded":"2024-09-11T01:21:07.320235+0000","last_fullsized":"2024-09-11T01:21:07.320235+0000","mapping_epoch":46,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8519680,"num_objects":2080,"num_object_clones":0,"num_object_copies":4160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14459,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18102","reported_seq":19878,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:07.676405+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:21:07.676405+0000","last_peered":"2024-09-11T01:21:07.676405+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:21:07.676405+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8495104,"num_objects":2074,"num_object_clones":0,"num_object_copies":4148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3225,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14711,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":816,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1151}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007560+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007560+0000","last_peered":"2024-09-11T01:20:22.007560+0000","last_clean":"2024-09-11T01:20:22.007560+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007560+0000","last_undegraded":"2024-09-11T01:20:22.007560+0000","last_fullsized":"2024-09-11T01:20:22.007560+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008171+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008171+0000","last_peered":"2024-09-11T01:20:22.008171+0000","last_clean":"2024-09-11T01:20:22.008171+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008171+0000","last_undegraded":"2024-09-11T01:20:22.008171+0000","last_fullsized":"2024-09-11T01:20:22.008171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'17938","reported_seq":18082,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:07.638691+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:21:07.638691+0000","last_peered":"2024-09-11T01:21:07.638691+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:21:07.638691+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8249961,"num_objects":2015,"num_object_clones":0,"num_object_copies":4030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15008,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1902}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":135197313,"num_objects":33009,"num_object_clones":0,"num_object_copies":66018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35018,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33009,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":249278,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12621,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":466210816,"data_stored":466147008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160828,"ondisk_log_size":160828,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182515,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":593068,"kb_used_data":164376,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93778772,"statfs":{"total":96636764160,"available":96029462528,"internally_reserved":0,"allocated":168321024,"data_stored":168040870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":438955691},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,1,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986288,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539752,"kb_used_data":192984,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93832088,"statfs":{"total":96636764160,"available":96084058112,"internally_reserved":0,"allocated":197615616,"data_stored":197335467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659232,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":484288,"kb_used_data":101424,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93887552,"statfs":{"total":96636764160,"available":96140853248,"internally_reserved":0,"allocated":103858176,"data_stored":103528723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":392031919},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,4,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":166719488,"data_stored":166716009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":102768640,"data_stored":102715862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":196722688,"data_stored":196715137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:11.027 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:11.777 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:21:11.778 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:21:11.824 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":346,"stamp":"2024-09-11T01:21:10.762126+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":135197332,"num_objects":33011,"num_object_clones":0,"num_object_copies":66022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35018,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33011,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":249280,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12622,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160831,"ondisk_log_size":160831,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1617108,"kb_used_data":458784,"kb_used_omap":12,"kb_used_meta":1158259,"kb_avail":281498412,"statfs":{"total":289910292480,"available":288254373888,"internally_reserved":0,"allocated":469794816,"data_stored":468905060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1186057220},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,6,6,6,5,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22102016,"num_objects":-5396,"num_object_clones":0,"num_object_copies":-10792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-172,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5396,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5396,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":172,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007777"},"pg_stats":[{"pgid":"2.f","version":"49'18200","reported_seq":18288,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:07.692111+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:21:07.692111+0000","last_peered":"2024-09-11T01:21:07.692111+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:21:07.692111+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8462336,"num_objects":2066,"num_object_clones":0,"num_object_copies":4132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18200,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2006}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18300","reported_seq":18412,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:10.055024+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:21:10.055024+0000","last_peered":"2024-09-11T01:21:10.055024+0000","last_clean":"2024-09-11T01:21:10.055024+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:21:10.055024+0000","last_undegraded":"2024-09-11T01:21:10.055024+0000","last_fullsized":"2024-09-11T01:21:10.055024+0000","mapping_epoch":35,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8404992,"num_objects":2052,"num_object_clones":0,"num_object_copies":4104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18035","reported_seq":18173,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:10.031517+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:21:10.031517+0000","last_peered":"2024-09-11T01:21:10.031517+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:21:10.031517+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8634392,"num_objects":2109,"num_object_clones":0,"num_object_copies":4218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2109,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16336,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1787}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'18099","reported_seq":18178,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:10.044932+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:21:10.044932+0000","last_peered":"2024-09-11T01:21:10.044932+0000","last_clean":"2024-09-11T01:21:10.044932+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:21:10.044932+0000","last_undegraded":"2024-09-11T01:21:10.044932+0000","last_fullsized":"2024-09-11T01:21:10.044932+0000","mapping_epoch":35,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8335360,"num_objects":2035,"num_object_clones":0,"num_object_copies":4070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18099,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'17951","reported_seq":18088,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:07.678951+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:21:07.678951+0000","last_peered":"2024-09-11T01:21:07.678951+0000","last_clean":"2024-09-11T01:21:07.678951+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:21:07.678951+0000","last_undegraded":"2024-09-11T01:21:07.678951+0000","last_fullsized":"2024-09-11T01:21:07.678951+0000","mapping_epoch":34,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8429568,"num_objects":2058,"num_object_clones":0,"num_object_copies":4116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2058,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14956,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18117","reported_seq":18297,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:07.632670+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:21:07.632670+0000","last_peered":"2024-09-11T01:21:07.632670+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:21:07.632670+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10017,"log_dups_size":0,"ondisk_log_size":10017,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8642560,"num_objects":2110,"num_object_clones":0,"num_object_copies":4220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2110,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14458,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1841}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'17848","reported_seq":17988,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:07.694685+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:21:07.694685+0000","last_peered":"2024-09-11T01:21:07.694685+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:21:07.694685+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8527872,"num_objects":2082,"num_object_clones":0,"num_object_copies":4164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14705,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1904}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18159","reported_seq":18299,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:07.645085+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:21:07.645085+0000","last_peered":"2024-09-11T01:21:07.645085+0000","last_clean":"2024-09-11T01:21:07.645085+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:21:07.645085+0000","last_undegraded":"2024-09-11T01:21:07.645085+0000","last_fullsized":"2024-09-11T01:21:07.645085+0000","mapping_epoch":34,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8581120,"num_objects":2095,"num_object_clones":0,"num_object_copies":4190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2095,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15035,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003131+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003131+0000","last_peered":"2024-09-11T01:20:22.003131+0000","last_clean":"2024-09-11T01:20:22.003131+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003131+0000","last_undegraded":"2024-09-11T01:20:22.003131+0000","last_fullsized":"2024-09-11T01:20:22.003131+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'17877","reported_seq":25895,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:07.332777+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:21:07.332777+0000","last_peered":"2024-09-11T01:21:07.332777+0000","last_clean":"2024-09-11T01:21:07.332777+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:21:07.332777+0000","last_undegraded":"2024-09-11T01:21:07.332777+0000","last_fullsized":"2024-09-11T01:21:07.332777+0000","mapping_epoch":48,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8585216,"num_objects":2096,"num_object_clones":0,"num_object_copies":4192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2096,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15848,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'17823","reported_seq":17961,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:07.682180+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:21:07.682180+0000","last_peered":"2024-09-11T01:21:07.682180+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:21:07.682180+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8376320,"num_objects":2045,"num_object_clones":0,"num_object_copies":4090,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2045,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14410,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1939}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'18158","reported_seq":18283,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:10.006717+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:21:10.006717+0000","last_peered":"2024-09-11T01:21:10.006717+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:21:10.006717+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8392704,"num_objects":2049,"num_object_clones":0,"num_object_copies":4098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15975,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1975}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003225+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003225+0000","last_peered":"2024-09-11T01:20:22.003225+0000","last_clean":"2024-09-11T01:20:22.003225+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003225+0000","last_undegraded":"2024-09-11T01:20:22.003225+0000","last_fullsized":"2024-09-11T01:20:22.003225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'17835","reported_seq":25826,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:10.065676+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:21:10.065676+0000","last_peered":"2024-09-11T01:21:10.065676+0000","last_clean":"2024-09-11T01:21:10.065676+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:21:10.065676+0000","last_undegraded":"2024-09-11T01:21:10.065676+0000","last_fullsized":"2024-09-11T01:21:10.065676+0000","mapping_epoch":44,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8032256,"num_objects":1961,"num_object_clones":0,"num_object_copies":3922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15620,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18182","reported_seq":18324,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:07.646002+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:21:07.646002+0000","last_peered":"2024-09-11T01:21:07.646002+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:21:07.646002+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8527872,"num_objects":2082,"num_object_clones":0,"num_object_copies":4164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2082,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14919,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1868}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007532+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007532+0000","last_peered":"2024-09-11T01:20:22.007532+0000","last_clean":"2024-09-11T01:20:22.007532+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007532+0000","last_undegraded":"2024-09-11T01:20:22.007532+0000","last_fullsized":"2024-09-11T01:20:22.007532+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'17804","reported_seq":25845,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:07.320235+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:21:07.320235+0000","last_peered":"2024-09-11T01:21:07.320235+0000","last_clean":"2024-09-11T01:21:07.320235+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:21:07.320235+0000","last_undegraded":"2024-09-11T01:21:07.320235+0000","last_fullsized":"2024-09-11T01:21:07.320235+0000","mapping_epoch":46,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8519680,"num_objects":2080,"num_object_clones":0,"num_object_copies":4160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2080,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14459,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18102","reported_seq":19878,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:07.676405+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:21:07.676405+0000","last_peered":"2024-09-11T01:21:07.676405+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:21:07.676405+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8495104,"num_objects":2074,"num_object_clones":0,"num_object_copies":4148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3225,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2074,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14711,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":816,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1151}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007560+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007560+0000","last_peered":"2024-09-11T01:20:22.007560+0000","last_clean":"2024-09-11T01:20:22.007560+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007560+0000","last_undegraded":"2024-09-11T01:20:22.007560+0000","last_fullsized":"2024-09-11T01:20:22.007560+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008171+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008171+0000","last_peered":"2024-09-11T01:20:22.008171+0000","last_clean":"2024-09-11T01:20:22.008171+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008171+0000","last_undegraded":"2024-09-11T01:20:22.008171+0000","last_fullsized":"2024-09-11T01:20:22.008171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'17938","reported_seq":18082,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:07.638691+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:21:07.638691+0000","last_peered":"2024-09-11T01:21:07.638691+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:21:07.638691+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8249961,"num_objects":2015,"num_object_clones":0,"num_object_copies":4030,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2015,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15008,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1902}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":135197313,"num_objects":33009,"num_object_clones":0,"num_object_copies":66018,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":35018,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":33009,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":249278,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12621,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":466210816,"data_stored":466147008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160828,"ondisk_log_size":160828,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182515,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":593068,"kb_used_data":164376,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93778772,"statfs":{"total":96636764160,"available":96029462528,"internally_reserved":0,"allocated":168321024,"data_stored":168040870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":438955691},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,4,1,2,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986288,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539752,"kb_used_data":192984,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93832088,"statfs":{"total":96636764160,"available":96084058112,"internally_reserved":0,"allocated":197615616,"data_stored":197335467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659232,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":484288,"kb_used_data":101424,"kb_used_omap":4,"kb_used_meta":382843,"kb_avail":93887552,"statfs":{"total":96636764160,"available":96140853248,"internally_reserved":0,"allocated":103858176,"data_stored":103528723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":392031919},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,4,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":166719488,"data_stored":166716009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":102768640,"data_stored":102715862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":196722688,"data_stored":196715137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:11.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:21:11.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'18200", 'reported_seq': 18288, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:07.692111+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:21:07.692111+0000', 'last_peered': '2024-09-11T01:21:07.692111+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:21:07.692111+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "43'8100", 'ondisk_log_start': "43'8100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8462336, 'num_objects': 2066, 'num_object_clones': 0, 'num_object_copies': 4132, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2066, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18200, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2006}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:11.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:21:11.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'18035", 'reported_seq': 18173, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:10.031517+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:21:10.031517+0000', 'last_peered': '2024-09-11T01:21:10.031517+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:21:10.031517+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "43'8000", 'ondisk_log_start': "43'8000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8634392, 'num_objects': 2109, 'num_object_clones': 0, 'num_object_copies': 4218, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2109, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16336, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1787}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:11.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:21:11.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'18117", 'reported_seq': 18297, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:07.632670+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:21:07.632670+0000', 'last_peered': '2024-09-11T01:21:07.632670+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:21:07.632670+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "43'8100", 'ondisk_log_start': "43'8100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10017, 'log_dups_size': 0, 'ondisk_log_size': 10017, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8642560, 'num_objects': 2110, 'num_object_clones': 0, 'num_object_copies': 4220, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2110, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14458, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1841}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:11.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:21:11.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'17848", 'reported_seq': 17988, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:07.694685+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:21:07.694685+0000', 'last_peered': '2024-09-11T01:21:07.694685+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:21:07.694685+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "43'7800", 'ondisk_log_start': "43'7800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8527872, 'num_objects': 2082, 'num_object_clones': 0, 'num_object_copies': 4164, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2082, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14705, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1904}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:11.826 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:21:11.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'17823", 'reported_seq': 17961, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:07.682180+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:21:07.682180+0000', 'last_peered': '2024-09-11T01:21:07.682180+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:21:07.682180+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "43'7800", 'ondisk_log_start': "43'7800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8376320, 'num_objects': 2045, 'num_object_clones': 0, 'num_object_copies': 4090, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2045, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14410, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1939}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:11.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:21:11.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'18158", 'reported_seq': 18283, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:10.006717+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:21:10.006717+0000', 'last_peered': '2024-09-11T01:21:10.006717+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:21:10.006717+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "43'8100", 'ondisk_log_start': "43'8100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8392704, 'num_objects': 2049, 'num_object_clones': 0, 'num_object_copies': 4098, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2049, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15975, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1975}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:11.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:21:11.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'18182", 'reported_seq': 18324, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:07.646002+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:21:07.646002+0000', 'last_peered': '2024-09-11T01:21:07.646002+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:21:07.646002+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "43'8100", 'ondisk_log_start': "43'8100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10082, 'log_dups_size': 0, 'ondisk_log_size': 10082, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8527872, 'num_objects': 2082, 'num_object_clones': 0, 'num_object_copies': 4164, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2082, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14919, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1868}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:11.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:21:11.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'18102", 'reported_seq': 19878, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:07.676405+0000', 'last_change': '2024-09-11T01:20:20.985862+0000', 'last_active': '2024-09-11T01:21:07.676405+0000', 'last_peered': '2024-09-11T01:21:07.676405+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:21:07.676405+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "43'8100", 'ondisk_log_start': "43'8100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10002, 'log_dups_size': 0, 'ondisk_log_size': 10002, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8495104, 'num_objects': 2074, 'num_object_clones': 0, 'num_object_copies': 4148, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3225, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2074, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14711, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 816, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1151}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:11.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:21:11.827 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'17938", 'reported_seq': 18082, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:07.638691+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:21:07.638691+0000', 'last_peered': '2024-09-11T01:21:07.638691+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:21:07.638691+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "43'7900", 'ondisk_log_start': "43'7900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8249961, 'num_objects': 2015, 'num_object_clones': 0, 'num_object_copies': 4030, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2015, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15008, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1902}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:11.828 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:21:12.618 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:21:12.618 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 35018/66022 objects degraded (53.040%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":609,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":33011,"data_bytes":135197332,"bytes_used":1655918592,"bytes_avail":288254373888,"bytes_total":289910292480,"degraded_objects":35018,"degraded_total":66022,"degraded_ratio":0.53039895792311653,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":172,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":449},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:21:12.656 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:21:12.656 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:13.499 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:21:13.500 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:21:13.587 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":347,"stamp":"2024-09-11T01:21:12.762557+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":128549524,"num_objects":31388,"num_object_clones":0,"num_object_copies":62776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34928,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31388,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":250903,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12712,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160954,"ondisk_log_size":160954,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1598736,"kb_used_data":444576,"kb_used_omap":12,"kb_used_meta":1154099,"kb_avail":281516784,"statfs":{"total":289910292480,"available":288273186816,"internally_reserved":0,"allocated":455245824,"data_stored":454356068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1181797380},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,8,6,9,1,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25808896,"num_objects":-6301,"num_object_clones":0,"num_object_copies":-12602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-262,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6301,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6301,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":262,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007736"},"pg_stats":[{"pgid":"2.f","version":"49'18357","reported_seq":18445,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:12.736640+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:21:12.736640+0000","last_peered":"2024-09-11T01:21:12.736640+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:21:12.736640+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7819264,"num_objects":1909,"num_object_clones":0,"num_object_copies":3818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18357,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2163}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18300","reported_seq":18412,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:10.055024+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:21:10.055024+0000","last_peered":"2024-09-11T01:21:10.055024+0000","last_clean":"2024-09-11T01:21:10.055024+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:21:10.055024+0000","last_undegraded":"2024-09-11T01:21:10.055024+0000","last_fullsized":"2024-09-11T01:21:10.055024+0000","mapping_epoch":35,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8404992,"num_objects":2052,"num_object_clones":0,"num_object_copies":4104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2052,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18035","reported_seq":18173,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:10.031517+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:21:10.031517+0000","last_peered":"2024-09-11T01:21:10.031517+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:21:10.031517+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8634392,"num_objects":2109,"num_object_clones":0,"num_object_copies":4218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2109,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16336,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1787}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'18099","reported_seq":18178,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:10.044932+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:21:10.044932+0000","last_peered":"2024-09-11T01:21:10.044932+0000","last_clean":"2024-09-11T01:21:10.044932+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:21:10.044932+0000","last_undegraded":"2024-09-11T01:21:10.044932+0000","last_fullsized":"2024-09-11T01:21:10.044932+0000","mapping_epoch":35,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8335360,"num_objects":2035,"num_object_clones":0,"num_object_copies":4070,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2035,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18099,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'18104","reported_seq":18242,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:12.716576+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:21:12.716576+0000","last_peered":"2024-09-11T01:21:12.716576+0000","last_clean":"2024-09-11T01:21:12.716576+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:21:12.716576+0000","last_undegraded":"2024-09-11T01:21:12.716576+0000","last_fullsized":"2024-09-11T01:21:12.716576+0000","mapping_epoch":34,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7802880,"num_objects":1905,"num_object_clones":0,"num_object_copies":3810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15109,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18261","reported_seq":18439,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:12.679810+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:21:12.679810+0000","last_peered":"2024-09-11T01:21:12.679810+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:21:12.679810+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8052736,"num_objects":1966,"num_object_clones":0,"num_object_copies":3932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14602,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1985}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'17993","reported_seq":18132,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:12.612222+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:21:12.612222+0000","last_peered":"2024-09-11T01:21:12.612222+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:21:12.612222+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7933952,"num_objects":1937,"num_object_clones":0,"num_object_copies":3874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1937,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14850,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2049}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18294","reported_seq":18434,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:12.644050+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:21:12.644050+0000","last_peered":"2024-09-11T01:21:12.644050+0000","last_clean":"2024-09-11T01:21:12.644050+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:21:12.644050+0000","last_undegraded":"2024-09-11T01:21:12.644050+0000","last_fullsized":"2024-09-11T01:21:12.644050+0000","mapping_epoch":34,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8028160,"num_objects":1960,"num_object_clones":0,"num_object_copies":3920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15170,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003209+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003209+0000","last_peered":"2024-09-11T01:20:22.003209+0000","last_clean":"2024-09-11T01:20:22.003209+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003209+0000","last_undegraded":"2024-09-11T01:20:22.003209+0000","last_fullsized":"2024-09-11T01:20:22.003209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18052","reported_seq":26069,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:12.464467+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:21:12.464467+0000","last_peered":"2024-09-11T01:21:12.464467+0000","last_clean":"2024-09-11T01:21:12.464467+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:21:12.464467+0000","last_undegraded":"2024-09-11T01:21:12.464467+0000","last_fullsized":"2024-09-11T01:21:12.464467+0000","mapping_epoch":48,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7868416,"num_objects":1921,"num_object_clones":0,"num_object_copies":3842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16023,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'17952","reported_seq":18089,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:12.706230+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:21:12.706230+0000","last_peered":"2024-09-11T01:21:12.706230+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:21:12.706230+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7847936,"num_objects":1916,"num_object_clones":0,"num_object_copies":3832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14539,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2068}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'18158","reported_seq":18283,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:10.006717+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:21:10.006717+0000","last_peered":"2024-09-11T01:21:10.006717+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:21:10.006717+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8392704,"num_objects":2049,"num_object_clones":0,"num_object_copies":4098,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2049,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15975,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1975}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003225+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003225+0000","last_peered":"2024-09-11T01:20:22.003225+0000","last_clean":"2024-09-11T01:20:22.003225+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003225+0000","last_undegraded":"2024-09-11T01:20:22.003225+0000","last_fullsized":"2024-09-11T01:20:22.003225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'17835","reported_seq":25826,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:10.065676+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:21:10.065676+0000","last_peered":"2024-09-11T01:21:10.065676+0000","last_clean":"2024-09-11T01:21:10.065676+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:21:10.065676+0000","last_undegraded":"2024-09-11T01:21:10.065676+0000","last_fullsized":"2024-09-11T01:21:10.065676+0000","mapping_epoch":44,"log_start":"43'7800","ondisk_log_start":"43'7800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8032256,"num_objects":1961,"num_object_clones":0,"num_object_copies":3922,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1961,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15620,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18338","reported_seq":18480,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:12.691509+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:21:12.691509+0000","last_peered":"2024-09-11T01:21:12.691509+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:21:12.691509+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7888896,"num_objects":1926,"num_object_clones":0,"num_object_copies":3852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15075,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2024}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007532+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007532+0000","last_peered":"2024-09-11T01:20:22.007532+0000","last_clean":"2024-09-11T01:20:22.007532+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007532+0000","last_undegraded":"2024-09-11T01:20:22.007532+0000","last_fullsized":"2024-09-11T01:20:22.007532+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'17949","reported_seq":25990,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:12.478605+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:21:12.478605+0000","last_peered":"2024-09-11T01:21:12.478605+0000","last_clean":"2024-09-11T01:21:12.478605+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:21:12.478605+0000","last_undegraded":"2024-09-11T01:21:12.478605+0000","last_fullsized":"2024-09-11T01:21:12.478605+0000","mapping_epoch":46,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7925760,"num_objects":1935,"num_object_clones":0,"num_object_copies":3870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14604,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18247","reported_seq":20205,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:12.686300+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:21:12.686300+0000","last_peered":"2024-09-11T01:21:12.686300+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:21:12.686300+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7901184,"num_objects":1929,"num_object_clones":0,"num_object_copies":3858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3135,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14856,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":906,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1206}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007560+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007560+0000","last_peered":"2024-09-11T01:20:22.007560+0000","last_clean":"2024-09-11T01:20:22.007560+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007560+0000","last_undegraded":"2024-09-11T01:20:22.007560+0000","last_fullsized":"2024-09-11T01:20:22.007560+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008171+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008171+0000","last_peered":"2024-09-11T01:20:22.008171+0000","last_clean":"2024-09-11T01:20:22.008171+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008171+0000","last_undegraded":"2024-09-11T01:20:22.008171+0000","last_fullsized":"2024-09-11T01:20:22.008171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18077","reported_seq":18220,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:12.691594+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:21:12.691594+0000","last_peered":"2024-09-11T01:21:12.691594+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:21:12.691594+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7680617,"num_objects":1876,"num_object_clones":0,"num_object_copies":3752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1876,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15147,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2041}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":128549505,"num_objects":31386,"num_object_clones":0,"num_object_copies":62772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34928,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31386,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":250901,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12711,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":451723264,"data_stored":451659456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160951,"ondisk_log_size":160951,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182516,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":590212,"kb_used_data":157364,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93781628,"statfs":{"total":96636764160,"available":96032387072,"internally_reserved":0,"allocated":161140736,"data_stored":160860582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":443215531},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986288,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":539752,"kb_used_data":192984,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93832088,"statfs":{"total":96636764160,"available":96084058112,"internally_reserved":0,"allocated":197615616,"data_stored":197335467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,2,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659233,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":468772,"kb_used_data":94228,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93903068,"statfs":{"total":96636764160,"available":96156741632,"internally_reserved":0,"allocated":96489472,"data_stored":96160019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":383512239},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,2,6,1,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":159592448,"data_stored":159588969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":95408128,"data_stored":95355350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":196722688,"data_stored":196715137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:16.589 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:17.478 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:21:17.480 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:21:17.536 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2024-09-11T01:21:16.763411+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":125530772,"num_objects":30651,"num_object_clones":0,"num_object_copies":61302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34928,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30651,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":251640,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12712,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160891,"ondisk_log_size":160891,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1581636,"kb_used_data":439956,"kb_used_omap":12,"kb_used_meta":1141619,"kb_avail":281533884,"statfs":{"total":289910292480,"available":288290697216,"internally_reserved":0,"allocated":450514944,"data_stored":449625188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1169017860},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,6,7,12,2,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22204416,"num_objects":-5421,"num_object_clones":0,"num_object_copies":-10842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5421,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002569"},"pg_stats":[{"pgid":"2.f","version":"49'18357","reported_seq":18445,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:12.736640+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:21:12.736640+0000","last_peered":"2024-09-11T01:21:12.736640+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:21:12.736640+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7819264,"num_objects":1909,"num_object_clones":0,"num_object_copies":3818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18357,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2163}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18442","reported_seq":18554,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:14.995099+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:21:14.995099+0000","last_peered":"2024-09-11T01:21:14.995099+0000","last_clean":"2024-09-11T01:21:14.995099+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:21:14.995099+0000","last_undegraded":"2024-09-11T01:21:14.995099+0000","last_fullsized":"2024-09-11T01:21:14.995099+0000","mapping_epoch":35,"log_start":"44'8400","ondisk_log_start":"44'8400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7823360,"num_objects":1910,"num_object_clones":0,"num_object_copies":3820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16681,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18185","reported_seq":18322,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:15.103964+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:21:15.103964+0000","last_peered":"2024-09-11T01:21:15.103964+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:21:15.103964+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8019992,"num_objects":1959,"num_object_clones":0,"num_object_copies":3918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1959,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16486,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1937}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'18244","reported_seq":18324,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:14.988281+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:21:14.988281+0000","last_peered":"2024-09-11T01:21:14.988281+0000","last_clean":"2024-09-11T01:21:14.988281+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:21:14.988281+0000","last_undegraded":"2024-09-11T01:21:14.988281+0000","last_fullsized":"2024-09-11T01:21:14.988281+0000","mapping_epoch":35,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7741440,"num_objects":1890,"num_object_clones":0,"num_object_copies":3780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18244,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'18104","reported_seq":18242,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:12.716576+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:21:12.716576+0000","last_peered":"2024-09-11T01:21:12.716576+0000","last_clean":"2024-09-11T01:21:12.716576+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:21:12.716576+0000","last_undegraded":"2024-09-11T01:21:12.716576+0000","last_fullsized":"2024-09-11T01:21:12.716576+0000","mapping_epoch":34,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7802880,"num_objects":1905,"num_object_clones":0,"num_object_copies":3810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15109,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18261","reported_seq":18439,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:12.679810+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:21:12.679810+0000","last_peered":"2024-09-11T01:21:12.679810+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:21:12.679810+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8052736,"num_objects":1966,"num_object_clones":0,"num_object_copies":3932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14602,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1985}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'17993","reported_seq":18132,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:12.612222+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:21:12.612222+0000","last_peered":"2024-09-11T01:21:12.612222+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:21:12.612222+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7933952,"num_objects":1937,"num_object_clones":0,"num_object_copies":3874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1937,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14850,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2049}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18294","reported_seq":18434,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:12.644050+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:21:12.644050+0000","last_peered":"2024-09-11T01:21:12.644050+0000","last_clean":"2024-09-11T01:21:12.644050+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:21:12.644050+0000","last_undegraded":"2024-09-11T01:21:12.644050+0000","last_fullsized":"2024-09-11T01:21:12.644050+0000","mapping_epoch":34,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8028160,"num_objects":1960,"num_object_clones":0,"num_object_copies":3920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15170,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003209+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003209+0000","last_peered":"2024-09-11T01:20:22.003209+0000","last_clean":"2024-09-11T01:20:22.003209+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003209+0000","last_undegraded":"2024-09-11T01:20:22.003209+0000","last_fullsized":"2024-09-11T01:20:22.003209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18052","reported_seq":26069,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:12.464467+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:21:12.464467+0000","last_peered":"2024-09-11T01:21:12.464467+0000","last_clean":"2024-09-11T01:21:12.464467+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:21:12.464467+0000","last_undegraded":"2024-09-11T01:21:12.464467+0000","last_fullsized":"2024-09-11T01:21:12.464467+0000","mapping_epoch":48,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7868416,"num_objects":1921,"num_object_clones":0,"num_object_copies":3842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16023,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'17952","reported_seq":18089,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:12.706230+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:21:12.706230+0000","last_peered":"2024-09-11T01:21:12.706230+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:21:12.706230+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7847936,"num_objects":1916,"num_object_clones":0,"num_object_copies":3832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14539,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2068}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'18314","reported_seq":18443,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:15.048985+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:21:15.048985+0000","last_peered":"2024-09-11T01:21:15.048985+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:21:15.048985+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7753728,"num_objects":1893,"num_object_clones":0,"num_object_copies":3786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16131,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2131}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003225+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003225+0000","last_peered":"2024-09-11T01:20:22.003225+0000","last_clean":"2024-09-11T01:20:22.003225+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003225+0000","last_undegraded":"2024-09-11T01:20:22.003225+0000","last_fullsized":"2024-09-11T01:20:22.003225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'17979","reported_seq":25970,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:15.078852+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:21:15.078852+0000","last_peered":"2024-09-11T01:21:15.078852+0000","last_clean":"2024-09-11T01:21:15.078852+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:21:15.078852+0000","last_undegraded":"2024-09-11T01:21:15.078852+0000","last_fullsized":"2024-09-11T01:21:15.078852+0000","mapping_epoch":44,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7442432,"num_objects":1817,"num_object_clones":0,"num_object_copies":3634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15764,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18338","reported_seq":18480,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:12.691509+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:21:12.691509+0000","last_peered":"2024-09-11T01:21:12.691509+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:21:12.691509+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7888896,"num_objects":1926,"num_object_clones":0,"num_object_copies":3852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15075,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2024}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007532+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007532+0000","last_peered":"2024-09-11T01:20:22.007532+0000","last_clean":"2024-09-11T01:20:22.007532+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007532+0000","last_undegraded":"2024-09-11T01:20:22.007532+0000","last_fullsized":"2024-09-11T01:20:22.007532+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'17949","reported_seq":25990,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:12.478605+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:21:12.478605+0000","last_peered":"2024-09-11T01:21:12.478605+0000","last_clean":"2024-09-11T01:21:12.478605+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:21:12.478605+0000","last_undegraded":"2024-09-11T01:21:12.478605+0000","last_fullsized":"2024-09-11T01:21:12.478605+0000","mapping_epoch":46,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7925760,"num_objects":1935,"num_object_clones":0,"num_object_copies":3870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14604,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18247","reported_seq":20205,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:12.686300+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:21:12.686300+0000","last_peered":"2024-09-11T01:21:12.686300+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:21:12.686300+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7901184,"num_objects":1929,"num_object_clones":0,"num_object_copies":3858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3135,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14856,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":906,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1206}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007560+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007560+0000","last_peered":"2024-09-11T01:20:22.007560+0000","last_clean":"2024-09-11T01:20:22.007560+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007560+0000","last_undegraded":"2024-09-11T01:20:22.007560+0000","last_fullsized":"2024-09-11T01:20:22.007560+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008171+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008171+0000","last_peered":"2024-09-11T01:20:22.008171+0000","last_clean":"2024-09-11T01:20:22.008171+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008171+0000","last_undegraded":"2024-09-11T01:20:22.008171+0000","last_fullsized":"2024-09-11T01:20:22.008171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18077","reported_seq":18220,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:12.691594+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:21:12.691594+0000","last_peered":"2024-09-11T01:21:12.691594+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:21:12.691594+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7680617,"num_objects":1876,"num_object_clones":0,"num_object_copies":3752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1876,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15147,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2041}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":125530753,"num_objects":30649,"num_object_clones":0,"num_object_copies":61298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34928,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30649,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":251638,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12711,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":447021056,"data_stored":446957248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160888,"ondisk_log_size":160888,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182516,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":590212,"kb_used_data":157364,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93781628,"statfs":{"total":96636764160,"available":96032387072,"internally_reserved":0,"allocated":161140736,"data_stored":160860582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":443215531},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986289,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":522652,"kb_used_data":188364,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93849188,"statfs":{"total":96636764160,"available":96101568512,"internally_reserved":0,"allocated":192884736,"data_stored":192604587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342290090},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659233,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":468772,"kb_used_data":94228,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93903068,"statfs":{"total":96636764160,"available":96156741632,"internally_reserved":0,"allocated":96489472,"data_stored":96160019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":383512239},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,2,6,1,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":159592448,"data_stored":159588969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":95408128,"data_stored":95355350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":192020480,"data_stored":192012929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:17.538 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:18.285 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:21:18.286 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:21:18.323 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":349,"stamp":"2024-09-11T01:21:16.763411+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":125530772,"num_objects":30651,"num_object_clones":0,"num_object_copies":61302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34928,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30651,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":251640,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12712,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160891,"ondisk_log_size":160891,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1581636,"kb_used_data":439956,"kb_used_omap":12,"kb_used_meta":1141619,"kb_avail":281533884,"statfs":{"total":289910292480,"available":288290697216,"internally_reserved":0,"allocated":450514944,"data_stored":449625188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1169017860},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,6,7,12,2,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22204416,"num_objects":-5421,"num_object_clones":0,"num_object_copies":-10842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-174,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5421,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5421,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":174,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002569"},"pg_stats":[{"pgid":"2.f","version":"49'18357","reported_seq":18445,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:12.736640+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:21:12.736640+0000","last_peered":"2024-09-11T01:21:12.736640+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:21:12.736640+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10057,"log_dups_size":0,"ondisk_log_size":10057,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7819264,"num_objects":1909,"num_object_clones":0,"num_object_copies":3818,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1909,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18357,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2163}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18442","reported_seq":18554,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:14.995099+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:21:14.995099+0000","last_peered":"2024-09-11T01:21:14.995099+0000","last_clean":"2024-09-11T01:21:14.995099+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:21:14.995099+0000","last_undegraded":"2024-09-11T01:21:14.995099+0000","last_fullsized":"2024-09-11T01:21:14.995099+0000","mapping_epoch":35,"log_start":"44'8400","ondisk_log_start":"44'8400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7823360,"num_objects":1910,"num_object_clones":0,"num_object_copies":3820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16681,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18185","reported_seq":18322,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:15.103964+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:21:15.103964+0000","last_peered":"2024-09-11T01:21:15.103964+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:21:15.103964+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8019992,"num_objects":1959,"num_object_clones":0,"num_object_copies":3918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1959,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16486,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1937}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'18244","reported_seq":18324,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:14.988281+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:21:14.988281+0000","last_peered":"2024-09-11T01:21:14.988281+0000","last_clean":"2024-09-11T01:21:14.988281+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:21:14.988281+0000","last_undegraded":"2024-09-11T01:21:14.988281+0000","last_fullsized":"2024-09-11T01:21:14.988281+0000","mapping_epoch":35,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7741440,"num_objects":1890,"num_object_clones":0,"num_object_copies":3780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18244,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'18104","reported_seq":18242,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:12.716576+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:21:12.716576+0000","last_peered":"2024-09-11T01:21:12.716576+0000","last_clean":"2024-09-11T01:21:12.716576+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:21:12.716576+0000","last_undegraded":"2024-09-11T01:21:12.716576+0000","last_fullsized":"2024-09-11T01:21:12.716576+0000","mapping_epoch":34,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7802880,"num_objects":1905,"num_object_clones":0,"num_object_copies":3810,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1905,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15109,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18261","reported_seq":18439,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:12.679810+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:21:12.679810+0000","last_peered":"2024-09-11T01:21:12.679810+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:21:12.679810+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8052736,"num_objects":1966,"num_object_clones":0,"num_object_copies":3932,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1966,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14602,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1985}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'17993","reported_seq":18132,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:12.612222+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:21:12.612222+0000","last_peered":"2024-09-11T01:21:12.612222+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:21:12.612222+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7933952,"num_objects":1937,"num_object_clones":0,"num_object_copies":3874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1937,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14850,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2049}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18294","reported_seq":18434,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:12.644050+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:21:12.644050+0000","last_peered":"2024-09-11T01:21:12.644050+0000","last_clean":"2024-09-11T01:21:12.644050+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:21:12.644050+0000","last_undegraded":"2024-09-11T01:21:12.644050+0000","last_fullsized":"2024-09-11T01:21:12.644050+0000","mapping_epoch":34,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8028160,"num_objects":1960,"num_object_clones":0,"num_object_copies":3920,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1960,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15170,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003209+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003209+0000","last_peered":"2024-09-11T01:20:22.003209+0000","last_clean":"2024-09-11T01:20:22.003209+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003209+0000","last_undegraded":"2024-09-11T01:20:22.003209+0000","last_fullsized":"2024-09-11T01:20:22.003209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18052","reported_seq":26069,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:12.464467+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:21:12.464467+0000","last_peered":"2024-09-11T01:21:12.464467+0000","last_clean":"2024-09-11T01:21:12.464467+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:21:12.464467+0000","last_undegraded":"2024-09-11T01:21:12.464467+0000","last_fullsized":"2024-09-11T01:21:12.464467+0000","mapping_epoch":48,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7868416,"num_objects":1921,"num_object_clones":0,"num_object_copies":3842,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1921,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16023,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'17952","reported_seq":18089,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:12.706230+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:21:12.706230+0000","last_peered":"2024-09-11T01:21:12.706230+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:21:12.706230+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7847936,"num_objects":1916,"num_object_clones":0,"num_object_copies":3832,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1916,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14539,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2068}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'18314","reported_seq":18443,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:15.048985+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:21:15.048985+0000","last_peered":"2024-09-11T01:21:15.048985+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:21:15.048985+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7753728,"num_objects":1893,"num_object_clones":0,"num_object_copies":3786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16131,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2131}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003225+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003225+0000","last_peered":"2024-09-11T01:20:22.003225+0000","last_clean":"2024-09-11T01:20:22.003225+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003225+0000","last_undegraded":"2024-09-11T01:20:22.003225+0000","last_fullsized":"2024-09-11T01:20:22.003225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'17979","reported_seq":25970,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:15.078852+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:21:15.078852+0000","last_peered":"2024-09-11T01:21:15.078852+0000","last_clean":"2024-09-11T01:21:15.078852+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:21:15.078852+0000","last_undegraded":"2024-09-11T01:21:15.078852+0000","last_fullsized":"2024-09-11T01:21:15.078852+0000","mapping_epoch":44,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7442432,"num_objects":1817,"num_object_clones":0,"num_object_copies":3634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15764,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18338","reported_seq":18480,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:12.691509+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:21:12.691509+0000","last_peered":"2024-09-11T01:21:12.691509+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:21:12.691509+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7888896,"num_objects":1926,"num_object_clones":0,"num_object_copies":3852,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1926,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15075,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2024}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007532+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007532+0000","last_peered":"2024-09-11T01:20:22.007532+0000","last_clean":"2024-09-11T01:20:22.007532+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007532+0000","last_undegraded":"2024-09-11T01:20:22.007532+0000","last_fullsized":"2024-09-11T01:20:22.007532+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'17949","reported_seq":25990,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:12.478605+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:21:12.478605+0000","last_peered":"2024-09-11T01:21:12.478605+0000","last_clean":"2024-09-11T01:21:12.478605+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:21:12.478605+0000","last_undegraded":"2024-09-11T01:21:12.478605+0000","last_fullsized":"2024-09-11T01:21:12.478605+0000","mapping_epoch":46,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7925760,"num_objects":1935,"num_object_clones":0,"num_object_copies":3870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14604,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18247","reported_seq":20205,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:12.686300+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:21:12.686300+0000","last_peered":"2024-09-11T01:21:12.686300+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:21:12.686300+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7901184,"num_objects":1929,"num_object_clones":0,"num_object_copies":3858,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3135,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1929,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14856,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":906,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1206}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007560+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007560+0000","last_peered":"2024-09-11T01:20:22.007560+0000","last_clean":"2024-09-11T01:20:22.007560+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007560+0000","last_undegraded":"2024-09-11T01:20:22.007560+0000","last_fullsized":"2024-09-11T01:20:22.007560+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008171+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008171+0000","last_peered":"2024-09-11T01:20:22.008171+0000","last_clean":"2024-09-11T01:20:22.008171+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008171+0000","last_undegraded":"2024-09-11T01:20:22.008171+0000","last_fullsized":"2024-09-11T01:20:22.008171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18077","reported_seq":18220,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:12.691594+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:21:12.691594+0000","last_peered":"2024-09-11T01:21:12.691594+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:21:12.691594+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7680617,"num_objects":1876,"num_object_clones":0,"num_object_copies":3752,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1876,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15147,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2041}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":125530753,"num_objects":30649,"num_object_clones":0,"num_object_copies":61298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34928,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":30649,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":251638,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12711,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":447021056,"data_stored":446957248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160888,"ondisk_log_size":160888,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182516,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":590212,"kb_used_data":157364,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93781628,"statfs":{"total":96636764160,"available":96032387072,"internally_reserved":0,"allocated":161140736,"data_stored":160860582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":443215531},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,3,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986289,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":522652,"kb_used_data":188364,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93849188,"statfs":{"total":96636764160,"available":96101568512,"internally_reserved":0,"allocated":192884736,"data_stored":192604587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342290090},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659233,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":468772,"kb_used_data":94228,"kb_used_omap":4,"kb_used_meta":374523,"kb_avail":93903068,"statfs":{"total":96636764160,"available":96156741632,"internally_reserved":0,"allocated":96489472,"data_stored":96160019,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":383512239},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,2,6,1,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":159592448,"data_stored":159588969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":95408128,"data_stored":95355350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":192020480,"data_stored":192012929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:18.324 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:21:18.324 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'18357", 'reported_seq': 18445, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:12.736640+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:21:12.736640+0000', 'last_peered': '2024-09-11T01:21:12.736640+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:21:12.736640+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "43'8300", 'ondisk_log_start': "43'8300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10057, 'log_dups_size': 0, 'ondisk_log_size': 10057, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7819264, 'num_objects': 1909, 'num_object_clones': 0, 'num_object_copies': 3818, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1909, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18357, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2163}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:18.325 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:21:18.325 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'18185", 'reported_seq': 18322, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:15.103964+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:21:15.103964+0000', 'last_peered': '2024-09-11T01:21:15.103964+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:21:15.103964+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "43'8100", 'ondisk_log_start': "43'8100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10085, 'log_dups_size': 0, 'ondisk_log_size': 10085, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8019992, 'num_objects': 1959, 'num_object_clones': 0, 'num_object_copies': 3918, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1959, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16486, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1937}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:18.325 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:21:18.325 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'18261", 'reported_seq': 18439, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:12.679810+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:21:12.679810+0000', 'last_peered': '2024-09-11T01:21:12.679810+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:21:12.679810+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "43'8200", 'ondisk_log_start': "43'8200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10061, 'log_dups_size': 0, 'ondisk_log_size': 10061, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 8052736, 'num_objects': 1966, 'num_object_clones': 0, 'num_object_copies': 3932, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1966, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14602, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1985}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:18.325 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:21:18.325 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'17993", 'reported_seq': 18132, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:12.612222+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:21:12.612222+0000', 'last_peered': '2024-09-11T01:21:12.612222+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:21:12.612222+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "43'7900", 'ondisk_log_start': "43'7900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7933952, 'num_objects': 1937, 'num_object_clones': 0, 'num_object_copies': 3874, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1937, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14850, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2049}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:18.325 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:21:18.325 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'17952", 'reported_seq': 18089, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:12.706230+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:21:12.706230+0000', 'last_peered': '2024-09-11T01:21:12.706230+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:21:12.706230+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "43'7900", 'ondisk_log_start': "43'7900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7847936, 'num_objects': 1916, 'num_object_clones': 0, 'num_object_copies': 3832, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1916, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14539, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2068}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:18.325 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:21:18.325 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'18314", 'reported_seq': 18443, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:15.048985+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:21:15.048985+0000', 'last_peered': '2024-09-11T01:21:15.048985+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:21:15.048985+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "43'8300", 'ondisk_log_start': "43'8300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10014, 'log_dups_size': 0, 'ondisk_log_size': 10014, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7753728, 'num_objects': 1893, 'num_object_clones': 0, 'num_object_copies': 3786, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1893, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16131, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2131}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:18.325 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:21:18.326 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'18338", 'reported_seq': 18480, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:12.691509+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:21:12.691509+0000', 'last_peered': '2024-09-11T01:21:12.691509+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:21:12.691509+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "43'8300", 'ondisk_log_start': "43'8300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7888896, 'num_objects': 1926, 'num_object_clones': 0, 'num_object_copies': 3852, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1926, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15075, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2024}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:18.326 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:21:18.326 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'18247", 'reported_seq': 20205, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:12.686300+0000', 'last_change': '2024-09-11T01:20:20.985862+0000', 'last_active': '2024-09-11T01:21:12.686300+0000', 'last_peered': '2024-09-11T01:21:12.686300+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:21:12.686300+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "43'8200", 'ondisk_log_start': "43'8200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7901184, 'num_objects': 1929, 'num_object_clones': 0, 'num_object_copies': 3858, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3135, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1929, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14856, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 906, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1206}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:18.326 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:21:18.326 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'18077", 'reported_seq': 18220, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:12.691594+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:21:12.691594+0000', 'last_peered': '2024-09-11T01:21:12.691594+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:21:12.691594+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "43'8000", 'ondisk_log_start': "43'8000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7680617, 'num_objects': 1876, 'num_object_clones': 0, 'num_object_copies': 3752, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1876, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15147, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2041}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:18.326 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:21:19.218 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:21:19.218 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34837/58106 objects degraded (59.954%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":615,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":29053,"data_bytes":118985364,"bytes_used":1601060864,"bytes_avail":288309231616,"bytes_total":289910292480,"degraded_objects":34837,"degraded_total":58106,"degraded_ratio":0.59954221595016011,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":265,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":523},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:21:19.283 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:21:19.283 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:20.037 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:21:20.038 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:21:20.075 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":350,"stamp":"2024-09-11T01:21:18.763864+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":118985364,"num_objects":29053,"num_object_clones":0,"num_object_copies":58106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34837,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29053,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":253238,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12803,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160689,"ondisk_log_size":160689,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1563536,"kb_used_data":425992,"kb_used_omap":12,"kb_used_meta":1137459,"kb_avail":281551984,"statfs":{"total":289910292480,"available":288309231616,"internally_reserved":0,"allocated":436215808,"data_stored":435326052,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1164758020},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,2,6,13,7,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25747456,"num_objects":-6286,"num_object_clones":0,"num_object_copies":-12572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-265,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6286,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6286,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":265,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002616"},"pg_stats":[{"pgid":"2.f","version":"49'18509","reported_seq":18595,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:17.779464+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:21:17.779464+0000","last_peered":"2024-09-11T01:21:17.779464+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:21:17.779464+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7196672,"num_objects":1757,"num_object_clones":0,"num_object_copies":3514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18509,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2315}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18442","reported_seq":18554,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:14.995099+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:21:14.995099+0000","last_peered":"2024-09-11T01:21:14.995099+0000","last_clean":"2024-09-11T01:21:14.995099+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:21:14.995099+0000","last_undegraded":"2024-09-11T01:21:14.995099+0000","last_fullsized":"2024-09-11T01:21:14.995099+0000","mapping_epoch":35,"log_start":"44'8400","ondisk_log_start":"44'8400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7823360,"num_objects":1910,"num_object_clones":0,"num_object_copies":3820,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1910,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16681,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18185","reported_seq":18322,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:15.103964+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:21:15.103964+0000","last_peered":"2024-09-11T01:21:15.103964+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:21:15.103964+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10085,"log_dups_size":0,"ondisk_log_size":10085,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":8019992,"num_objects":1959,"num_object_clones":0,"num_object_copies":3918,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1959,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16486,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1937}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'18244","reported_seq":18324,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:14.988281+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:21:14.988281+0000","last_peered":"2024-09-11T01:21:14.988281+0000","last_clean":"2024-09-11T01:21:14.988281+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:21:14.988281+0000","last_undegraded":"2024-09-11T01:21:14.988281+0000","last_fullsized":"2024-09-11T01:21:14.988281+0000","mapping_epoch":35,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7741440,"num_objects":1890,"num_object_clones":0,"num_object_copies":3780,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1890,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18244,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'18245","reported_seq":18383,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:17.674749+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:21:17.674749+0000","last_peered":"2024-09-11T01:21:17.674749+0000","last_clean":"2024-09-11T01:21:17.674749+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:21:17.674749+0000","last_undegraded":"2024-09-11T01:21:17.674749+0000","last_fullsized":"2024-09-11T01:21:17.674749+0000","mapping_epoch":34,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7225344,"num_objects":1764,"num_object_clones":0,"num_object_copies":3528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15250,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18420","reported_seq":18600,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:17.680100+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:21:17.680100+0000","last_peered":"2024-09-11T01:21:17.680100+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:21:17.680100+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'8400","ondisk_log_start":"45'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7401472,"num_objects":1807,"num_object_clones":0,"num_object_copies":3614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14761,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2144}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'18127","reported_seq":18267,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:17.703905+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:21:17.703905+0000","last_peered":"2024-09-11T01:21:17.703905+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:21:17.703905+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7385088,"num_objects":1803,"num_object_clones":0,"num_object_copies":3606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14984,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2183}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18433","reported_seq":18572,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:17.766198+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:21:17.766198+0000","last_peered":"2024-09-11T01:21:17.766198+0000","last_clean":"2024-09-11T01:21:17.766198+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:21:17.766198+0000","last_undegraded":"2024-09-11T01:21:17.766198+0000","last_fullsized":"2024-09-11T01:21:17.766198+0000","mapping_epoch":34,"log_start":"45'8400","ondisk_log_start":"45'8400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7458816,"num_objects":1821,"num_object_clones":0,"num_object_copies":3642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1821,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15309,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003209+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003209+0000","last_peered":"2024-09-11T01:20:22.003209+0000","last_clean":"2024-09-11T01:20:22.003209+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003209+0000","last_undegraded":"2024-09-11T01:20:22.003209+0000","last_fullsized":"2024-09-11T01:20:22.003209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18201","reported_seq":26219,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:17.468372+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:21:17.468372+0000","last_peered":"2024-09-11T01:21:17.468372+0000","last_clean":"2024-09-11T01:21:17.468372+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:21:17.468372+0000","last_undegraded":"2024-09-11T01:21:17.468372+0000","last_fullsized":"2024-09-11T01:21:17.468372+0000","mapping_epoch":48,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7258112,"num_objects":1772,"num_object_clones":0,"num_object_copies":3544,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1772,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16172,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18110","reported_seq":18248,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:17.727803+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:21:17.727803+0000","last_peered":"2024-09-11T01:21:17.727803+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:21:17.727803+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7200768,"num_objects":1758,"num_object_clones":0,"num_object_copies":3516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14697,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2226}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'18314","reported_seq":18443,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:15.048985+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:21:15.048985+0000","last_peered":"2024-09-11T01:21:15.048985+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:21:15.048985+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7753728,"num_objects":1893,"num_object_clones":0,"num_object_copies":3786,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1893,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16131,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2131}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003225+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003225+0000","last_peered":"2024-09-11T01:20:22.003225+0000","last_clean":"2024-09-11T01:20:22.003225+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003225+0000","last_undegraded":"2024-09-11T01:20:22.003225+0000","last_fullsized":"2024-09-11T01:20:22.003225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'17979","reported_seq":25970,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:15.078852+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:21:15.078852+0000","last_peered":"2024-09-11T01:21:15.078852+0000","last_clean":"2024-09-11T01:21:15.078852+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:21:15.078852+0000","last_undegraded":"2024-09-11T01:21:15.078852+0000","last_fullsized":"2024-09-11T01:21:15.078852+0000","mapping_epoch":44,"log_start":"43'7900","ondisk_log_start":"43'7900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7442432,"num_objects":1817,"num_object_clones":0,"num_object_copies":3634,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1817,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15764,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18480","reported_seq":18622,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:17.616709+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:21:17.616709+0000","last_peered":"2024-09-11T01:21:17.616709+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:21:17.616709+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"44'8400","ondisk_log_start":"44'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7307264,"num_objects":1784,"num_object_clones":0,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15217,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2166}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007532+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007532+0000","last_peered":"2024-09-11T01:20:22.007532+0000","last_clean":"2024-09-11T01:20:22.007532+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007532+0000","last_undegraded":"2024-09-11T01:20:22.007532+0000","last_fullsized":"2024-09-11T01:20:22.007532+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18092","reported_seq":26133,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:17.480872+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:21:17.480872+0000","last_peered":"2024-09-11T01:21:17.480872+0000","last_clean":"2024-09-11T01:21:17.480872+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:21:17.480872+0000","last_undegraded":"2024-09-11T01:21:17.480872+0000","last_fullsized":"2024-09-11T01:21:17.480872+0000","mapping_epoch":46,"log_start":"43'8000","ondisk_log_start":"43'8000","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7340032,"num_objects":1792,"num_object_clones":0,"num_object_copies":3584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1792,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14747,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18390","reported_seq":20530,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:17.751221+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:21:17.751221+0000","last_peered":"2024-09-11T01:21:17.751221+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:21:17.751221+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7315456,"num_objects":1786,"num_object_clones":0,"num_object_copies":3572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3044,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14999,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":997,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1258}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007560+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007560+0000","last_peered":"2024-09-11T01:20:22.007560+0000","last_clean":"2024-09-11T01:20:22.007560+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007560+0000","last_undegraded":"2024-09-11T01:20:22.007560+0000","last_fullsized":"2024-09-11T01:20:22.007560+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008171+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008171+0000","last_peered":"2024-09-11T01:20:22.008171+0000","last_clean":"2024-09-11T01:20:22.008171+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008171+0000","last_undegraded":"2024-09-11T01:20:22.008171+0000","last_fullsized":"2024-09-11T01:20:22.008171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18215","reported_seq":18359,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:17.751957+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:21:17.751957+0000","last_peered":"2024-09-11T01:21:17.751957+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:21:17.751957+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7115369,"num_objects":1738,"num_object_clones":0,"num_object_copies":3476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1738,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15285,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2179}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":118985345,"num_objects":29051,"num_object_clones":0,"num_object_copies":58102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34837,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29051,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":253236,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12802,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":432529408,"data_stored":432465600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160686,"ondisk_log_size":160686,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182517,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":575048,"kb_used_data":150520,"kb_used_omap":4,"kb_used_meta":424507,"kb_avail":93796792,"statfs":{"total":96636764160,"available":96047915008,"internally_reserved":0,"allocated":154132480,"data_stored":153852326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":434695851},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,8,2,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986289,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":522652,"kb_used_data":188364,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93849188,"statfs":{"total":96636764160,"available":96101568512,"internally_reserved":0,"allocated":192884736,"data_stored":192604587,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342290090},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659234,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465836,"kb_used_data":87108,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93906004,"statfs":{"total":96636764160,"available":96159748096,"internally_reserved":0,"allocated":89198592,"data_stored":88869139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":387772079},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,6,0,2,1,4,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":152449024,"data_stored":152445545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":88059904,"data_stored":88007126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":192020480,"data_stored":192012929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:23.077 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:24.167 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:21:24.167 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:21:24.276 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":352,"stamp":"2024-09-11T01:21:22.764753+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":114639508,"num_objects":27992,"num_object_clones":0,"num_object_copies":55984,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34837,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27992,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":254299,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12803,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160750,"ondisk_log_size":160750,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1547744,"kb_used_data":414380,"kb_used_omap":12,"kb_used_meta":1133299,"kb_avail":281567776,"statfs":{"total":289910292480,"available":288325402624,"internally_reserved":0,"allocated":424325120,"data_stored":423435364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1160498180},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,9,3,14,9,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20557824,"num_objects":-5019,"num_object_clones":0,"num_object_copies":-10038,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-181,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5019,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5019,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":181,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002627"},"pg_stats":[{"pgid":"2.f","version":"49'18509","reported_seq":18595,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:17.779464+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:21:17.779464+0000","last_peered":"2024-09-11T01:21:17.779464+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:21:17.779464+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7196672,"num_objects":1757,"num_object_clones":0,"num_object_copies":3514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1757,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18509,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2315}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18584","reported_seq":18696,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:20.069747+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:21:20.069747+0000","last_peered":"2024-09-11T01:21:20.069747+0000","last_clean":"2024-09-11T01:21:20.069747+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:21:20.069747+0000","last_undegraded":"2024-09-11T01:21:20.069747+0000","last_fullsized":"2024-09-11T01:21:20.069747+0000","mapping_epoch":35,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7241728,"num_objects":1768,"num_object_clones":0,"num_object_copies":3536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16823,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18349","reported_seq":18484,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:20.100277+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:21:20.100277+0000","last_peered":"2024-09-11T01:21:20.100277+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:21:20.100277+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'8300","ondisk_log_start":"45'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7348248,"num_objects":1795,"num_object_clones":0,"num_object_copies":3590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1795,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16650,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2101}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'18395","reported_seq":18475,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:20.097008+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:21:20.097008+0000","last_peered":"2024-09-11T01:21:20.097008+0000","last_clean":"2024-09-11T01:21:20.097008+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:21:20.097008+0000","last_undegraded":"2024-09-11T01:21:20.097008+0000","last_fullsized":"2024-09-11T01:21:20.097008+0000","mapping_epoch":35,"log_start":"44'8300","ondisk_log_start":"44'8300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7122944,"num_objects":1739,"num_object_clones":0,"num_object_copies":3478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18395,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'18245","reported_seq":18383,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:17.674749+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:21:17.674749+0000","last_peered":"2024-09-11T01:21:17.674749+0000","last_clean":"2024-09-11T01:21:17.674749+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:21:17.674749+0000","last_undegraded":"2024-09-11T01:21:17.674749+0000","last_fullsized":"2024-09-11T01:21:17.674749+0000","mapping_epoch":34,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7225344,"num_objects":1764,"num_object_clones":0,"num_object_copies":3528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1764,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15250,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18420","reported_seq":18600,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:17.680100+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:21:17.680100+0000","last_peered":"2024-09-11T01:21:17.680100+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:21:17.680100+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'8400","ondisk_log_start":"45'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10020,"log_dups_size":0,"ondisk_log_size":10020,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7401472,"num_objects":1807,"num_object_clones":0,"num_object_copies":3614,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1807,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14761,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2144}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'18127","reported_seq":18267,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:17.703905+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:21:17.703905+0000","last_peered":"2024-09-11T01:21:17.703905+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:21:17.703905+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7385088,"num_objects":1803,"num_object_clones":0,"num_object_copies":3606,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1803,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14984,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2183}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18433","reported_seq":18572,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:17.766198+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:21:17.766198+0000","last_peered":"2024-09-11T01:21:17.766198+0000","last_clean":"2024-09-11T01:21:17.766198+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:21:17.766198+0000","last_undegraded":"2024-09-11T01:21:17.766198+0000","last_fullsized":"2024-09-11T01:21:17.766198+0000","mapping_epoch":34,"log_start":"45'8400","ondisk_log_start":"45'8400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7458816,"num_objects":1821,"num_object_clones":0,"num_object_copies":3642,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1821,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15309,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003209+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003209+0000","last_peered":"2024-09-11T01:20:22.003209+0000","last_clean":"2024-09-11T01:20:22.003209+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003209+0000","last_undegraded":"2024-09-11T01:20:22.003209+0000","last_fullsized":"2024-09-11T01:20:22.003209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18371","reported_seq":26389,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:22.538685+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:21:22.538685+0000","last_peered":"2024-09-11T01:21:22.538685+0000","last_clean":"2024-09-11T01:21:22.538685+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:21:22.538685+0000","last_undegraded":"2024-09-11T01:21:22.538685+0000","last_fullsized":"2024-09-11T01:21:22.538685+0000","mapping_epoch":48,"log_start":"45'8300","ondisk_log_start":"45'8300","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6561792,"num_objects":1602,"num_object_clones":0,"num_object_copies":3204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16342,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18110","reported_seq":18248,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:17.727803+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:21:17.727803+0000","last_peered":"2024-09-11T01:21:17.727803+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:21:17.727803+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7200768,"num_objects":1758,"num_object_clones":0,"num_object_copies":3516,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1758,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14697,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2226}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'18481","reported_seq":18610,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:19.899523+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:21:19.899523+0000","last_peered":"2024-09-11T01:21:19.899523+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:21:19.899523+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'8400","ondisk_log_start":"45'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7069696,"num_objects":1726,"num_object_clones":0,"num_object_copies":3452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16298,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2298}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003225+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003225+0000","last_peered":"2024-09-11T01:20:22.003225+0000","last_clean":"2024-09-11T01:20:22.003225+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003225+0000","last_undegraded":"2024-09-11T01:20:22.003225+0000","last_fullsized":"2024-09-11T01:20:22.003225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18108","reported_seq":26099,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:20.024368+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:21:20.024368+0000","last_peered":"2024-09-11T01:21:20.024368+0000","last_clean":"2024-09-11T01:21:20.024368+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:21:20.024368+0000","last_undegraded":"2024-09-11T01:21:20.024368+0000","last_fullsized":"2024-09-11T01:21:20.024368+0000","mapping_epoch":44,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6914048,"num_objects":1688,"num_object_clones":0,"num_object_copies":3376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15893,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18480","reported_seq":18622,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:17.616709+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:21:17.616709+0000","last_peered":"2024-09-11T01:21:17.616709+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:21:17.616709+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"44'8400","ondisk_log_start":"44'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7307264,"num_objects":1784,"num_object_clones":0,"num_object_copies":3568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15217,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2166}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007532+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007532+0000","last_peered":"2024-09-11T01:20:22.007532+0000","last_clean":"2024-09-11T01:20:22.007532+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007532+0000","last_undegraded":"2024-09-11T01:20:22.007532+0000","last_fullsized":"2024-09-11T01:20:22.007532+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18230","reported_seq":26271,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:22.581577+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:21:22.581577+0000","last_peered":"2024-09-11T01:21:22.581577+0000","last_clean":"2024-09-11T01:21:22.581577+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:21:22.581577+0000","last_undegraded":"2024-09-11T01:21:22.581577+0000","last_fullsized":"2024-09-11T01:21:22.581577+0000","mapping_epoch":46,"log_start":"44'8200","ondisk_log_start":"44'8200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6774784,"num_objects":1654,"num_object_clones":0,"num_object_copies":3308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14885,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18390","reported_seq":20530,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:17.751221+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:21:17.751221+0000","last_peered":"2024-09-11T01:21:17.751221+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:21:17.751221+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"43'8300","ondisk_log_start":"43'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7315456,"num_objects":1786,"num_object_clones":0,"num_object_copies":3572,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3044,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1786,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14999,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":997,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1258}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007560+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007560+0000","last_peered":"2024-09-11T01:20:22.007560+0000","last_clean":"2024-09-11T01:20:22.007560+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007560+0000","last_undegraded":"2024-09-11T01:20:22.007560+0000","last_fullsized":"2024-09-11T01:20:22.007560+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008171+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008171+0000","last_peered":"2024-09-11T01:20:22.008171+0000","last_clean":"2024-09-11T01:20:22.008171+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008171+0000","last_undegraded":"2024-09-11T01:20:22.008171+0000","last_fullsized":"2024-09-11T01:20:22.008171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18215","reported_seq":18359,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:17.751957+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:21:17.751957+0000","last_peered":"2024-09-11T01:21:17.751957+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:21:17.751957+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7115369,"num_objects":1738,"num_object_clones":0,"num_object_copies":3476,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1738,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15285,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2179}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":114639489,"num_objects":27990,"num_object_clones":0,"num_object_copies":55980,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34837,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27990,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":254297,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12802,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":420503552,"data_stored":420439744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160747,"ondisk_log_size":160747,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182517,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":575048,"kb_used_data":150520,"kb_used_omap":4,"kb_used_meta":424507,"kb_avail":93796792,"statfs":{"total":96636764160,"available":96047915008,"internally_reserved":0,"allocated":154132480,"data_stored":153852326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":434695851},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,8,2,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986290,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":522068,"kb_used_data":183620,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93849772,"statfs":{"total":96636764160,"available":96102166528,"internally_reserved":0,"allocated":188026880,"data_stored":187746731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659235,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":450628,"kb_used_data":80240,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93921212,"statfs":{"total":96636764160,"available":96175321088,"internally_reserved":0,"allocated":82165760,"data_stored":81836307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":379252399},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,6,0,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":152449024,"data_stored":152445545,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":80793600,"data_stored":80740822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":187260928,"data_stored":187253377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:24.277 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:26.149 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:21:26.150 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:21:26.717 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":353,"stamp":"2024-09-11T01:21:24.765147+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":109425300,"num_objects":26719,"num_object_clones":0,"num_object_copies":53438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34753,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26719,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":255572,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12887,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160923,"ondisk_log_size":160923,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1545060,"kb_used_data":407532,"kb_used_omap":12,"kb_used_meta":1137459,"kb_avail":281570460,"statfs":{"total":289910292480,"available":288328151040,"internally_reserved":0,"allocated":417312768,"data_stored":416423012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1164758020},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,4,10,5,9,12,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19124224,"num_objects":-4669,"num_object_clones":0,"num_object_copies":-9338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4669,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":175,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002590"},"pg_stats":[{"pgid":"2.f","version":"49'18646","reported_seq":18734,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:22.620652+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:21:22.620652+0000","last_peered":"2024-09-11T01:21:22.620652+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:21:22.620652+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6635520,"num_objects":1620,"num_object_clones":0,"num_object_copies":3240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1620,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18646,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2452}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18584","reported_seq":18696,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:20.069747+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:21:20.069747+0000","last_peered":"2024-09-11T01:21:20.069747+0000","last_clean":"2024-09-11T01:21:20.069747+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:21:20.069747+0000","last_undegraded":"2024-09-11T01:21:20.069747+0000","last_fullsized":"2024-09-11T01:21:20.069747+0000","mapping_epoch":35,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7241728,"num_objects":1768,"num_object_clones":0,"num_object_copies":3536,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1768,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16823,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18349","reported_seq":18484,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:20.100277+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:21:20.100277+0000","last_peered":"2024-09-11T01:21:20.100277+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:21:20.100277+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'8300","ondisk_log_start":"45'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7348248,"num_objects":1795,"num_object_clones":0,"num_object_copies":3590,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1795,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16650,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2101}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'18395","reported_seq":18475,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:20.097008+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:21:20.097008+0000","last_peered":"2024-09-11T01:21:20.097008+0000","last_clean":"2024-09-11T01:21:20.097008+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:21:20.097008+0000","last_undegraded":"2024-09-11T01:21:20.097008+0000","last_fullsized":"2024-09-11T01:21:20.097008+0000","mapping_epoch":35,"log_start":"44'8300","ondisk_log_start":"44'8300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10095,"log_dups_size":0,"ondisk_log_size":10095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7122944,"num_objects":1739,"num_object_clones":0,"num_object_copies":3478,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1739,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18395,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'18386","reported_seq":18524,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:22.735952+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:21:22.735952+0000","last_peered":"2024-09-11T01:21:22.735952+0000","last_clean":"2024-09-11T01:21:22.735952+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:21:22.735952+0000","last_undegraded":"2024-09-11T01:21:22.735952+0000","last_fullsized":"2024-09-11T01:21:22.735952+0000","mapping_epoch":34,"log_start":"45'8300","ondisk_log_start":"45'8300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6647808,"num_objects":1623,"num_object_clones":0,"num_object_copies":3246,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1623,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15391,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18546","reported_seq":18724,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:22.740720+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:21:22.740720+0000","last_peered":"2024-09-11T01:21:22.740720+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:21:22.740720+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6885376,"num_objects":1681,"num_object_clones":0,"num_object_copies":3362,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1681,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14887,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2270}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'18266","reported_seq":18405,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:22.814482+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:21:22.814482+0000","last_peered":"2024-09-11T01:21:22.814482+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:21:22.814482+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"44'8200","ondisk_log_start":"44'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6815744,"num_objects":1664,"num_object_clones":0,"num_object_copies":3328,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1664,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15123,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2322}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18582","reported_seq":18722,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:22.684438+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:21:22.684438+0000","last_peered":"2024-09-11T01:21:22.684438+0000","last_clean":"2024-09-11T01:21:22.684438+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:21:22.684438+0000","last_undegraded":"2024-09-11T01:21:22.684438+0000","last_fullsized":"2024-09-11T01:21:22.684438+0000","mapping_epoch":34,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6848512,"num_objects":1672,"num_object_clones":0,"num_object_copies":3344,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1672,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15458,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003209+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003209+0000","last_peered":"2024-09-11T01:20:22.003209+0000","last_clean":"2024-09-11T01:20:22.003209+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003209+0000","last_undegraded":"2024-09-11T01:20:22.003209+0000","last_fullsized":"2024-09-11T01:20:22.003209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18371","reported_seq":26389,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:22.538685+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:21:22.538685+0000","last_peered":"2024-09-11T01:21:22.538685+0000","last_clean":"2024-09-11T01:21:22.538685+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:21:22.538685+0000","last_undegraded":"2024-09-11T01:21:22.538685+0000","last_fullsized":"2024-09-11T01:21:22.538685+0000","mapping_epoch":48,"log_start":"45'8300","ondisk_log_start":"45'8300","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10071,"log_dups_size":0,"ondisk_log_size":10071,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6561792,"num_objects":1602,"num_object_clones":0,"num_object_copies":3204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1602,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16342,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18256","reported_seq":18393,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:22.815728+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:21:22.815728+0000","last_peered":"2024-09-11T01:21:22.815728+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:21:22.815728+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"43'8200","ondisk_log_start":"43'8200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6602752,"num_objects":1612,"num_object_clones":0,"num_object_copies":3224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1612,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14843,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2372}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'18481","reported_seq":18610,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:19.899523+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:21:19.899523+0000","last_peered":"2024-09-11T01:21:19.899523+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:21:19.899523+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'8400","ondisk_log_start":"45'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7069696,"num_objects":1726,"num_object_clones":0,"num_object_copies":3452,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1726,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16298,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2298}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003225+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003225+0000","last_peered":"2024-09-11T01:20:22.003225+0000","last_clean":"2024-09-11T01:20:22.003225+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003225+0000","last_undegraded":"2024-09-11T01:20:22.003225+0000","last_fullsized":"2024-09-11T01:20:22.003225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18108","reported_seq":26099,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:20.024368+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:21:20.024368+0000","last_peered":"2024-09-11T01:21:20.024368+0000","last_clean":"2024-09-11T01:21:20.024368+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:21:20.024368+0000","last_undegraded":"2024-09-11T01:21:20.024368+0000","last_fullsized":"2024-09-11T01:21:20.024368+0000","mapping_epoch":44,"log_start":"43'8100","ondisk_log_start":"43'8100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6914048,"num_objects":1688,"num_object_clones":0,"num_object_copies":3376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1688,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15893,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18616","reported_seq":18757,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:22.762104+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:21:22.762104+0000","last_peered":"2024-09-11T01:21:22.762104+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:21:22.762104+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6750208,"num_objects":1648,"num_object_clones":0,"num_object_copies":3296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1648,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15353,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2302}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007532+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007532+0000","last_peered":"2024-09-11T01:20:22.007532+0000","last_clean":"2024-09-11T01:20:22.007532+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007532+0000","last_undegraded":"2024-09-11T01:20:22.007532+0000","last_fullsized":"2024-09-11T01:20:22.007532+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18230","reported_seq":26271,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:22.581577+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:21:22.581577+0000","last_peered":"2024-09-11T01:21:22.581577+0000","last_clean":"2024-09-11T01:21:22.581577+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:21:22.581577+0000","last_undegraded":"2024-09-11T01:21:22.581577+0000","last_fullsized":"2024-09-11T01:21:22.581577+0000","mapping_epoch":46,"log_start":"44'8200","ondisk_log_start":"44'8200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6774784,"num_objects":1654,"num_object_clones":0,"num_object_copies":3308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1654,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14885,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18525","reported_seq":20833,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:22.795077+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:21:22.795077+0000","last_peered":"2024-09-11T01:21:22.795077+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:21:22.795077+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6762496,"num_objects":1651,"num_object_clones":0,"num_object_copies":3302,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2960,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1651,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15134,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1081,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1309}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007560+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007560+0000","last_peered":"2024-09-11T01:20:22.007560+0000","last_clean":"2024-09-11T01:20:22.007560+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007560+0000","last_undegraded":"2024-09-11T01:20:22.007560+0000","last_fullsized":"2024-09-11T01:20:22.007560+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008171+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008171+0000","last_peered":"2024-09-11T01:20:22.008171+0000","last_clean":"2024-09-11T01:20:22.008171+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008171+0000","last_undegraded":"2024-09-11T01:20:22.008171+0000","last_fullsized":"2024-09-11T01:20:22.008171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18379","reported_seq":18523,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:22.785740+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:21:22.785740+0000","last_peered":"2024-09-11T01:21:22.785740+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:21:22.785740+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"44'8300","ondisk_log_start":"44'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6443625,"num_objects":1574,"num_object_clones":0,"num_object_copies":3148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1574,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15449,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2343}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":109425281,"num_objects":26717,"num_object_clones":0,"num_object_copies":53434,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34753,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26717,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":255570,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12886,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":413425664,"data_stored":413361856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160920,"ondisk_log_size":160920,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182518,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":572364,"kb_used_data":143672,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93799476,"statfs":{"total":96636764160,"available":96050663424,"internally_reserved":0,"allocated":147120128,"data_stored":146839974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":438955691},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,4,3,4,3,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986290,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":522068,"kb_used_data":183620,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93849772,"statfs":{"total":96636764160,"available":96102166528,"internally_reserved":0,"allocated":188026880,"data_stored":187746731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,1,2,3,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659235,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":450628,"kb_used_data":80240,"kb_used_omap":4,"kb_used_meta":370363,"kb_avail":93921212,"statfs":{"total":96636764160,"available":96175321088,"internally_reserved":0,"allocated":82165760,"data_stored":81836307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":379252399},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,6,0,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":145371136,"data_stored":145367657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":80793600,"data_stored":80740822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":187260928,"data_stored":187253377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:26.718 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:21:26.719 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'18646", 'reported_seq': 18734, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:22.620652+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:21:22.620652+0000', 'last_peered': '2024-09-11T01:21:22.620652+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:21:22.620652+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "45'8600", 'ondisk_log_start': "45'8600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6635520, 'num_objects': 1620, 'num_object_clones': 0, 'num_object_copies': 3240, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1620, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18646, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2452}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:26.719 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:21:26.719 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'18349", 'reported_seq': 18484, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:20.100277+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:21:20.100277+0000', 'last_peered': '2024-09-11T01:21:20.100277+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:21:20.100277+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "45'8300", 'ondisk_log_start': "45'8300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7348248, 'num_objects': 1795, 'num_object_clones': 0, 'num_object_copies': 3590, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1795, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16650, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2101}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:26.719 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:21:26.719 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'18546", 'reported_seq': 18724, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:22.740720+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:21:22.740720+0000', 'last_peered': '2024-09-11T01:21:22.740720+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:21:22.740720+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'8500", 'ondisk_log_start': "45'8500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6885376, 'num_objects': 1681, 'num_object_clones': 0, 'num_object_copies': 3362, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1681, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14887, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2270}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:26.719 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:21:26.719 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'18266", 'reported_seq': 18405, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:22.814482+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:21:22.814482+0000', 'last_peered': '2024-09-11T01:21:22.814482+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:21:22.814482+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "44'8200", 'ondisk_log_start': "44'8200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6815744, 'num_objects': 1664, 'num_object_clones': 0, 'num_object_copies': 3328, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1664, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15123, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2322}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:26.719 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:21:26.720 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'18256", 'reported_seq': 18393, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:22.815728+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:21:22.815728+0000', 'last_peered': '2024-09-11T01:21:22.815728+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:21:22.815728+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "43'8200", 'ondisk_log_start': "43'8200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10056, 'log_dups_size': 0, 'ondisk_log_size': 10056, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6602752, 'num_objects': 1612, 'num_object_clones': 0, 'num_object_copies': 3224, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1612, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14843, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2372}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:26.720 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:21:26.720 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'18481", 'reported_seq': 18610, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:19.899523+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:21:19.899523+0000', 'last_peered': '2024-09-11T01:21:19.899523+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:21:19.899523+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "45'8400", 'ondisk_log_start': "45'8400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10081, 'log_dups_size': 0, 'ondisk_log_size': 10081, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 7069696, 'num_objects': 1726, 'num_object_clones': 0, 'num_object_copies': 3452, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1726, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16298, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2298}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:26.720 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:21:26.720 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'18616", 'reported_seq': 18757, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:22.762104+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:21:22.762104+0000', 'last_peered': '2024-09-11T01:21:22.762104+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:21:22.762104+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'8600", 'ondisk_log_start': "45'8600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6750208, 'num_objects': 1648, 'num_object_clones': 0, 'num_object_copies': 3296, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1648, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15353, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2302}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:26.720 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:21:26.720 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'18525", 'reported_seq': 20833, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:22.795077+0000', 'last_change': '2024-09-11T01:20:20.985862+0000', 'last_active': '2024-09-11T01:21:22.795077+0000', 'last_peered': '2024-09-11T01:21:22.795077+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:21:22.795077+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "45'8500", 'ondisk_log_start': "45'8500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6762496, 'num_objects': 1651, 'num_object_clones': 0, 'num_object_copies': 3302, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2960, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1651, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15134, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1081, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1309}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:26.720 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:21:26.720 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'18379", 'reported_seq': 18523, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:22.785740+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:21:22.785740+0000', 'last_peered': '2024-09-11T01:21:22.785740+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:21:22.785740+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "44'8300", 'ondisk_log_start': "44'8300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6443625, 'num_objects': 1574, 'num_object_clones': 0, 'num_object_copies': 3148, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1574, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15449, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2343}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:26.721 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:21:28.103 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:21:28.103 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34753/51898 objects degraded (66.964%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":624,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":25949,"data_bytes":106271380,"bytes_used":1581326336,"bytes_avail":288328966144,"bytes_total":289910292480,"degraded_objects":34753,"degraded_total":51898,"degraded_ratio":0.66964044857219929,"recovering_objects_per_sec":14,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":175,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":453},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:21:28.147 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:21:28.148 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:29.026 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:21:29.027 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:21:29.082 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-09-11T01:21:28.765917+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":99685012,"num_objects":24341,"num_object_clones":0,"num_object_copies":48682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24341,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":257950,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12983,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160701,"ondisk_log_size":160701,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1546620,"kb_used_data":388292,"kb_used_omap":12,"kb_used_meta":1158259,"kb_avail":281568900,"statfs":{"total":289910292480,"available":288326553600,"internally_reserved":0,"allocated":397611008,"data_stored":396721252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1186057220},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,6,2,8,4,10,6,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25845760,"num_objects":-6310,"num_object_clones":0,"num_object_copies":-12620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-271,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6310,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":271,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002505"},"pg_stats":[{"pgid":"2.f","version":"49'18794","reported_seq":18882,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:27.754443+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:21:27.754443+0000","last_peered":"2024-09-11T01:21:27.754443+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:21:27.754443+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6029312,"num_objects":1472,"num_object_clones":0,"num_object_copies":2944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18794,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2600}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18727","reported_seq":18839,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:25.087287+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:21:25.087287+0000","last_peered":"2024-09-11T01:21:25.087287+0000","last_clean":"2024-09-11T01:21:25.087287+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:21:25.087287+0000","last_undegraded":"2024-09-11T01:21:25.087287+0000","last_fullsized":"2024-09-11T01:21:25.087287+0000","mapping_epoch":35,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6656000,"num_objects":1625,"num_object_clones":0,"num_object_copies":3250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1625,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16966,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18518","reported_seq":18657,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:25.103556+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:21:25.103556+0000","last_peered":"2024-09-11T01:21:25.103556+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:21:25.103556+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6656024,"num_objects":1626,"num_object_clones":0,"num_object_copies":3252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1626,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16819,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2270}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'18558","reported_seq":18638,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:25.106870+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:21:25.106870+0000","last_peered":"2024-09-11T01:21:25.106870+0000","last_clean":"2024-09-11T01:21:25.106870+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:21:25.106870+0000","last_undegraded":"2024-09-11T01:21:25.106870+0000","last_fullsized":"2024-09-11T01:21:25.106870+0000","mapping_epoch":35,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6455296,"num_objects":1576,"num_object_clones":0,"num_object_copies":3152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1576,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18558,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'18514","reported_seq":18652,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:27.842717+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:21:27.842717+0000","last_peered":"2024-09-11T01:21:27.842717+0000","last_clean":"2024-09-11T01:21:27.842717+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:21:27.842717+0000","last_undegraded":"2024-09-11T01:21:27.842717+0000","last_fullsized":"2024-09-11T01:21:27.842717+0000","mapping_epoch":34,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6123520,"num_objects":1495,"num_object_clones":0,"num_object_copies":2990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15519,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18704","reported_seq":18884,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:27.806414+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:21:27.806414+0000","last_peered":"2024-09-11T01:21:27.806414+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:21:27.806414+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6238208,"num_objects":1523,"num_object_clones":0,"num_object_copies":3046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15045,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2428}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'18412","reported_seq":18551,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:27.895050+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:21:27.895050+0000","last_peered":"2024-09-11T01:21:27.895050+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:21:27.895050+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'8400","ondisk_log_start":"45'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6217728,"num_objects":1518,"num_object_clones":0,"num_object_copies":3036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15269,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18725","reported_seq":18864,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:27.859856+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:21:27.859856+0000","last_peered":"2024-09-11T01:21:27.859856+0000","last_clean":"2024-09-11T01:21:27.859856+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:21:27.859856+0000","last_undegraded":"2024-09-11T01:21:27.859856+0000","last_fullsized":"2024-09-11T01:21:27.859856+0000","mapping_epoch":34,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6262784,"num_objects":1529,"num_object_clones":0,"num_object_copies":3058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1529,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15601,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003209+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003209+0000","last_peered":"2024-09-11T01:20:22.003209+0000","last_clean":"2024-09-11T01:20:22.003209+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003209+0000","last_undegraded":"2024-09-11T01:20:22.003209+0000","last_fullsized":"2024-09-11T01:20:22.003209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18530","reported_seq":26548,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:27.480282+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:21:27.480282+0000","last_peered":"2024-09-11T01:21:27.480282+0000","last_clean":"2024-09-11T01:21:27.480282+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:21:27.480282+0000","last_undegraded":"2024-09-11T01:21:27.480282+0000","last_fullsized":"2024-09-11T01:21:27.480282+0000","mapping_epoch":48,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5910528,"num_objects":1443,"num_object_clones":0,"num_object_copies":2886,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1443,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16501,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18387","reported_seq":18526,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:27.803723+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:21:27.803723+0000","last_peered":"2024-09-11T01:21:27.803723+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:21:27.803723+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'8300","ondisk_log_start":"45'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6066176,"num_objects":1481,"num_object_clones":0,"num_object_copies":2962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1481,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14974,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2503}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'18608","reported_seq":18734,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:25.144115+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:21:25.144115+0000","last_peered":"2024-09-11T01:21:25.144115+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:21:25.144115+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6549504,"num_objects":1599,"num_object_clones":0,"num_object_copies":3198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1599,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2425}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":89,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003225+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003225+0000","last_peered":"2024-09-11T01:20:22.003225+0000","last_clean":"2024-09-11T01:20:22.003225+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003225+0000","last_undegraded":"2024-09-11T01:20:22.003225+0000","last_fullsized":"2024-09-11T01:20:22.003225+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18276","reported_seq":26267,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:25.132245+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:21:25.132245+0000","last_peered":"2024-09-11T01:21:25.132245+0000","last_clean":"2024-09-11T01:21:25.132245+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:21:25.132245+0000","last_undegraded":"2024-09-11T01:21:25.132245+0000","last_fullsized":"2024-09-11T01:21:25.132245+0000","mapping_epoch":44,"log_start":"45'8200","ondisk_log_start":"45'8200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6225920,"num_objects":1520,"num_object_clones":0,"num_object_copies":3040,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1520,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16061,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18766","reported_seq":18905,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:27.878477+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:21:27.878477+0000","last_peered":"2024-09-11T01:21:27.878477+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:21:27.878477+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6135808,"num_objects":1498,"num_object_clones":0,"num_object_copies":2996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15503,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2452}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007532+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007532+0000","last_peered":"2024-09-11T01:20:22.007532+0000","last_clean":"2024-09-11T01:20:22.007532+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007532+0000","last_undegraded":"2024-09-11T01:20:22.007532+0000","last_fullsized":"2024-09-11T01:20:22.007532+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18380","reported_seq":26421,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:27.546664+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:21:27.546664+0000","last_peered":"2024-09-11T01:21:27.546664+0000","last_clean":"2024-09-11T01:21:27.546664+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:21:27.546664+0000","last_undegraded":"2024-09-11T01:21:27.546664+0000","last_fullsized":"2024-09-11T01:21:27.546664+0000","mapping_epoch":46,"log_start":"45'8300","ondisk_log_start":"45'8300","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6160384,"num_objects":1504,"num_object_clones":0,"num_object_copies":3008,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1504,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15035,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18660","reported_seq":21159,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:27.914915+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:21:27.914915+0000","last_peered":"2024-09-11T01:21:27.914915+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:21:27.914915+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6209536,"num_objects":1516,"num_object_clones":0,"num_object_copies":3032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2864,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15269,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1177,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1348}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":79,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007560+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007560+0000","last_peered":"2024-09-11T01:20:22.007560+0000","last_clean":"2024-09-11T01:20:22.007560+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007560+0000","last_undegraded":"2024-09-11T01:20:22.007560+0000","last_fullsized":"2024-09-11T01:20:22.007560+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008171+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008171+0000","last_peered":"2024-09-11T01:20:22.008171+0000","last_clean":"2024-09-11T01:20:22.008171+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008171+0000","last_undegraded":"2024-09-11T01:20:22.008171+0000","last_fullsized":"2024-09-11T01:20:22.008171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18539","reported_seq":18681,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:27.887759+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:21:27.887759+0000","last_peered":"2024-09-11T01:21:27.887759+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:21:27.887759+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5788265,"num_objects":1414,"num_object_clones":0,"num_object_copies":2828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15609,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2503}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":99684993,"num_objects":24339,"num_object_clones":0,"num_object_copies":48678,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24339,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":257948,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12982,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393736192,"data_stored":393672384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160698,"ondisk_log_size":160698,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182519,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":573664,"kb_used_data":136652,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93798176,"statfs":{"total":96636764160,"available":96049332224,"internally_reserved":0,"allocated":139931648,"data_stored":139651494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":447475371},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,1,6,1,1,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986291,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":521272,"kb_used_data":178664,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93850568,"statfs":{"total":96636764160,"available":96102981632,"internally_reserved":0,"allocated":182951936,"data_stored":182671787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,1,4,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659236,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451684,"kb_used_data":72976,"kb_used_omap":4,"kb_used_meta":378683,"kb_avail":93920156,"statfs":{"total":96636764160,"available":96174239744,"internally_reserved":0,"allocated":74727424,"data_stored":74397971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":387772079},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,5,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":138104832,"data_stored":138101353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73502720,"data_stored":73449942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":182128640,"data_stored":182121089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:32.084 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:32.939 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:21:32.945 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:21:33.001 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":357,"stamp":"2024-09-11T01:21:32.772416+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":95261332,"num_objects":23261,"num_object_clones":0,"num_object_copies":46522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23261,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":259030,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12983,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160781,"ondisk_log_size":160781,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1561076,"kb_used_data":376380,"kb_used_omap":12,"kb_used_meta":1184627,"kb_avail":281554444,"statfs":{"total":289910292480,"available":288311750656,"internally_reserved":0,"allocated":385413120,"data_stored":384523364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1213058052},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,1,8,3,5,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20639744,"num_objects":-5039,"num_object_clones":0,"num_object_copies":-10078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-180,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5039,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008087"},"pg_stats":[{"pgid":"2.f","version":"49'18794","reported_seq":18882,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:27.754443+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:21:27.754443+0000","last_peered":"2024-09-11T01:21:27.754443+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:21:27.754443+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6029312,"num_objects":1472,"num_object_clones":0,"num_object_copies":2944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18794,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2600}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18898","reported_seq":19010,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:30.170821+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:21:30.170821+0000","last_peered":"2024-09-11T01:21:30.170821+0000","last_clean":"2024-09-11T01:21:30.170821+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:21:30.170821+0000","last_undegraded":"2024-09-11T01:21:30.170821+0000","last_fullsized":"2024-09-11T01:21:30.170821+0000","mapping_epoch":35,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5955584,"num_objects":1454,"num_object_clones":0,"num_object_copies":2908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17137,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18675","reported_seq":18814,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:30.067973+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:21:30.067973+0000","last_peered":"2024-09-11T01:21:30.067973+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:21:30.067973+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6012952,"num_objects":1469,"num_object_clones":0,"num_object_copies":2938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1469,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16976,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2427}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'18719","reported_seq":18798,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:30.188210+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:21:30.188210+0000","last_peered":"2024-09-11T01:21:30.188210+0000","last_clean":"2024-09-11T01:21:30.188210+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:21:30.188210+0000","last_undegraded":"2024-09-11T01:21:30.188210+0000","last_fullsized":"2024-09-11T01:21:30.188210+0000","mapping_epoch":35,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5795840,"num_objects":1415,"num_object_clones":0,"num_object_copies":2830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18719,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'18514","reported_seq":18652,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:27.842717+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:21:27.842717+0000","last_peered":"2024-09-11T01:21:27.842717+0000","last_clean":"2024-09-11T01:21:27.842717+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:21:27.842717+0000","last_undegraded":"2024-09-11T01:21:27.842717+0000","last_fullsized":"2024-09-11T01:21:27.842717+0000","mapping_epoch":34,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6123520,"num_objects":1495,"num_object_clones":0,"num_object_copies":2990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15519,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18704","reported_seq":18884,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:27.806414+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:21:27.806414+0000","last_peered":"2024-09-11T01:21:27.806414+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:21:27.806414+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6238208,"num_objects":1523,"num_object_clones":0,"num_object_copies":3046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15045,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2428}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'18412","reported_seq":18551,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:27.895050+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:21:27.895050+0000","last_peered":"2024-09-11T01:21:27.895050+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:21:27.895050+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'8400","ondisk_log_start":"45'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6217728,"num_objects":1518,"num_object_clones":0,"num_object_copies":3036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15269,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18725","reported_seq":18864,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:27.859856+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:21:27.859856+0000","last_peered":"2024-09-11T01:21:27.859856+0000","last_clean":"2024-09-11T01:21:27.859856+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:21:27.859856+0000","last_undegraded":"2024-09-11T01:21:27.859856+0000","last_fullsized":"2024-09-11T01:21:27.859856+0000","mapping_epoch":34,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6262784,"num_objects":1529,"num_object_clones":0,"num_object_copies":3058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1529,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15601,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003209+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003209+0000","last_peered":"2024-09-11T01:20:22.003209+0000","last_clean":"2024-09-11T01:20:22.003209+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003209+0000","last_undegraded":"2024-09-11T01:20:22.003209+0000","last_fullsized":"2024-09-11T01:20:22.003209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18668","reported_seq":26686,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:32.605856+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:21:32.605856+0000","last_peered":"2024-09-11T01:21:32.605856+0000","last_clean":"2024-09-11T01:21:32.605856+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:21:32.605856+0000","last_undegraded":"2024-09-11T01:21:32.605856+0000","last_fullsized":"2024-09-11T01:21:32.605856+0000","mapping_epoch":48,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5345280,"num_objects":1305,"num_object_clones":0,"num_object_copies":2610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16639,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18387","reported_seq":18526,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:27.803723+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:21:27.803723+0000","last_peered":"2024-09-11T01:21:27.803723+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:21:27.803723+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'8300","ondisk_log_start":"45'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6066176,"num_objects":1481,"num_object_clones":0,"num_object_copies":2962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1481,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14974,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2503}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'18767","reported_seq":18894,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:30.172189+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:21:30.172189+0000","last_peered":"2024-09-11T01:21:30.172189+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:21:30.172189+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5898240,"num_objects":1440,"num_object_clones":0,"num_object_copies":2880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16584,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2584}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003275+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003275+0000","last_peered":"2024-09-11T01:20:22.003275+0000","last_clean":"2024-09-11T01:20:22.003275+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003275+0000","last_undegraded":"2024-09-11T01:20:22.003275+0000","last_fullsized":"2024-09-11T01:20:22.003275+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18410","reported_seq":26400,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:30.118888+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:21:30.118888+0000","last_peered":"2024-09-11T01:21:30.118888+0000","last_clean":"2024-09-11T01:21:30.118888+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:21:30.118888+0000","last_undegraded":"2024-09-11T01:21:30.118888+0000","last_fullsized":"2024-09-11T01:21:30.118888+0000","mapping_epoch":44,"log_start":"45'8400","ondisk_log_start":"45'8400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5677056,"num_objects":1386,"num_object_clones":0,"num_object_copies":2772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16195,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18766","reported_seq":18905,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:27.878477+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:21:27.878477+0000","last_peered":"2024-09-11T01:21:27.878477+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:21:27.878477+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6135808,"num_objects":1498,"num_object_clones":0,"num_object_copies":2996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15503,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2452}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007577+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007577+0000","last_peered":"2024-09-11T01:20:22.007577+0000","last_clean":"2024-09-11T01:20:22.007577+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007577+0000","last_undegraded":"2024-09-11T01:20:22.007577+0000","last_fullsized":"2024-09-11T01:20:22.007577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18540","reported_seq":26581,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:32.635504+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:21:32.635504+0000","last_peered":"2024-09-11T01:21:32.635504+0000","last_clean":"2024-09-11T01:21:32.635504+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:21:32.635504+0000","last_undegraded":"2024-09-11T01:21:32.635504+0000","last_fullsized":"2024-09-11T01:21:32.635504+0000","mapping_epoch":46,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5505024,"num_objects":1344,"num_object_clones":0,"num_object_copies":2688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15195,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18660","reported_seq":21159,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:27.914915+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:21:27.914915+0000","last_peered":"2024-09-11T01:21:27.914915+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:21:27.914915+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6209536,"num_objects":1516,"num_object_clones":0,"num_object_copies":3032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2864,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15269,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1177,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1348}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007616+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007616+0000","last_peered":"2024-09-11T01:20:22.007616+0000","last_clean":"2024-09-11T01:20:22.007616+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007616+0000","last_undegraded":"2024-09-11T01:20:22.007616+0000","last_fullsized":"2024-09-11T01:20:22.007616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008171+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008171+0000","last_peered":"2024-09-11T01:20:22.008171+0000","last_clean":"2024-09-11T01:20:22.008171+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008171+0000","last_undegraded":"2024-09-11T01:20:22.008171+0000","last_fullsized":"2024-09-11T01:20:22.008171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18539","reported_seq":18681,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:27.887759+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:21:27.887759+0000","last_peered":"2024-09-11T01:21:27.887759+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:21:27.887759+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5788265,"num_objects":1414,"num_object_clones":0,"num_object_copies":2828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15609,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2503}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":95261313,"num_objects":23259,"num_object_clones":0,"num_object_copies":46518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23259,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":259028,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12982,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":381497344,"data_stored":381433536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160778,"ondisk_log_size":160778,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182519,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":573664,"kb_used_data":136652,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93798176,"statfs":{"total":96636764160,"available":96049332224,"internally_reserved":0,"allocated":139931648,"data_stored":139651494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":447475371},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,1,6,1,1,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986292,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520648,"kb_used_data":173876,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93851192,"statfs":{"total":96636764160,"available":96103620608,"internally_reserved":0,"allocated":178049024,"data_stored":177768875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659237,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":466764,"kb_used_data":65852,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93905076,"statfs":{"total":96636764160,"available":96158797824,"internally_reserved":0,"allocated":67432448,"data_stored":67102995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":410513071},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":138104832,"data_stored":138101353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":66183168,"data_stored":66130390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":177209344,"data_stored":177201793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:33.002 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:33.951 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:21:33.951 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:21:34.004 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":357,"stamp":"2024-09-11T01:21:32.772416+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":95261332,"num_objects":23261,"num_object_clones":0,"num_object_copies":46522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23261,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":259030,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12983,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160781,"ondisk_log_size":160781,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1561076,"kb_used_data":376380,"kb_used_omap":12,"kb_used_meta":1184627,"kb_avail":281554444,"statfs":{"total":289910292480,"available":288311750656,"internally_reserved":0,"allocated":385413120,"data_stored":384523364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1213058052},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,5,1,8,3,5,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-20639744,"num_objects":-5039,"num_object_clones":0,"num_object_copies":-10078,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-180,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5039,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5039,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":180,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008087"},"pg_stats":[{"pgid":"2.f","version":"49'18794","reported_seq":18882,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:27.754443+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:21:27.754443+0000","last_peered":"2024-09-11T01:21:27.754443+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:21:27.754443+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6029312,"num_objects":1472,"num_object_clones":0,"num_object_copies":2944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18794,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2600}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18898","reported_seq":19010,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:30.170821+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:21:30.170821+0000","last_peered":"2024-09-11T01:21:30.170821+0000","last_clean":"2024-09-11T01:21:30.170821+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:21:30.170821+0000","last_undegraded":"2024-09-11T01:21:30.170821+0000","last_fullsized":"2024-09-11T01:21:30.170821+0000","mapping_epoch":35,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5955584,"num_objects":1454,"num_object_clones":0,"num_object_copies":2908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17137,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18675","reported_seq":18814,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:30.067973+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:21:30.067973+0000","last_peered":"2024-09-11T01:21:30.067973+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:21:30.067973+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6012952,"num_objects":1469,"num_object_clones":0,"num_object_copies":2938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1469,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16976,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2427}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'18719","reported_seq":18798,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:30.188210+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:21:30.188210+0000","last_peered":"2024-09-11T01:21:30.188210+0000","last_clean":"2024-09-11T01:21:30.188210+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:21:30.188210+0000","last_undegraded":"2024-09-11T01:21:30.188210+0000","last_fullsized":"2024-09-11T01:21:30.188210+0000","mapping_epoch":35,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5795840,"num_objects":1415,"num_object_clones":0,"num_object_copies":2830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18719,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'18514","reported_seq":18652,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:27.842717+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:21:27.842717+0000","last_peered":"2024-09-11T01:21:27.842717+0000","last_clean":"2024-09-11T01:21:27.842717+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:21:27.842717+0000","last_undegraded":"2024-09-11T01:21:27.842717+0000","last_fullsized":"2024-09-11T01:21:27.842717+0000","mapping_epoch":34,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6123520,"num_objects":1495,"num_object_clones":0,"num_object_copies":2990,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1495,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15519,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18704","reported_seq":18884,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:27.806414+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:21:27.806414+0000","last_peered":"2024-09-11T01:21:27.806414+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:21:27.806414+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6238208,"num_objects":1523,"num_object_clones":0,"num_object_copies":3046,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1523,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15045,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2428}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'18412","reported_seq":18551,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:27.895050+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:21:27.895050+0000","last_peered":"2024-09-11T01:21:27.895050+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:21:27.895050+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'8400","ondisk_log_start":"45'8400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6217728,"num_objects":1518,"num_object_clones":0,"num_object_copies":3036,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1518,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15269,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2468}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18725","reported_seq":18864,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:27.859856+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:21:27.859856+0000","last_peered":"2024-09-11T01:21:27.859856+0000","last_clean":"2024-09-11T01:21:27.859856+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:21:27.859856+0000","last_undegraded":"2024-09-11T01:21:27.859856+0000","last_fullsized":"2024-09-11T01:21:27.859856+0000","mapping_epoch":34,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6262784,"num_objects":1529,"num_object_clones":0,"num_object_copies":3058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1529,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15601,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003209+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003209+0000","last_peered":"2024-09-11T01:20:22.003209+0000","last_clean":"2024-09-11T01:20:22.003209+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003209+0000","last_undegraded":"2024-09-11T01:20:22.003209+0000","last_fullsized":"2024-09-11T01:20:22.003209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18668","reported_seq":26686,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:32.605856+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:21:32.605856+0000","last_peered":"2024-09-11T01:21:32.605856+0000","last_clean":"2024-09-11T01:21:32.605856+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:21:32.605856+0000","last_undegraded":"2024-09-11T01:21:32.605856+0000","last_fullsized":"2024-09-11T01:21:32.605856+0000","mapping_epoch":48,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5345280,"num_objects":1305,"num_object_clones":0,"num_object_copies":2610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16639,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18387","reported_seq":18526,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:27.803723+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:21:27.803723+0000","last_peered":"2024-09-11T01:21:27.803723+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:21:27.803723+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'8300","ondisk_log_start":"45'8300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10087,"log_dups_size":0,"ondisk_log_size":10087,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6066176,"num_objects":1481,"num_object_clones":0,"num_object_copies":2962,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1481,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":14974,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2503}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'18767","reported_seq":18894,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:30.172189+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:21:30.172189+0000","last_peered":"2024-09-11T01:21:30.172189+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:21:30.172189+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5898240,"num_objects":1440,"num_object_clones":0,"num_object_copies":2880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16584,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2584}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003275+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003275+0000","last_peered":"2024-09-11T01:20:22.003275+0000","last_clean":"2024-09-11T01:20:22.003275+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003275+0000","last_undegraded":"2024-09-11T01:20:22.003275+0000","last_fullsized":"2024-09-11T01:20:22.003275+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18410","reported_seq":26400,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:30.118888+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:21:30.118888+0000","last_peered":"2024-09-11T01:21:30.118888+0000","last_clean":"2024-09-11T01:21:30.118888+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:21:30.118888+0000","last_undegraded":"2024-09-11T01:21:30.118888+0000","last_fullsized":"2024-09-11T01:21:30.118888+0000","mapping_epoch":44,"log_start":"45'8400","ondisk_log_start":"45'8400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5677056,"num_objects":1386,"num_object_clones":0,"num_object_copies":2772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16195,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18766","reported_seq":18905,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:27.878477+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:21:27.878477+0000","last_peered":"2024-09-11T01:21:27.878477+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:21:27.878477+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6135808,"num_objects":1498,"num_object_clones":0,"num_object_copies":2996,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1498,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15503,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2452}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007577+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007577+0000","last_peered":"2024-09-11T01:20:22.007577+0000","last_clean":"2024-09-11T01:20:22.007577+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007577+0000","last_undegraded":"2024-09-11T01:20:22.007577+0000","last_fullsized":"2024-09-11T01:20:22.007577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18540","reported_seq":26581,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:32.635504+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:21:32.635504+0000","last_peered":"2024-09-11T01:21:32.635504+0000","last_clean":"2024-09-11T01:21:32.635504+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:21:32.635504+0000","last_undegraded":"2024-09-11T01:21:32.635504+0000","last_fullsized":"2024-09-11T01:21:32.635504+0000","mapping_epoch":46,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5505024,"num_objects":1344,"num_object_clones":0,"num_object_copies":2688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15195,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18660","reported_seq":21159,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:27.914915+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:21:27.914915+0000","last_peered":"2024-09-11T01:21:27.914915+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:21:27.914915+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6209536,"num_objects":1516,"num_object_clones":0,"num_object_copies":3032,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2864,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1516,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15269,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1177,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1348}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007616+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007616+0000","last_peered":"2024-09-11T01:20:22.007616+0000","last_clean":"2024-09-11T01:20:22.007616+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007616+0000","last_undegraded":"2024-09-11T01:20:22.007616+0000","last_fullsized":"2024-09-11T01:20:22.007616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008171+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008171+0000","last_peered":"2024-09-11T01:20:22.008171+0000","last_clean":"2024-09-11T01:20:22.008171+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008171+0000","last_undegraded":"2024-09-11T01:20:22.008171+0000","last_fullsized":"2024-09-11T01:20:22.008171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18539","reported_seq":18681,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:27.887759+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:21:27.887759+0000","last_peered":"2024-09-11T01:21:27.887759+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:21:27.887759+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5788265,"num_objects":1414,"num_object_clones":0,"num_object_copies":2828,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1414,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15609,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2503}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":95261313,"num_objects":23259,"num_object_clones":0,"num_object_copies":46518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34657,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23259,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":259028,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12982,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":381497344,"data_stored":381433536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160778,"ondisk_log_size":160778,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182519,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":573664,"kb_used_data":136652,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93798176,"statfs":{"total":96636764160,"available":96049332224,"internally_reserved":0,"allocated":139931648,"data_stored":139651494,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":447475371},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,4,1,6,1,1,1,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986292,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520648,"kb_used_data":173876,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93851192,"statfs":{"total":96636764160,"available":96103620608,"internally_reserved":0,"allocated":178049024,"data_stored":177768875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659237,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":466764,"kb_used_data":65852,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93905076,"statfs":{"total":96636764160,"available":96158797824,"internally_reserved":0,"allocated":67432448,"data_stored":67102995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":410513071},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":138104832,"data_stored":138101353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":66183168,"data_stored":66130390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":177209344,"data_stored":177201793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:34.006 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:21:34.006 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'18794", 'reported_seq': 18882, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:27.754443+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:21:27.754443+0000', 'last_peered': '2024-09-11T01:21:27.754443+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:21:27.754443+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "45'8700", 'ondisk_log_start': "45'8700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6029312, 'num_objects': 1472, 'num_object_clones': 0, 'num_object_copies': 2944, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1472, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18794, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2600}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:34.006 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:21:34.006 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'18675", 'reported_seq': 18814, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:30.067973+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:21:30.067973+0000', 'last_peered': '2024-09-11T01:21:30.067973+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:21:30.067973+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "45'8600", 'ondisk_log_start': "45'8600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10075, 'log_dups_size': 0, 'ondisk_log_size': 10075, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6012952, 'num_objects': 1469, 'num_object_clones': 0, 'num_object_copies': 2938, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1469, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 16976, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2427}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:34.006 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:21:34.006 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'18704", 'reported_seq': 18884, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:27.806414+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:21:27.806414+0000', 'last_peered': '2024-09-11T01:21:27.806414+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:21:27.806414+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'8700", 'ondisk_log_start': "45'8700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10004, 'log_dups_size': 0, 'ondisk_log_size': 10004, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6238208, 'num_objects': 1523, 'num_object_clones': 0, 'num_object_copies': 3046, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1523, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15045, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2428}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:34.006 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:21:34.006 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'18412", 'reported_seq': 18551, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:27.895050+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:21:27.895050+0000', 'last_peered': '2024-09-11T01:21:27.895050+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:21:27.895050+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'8400", 'ondisk_log_start': "45'8400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6217728, 'num_objects': 1518, 'num_object_clones': 0, 'num_object_copies': 3036, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1518, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15269, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2468}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:34.007 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:21:34.007 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'18387", 'reported_seq': 18526, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:27.803723+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:21:27.803723+0000', 'last_peered': '2024-09-11T01:21:27.803723+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:21:27.803723+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'8300", 'ondisk_log_start': "45'8300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10087, 'log_dups_size': 0, 'ondisk_log_size': 10087, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6066176, 'num_objects': 1481, 'num_object_clones': 0, 'num_object_copies': 2962, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1481, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 14974, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2503}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:34.007 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:21:34.007 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'18767", 'reported_seq': 18894, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:30.172189+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:21:30.172189+0000', 'last_peered': '2024-09-11T01:21:30.172189+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:21:30.172189+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "45'8700", 'ondisk_log_start': "45'8700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10067, 'log_dups_size': 0, 'ondisk_log_size': 10067, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5898240, 'num_objects': 1440, 'num_object_clones': 0, 'num_object_copies': 2880, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1440, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16584, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2584}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:34.007 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:21:34.007 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'18766", 'reported_seq': 18905, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:27.878477+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:21:27.878477+0000', 'last_peered': '2024-09-11T01:21:27.878477+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:21:27.878477+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'8700", 'ondisk_log_start': "45'8700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6135808, 'num_objects': 1498, 'num_object_clones': 0, 'num_object_copies': 2996, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1498, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15503, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2452}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:34.007 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:21:34.007 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'18660", 'reported_seq': 21159, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:27.914915+0000', 'last_change': '2024-09-11T01:20:20.985862+0000', 'last_active': '2024-09-11T01:21:27.914915+0000', 'last_peered': '2024-09-11T01:21:27.914915+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:21:27.914915+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "45'8600", 'ondisk_log_start': "45'8600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10060, 'log_dups_size': 0, 'ondisk_log_size': 10060, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 6209536, 'num_objects': 1516, 'num_object_clones': 0, 'num_object_copies': 3032, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2864, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1516, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15269, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1177, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1348}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:34.007 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:21:34.008 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'18539", 'reported_seq': 18681, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:27.887759+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:21:27.887759+0000', 'last_peered': '2024-09-11T01:21:27.887759+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:21:27.887759+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'8500", 'ondisk_log_start': "45'8500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5788265, 'num_objects': 1414, 'num_object_clones': 0, 'num_object_copies': 2828, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1414, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15609, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2503}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:34.008 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:21:34.978 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:21:34.978 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34554/43874 objects degraded (78.757%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":631,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":21937,"data_bytes":89838228,"bytes_used":1582575616,"bytes_avail":288327716864,"bytes_total":289910292480,"degraded_objects":34554,"degraded_total":43874,"degraded_ratio":0.78757350594885356,"recovering_objects_per_sec":23,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":283,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":504},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:21:35.037 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:21:35.037 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:35.894 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:21:35.896 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:21:35.942 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":358,"stamp":"2024-09-11T01:21:34.772867+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":89838228,"num_objects":21937,"num_object_clones":0,"num_object_copies":43874,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34554,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21937,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":260354,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13086,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160905,"ondisk_log_size":160905,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1545484,"kb_used_data":369112,"kb_used_omap":12,"kb_used_meta":1176307,"kb_avail":281570036,"statfs":{"total":289910292480,"available":288327716864,"internally_reserved":0,"allocated":377970688,"data_stored":377080932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1204538372},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2,3,6,6,8,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-24801280,"num_objects":-6055,"num_object_clones":0,"num_object_copies":-12110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6055,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6055,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":283,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008114"},"pg_stats":[{"pgid":"2.f","version":"49'18943","reported_seq":19031,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:32.740954+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:21:32.740954+0000","last_peered":"2024-09-11T01:21:32.740954+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:21:32.740954+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10043,"log_dups_size":0,"ondisk_log_size":10043,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5419008,"num_objects":1323,"num_object_clones":0,"num_object_copies":2646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1323,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18943,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2749}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'18898","reported_seq":19010,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:30.170821+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:21:30.170821+0000","last_peered":"2024-09-11T01:21:30.170821+0000","last_clean":"2024-09-11T01:21:30.170821+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:21:30.170821+0000","last_undegraded":"2024-09-11T01:21:30.170821+0000","last_fullsized":"2024-09-11T01:21:30.170821+0000","mapping_epoch":35,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10098,"log_dups_size":0,"ondisk_log_size":10098,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5955584,"num_objects":1454,"num_object_clones":0,"num_object_copies":2908,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1454,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17137,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18675","reported_seq":18814,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:30.067973+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:21:30.067973+0000","last_peered":"2024-09-11T01:21:30.067973+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:21:30.067973+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":6012952,"num_objects":1469,"num_object_clones":0,"num_object_copies":2938,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1469,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":16976,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2427}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'18719","reported_seq":18798,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:30.188210+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:21:30.188210+0000","last_peered":"2024-09-11T01:21:30.188210+0000","last_clean":"2024-09-11T01:21:30.188210+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:21:30.188210+0000","last_undegraded":"2024-09-11T01:21:30.188210+0000","last_fullsized":"2024-09-11T01:21:30.188210+0000","mapping_epoch":35,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5795840,"num_objects":1415,"num_object_clones":0,"num_object_copies":2830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1415,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18719,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'18659","reported_seq":18797,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:32.791125+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:21:32.791125+0000","last_peered":"2024-09-11T01:21:32.791125+0000","last_clean":"2024-09-11T01:21:32.791125+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:21:32.791125+0000","last_undegraded":"2024-09-11T01:21:32.791125+0000","last_fullsized":"2024-09-11T01:21:32.791125+0000","mapping_epoch":34,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5529600,"num_objects":1350,"num_object_clones":0,"num_object_copies":2700,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1350,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15664,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'18863","reported_seq":19043,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:32.901773+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:21:32.901773+0000","last_peered":"2024-09-11T01:21:32.901773+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:21:32.901773+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5586944,"num_objects":1364,"num_object_clones":0,"num_object_copies":2728,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1364,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15204,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2587}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'18558","reported_seq":18695,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:32.838050+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:21:32.838050+0000","last_peered":"2024-09-11T01:21:32.838050+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:21:32.838050+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5619712,"num_objects":1372,"num_object_clones":0,"num_object_copies":2744,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1372,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15415,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2614}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'18859","reported_seq":18999,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:32.834854+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:21:32.834854+0000","last_peered":"2024-09-11T01:21:32.834854+0000","last_clean":"2024-09-11T01:21:32.834854+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:21:32.834854+0000","last_undegraded":"2024-09-11T01:21:32.834854+0000","last_fullsized":"2024-09-11T01:21:32.834854+0000","mapping_epoch":34,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5713920,"num_objects":1395,"num_object_clones":0,"num_object_copies":2790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15735,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003209+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003209+0000","last_peered":"2024-09-11T01:20:22.003209+0000","last_clean":"2024-09-11T01:20:22.003209+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003209+0000","last_undegraded":"2024-09-11T01:20:22.003209+0000","last_fullsized":"2024-09-11T01:20:22.003209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18668","reported_seq":26686,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:32.605856+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:21:32.605856+0000","last_peered":"2024-09-11T01:21:32.605856+0000","last_clean":"2024-09-11T01:21:32.605856+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:21:32.605856+0000","last_undegraded":"2024-09-11T01:21:32.605856+0000","last_fullsized":"2024-09-11T01:21:32.605856+0000","mapping_epoch":48,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5345280,"num_objects":1305,"num_object_clones":0,"num_object_copies":2610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16639,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18535","reported_seq":18674,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:32.843676+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:21:32.843676+0000","last_peered":"2024-09-11T01:21:32.843676+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:21:32.843676+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5459968,"num_objects":1333,"num_object_clones":0,"num_object_copies":2666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15122,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2651}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'18767","reported_seq":18894,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:30.172189+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:21:30.172189+0000","last_peered":"2024-09-11T01:21:30.172189+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:21:30.172189+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5898240,"num_objects":1440,"num_object_clones":0,"num_object_copies":2880,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1440,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16584,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2584}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003275+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003275+0000","last_peered":"2024-09-11T01:20:22.003275+0000","last_clean":"2024-09-11T01:20:22.003275+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003275+0000","last_undegraded":"2024-09-11T01:20:22.003275+0000","last_fullsized":"2024-09-11T01:20:22.003275+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18410","reported_seq":26400,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:30.118888+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:21:30.118888+0000","last_peered":"2024-09-11T01:21:30.118888+0000","last_clean":"2024-09-11T01:21:30.118888+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:21:30.118888+0000","last_undegraded":"2024-09-11T01:21:30.118888+0000","last_fullsized":"2024-09-11T01:21:30.118888+0000","mapping_epoch":44,"log_start":"45'8400","ondisk_log_start":"45'8400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5677056,"num_objects":1386,"num_object_clones":0,"num_object_copies":2772,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1386,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16195,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'18926","reported_seq":19064,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:32.903877+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:21:32.903877+0000","last_peered":"2024-09-11T01:21:32.903877+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:21:32.903877+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5480448,"num_objects":1338,"num_object_clones":0,"num_object_copies":2676,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1338,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15663,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2612}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007577+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007577+0000","last_peered":"2024-09-11T01:20:22.007577+0000","last_clean":"2024-09-11T01:20:22.007577+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007577+0000","last_undegraded":"2024-09-11T01:20:22.007577+0000","last_fullsized":"2024-09-11T01:20:22.007577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18540","reported_seq":26581,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:32.635504+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:21:32.635504+0000","last_peered":"2024-09-11T01:21:32.635504+0000","last_clean":"2024-09-11T01:21:32.635504+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:21:32.635504+0000","last_undegraded":"2024-09-11T01:21:32.635504+0000","last_fullsized":"2024-09-11T01:21:32.635504+0000","mapping_epoch":46,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10040,"log_dups_size":0,"ondisk_log_size":10040,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5505024,"num_objects":1344,"num_object_clones":0,"num_object_copies":2688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15195,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18799","reported_seq":21505,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:32.917250+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:21:32.917250+0000","last_peered":"2024-09-11T01:21:32.917250+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:21:32.917250+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5640192,"num_objects":1377,"num_object_clones":0,"num_object_copies":2754,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2761,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1377,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15408,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1280,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1384}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007616+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007616+0000","last_peered":"2024-09-11T01:20:22.007616+0000","last_clean":"2024-09-11T01:20:22.007616+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007616+0000","last_undegraded":"2024-09-11T01:20:22.007616+0000","last_fullsized":"2024-09-11T01:20:22.007616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008171+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008171+0000","last_peered":"2024-09-11T01:20:22.008171+0000","last_clean":"2024-09-11T01:20:22.008171+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008171+0000","last_undegraded":"2024-09-11T01:20:22.008171+0000","last_fullsized":"2024-09-11T01:20:22.008171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18683","reported_seq":18827,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:32.921107+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:21:32.921107+0000","last_peered":"2024-09-11T01:21:32.921107+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:21:32.921107+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10083,"log_dups_size":0,"ondisk_log_size":10083,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5198441,"num_objects":1270,"num_object_clones":0,"num_object_copies":2540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15753,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2647}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":89838209,"num_objects":21935,"num_object_clones":0,"num_object_copies":43870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34554,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21935,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":260352,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13085,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":374226944,"data_stored":374163136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160902,"ondisk_log_size":160902,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182520,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":558072,"kb_used_data":129384,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93813768,"statfs":{"total":96636764160,"available":96065298432,"internally_reserved":0,"allocated":132489216,"data_stored":132209062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":438955691},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,4,2,6,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986292,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":520648,"kb_used_data":173876,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93851192,"statfs":{"total":96636764160,"available":96103620608,"internally_reserved":0,"allocated":178049024,"data_stored":177768875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,3,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659237,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":466764,"kb_used_data":65852,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93905076,"statfs":{"total":96636764160,"available":96158797824,"internally_reserved":0,"allocated":67432448,"data_stored":67102995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":410513071},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,1,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":130834432,"data_stored":130830953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":66183168,"data_stored":66130390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":177209344,"data_stored":177201793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:38.944 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:40.705 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:21:40.707 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:21:40.784 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":360,"stamp":"2024-09-11T01:21:38.773752+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":80315028,"num_objects":19612,"num_object_clones":0,"num_object_copies":39224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34464,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19612,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":262679,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13176,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160830,"ondisk_log_size":160830,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1514564,"kb_used_data":350656,"kb_used_omap":12,"kb_used_meta":1163827,"kb_avail":281600956,"statfs":{"total":289910292480,"available":288359378944,"internally_reserved":0,"allocated":359071744,"data_stored":358181988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1191758852},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,4,7,8,6,10],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25956352,"num_objects":-6337,"num_object_clones":0,"num_object_copies":-12674,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-289,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6337,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6337,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":289,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008173"},"pg_stats":[{"pgid":"2.f","version":"49'19086","reported_seq":19171,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:37.911921+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:21:37.911921+0000","last_peered":"2024-09-11T01:21:37.911921+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:21:37.911921+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4833280,"num_objects":1180,"num_object_clones":0,"num_object_copies":2360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19086,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2892}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19047","reported_seq":19159,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:35.184527+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:21:35.184527+0000","last_peered":"2024-09-11T01:21:35.184527+0000","last_clean":"2024-09-11T01:21:35.184527+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:21:35.184527+0000","last_undegraded":"2024-09-11T01:21:35.184527+0000","last_fullsized":"2024-09-11T01:21:35.184527+0000","mapping_epoch":35,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5345280,"num_objects":1305,"num_object_clones":0,"num_object_copies":2610,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1305,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17286,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18822","reported_seq":18958,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:35.184847+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:21:35.184847+0000","last_peered":"2024-09-11T01:21:35.184847+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:21:35.184847+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10022,"log_dups_size":0,"ondisk_log_size":10022,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5410840,"num_objects":1322,"num_object_clones":0,"num_object_copies":2644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1322,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17123,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2574}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'18864","reported_seq":18944,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:35.161591+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:21:35.161591+0000","last_peered":"2024-09-11T01:21:35.161591+0000","last_clean":"2024-09-11T01:21:35.161591+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:21:35.161591+0000","last_undegraded":"2024-09-11T01:21:35.161591+0000","last_fullsized":"2024-09-11T01:21:35.161591+0000","mapping_epoch":35,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5201920,"num_objects":1270,"num_object_clones":0,"num_object_copies":2540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1270,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18864,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'18808","reported_seq":18946,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:37.871027+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:21:37.871027+0000","last_peered":"2024-09-11T01:21:37.871027+0000","last_clean":"2024-09-11T01:21:37.871027+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:21:37.871027+0000","last_undegraded":"2024-09-11T01:21:37.871027+0000","last_fullsized":"2024-09-11T01:21:37.871027+0000","mapping_epoch":34,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4919296,"num_objects":1201,"num_object_clones":0,"num_object_copies":2402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15813,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'19016","reported_seq":19195,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:37.903948+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:21:37.903948+0000","last_peered":"2024-09-11T01:21:37.903948+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:21:37.903948+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4960256,"num_objects":1211,"num_object_clones":0,"num_object_copies":2422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1211,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15357,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2740}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'18679","reported_seq":18820,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:37.951310+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:21:37.951310+0000","last_peered":"2024-09-11T01:21:37.951310+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:21:37.951310+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5124096,"num_objects":1251,"num_object_clones":0,"num_object_copies":2502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15536,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2735}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19000","reported_seq":19140,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:37.936328+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:21:37.936328+0000","last_peered":"2024-09-11T01:21:37.936328+0000","last_clean":"2024-09-11T01:21:37.936328+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:21:37.936328+0000","last_undegraded":"2024-09-11T01:21:37.936328+0000","last_fullsized":"2024-09-11T01:21:37.936328+0000","mapping_epoch":34,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5136384,"num_objects":1254,"num_object_clones":0,"num_object_copies":2508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15876,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003209+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003209+0000","last_peered":"2024-09-11T01:20:22.003209+0000","last_clean":"2024-09-11T01:20:22.003209+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003209+0000","last_undegraded":"2024-09-11T01:20:22.003209+0000","last_fullsized":"2024-09-11T01:20:22.003209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18813","reported_seq":26831,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:37.739595+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:21:37.739595+0000","last_peered":"2024-09-11T01:21:37.739595+0000","last_clean":"2024-09-11T01:21:37.739595+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:21:37.739595+0000","last_undegraded":"2024-09-11T01:21:37.739595+0000","last_fullsized":"2024-09-11T01:21:37.739595+0000","mapping_epoch":48,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4751360,"num_objects":1160,"num_object_clones":0,"num_object_copies":2320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16784,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18680","reported_seq":18815,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:37.947710+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:21:37.947710+0000","last_peered":"2024-09-11T01:21:37.947710+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:21:37.947710+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4866048,"num_objects":1188,"num_object_clones":0,"num_object_copies":2376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15267,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2796}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'18930","reported_seq":19059,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:35.115757+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:21:35.115757+0000","last_peered":"2024-09-11T01:21:35.115757+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:21:35.115757+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5230592,"num_objects":1277,"num_object_clones":0,"num_object_copies":2554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1277,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16747,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2747}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003275+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003275+0000","last_peered":"2024-09-11T01:20:22.003275+0000","last_clean":"2024-09-11T01:20:22.003275+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003275+0000","last_undegraded":"2024-09-11T01:20:22.003275+0000","last_fullsized":"2024-09-11T01:20:22.003275+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18554","reported_seq":26545,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:35.122256+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:21:35.122256+0000","last_peered":"2024-09-11T01:21:35.122256+0000","last_clean":"2024-09-11T01:21:35.122256+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:21:35.122256+0000","last_undegraded":"2024-09-11T01:21:35.122256+0000","last_fullsized":"2024-09-11T01:21:35.122256+0000","mapping_epoch":44,"log_start":"45'8500","ondisk_log_start":"45'8500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5087232,"num_objects":1242,"num_object_clones":0,"num_object_copies":2484,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1242,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16339,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19078","reported_seq":19220,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:37.903526+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:21:37.903526+0000","last_peered":"2024-09-11T01:21:37.903526+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:21:37.903526+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4857856,"num_objects":1186,"num_object_clones":0,"num_object_copies":2372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15815,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2764}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007577+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007577+0000","last_peered":"2024-09-11T01:20:22.007577+0000","last_clean":"2024-09-11T01:20:22.007577+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007577+0000","last_undegraded":"2024-09-11T01:20:22.007577+0000","last_fullsized":"2024-09-11T01:20:22.007577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18684","reported_seq":26725,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:37.712136+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:21:37.712136+0000","last_peered":"2024-09-11T01:21:37.712136+0000","last_clean":"2024-09-11T01:21:37.712136+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:21:37.712136+0000","last_undegraded":"2024-09-11T01:21:37.712136+0000","last_fullsized":"2024-09-11T01:21:37.712136+0000","mapping_epoch":46,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4915200,"num_objects":1200,"num_object_clones":0,"num_object_copies":2400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15339,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18948","reported_seq":21834,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:37.964674+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:21:37.964674+0000","last_peered":"2024-09-11T01:21:37.964674+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:21:37.964674+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5029888,"num_objects":1228,"num_object_clones":0,"num_object_copies":2456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2671,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15557,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1370,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1443}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007616+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007616+0000","last_peered":"2024-09-11T01:20:22.007616+0000","last_clean":"2024-09-11T01:20:22.007616+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007616+0000","last_undegraded":"2024-09-11T01:20:22.007616+0000","last_fullsized":"2024-09-11T01:20:22.007616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008171+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008171+0000","last_peered":"2024-09-11T01:20:22.008171+0000","last_clean":"2024-09-11T01:20:22.008171+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008171+0000","last_undegraded":"2024-09-11T01:20:22.008171+0000","last_fullsized":"2024-09-11T01:20:22.008171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18818","reported_seq":18962,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:37.870875+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:21:37.870875+0000","last_peered":"2024-09-11T01:21:37.870875+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:21:37.870875+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4645481,"num_objects":1135,"num_object_clones":0,"num_object_copies":2270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15888,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2782}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":80315009,"num_objects":19610,"num_object_clones":0,"num_object_copies":39220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34464,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19610,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":262677,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13175,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":355078144,"data_stored":355014336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160827,"ondisk_log_size":160827,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182521,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":555304,"kb_used_data":122456,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93816536,"statfs":{"total":96636764160,"available":96068132864,"internally_reserved":0,"allocated":125394944,"data_stored":125114790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":443215531},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,1,3,3,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986293,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":507752,"kb_used_data":169300,"kb_used_omap":4,"kb_used_meta":338427,"kb_avail":93864088,"statfs":{"total":96636764160,"available":96116826112,"internally_reserved":0,"allocated":173363200,"data_stored":173083051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":346549930},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,2,1,2,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659238,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451508,"kb_used_data":58900,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93920332,"statfs":{"total":96636764160,"available":96174419968,"internally_reserved":0,"allocated":60313600,"data_stored":59984147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":401993391},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,4,1,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":123711488,"data_stored":123708009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":58908672,"data_stored":58855894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":172457984,"data_stored":172450433,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:40.786 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:41.587 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:21:41.589 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:21:41.653 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":361,"stamp":"2024-09-11T01:21:40.774215+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":77103764,"num_objects":18828,"num_object_clones":0,"num_object_copies":37656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34464,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18828,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":263463,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13176,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160914,"ondisk_log_size":160914,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1513656,"kb_used_data":345592,"kb_used_omap":12,"kb_used_meta":1167987,"kb_avail":281601864,"statfs":{"total":289910292480,"available":288360308736,"internally_reserved":0,"allocated":353886208,"data_stored":352996452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1196018692},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,4,6,7,5,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22581248,"num_objects":-5513,"num_object_clones":0,"num_object_copies":-11026,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5513,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5513,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008297"},"pg_stats":[{"pgid":"2.f","version":"49'19086","reported_seq":19171,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:37.911921+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:21:37.911921+0000","last_peered":"2024-09-11T01:21:37.911921+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:21:37.911921+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4833280,"num_objects":1180,"num_object_clones":0,"num_object_copies":2360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19086,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2892}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19179","reported_seq":19291,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:40.103066+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:21:40.103066+0000","last_peered":"2024-09-11T01:21:40.103066+0000","last_clean":"2024-09-11T01:21:40.103066+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:21:40.103066+0000","last_undegraded":"2024-09-11T01:21:40.103066+0000","last_fullsized":"2024-09-11T01:21:40.103066+0000","mapping_epoch":35,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4804608,"num_objects":1173,"num_object_clones":0,"num_object_copies":2346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17418,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18976","reported_seq":19109,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:40.204427+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:21:40.204427+0000","last_peered":"2024-09-11T01:21:40.204427+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:21:40.204427+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4780056,"num_objects":1168,"num_object_clones":0,"num_object_copies":2336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1168,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17277,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2728}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'19026","reported_seq":19106,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:40.134714+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:21:40.134714+0000","last_peered":"2024-09-11T01:21:40.134714+0000","last_clean":"2024-09-11T01:21:40.134714+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:21:40.134714+0000","last_undegraded":"2024-09-11T01:21:40.134714+0000","last_fullsized":"2024-09-11T01:21:40.134714+0000","mapping_epoch":35,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4538368,"num_objects":1108,"num_object_clones":0,"num_object_copies":2216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1108,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19026,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'18808","reported_seq":18946,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:37.871027+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:21:37.871027+0000","last_peered":"2024-09-11T01:21:37.871027+0000","last_clean":"2024-09-11T01:21:37.871027+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:21:37.871027+0000","last_undegraded":"2024-09-11T01:21:37.871027+0000","last_fullsized":"2024-09-11T01:21:37.871027+0000","mapping_epoch":34,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4919296,"num_objects":1201,"num_object_clones":0,"num_object_copies":2402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15813,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'19016","reported_seq":19195,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:37.903948+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:21:37.903948+0000","last_peered":"2024-09-11T01:21:37.903948+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:21:37.903948+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4960256,"num_objects":1211,"num_object_clones":0,"num_object_copies":2422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1211,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15357,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2740}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'18679","reported_seq":18820,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:37.951310+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:21:37.951310+0000","last_peered":"2024-09-11T01:21:37.951310+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:21:37.951310+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5124096,"num_objects":1251,"num_object_clones":0,"num_object_copies":2502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15536,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2735}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19000","reported_seq":19140,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:37.936328+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:21:37.936328+0000","last_peered":"2024-09-11T01:21:37.936328+0000","last_clean":"2024-09-11T01:21:37.936328+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:21:37.936328+0000","last_undegraded":"2024-09-11T01:21:37.936328+0000","last_fullsized":"2024-09-11T01:21:37.936328+0000","mapping_epoch":34,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5136384,"num_objects":1254,"num_object_clones":0,"num_object_copies":2508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15876,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003209+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003209+0000","last_peered":"2024-09-11T01:20:22.003209+0000","last_clean":"2024-09-11T01:20:22.003209+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003209+0000","last_undegraded":"2024-09-11T01:20:22.003209+0000","last_fullsized":"2024-09-11T01:20:22.003209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18813","reported_seq":26831,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:37.739595+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:21:37.739595+0000","last_peered":"2024-09-11T01:21:37.739595+0000","last_clean":"2024-09-11T01:21:37.739595+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:21:37.739595+0000","last_undegraded":"2024-09-11T01:21:37.739595+0000","last_fullsized":"2024-09-11T01:21:37.739595+0000","mapping_epoch":48,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4751360,"num_objects":1160,"num_object_clones":0,"num_object_copies":2320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16784,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18680","reported_seq":18815,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:37.947710+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:21:37.947710+0000","last_peered":"2024-09-11T01:21:37.947710+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:21:37.947710+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4866048,"num_objects":1188,"num_object_clones":0,"num_object_copies":2376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15267,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2796}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19096","reported_seq":19225,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:40.154279+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:21:40.154279+0000","last_peered":"2024-09-11T01:21:40.154279+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:21:40.154279+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4550656,"num_objects":1111,"num_object_clones":0,"num_object_copies":2222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16913,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2913}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003275+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003275+0000","last_peered":"2024-09-11T01:20:22.003275+0000","last_clean":"2024-09-11T01:20:22.003275+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003275+0000","last_undegraded":"2024-09-11T01:20:22.003275+0000","last_fullsized":"2024-09-11T01:20:22.003275+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18724","reported_seq":26715,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:40.176601+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:21:40.176601+0000","last_peered":"2024-09-11T01:21:40.176601+0000","last_clean":"2024-09-11T01:21:40.176601+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:21:40.176601+0000","last_undegraded":"2024-09-11T01:21:40.176601+0000","last_fullsized":"2024-09-11T01:21:40.176601+0000","mapping_epoch":44,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4390912,"num_objects":1072,"num_object_clones":0,"num_object_copies":2144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16509,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19078","reported_seq":19220,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:37.903526+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:21:37.903526+0000","last_peered":"2024-09-11T01:21:37.903526+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:21:37.903526+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4857856,"num_objects":1186,"num_object_clones":0,"num_object_copies":2372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15815,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2764}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007577+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007577+0000","last_peered":"2024-09-11T01:20:22.007577+0000","last_clean":"2024-09-11T01:20:22.007577+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007577+0000","last_undegraded":"2024-09-11T01:20:22.007577+0000","last_fullsized":"2024-09-11T01:20:22.007577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18684","reported_seq":26725,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:37.712136+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:21:37.712136+0000","last_peered":"2024-09-11T01:21:37.712136+0000","last_clean":"2024-09-11T01:21:37.712136+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:21:37.712136+0000","last_undegraded":"2024-09-11T01:21:37.712136+0000","last_fullsized":"2024-09-11T01:21:37.712136+0000","mapping_epoch":46,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4915200,"num_objects":1200,"num_object_clones":0,"num_object_copies":2400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15339,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18948","reported_seq":21834,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:37.964674+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:21:37.964674+0000","last_peered":"2024-09-11T01:21:37.964674+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:21:37.964674+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5029888,"num_objects":1228,"num_object_clones":0,"num_object_copies":2456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2671,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15557,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1370,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1443}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007616+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007616+0000","last_peered":"2024-09-11T01:20:22.007616+0000","last_clean":"2024-09-11T01:20:22.007616+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007616+0000","last_undegraded":"2024-09-11T01:20:22.007616+0000","last_fullsized":"2024-09-11T01:20:22.007616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008171+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008171+0000","last_peered":"2024-09-11T01:20:22.008171+0000","last_clean":"2024-09-11T01:20:22.008171+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008171+0000","last_undegraded":"2024-09-11T01:20:22.008171+0000","last_fullsized":"2024-09-11T01:20:22.008171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18818","reported_seq":18962,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:37.870875+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:21:37.870875+0000","last_peered":"2024-09-11T01:21:37.870875+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:21:37.870875+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4645481,"num_objects":1135,"num_object_clones":0,"num_object_copies":2270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15888,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2782}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":77103745,"num_objects":18826,"num_object_clones":0,"num_object_copies":37652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34464,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18826,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":263461,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13175,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":350007296,"data_stored":349943488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160911,"ondisk_log_size":160911,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182521,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":555304,"kb_used_data":122456,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93816536,"statfs":{"total":96636764160,"available":96068132864,"internally_reserved":0,"allocated":125394944,"data_stored":125114790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":443215531},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,1,3,3,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986294,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506844,"kb_used_data":164236,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93864996,"statfs":{"total":96636764160,"available":96117755904,"internally_reserved":0,"allocated":168177664,"data_stored":167897515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659238,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451508,"kb_used_data":58900,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93920332,"statfs":{"total":96636764160,"available":96174419968,"internally_reserved":0,"allocated":60313600,"data_stored":59984147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":401993391},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,4,1,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":123711488,"data_stored":123708009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":58908672,"data_stored":58855894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":167387136,"data_stored":167379585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:41.654 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:21:41.654 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'19086", 'reported_seq': 19171, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:37.911921+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:21:37.911921+0000', 'last_peered': '2024-09-11T01:21:37.911921+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:21:37.911921+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "45'9000", 'ondisk_log_start': "45'9000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10086, 'log_dups_size': 0, 'ondisk_log_size': 10086, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4833280, 'num_objects': 1180, 'num_object_clones': 0, 'num_object_copies': 2360, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1180, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19086, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2892}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:41.654 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:21:41.654 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'18976", 'reported_seq': 19109, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:40.204427+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:21:40.204427+0000', 'last_peered': '2024-09-11T01:21:40.204427+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:21:40.204427+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "45'8900", 'ondisk_log_start': "45'8900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4780056, 'num_objects': 1168, 'num_object_clones': 0, 'num_object_copies': 2336, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1168, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17277, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2728}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:41.655 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:21:41.655 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'19016", 'reported_seq': 19195, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:37.903948+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:21:37.903948+0000', 'last_peered': '2024-09-11T01:21:37.903948+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:21:37.903948+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'9000", 'ondisk_log_start': "45'9000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10016, 'log_dups_size': 0, 'ondisk_log_size': 10016, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4960256, 'num_objects': 1211, 'num_object_clones': 0, 'num_object_copies': 2422, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1211, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15357, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2740}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:41.655 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:21:41.655 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'18679", 'reported_seq': 18820, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:37.951310+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:21:37.951310+0000', 'last_peered': '2024-09-11T01:21:37.951310+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:21:37.951310+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'8600", 'ondisk_log_start': "45'8600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10079, 'log_dups_size': 0, 'ondisk_log_size': 10079, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5124096, 'num_objects': 1251, 'num_object_clones': 0, 'num_object_copies': 2502, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1251, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15536, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2735}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:41.655 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:21:41.655 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'18680", 'reported_seq': 18815, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:37.947710+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:21:37.947710+0000', 'last_peered': '2024-09-11T01:21:37.947710+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:21:37.947710+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'8600", 'ondisk_log_start': "45'8600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4866048, 'num_objects': 1188, 'num_object_clones': 0, 'num_object_copies': 2376, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1188, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15267, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2796}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:41.655 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:21:41.655 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'19096", 'reported_seq': 19225, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:40.154279+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:21:40.154279+0000', 'last_peered': '2024-09-11T01:21:40.154279+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:21:40.154279+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "45'9000", 'ondisk_log_start': "45'9000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10096, 'log_dups_size': 0, 'ondisk_log_size': 10096, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4550656, 'num_objects': 1111, 'num_object_clones': 0, 'num_object_copies': 2222, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1111, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16913, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2913}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:41.656 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:21:41.656 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'19078", 'reported_seq': 19220, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:37.903526+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:21:37.903526+0000', 'last_peered': '2024-09-11T01:21:37.903526+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:21:37.903526+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'9000", 'ondisk_log_start': "45'9000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4857856, 'num_objects': 1186, 'num_object_clones': 0, 'num_object_copies': 2372, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1186, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15815, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2764}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:41.656 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:21:41.656 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'18948", 'reported_seq': 21834, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:37.964674+0000', 'last_change': '2024-09-11T01:20:20.985862+0000', 'last_active': '2024-09-11T01:21:37.964674+0000', 'last_peered': '2024-09-11T01:21:37.964674+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:21:37.964674+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "45'8900", 'ondisk_log_start': "45'8900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10048, 'log_dups_size': 0, 'ondisk_log_size': 10048, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 5029888, 'num_objects': 1228, 'num_object_clones': 0, 'num_object_copies': 2456, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2671, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1228, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15557, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1370, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1443}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:41.656 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:21:41.656 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'18818", 'reported_seq': 18962, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:37.870875+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:21:37.870875+0000', 'last_peered': '2024-09-11T01:21:37.870875+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:21:37.870875+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'8800", 'ondisk_log_start': "45'8800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10018, 'log_dups_size': 0, 'ondisk_log_size': 10018, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4645481, 'num_objects': 1135, 'num_object_clones': 0, 'num_object_copies': 2270, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1135, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15888, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2782}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:41.656 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:21:42.563 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:21:42.563 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34464/37656 objects degraded (91.523%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":639,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":18828,"data_bytes":77103764,"bytes_used":1549983744,"bytes_avail":288360308736,"bytes_total":289910292480,"degraded_objects":34464,"degraded_total":37656,"degraded_ratio":0.91523263224984064,"recovering_objects_per_sec":16,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":193,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":459},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:21:42.985 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:21:42.986 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:43.890 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:21:43.891 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:21:43.939 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":362,"stamp":"2024-09-11T01:21:42.774672+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":77103764,"num_objects":18828,"num_object_clones":0,"num_object_copies":37656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34464,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18828,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":263463,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13176,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160914,"ondisk_log_size":160914,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1513656,"kb_used_data":345592,"kb_used_omap":12,"kb_used_meta":1167987,"kb_avail":281601864,"statfs":{"total":289910292480,"available":288360308736,"internally_reserved":0,"allocated":353886208,"data_stored":352996452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1196018692},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,4,6,7,5,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19378176,"num_objects":-4731,"num_object_clones":0,"num_object_copies":-9462,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-193,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4731,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4731,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":193,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.008312"},"pg_stats":[{"pgid":"2.f","version":"49'19086","reported_seq":19171,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:37.911921+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:21:37.911921+0000","last_peered":"2024-09-11T01:21:37.911921+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:21:37.911921+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4833280,"num_objects":1180,"num_object_clones":0,"num_object_copies":2360,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1180,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19086,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2892}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19179","reported_seq":19291,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:40.103066+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:21:40.103066+0000","last_peered":"2024-09-11T01:21:40.103066+0000","last_clean":"2024-09-11T01:21:40.103066+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:21:40.103066+0000","last_undegraded":"2024-09-11T01:21:40.103066+0000","last_fullsized":"2024-09-11T01:21:40.103066+0000","mapping_epoch":35,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4804608,"num_objects":1173,"num_object_clones":0,"num_object_copies":2346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1173,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17418,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'18976","reported_seq":19109,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:40.204427+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:21:40.204427+0000","last_peered":"2024-09-11T01:21:40.204427+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:21:40.204427+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4780056,"num_objects":1168,"num_object_clones":0,"num_object_copies":2336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1168,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17277,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2728}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'19026","reported_seq":19106,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:40.134714+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:21:40.134714+0000","last_peered":"2024-09-11T01:21:40.134714+0000","last_clean":"2024-09-11T01:21:40.134714+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:21:40.134714+0000","last_undegraded":"2024-09-11T01:21:40.134714+0000","last_fullsized":"2024-09-11T01:21:40.134714+0000","mapping_epoch":35,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4538368,"num_objects":1108,"num_object_clones":0,"num_object_copies":2216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1108,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19026,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'18808","reported_seq":18946,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:37.871027+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:21:37.871027+0000","last_peered":"2024-09-11T01:21:37.871027+0000","last_clean":"2024-09-11T01:21:37.871027+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:21:37.871027+0000","last_undegraded":"2024-09-11T01:21:37.871027+0000","last_fullsized":"2024-09-11T01:21:37.871027+0000","mapping_epoch":34,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4919296,"num_objects":1201,"num_object_clones":0,"num_object_copies":2402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1201,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15813,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'19016","reported_seq":19195,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:37.903948+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:21:37.903948+0000","last_peered":"2024-09-11T01:21:37.903948+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:21:37.903948+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4960256,"num_objects":1211,"num_object_clones":0,"num_object_copies":2422,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1211,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15357,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2740}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'18679","reported_seq":18820,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:37.951310+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:21:37.951310+0000","last_peered":"2024-09-11T01:21:37.951310+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:21:37.951310+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10079,"log_dups_size":0,"ondisk_log_size":10079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5124096,"num_objects":1251,"num_object_clones":0,"num_object_copies":2502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1251,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15536,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2735}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19000","reported_seq":19140,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:37.936328+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:21:37.936328+0000","last_peered":"2024-09-11T01:21:37.936328+0000","last_clean":"2024-09-11T01:21:37.936328+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:21:37.936328+0000","last_undegraded":"2024-09-11T01:21:37.936328+0000","last_fullsized":"2024-09-11T01:21:37.936328+0000","mapping_epoch":34,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5136384,"num_objects":1254,"num_object_clones":0,"num_object_copies":2508,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1254,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15876,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003209+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003209+0000","last_peered":"2024-09-11T01:20:22.003209+0000","last_clean":"2024-09-11T01:20:22.003209+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003209+0000","last_undegraded":"2024-09-11T01:20:22.003209+0000","last_fullsized":"2024-09-11T01:20:22.003209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18813","reported_seq":26831,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:37.739595+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:21:37.739595+0000","last_peered":"2024-09-11T01:21:37.739595+0000","last_clean":"2024-09-11T01:21:37.739595+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:21:37.739595+0000","last_undegraded":"2024-09-11T01:21:37.739595+0000","last_fullsized":"2024-09-11T01:21:37.739595+0000","mapping_epoch":48,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4751360,"num_objects":1160,"num_object_clones":0,"num_object_copies":2320,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1160,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16784,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18680","reported_seq":18815,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:37.947710+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:21:37.947710+0000","last_peered":"2024-09-11T01:21:37.947710+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:21:37.947710+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4866048,"num_objects":1188,"num_object_clones":0,"num_object_copies":2376,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1188,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15267,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2796}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19096","reported_seq":19225,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:40.154279+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:21:40.154279+0000","last_peered":"2024-09-11T01:21:40.154279+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:21:40.154279+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4550656,"num_objects":1111,"num_object_clones":0,"num_object_copies":2222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1111,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16913,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2913}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003275+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003275+0000","last_peered":"2024-09-11T01:20:22.003275+0000","last_clean":"2024-09-11T01:20:22.003275+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003275+0000","last_undegraded":"2024-09-11T01:20:22.003275+0000","last_fullsized":"2024-09-11T01:20:22.003275+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18724","reported_seq":26715,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:40.176601+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:21:40.176601+0000","last_peered":"2024-09-11T01:21:40.176601+0000","last_clean":"2024-09-11T01:21:40.176601+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:21:40.176601+0000","last_undegraded":"2024-09-11T01:21:40.176601+0000","last_fullsized":"2024-09-11T01:21:40.176601+0000","mapping_epoch":44,"log_start":"45'8700","ondisk_log_start":"45'8700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4390912,"num_objects":1072,"num_object_clones":0,"num_object_copies":2144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1072,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16509,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19078","reported_seq":19220,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:37.903526+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:21:37.903526+0000","last_peered":"2024-09-11T01:21:37.903526+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:21:37.903526+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4857856,"num_objects":1186,"num_object_clones":0,"num_object_copies":2372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15815,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2764}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007577+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007577+0000","last_peered":"2024-09-11T01:20:22.007577+0000","last_clean":"2024-09-11T01:20:22.007577+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007577+0000","last_undegraded":"2024-09-11T01:20:22.007577+0000","last_fullsized":"2024-09-11T01:20:22.007577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18684","reported_seq":26725,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:37.712136+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:21:37.712136+0000","last_peered":"2024-09-11T01:21:37.712136+0000","last_clean":"2024-09-11T01:21:37.712136+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:21:37.712136+0000","last_undegraded":"2024-09-11T01:21:37.712136+0000","last_fullsized":"2024-09-11T01:21:37.712136+0000","mapping_epoch":46,"log_start":"45'8600","ondisk_log_start":"45'8600","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4915200,"num_objects":1200,"num_object_clones":0,"num_object_copies":2400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1200,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15339,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'18948","reported_seq":21834,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:37.964674+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:21:37.964674+0000","last_peered":"2024-09-11T01:21:37.964674+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:21:37.964674+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5029888,"num_objects":1228,"num_object_clones":0,"num_object_copies":2456,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2671,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1228,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15557,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1370,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1443}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007616+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007616+0000","last_peered":"2024-09-11T01:20:22.007616+0000","last_clean":"2024-09-11T01:20:22.007616+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007616+0000","last_undegraded":"2024-09-11T01:20:22.007616+0000","last_fullsized":"2024-09-11T01:20:22.007616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008171+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008171+0000","last_peered":"2024-09-11T01:20:22.008171+0000","last_clean":"2024-09-11T01:20:22.008171+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008171+0000","last_undegraded":"2024-09-11T01:20:22.008171+0000","last_fullsized":"2024-09-11T01:20:22.008171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18818","reported_seq":18962,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:37.870875+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:21:37.870875+0000","last_peered":"2024-09-11T01:21:37.870875+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:21:37.870875+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4645481,"num_objects":1135,"num_object_clones":0,"num_object_copies":2270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1135,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15888,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2782}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":77103745,"num_objects":18826,"num_object_clones":0,"num_object_copies":37652,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34464,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18826,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":263461,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13175,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":350007296,"data_stored":349943488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160911,"ondisk_log_size":160911,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182521,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":555304,"kb_used_data":122456,"kb_used_omap":4,"kb_used_meta":432827,"kb_avail":93816536,"statfs":{"total":96636764160,"available":96068132864,"internally_reserved":0,"allocated":125394944,"data_stored":125114790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":443215531},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,4,1,3,3,3,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986294,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506844,"kb_used_data":164236,"kb_used_omap":4,"kb_used_meta":342587,"kb_avail":93864996,"statfs":{"total":96636764160,"available":96117755904,"internally_reserved":0,"allocated":168177664,"data_stored":167897515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":350809770},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,0,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659238,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451508,"kb_used_data":58900,"kb_used_omap":4,"kb_used_meta":392571,"kb_avail":93920332,"statfs":{"total":96636764160,"available":96174419968,"internally_reserved":0,"allocated":60313600,"data_stored":59984147,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":401993391},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,2,4,1,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":123711488,"data_stored":123708009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":58908672,"data_stored":58855894,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":167387136,"data_stored":167379585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:46.942 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:47.719 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:21:47.720 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:21:47.803 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2024-09-11T01:21:46.776403+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":67445396,"num_objects":16470,"num_object_clones":0,"num_object_copies":32940,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34376,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16470,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":265821,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13264,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160772,"ondisk_log_size":160772,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1511492,"kb_used_data":326796,"kb_used_omap":12,"kb_used_meta":1184627,"kb_avail":281604028,"statfs":{"total":289910292480,"available":288362524672,"internally_reserved":0,"allocated":334639104,"data_stored":333749348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1213058052},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,2,2,7,10,8,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22392832,"num_objects":-5467,"num_object_clones":0,"num_object_copies":-10934,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-178,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5467,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5467,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":178,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003536"},"pg_stats":[{"pgid":"2.f","version":"49'19225","reported_seq":19313,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:42.904249+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:21:42.904249+0000","last_peered":"2024-09-11T01:21:42.904249+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:21:42.904249+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4263936,"num_objects":1041,"num_object_clones":0,"num_object_copies":2082,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1041,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19225,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3031}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19355","reported_seq":19467,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:45.218924+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:21:45.218924+0000","last_peered":"2024-09-11T01:21:45.218924+0000","last_clean":"2024-09-11T01:21:45.218924+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:21:45.218924+0000","last_undegraded":"2024-09-11T01:21:45.218924+0000","last_fullsized":"2024-09-11T01:21:45.218924+0000","mapping_epoch":35,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4083712,"num_objects":997,"num_object_clones":0,"num_object_copies":1994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":997,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17594,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19133","reported_seq":19270,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:45.168383+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:21:45.168383+0000","last_peered":"2024-09-11T01:21:45.168383+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:21:45.168383+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4136984,"num_objects":1011,"num_object_clones":0,"num_object_copies":2022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1011,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17434,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2885}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'19184","reported_seq":19264,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:45.185113+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:21:45.185113+0000","last_peered":"2024-09-11T01:21:45.185113+0000","last_clean":"2024-09-11T01:21:45.185113+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:21:45.185113+0000","last_undegraded":"2024-09-11T01:21:45.185113+0000","last_fullsized":"2024-09-11T01:21:45.185113+0000","mapping_epoch":35,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3891200,"num_objects":950,"num_object_clones":0,"num_object_copies":1900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19184,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'18963","reported_seq":19101,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:42.955758+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:21:42.955758+0000","last_peered":"2024-09-11T01:21:42.955758+0000","last_clean":"2024-09-11T01:21:42.955758+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:21:42.955758+0000","last_undegraded":"2024-09-11T01:21:42.955758+0000","last_fullsized":"2024-09-11T01:21:42.955758+0000","mapping_epoch":34,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4284416,"num_objects":1046,"num_object_clones":0,"num_object_copies":2092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1046,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15968,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'19161","reported_seq":19341,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:42.950431+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:21:42.950431+0000","last_peered":"2024-09-11T01:21:42.950431+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:21:42.950431+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4366336,"num_objects":1066,"num_object_clones":0,"num_object_copies":2132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1066,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15502,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2885}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'18824","reported_seq":18965,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:42.970621+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:21:42.970621+0000","last_peered":"2024-09-11T01:21:42.970621+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:21:42.970621+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10024,"log_dups_size":0,"ondisk_log_size":10024,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4530176,"num_objects":1106,"num_object_clones":0,"num_object_copies":2212,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1106,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15681,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2880}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19150","reported_seq":19290,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:42.911910+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:21:42.911910+0000","last_peered":"2024-09-11T01:21:42.911910+0000","last_clean":"2024-09-11T01:21:42.911910+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:21:42.911910+0000","last_undegraded":"2024-09-11T01:21:42.911910+0000","last_fullsized":"2024-09-11T01:21:42.911910+0000","mapping_epoch":34,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4521984,"num_objects":1104,"num_object_clones":0,"num_object_copies":2208,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1104,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16026,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003209+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003209+0000","last_peered":"2024-09-11T01:20:22.003209+0000","last_clean":"2024-09-11T01:20:22.003209+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003209+0000","last_undegraded":"2024-09-11T01:20:22.003209+0000","last_fullsized":"2024-09-11T01:20:22.003209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'18967","reported_seq":26985,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:42.744582+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:21:42.744582+0000","last_peered":"2024-09-11T01:21:42.744582+0000","last_clean":"2024-09-11T01:21:42.744582+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:21:42.744582+0000","last_undegraded":"2024-09-11T01:21:42.744582+0000","last_fullsized":"2024-09-11T01:21:42.744582+0000","mapping_epoch":48,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10067,"log_dups_size":0,"ondisk_log_size":10067,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4120576,"num_objects":1006,"num_object_clones":0,"num_object_copies":2012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1006,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16938,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18831","reported_seq":18970,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:43.002906+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:21:43.002906+0000","last_peered":"2024-09-11T01:21:43.002906+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:21:43.002906+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4247552,"num_objects":1037,"num_object_clones":0,"num_object_copies":2074,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1037,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15418,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2947}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19225","reported_seq":19354,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:45.141127+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:21:45.141127+0000","last_peered":"2024-09-11T01:21:45.141127+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:21:45.141127+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4022272,"num_objects":982,"num_object_clones":0,"num_object_copies":1964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3042}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003275+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003275+0000","last_peered":"2024-09-11T01:20:22.003275+0000","last_clean":"2024-09-11T01:20:22.003275+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003275+0000","last_undegraded":"2024-09-11T01:20:22.003275+0000","last_fullsized":"2024-09-11T01:20:22.003275+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18855","reported_seq":26846,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:45.198100+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:21:45.198100+0000","last_peered":"2024-09-11T01:21:45.198100+0000","last_clean":"2024-09-11T01:21:45.198100+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:21:45.198100+0000","last_undegraded":"2024-09-11T01:21:45.198100+0000","last_fullsized":"2024-09-11T01:21:45.198100+0000","mapping_epoch":44,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3854336,"num_objects":941,"num_object_clones":0,"num_object_copies":1882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16640,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19218","reported_seq":19360,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:42.919585+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:21:42.919585+0000","last_peered":"2024-09-11T01:21:42.919585+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:21:42.919585+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10018,"log_dups_size":0,"ondisk_log_size":10018,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4284416,"num_objects":1046,"num_object_clones":0,"num_object_copies":2092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1046,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15955,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2904}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007577+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007577+0000","last_peered":"2024-09-11T01:20:22.007577+0000","last_clean":"2024-09-11T01:20:22.007577+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007577+0000","last_undegraded":"2024-09-11T01:20:22.007577+0000","last_fullsized":"2024-09-11T01:20:22.007577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18819","reported_seq":26860,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:42.674726+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:21:42.674726+0000","last_peered":"2024-09-11T01:21:42.674726+0000","last_clean":"2024-09-11T01:21:42.674726+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:21:42.674726+0000","last_undegraded":"2024-09-11T01:21:42.674726+0000","last_fullsized":"2024-09-11T01:21:42.674726+0000","mapping_epoch":46,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4362240,"num_objects":1065,"num_object_clones":0,"num_object_copies":2130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1065,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15474,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19099","reported_seq":22161,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:42.975330+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:21:42.975330+0000","last_peered":"2024-09-11T01:21:42.975330+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:21:42.975330+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"45'9000","ondisk_log_start":"45'9000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10099,"log_dups_size":0,"ondisk_log_size":10099,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4411392,"num_objects":1077,"num_object_clones":0,"num_object_copies":2154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2583,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1077,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15708,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1458,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1506}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007616+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007616+0000","last_peered":"2024-09-11T01:20:22.007616+0000","last_clean":"2024-09-11T01:20:22.007616+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007616+0000","last_undegraded":"2024-09-11T01:20:22.007616+0000","last_fullsized":"2024-09-11T01:20:22.007616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008171+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008171+0000","last_peered":"2024-09-11T01:20:22.008171+0000","last_clean":"2024-09-11T01:20:22.008171+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008171+0000","last_undegraded":"2024-09-11T01:20:22.008171+0000","last_fullsized":"2024-09-11T01:20:22.008171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'18960","reported_seq":19104,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:42.926747+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:21:42.926747+0000","last_peered":"2024-09-11T01:21:42.926747+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:21:42.926747+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10060,"log_dups_size":0,"ondisk_log_size":10060,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4063849,"num_objects":993,"num_object_clones":0,"num_object_copies":1986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":993,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16030,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2924}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":67445377,"num_objects":16468,"num_object_clones":0,"num_object_copies":32936,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34376,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16468,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":265819,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13263,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":330715136,"data_stored":330651328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160769,"ondisk_log_size":160769,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182522,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":552424,"kb_used_data":115416,"kb_used_omap":4,"kb_used_meta":436987,"kb_avail":93819416,"statfs":{"total":96636764160,"available":96071081984,"internally_reserved":0,"allocated":118185984,"data_stored":117905830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":447475371},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,1,1,3,2,3,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986295,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506260,"kb_used_data":159484,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93865580,"statfs":{"total":96636764160,"available":96118353920,"internally_reserved":0,"allocated":163311616,"data_stored":163031467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659239,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":452808,"kb_used_data":51896,"kb_used_omap":4,"kb_used_meta":400891,"kb_avail":93919032,"statfs":{"total":96636764160,"available":96173088768,"internally_reserved":0,"allocated":53141504,"data_stored":52812051,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":410513071},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,2,5,3,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":116527104,"data_stored":116523625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":51724288,"data_stored":51671510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":162463744,"data_stored":162456193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:47.805 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:48.808 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:21:48.809 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:21:48.873 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-09-11T01:21:48.776778+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":60740244,"num_objects":14833,"num_object_clones":0,"num_object_copies":29666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34293,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14833,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267458,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13347,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160709,"ondisk_log_size":160709,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1493292,"kb_used_data":311340,"kb_used_omap":12,"kb_used_meta":1181875,"kb_avail":281622228,"statfs":{"total":289910292480,"available":288381161472,"internally_reserved":0,"allocated":318812160,"data_stored":317922404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1210240004},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,3,9,8,7,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26034176,"num_objects":-6356,"num_object_clones":0,"num_object_copies":-12712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-261,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6356,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003459"},"pg_stats":[{"pgid":"2.f","version":"49'19363","reported_seq":19451,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:47.991237+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:21:47.991237+0000","last_peered":"2024-09-11T01:21:47.991237+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:21:47.991237+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3698688,"num_objects":903,"num_object_clones":0,"num_object_copies":1806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19363,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3169}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19355","reported_seq":19467,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:45.218924+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:21:45.218924+0000","last_peered":"2024-09-11T01:21:45.218924+0000","last_clean":"2024-09-11T01:21:45.218924+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:21:45.218924+0000","last_undegraded":"2024-09-11T01:21:45.218924+0000","last_fullsized":"2024-09-11T01:21:45.218924+0000","mapping_epoch":35,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4083712,"num_objects":997,"num_object_clones":0,"num_object_copies":1994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":997,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17594,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19133","reported_seq":19270,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:45.168383+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:21:45.168383+0000","last_peered":"2024-09-11T01:21:45.168383+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:21:45.168383+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4136984,"num_objects":1011,"num_object_clones":0,"num_object_copies":2022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1011,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17434,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2885}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'19184","reported_seq":19264,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:45.185113+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:21:45.185113+0000","last_peered":"2024-09-11T01:21:45.185113+0000","last_clean":"2024-09-11T01:21:45.185113+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:21:45.185113+0000","last_undegraded":"2024-09-11T01:21:45.185113+0000","last_fullsized":"2024-09-11T01:21:45.185113+0000","mapping_epoch":35,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3891200,"num_objects":950,"num_object_clones":0,"num_object_copies":1900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19184,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19113","reported_seq":19251,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:47.889103+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:21:47.889103+0000","last_peered":"2024-09-11T01:21:47.889103+0000","last_clean":"2024-09-11T01:21:47.889103+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:21:47.889103+0000","last_undegraded":"2024-09-11T01:21:47.889103+0000","last_fullsized":"2024-09-11T01:21:47.889103+0000","mapping_epoch":34,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3670016,"num_objects":896,"num_object_clones":0,"num_object_copies":1792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16118,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'19303","reported_seq":19484,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:47.998013+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:21:47.998013+0000","last_peered":"2024-09-11T01:21:47.998013+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:21:47.998013+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3784704,"num_objects":924,"num_object_clones":0,"num_object_copies":1848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15644,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3027}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'18994","reported_seq":19135,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:47.906351+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:21:47.906351+0000","last_peered":"2024-09-11T01:21:47.906351+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:21:47.906351+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3833856,"num_objects":936,"num_object_clones":0,"num_object_copies":1872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15851,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3050}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19306","reported_seq":19446,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:47.990562+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:21:47.990562+0000","last_peered":"2024-09-11T01:21:47.990562+0000","last_clean":"2024-09-11T01:21:47.990562+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:21:47.990562+0000","last_undegraded":"2024-09-11T01:21:47.990562+0000","last_fullsized":"2024-09-11T01:21:47.990562+0000","mapping_epoch":34,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3883008,"num_objects":948,"num_object_clones":0,"num_object_copies":1896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16182,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003209+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003209+0000","last_peered":"2024-09-11T01:20:22.003209+0000","last_clean":"2024-09-11T01:20:22.003209+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003209+0000","last_undegraded":"2024-09-11T01:20:22.003209+0000","last_fullsized":"2024-09-11T01:20:22.003209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19103","reported_seq":27121,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:47.778611+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:21:47.778611+0000","last_peered":"2024-09-11T01:21:47.778611+0000","last_clean":"2024-09-11T01:21:47.778611+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:21:47.778611+0000","last_undegraded":"2024-09-11T01:21:47.778611+0000","last_fullsized":"2024-09-11T01:21:47.778611+0000","mapping_epoch":48,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3563520,"num_objects":870,"num_object_clones":0,"num_object_copies":1740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18980","reported_seq":19119,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:47.997843+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:21:47.997843+0000","last_peered":"2024-09-11T01:21:47.997843+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:21:47.997843+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3637248,"num_objects":888,"num_object_clones":0,"num_object_copies":1776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15567,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3096}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19225","reported_seq":19354,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:45.141127+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:21:45.141127+0000","last_peered":"2024-09-11T01:21:45.141127+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:21:45.141127+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4022272,"num_objects":982,"num_object_clones":0,"num_object_copies":1964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3042}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003275+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003275+0000","last_peered":"2024-09-11T01:20:22.003275+0000","last_clean":"2024-09-11T01:20:22.003275+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003275+0000","last_undegraded":"2024-09-11T01:20:22.003275+0000","last_fullsized":"2024-09-11T01:20:22.003275+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18855","reported_seq":26846,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:45.198100+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:21:45.198100+0000","last_peered":"2024-09-11T01:21:45.198100+0000","last_clean":"2024-09-11T01:21:45.198100+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:21:45.198100+0000","last_undegraded":"2024-09-11T01:21:45.198100+0000","last_fullsized":"2024-09-11T01:21:45.198100+0000","mapping_epoch":44,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3854336,"num_objects":941,"num_object_clones":0,"num_object_copies":1882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16640,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19363","reported_seq":19503,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:47.992162+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:21:47.992162+0000","last_peered":"2024-09-11T01:21:47.992162+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:21:47.992162+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3690496,"num_objects":901,"num_object_clones":0,"num_object_copies":1802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16100,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3049}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007577+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007577+0000","last_peered":"2024-09-11T01:20:22.007577+0000","last_clean":"2024-09-11T01:20:22.007577+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007577+0000","last_undegraded":"2024-09-11T01:20:22.007577+0000","last_fullsized":"2024-09-11T01:20:22.007577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18969","reported_seq":27010,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:47.806666+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:21:47.806666+0000","last_peered":"2024-09-11T01:21:47.806666+0000","last_clean":"2024-09-11T01:21:47.806666+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:21:47.806666+0000","last_undegraded":"2024-09-11T01:21:47.806666+0000","last_fullsized":"2024-09-11T01:21:47.806666+0000","mapping_epoch":46,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3747840,"num_objects":915,"num_object_clones":0,"num_object_copies":1830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15624,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19241","reported_seq":22469,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:48.045546+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:21:48.045546+0000","last_peered":"2024-09-11T01:21:48.045546+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:21:48.045546+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3829760,"num_objects":935,"num_object_clones":0,"num_object_copies":1870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2500,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15850,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1541,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007616+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007616+0000","last_peered":"2024-09-11T01:20:22.007616+0000","last_clean":"2024-09-11T01:20:22.007616+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007616+0000","last_undegraded":"2024-09-11T01:20:22.007616+0000","last_fullsized":"2024-09-11T01:20:22.007616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008171+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008171+0000","last_peered":"2024-09-11T01:20:22.008171+0000","last_clean":"2024-09-11T01:20:22.008171+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008171+0000","last_undegraded":"2024-09-11T01:20:22.008171+0000","last_fullsized":"2024-09-11T01:20:22.008171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19119","reported_seq":19263,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:48.038636+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:21:48.038636+0000","last_peered":"2024-09-11T01:21:48.038636+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:21:48.038636+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3412585,"num_objects":834,"num_object_clones":0,"num_object_copies":1668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":834,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3083}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":60740225,"num_objects":14831,"num_object_clones":0,"num_object_copies":29662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34293,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14831,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267456,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13346,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":315953152,"data_stored":315889344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160706,"ondisk_log_size":160706,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182523,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":553772,"kb_used_data":107032,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93818068,"statfs":{"total":96636764160,"available":96069701632,"internally_reserved":0,"allocated":109600768,"data_stored":109320614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":457436843},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,3,2,3,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986295,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506260,"kb_used_data":159484,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93865580,"statfs":{"total":96636764160,"available":96118353920,"internally_reserved":0,"allocated":163311616,"data_stored":163031467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659240,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":433260,"kb_used_data":44824,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93938580,"statfs":{"total":96636764160,"available":96193105920,"internally_reserved":0,"allocated":45899776,"data_stored":45570323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":397733551},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,4,3,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":109092864,"data_stored":109089385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":44396544,"data_stored":44343766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":162463744,"data_stored":162456193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:48.875 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:21:48.875 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'19363", 'reported_seq': 19451, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:47.991237+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:21:47.991237+0000', 'last_peered': '2024-09-11T01:21:47.991237+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:21:47.991237+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "45'9300", 'ondisk_log_start': "45'9300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3698688, 'num_objects': 903, 'num_object_clones': 0, 'num_object_copies': 1806, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 903, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19363, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3169}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:48.875 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:21:48.875 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'19133", 'reported_seq': 19270, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:45.168383+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:21:45.168383+0000', 'last_peered': '2024-09-11T01:21:45.168383+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:21:45.168383+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "45'9100", 'ondisk_log_start': "45'9100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10033, 'log_dups_size': 0, 'ondisk_log_size': 10033, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4136984, 'num_objects': 1011, 'num_object_clones': 0, 'num_object_copies': 2022, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1011, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17434, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2885}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:48.875 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:21:48.875 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'19303", 'reported_seq': 19484, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:47.998013+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:21:47.998013+0000', 'last_peered': '2024-09-11T01:21:47.998013+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:21:47.998013+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'9300", 'ondisk_log_start': "45'9300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3784704, 'num_objects': 924, 'num_object_clones': 0, 'num_object_copies': 1848, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 924, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15644, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3027}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:48.875 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:21:48.875 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'18994", 'reported_seq': 19135, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:47.906351+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:21:47.906351+0000', 'last_peered': '2024-09-11T01:21:47.906351+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:21:47.906351+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'8900", 'ondisk_log_start': "45'8900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3833856, 'num_objects': 936, 'num_object_clones': 0, 'num_object_copies': 1872, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 936, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15851, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3050}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:48.875 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:21:48.876 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'18980", 'reported_seq': 19119, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:47.997843+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:21:47.997843+0000', 'last_peered': '2024-09-11T01:21:47.997843+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:21:47.997843+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'8900", 'ondisk_log_start': "45'8900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10080, 'log_dups_size': 0, 'ondisk_log_size': 10080, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3637248, 'num_objects': 888, 'num_object_clones': 0, 'num_object_copies': 1776, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 888, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15567, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3096}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:48.876 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:21:48.876 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'19225", 'reported_seq': 19354, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:45.141127+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:21:45.141127+0000', 'last_peered': '2024-09-11T01:21:45.141127+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:21:45.141127+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "45'9200", 'ondisk_log_start': "45'9200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10025, 'log_dups_size': 0, 'ondisk_log_size': 10025, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 4022272, 'num_objects': 982, 'num_object_clones': 0, 'num_object_copies': 1964, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 982, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17042, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3042}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:48.876 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:21:48.876 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'19363", 'reported_seq': 19503, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:47.992162+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:21:47.992162+0000', 'last_peered': '2024-09-11T01:21:47.992162+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:21:47.992162+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'9300", 'ondisk_log_start': "45'9300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10063, 'log_dups_size': 0, 'ondisk_log_size': 10063, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3690496, 'num_objects': 901, 'num_object_clones': 0, 'num_object_copies': 1802, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 901, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16100, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3049}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:48.876 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:21:48.876 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19241", 'reported_seq': 22469, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:48.045546+0000', 'last_change': '2024-09-11T01:20:20.985862+0000', 'last_active': '2024-09-11T01:21:48.045546+0000', 'last_peered': '2024-09-11T01:21:48.045546+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:21:48.045546+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "45'9200", 'ondisk_log_start': "45'9200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10041, 'log_dups_size': 0, 'ondisk_log_size': 10041, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3829760, 'num_objects': 935, 'num_object_clones': 0, 'num_object_copies': 1870, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2500, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 935, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15850, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1541, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1565}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:48.876 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:21:48.876 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19119", 'reported_seq': 19263, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:48.038636+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:21:48.038636+0000', 'last_peered': '2024-09-11T01:21:48.038636+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:21:48.038636+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9100", 'ondisk_log_start': "45'9100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10019, 'log_dups_size': 0, 'ondisk_log_size': 10019, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3412585, 'num_objects': 834, 'num_object_clones': 0, 'num_object_copies': 1668, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 834, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16189, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3083}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:48.877 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:21:49.863 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:21:49.863 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34293/29666 objects degraded (115.597%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":646,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":14833,"data_bytes":60740244,"bytes_used":1529131008,"bytes_avail":288381161472,"bytes_total":289910292480,"degraded_objects":34293,"degraded_total":29666,"degraded_ratio":1.1559697970740916,"recovering_objects_per_sec":21,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":261,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":529},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:21:49.911 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:21:49.911 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:50.687 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:21:50.689 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:21:50.758 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":365,"stamp":"2024-09-11T01:21:48.776778+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":60740244,"num_objects":14833,"num_object_clones":0,"num_object_copies":29666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34293,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14833,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267458,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13347,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160709,"ondisk_log_size":160709,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1493292,"kb_used_data":311340,"kb_used_omap":12,"kb_used_meta":1181875,"kb_avail":281622228,"statfs":{"total":289910292480,"available":288381161472,"internally_reserved":0,"allocated":318812160,"data_stored":317922404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1210240004},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,3,3,9,8,7,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-26034176,"num_objects":-6356,"num_object_clones":0,"num_object_copies":-12712,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-261,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6356,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6356,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":261,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003459"},"pg_stats":[{"pgid":"2.f","version":"49'19363","reported_seq":19451,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:47.991237+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:21:47.991237+0000","last_peered":"2024-09-11T01:21:47.991237+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:21:47.991237+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3698688,"num_objects":903,"num_object_clones":0,"num_object_copies":1806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19363,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3169}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19355","reported_seq":19467,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:45.218924+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:21:45.218924+0000","last_peered":"2024-09-11T01:21:45.218924+0000","last_clean":"2024-09-11T01:21:45.218924+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:21:45.218924+0000","last_undegraded":"2024-09-11T01:21:45.218924+0000","last_fullsized":"2024-09-11T01:21:45.218924+0000","mapping_epoch":35,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4083712,"num_objects":997,"num_object_clones":0,"num_object_copies":1994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":997,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17594,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19133","reported_seq":19270,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:45.168383+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:21:45.168383+0000","last_peered":"2024-09-11T01:21:45.168383+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:21:45.168383+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10033,"log_dups_size":0,"ondisk_log_size":10033,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4136984,"num_objects":1011,"num_object_clones":0,"num_object_copies":2022,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1011,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17434,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2885}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'19184","reported_seq":19264,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:45.185113+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:21:45.185113+0000","last_peered":"2024-09-11T01:21:45.185113+0000","last_clean":"2024-09-11T01:21:45.185113+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:21:45.185113+0000","last_undegraded":"2024-09-11T01:21:45.185113+0000","last_fullsized":"2024-09-11T01:21:45.185113+0000","mapping_epoch":35,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3891200,"num_objects":950,"num_object_clones":0,"num_object_copies":1900,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":950,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19184,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19113","reported_seq":19251,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:47.889103+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:21:47.889103+0000","last_peered":"2024-09-11T01:21:47.889103+0000","last_clean":"2024-09-11T01:21:47.889103+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:21:47.889103+0000","last_undegraded":"2024-09-11T01:21:47.889103+0000","last_fullsized":"2024-09-11T01:21:47.889103+0000","mapping_epoch":34,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3670016,"num_objects":896,"num_object_clones":0,"num_object_copies":1792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16118,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'19303","reported_seq":19484,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:47.998013+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:21:47.998013+0000","last_peered":"2024-09-11T01:21:47.998013+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:21:47.998013+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3784704,"num_objects":924,"num_object_clones":0,"num_object_copies":1848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15644,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3027}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'18994","reported_seq":19135,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:47.906351+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:21:47.906351+0000","last_peered":"2024-09-11T01:21:47.906351+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:21:47.906351+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3833856,"num_objects":936,"num_object_clones":0,"num_object_copies":1872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15851,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3050}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19306","reported_seq":19446,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:47.990562+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:21:47.990562+0000","last_peered":"2024-09-11T01:21:47.990562+0000","last_clean":"2024-09-11T01:21:47.990562+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:21:47.990562+0000","last_undegraded":"2024-09-11T01:21:47.990562+0000","last_fullsized":"2024-09-11T01:21:47.990562+0000","mapping_epoch":34,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3883008,"num_objects":948,"num_object_clones":0,"num_object_copies":1896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16182,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003209+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003209+0000","last_peered":"2024-09-11T01:20:22.003209+0000","last_clean":"2024-09-11T01:20:22.003209+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003209+0000","last_undegraded":"2024-09-11T01:20:22.003209+0000","last_fullsized":"2024-09-11T01:20:22.003209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19103","reported_seq":27121,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:47.778611+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:21:47.778611+0000","last_peered":"2024-09-11T01:21:47.778611+0000","last_clean":"2024-09-11T01:21:47.778611+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:21:47.778611+0000","last_undegraded":"2024-09-11T01:21:47.778611+0000","last_fullsized":"2024-09-11T01:21:47.778611+0000","mapping_epoch":48,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3563520,"num_objects":870,"num_object_clones":0,"num_object_copies":1740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18980","reported_seq":19119,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:47.997843+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:21:47.997843+0000","last_peered":"2024-09-11T01:21:47.997843+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:21:47.997843+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3637248,"num_objects":888,"num_object_clones":0,"num_object_copies":1776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15567,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3096}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19225","reported_seq":19354,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:45.141127+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:21:45.141127+0000","last_peered":"2024-09-11T01:21:45.141127+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:21:45.141127+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4022272,"num_objects":982,"num_object_clones":0,"num_object_copies":1964,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":982,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17042,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3042}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003275+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003275+0000","last_peered":"2024-09-11T01:20:22.003275+0000","last_clean":"2024-09-11T01:20:22.003275+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003275+0000","last_undegraded":"2024-09-11T01:20:22.003275+0000","last_fullsized":"2024-09-11T01:20:22.003275+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18855","reported_seq":26846,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:45.198100+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:21:45.198100+0000","last_peered":"2024-09-11T01:21:45.198100+0000","last_clean":"2024-09-11T01:21:45.198100+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:21:45.198100+0000","last_undegraded":"2024-09-11T01:21:45.198100+0000","last_fullsized":"2024-09-11T01:21:45.198100+0000","mapping_epoch":44,"log_start":"45'8800","ondisk_log_start":"45'8800","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3854336,"num_objects":941,"num_object_clones":0,"num_object_copies":1882,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":941,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16640,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19363","reported_seq":19503,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:47.992162+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:21:47.992162+0000","last_peered":"2024-09-11T01:21:47.992162+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:21:47.992162+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3690496,"num_objects":901,"num_object_clones":0,"num_object_copies":1802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16100,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3049}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007577+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007577+0000","last_peered":"2024-09-11T01:20:22.007577+0000","last_clean":"2024-09-11T01:20:22.007577+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007577+0000","last_undegraded":"2024-09-11T01:20:22.007577+0000","last_fullsized":"2024-09-11T01:20:22.007577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18969","reported_seq":27010,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:47.806666+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:21:47.806666+0000","last_peered":"2024-09-11T01:21:47.806666+0000","last_clean":"2024-09-11T01:21:47.806666+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:21:47.806666+0000","last_undegraded":"2024-09-11T01:21:47.806666+0000","last_fullsized":"2024-09-11T01:21:47.806666+0000","mapping_epoch":46,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3747840,"num_objects":915,"num_object_clones":0,"num_object_copies":1830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15624,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19241","reported_seq":22469,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:48.045546+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:21:48.045546+0000","last_peered":"2024-09-11T01:21:48.045546+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:21:48.045546+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3829760,"num_objects":935,"num_object_clones":0,"num_object_copies":1870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2500,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15850,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1541,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007616+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007616+0000","last_peered":"2024-09-11T01:20:22.007616+0000","last_clean":"2024-09-11T01:20:22.007616+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007616+0000","last_undegraded":"2024-09-11T01:20:22.007616+0000","last_fullsized":"2024-09-11T01:20:22.007616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008171+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008171+0000","last_peered":"2024-09-11T01:20:22.008171+0000","last_clean":"2024-09-11T01:20:22.008171+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008171+0000","last_undegraded":"2024-09-11T01:20:22.008171+0000","last_fullsized":"2024-09-11T01:20:22.008171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19119","reported_seq":19263,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:48.038636+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:21:48.038636+0000","last_peered":"2024-09-11T01:21:48.038636+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:21:48.038636+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3412585,"num_objects":834,"num_object_clones":0,"num_object_copies":1668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":834,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3083}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":60740225,"num_objects":14831,"num_object_clones":0,"num_object_copies":29662,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34293,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14831,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":267456,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13346,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":315953152,"data_stored":315889344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160706,"ondisk_log_size":160706,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182523,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":553772,"kb_used_data":107032,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93818068,"statfs":{"total":96636764160,"available":96069701632,"internally_reserved":0,"allocated":109600768,"data_stored":109320614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":457436843},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,3,2,3,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986295,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506260,"kb_used_data":159484,"kb_used_omap":4,"kb_used_meta":346747,"kb_avail":93865580,"statfs":{"total":96636764160,"available":96118353920,"internally_reserved":0,"allocated":163311616,"data_stored":163031467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":355069610},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,1,2,3,2],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659240,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":433260,"kb_used_data":44824,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93938580,"statfs":{"total":96636764160,"available":96193105920,"internally_reserved":0,"allocated":45899776,"data_stored":45570323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":397733551},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,4,3,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":109092864,"data_stored":109089385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":44396544,"data_stored":44343766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":162463744,"data_stored":162456193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:53.760 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:54.635 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:21:54.636 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:21:54.729 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":367,"stamp":"2024-09-11T01:21:52.777679+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":57848468,"num_objects":14127,"num_object_clones":0,"num_object_copies":28254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34293,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14127,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":268164,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13347,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160815,"ondisk_log_size":160815,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1476208,"kb_used_data":306740,"kb_used_omap":12,"kb_used_meta":1169395,"kb_avail":281639312,"statfs":{"total":289910292480,"available":288398655488,"internally_reserved":0,"allocated":314101760,"data_stored":313212004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1197460484},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,3,3,7,9,9,6],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19255296,"num_objects":-4701,"num_object_clones":0,"num_object_copies":-9402,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-171,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4701,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4701,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":171,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003464"},"pg_stats":[{"pgid":"2.f","version":"49'19363","reported_seq":19451,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:47.991237+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:21:47.991237+0000","last_peered":"2024-09-11T01:21:47.991237+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:21:47.991237+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3698688,"num_objects":903,"num_object_clones":0,"num_object_copies":1806,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":903,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19363,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3169}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19482","reported_seq":19594,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:50.238577+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:21:50.238577+0000","last_peered":"2024-09-11T01:21:50.238577+0000","last_clean":"2024-09-11T01:21:50.238577+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:21:50.238577+0000","last_undegraded":"2024-09-11T01:21:50.238577+0000","last_fullsized":"2024-09-11T01:21:50.238577+0000","mapping_epoch":35,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3563520,"num_objects":870,"num_object_clones":0,"num_object_copies":1740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17721,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19290","reported_seq":19429,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:50.158900+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:21:50.158900+0000","last_peered":"2024-09-11T01:21:50.158900+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:21:50.158900+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3493912,"num_objects":854,"num_object_clones":0,"num_object_copies":1708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":854,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17591,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3042}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'19311","reported_seq":19391,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:50.180293+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:21:50.180293+0000","last_peered":"2024-09-11T01:21:50.180293+0000","last_clean":"2024-09-11T01:21:50.180293+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:21:50.180293+0000","last_undegraded":"2024-09-11T01:21:50.180293+0000","last_fullsized":"2024-09-11T01:21:50.180293+0000","mapping_epoch":35,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3371008,"num_objects":823,"num_object_clones":0,"num_object_copies":1646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19311,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19113","reported_seq":19251,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:47.889103+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:21:47.889103+0000","last_peered":"2024-09-11T01:21:47.889103+0000","last_clean":"2024-09-11T01:21:47.889103+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:21:47.889103+0000","last_undegraded":"2024-09-11T01:21:47.889103+0000","last_fullsized":"2024-09-11T01:21:47.889103+0000","mapping_epoch":34,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10013,"log_dups_size":0,"ondisk_log_size":10013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3670016,"num_objects":896,"num_object_clones":0,"num_object_copies":1792,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":896,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16118,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'19303","reported_seq":19484,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:47.998013+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:21:47.998013+0000","last_peered":"2024-09-11T01:21:47.998013+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:21:47.998013+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3784704,"num_objects":924,"num_object_clones":0,"num_object_copies":1848,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":924,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15644,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3027}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'18994","reported_seq":19135,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:47.906351+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:21:47.906351+0000","last_peered":"2024-09-11T01:21:47.906351+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:21:47.906351+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3833856,"num_objects":936,"num_object_clones":0,"num_object_copies":1872,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":936,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15851,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3050}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19306","reported_seq":19446,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:47.990562+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:21:47.990562+0000","last_peered":"2024-09-11T01:21:47.990562+0000","last_clean":"2024-09-11T01:21:47.990562+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:21:47.990562+0000","last_undegraded":"2024-09-11T01:21:47.990562+0000","last_fullsized":"2024-09-11T01:21:47.990562+0000","mapping_epoch":34,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3883008,"num_objects":948,"num_object_clones":0,"num_object_copies":1896,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":948,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16182,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003209+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003209+0000","last_peered":"2024-09-11T01:20:22.003209+0000","last_clean":"2024-09-11T01:20:22.003209+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003209+0000","last_undegraded":"2024-09-11T01:20:22.003209+0000","last_fullsized":"2024-09-11T01:20:22.003209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19103","reported_seq":27121,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:47.778611+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:21:47.778611+0000","last_peered":"2024-09-11T01:21:47.778611+0000","last_clean":"2024-09-11T01:21:47.778611+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:21:47.778611+0000","last_undegraded":"2024-09-11T01:21:47.778611+0000","last_fullsized":"2024-09-11T01:21:47.778611+0000","mapping_epoch":48,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3563520,"num_objects":870,"num_object_clones":0,"num_object_copies":1740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17074,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'18980","reported_seq":19119,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:47.997843+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:21:47.997843+0000","last_peered":"2024-09-11T01:21:47.997843+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:21:47.997843+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10080,"log_dups_size":0,"ondisk_log_size":10080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3637248,"num_objects":888,"num_object_clones":0,"num_object_copies":1776,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":888,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15567,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3096}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19378","reported_seq":19507,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:50.192949+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:21:50.192949+0000","last_peered":"2024-09-11T01:21:50.192949+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:21:50.192949+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3395584,"num_objects":829,"num_object_clones":0,"num_object_copies":1658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17195,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3195}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003275+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003275+0000","last_peered":"2024-09-11T01:20:22.003275+0000","last_clean":"2024-09-11T01:20:22.003275+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003275+0000","last_undegraded":"2024-09-11T01:20:22.003275+0000","last_fullsized":"2024-09-11T01:20:22.003275+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18997","reported_seq":26988,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:50.242321+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:21:50.242321+0000","last_peered":"2024-09-11T01:21:50.242321+0000","last_clean":"2024-09-11T01:21:50.242321+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:21:50.242321+0000","last_undegraded":"2024-09-11T01:21:50.242321+0000","last_fullsized":"2024-09-11T01:21:50.242321+0000","mapping_epoch":44,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3272704,"num_objects":799,"num_object_clones":0,"num_object_copies":1598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":799,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16782,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19363","reported_seq":19503,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:47.992162+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:21:47.992162+0000","last_peered":"2024-09-11T01:21:47.992162+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:21:47.992162+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3690496,"num_objects":901,"num_object_clones":0,"num_object_copies":1802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":901,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16100,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3049}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007577+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007577+0000","last_peered":"2024-09-11T01:20:22.007577+0000","last_clean":"2024-09-11T01:20:22.007577+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007577+0000","last_undegraded":"2024-09-11T01:20:22.007577+0000","last_fullsized":"2024-09-11T01:20:22.007577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'18969","reported_seq":27010,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:47.806666+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:21:47.806666+0000","last_peered":"2024-09-11T01:21:47.806666+0000","last_clean":"2024-09-11T01:21:47.806666+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:21:47.806666+0000","last_undegraded":"2024-09-11T01:21:47.806666+0000","last_fullsized":"2024-09-11T01:21:47.806666+0000","mapping_epoch":46,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10069,"log_dups_size":0,"ondisk_log_size":10069,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3747840,"num_objects":915,"num_object_clones":0,"num_object_copies":1830,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":915,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15624,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19241","reported_seq":22469,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:48.045546+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:21:48.045546+0000","last_peered":"2024-09-11T01:21:48.045546+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:21:48.045546+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10041,"log_dups_size":0,"ondisk_log_size":10041,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3829760,"num_objects":935,"num_object_clones":0,"num_object_copies":1870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2500,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":935,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15850,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1541,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1565}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007616+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007616+0000","last_peered":"2024-09-11T01:20:22.007616+0000","last_clean":"2024-09-11T01:20:22.007616+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007616+0000","last_undegraded":"2024-09-11T01:20:22.007616+0000","last_fullsized":"2024-09-11T01:20:22.007616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":84,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008171+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008171+0000","last_peered":"2024-09-11T01:20:22.008171+0000","last_clean":"2024-09-11T01:20:22.008171+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008171+0000","last_undegraded":"2024-09-11T01:20:22.008171+0000","last_fullsized":"2024-09-11T01:20:22.008171+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19119","reported_seq":19263,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:48.038636+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:21:48.038636+0000","last_peered":"2024-09-11T01:21:48.038636+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:21:48.038636+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10019,"log_dups_size":0,"ondisk_log_size":10019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3412585,"num_objects":834,"num_object_clones":0,"num_object_copies":1668,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":834,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16189,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3083}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":57848449,"num_objects":14125,"num_object_clones":0,"num_object_copies":28250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34293,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14125,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":268162,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13346,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":311300096,"data_stored":311236288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160812,"ondisk_log_size":160812,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182523,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":553772,"kb_used_data":107032,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93818068,"statfs":{"total":96636764160,"available":96069701632,"internally_reserved":0,"allocated":109600768,"data_stored":109320614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":457436843},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,2,3,2,3,5],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986296,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":489176,"kb_used_data":154884,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93882664,"statfs":{"total":96636764160,"available":96135847936,"internally_reserved":0,"allocated":158601216,"data_stored":158321067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342290090},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659240,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":433260,"kb_used_data":44824,"kb_used_omap":4,"kb_used_meta":388411,"kb_avail":93938580,"statfs":{"total":96636764160,"available":96193105920,"internally_reserved":0,"allocated":45899776,"data_stored":45570323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":397733551},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,0,4,3,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":109092864,"data_stored":109089385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":44396544,"data_stored":44343766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":157810688,"data_stored":157803137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:54.730 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:55.358 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:21:55.361 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:21:55.440 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":368,"stamp":"2024-09-11T01:21:54.778260+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":51217044,"num_objects":12508,"num_object_clones":0,"num_object_copies":25016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12508,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":269783,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13448,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160834,"ondisk_log_size":160834,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1478944,"kb_used_data":292844,"kb_used_omap":12,"kb_used_meta":1186035,"kb_avail":281636576,"statfs":{"total":289910292480,"available":288395853824,"internally_reserved":0,"allocated":299872256,"data_stored":298982500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1214499844},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,2,7,14,14,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25886720,"num_objects":-6320,"num_object_clones":0,"num_object_copies":-12640,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-272,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6320,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6320,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":272,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003587"},"pg_stats":[{"pgid":"2.f","version":"49'19511","reported_seq":19599,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:53.005054+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:21:53.005054+0000","last_peered":"2024-09-11T01:21:53.005054+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:21:53.005054+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3092480,"num_objects":755,"num_object_clones":0,"num_object_copies":1510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19511,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3317}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19482","reported_seq":19594,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:50.238577+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:21:50.238577+0000","last_peered":"2024-09-11T01:21:50.238577+0000","last_clean":"2024-09-11T01:21:50.238577+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:21:50.238577+0000","last_undegraded":"2024-09-11T01:21:50.238577+0000","last_fullsized":"2024-09-11T01:21:50.238577+0000","mapping_epoch":35,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3563520,"num_objects":870,"num_object_clones":0,"num_object_copies":1740,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":870,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17721,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19290","reported_seq":19429,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:50.158900+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:21:50.158900+0000","last_peered":"2024-09-11T01:21:50.158900+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:21:50.158900+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10090,"log_dups_size":0,"ondisk_log_size":10090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3493912,"num_objects":854,"num_object_clones":0,"num_object_copies":1708,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":854,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17591,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3042}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'19311","reported_seq":19391,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:50.180293+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:21:50.180293+0000","last_peered":"2024-09-11T01:21:50.180293+0000","last_clean":"2024-09-11T01:21:50.180293+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:21:50.180293+0000","last_undegraded":"2024-09-11T01:21:50.180293+0000","last_fullsized":"2024-09-11T01:21:50.180293+0000","mapping_epoch":35,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3371008,"num_objects":823,"num_object_clones":0,"num_object_copies":1646,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":823,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19311,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19273","reported_seq":19411,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:53.076129+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:21:53.076129+0000","last_peered":"2024-09-11T01:21:53.076129+0000","last_clean":"2024-09-11T01:21:53.076129+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:21:53.076129+0000","last_undegraded":"2024-09-11T01:21:53.076129+0000","last_fullsized":"2024-09-11T01:21:53.076129+0000","mapping_epoch":34,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3014656,"num_objects":736,"num_object_clones":0,"num_object_copies":1472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'19458","reported_seq":19637,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:53.010502+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:21:53.010502+0000","last_peered":"2024-09-11T01:21:53.010502+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:21:53.010502+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3149824,"num_objects":769,"num_object_clones":0,"num_object_copies":1538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15799,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3182}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19137","reported_seq":19278,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:53.009920+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:21:53.009920+0000","last_peered":"2024-09-11T01:21:53.009920+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:21:53.009920+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3248128,"num_objects":793,"num_object_clones":0,"num_object_copies":1586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15994,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3193}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19458","reported_seq":19598,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:53.048767+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:21:53.048767+0000","last_peered":"2024-09-11T01:21:53.048767+0000","last_clean":"2024-09-11T01:21:53.048767+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:21:53.048767+0000","last_undegraded":"2024-09-11T01:21:53.048767+0000","last_fullsized":"2024-09-11T01:21:53.048767+0000","mapping_epoch":34,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3260416,"num_objects":796,"num_object_clones":0,"num_object_copies":1592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16334,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003209+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003209+0000","last_peered":"2024-09-11T01:20:22.003209+0000","last_clean":"2024-09-11T01:20:22.003209+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003209+0000","last_undegraded":"2024-09-11T01:20:22.003209+0000","last_fullsized":"2024-09-11T01:20:22.003209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19238","reported_seq":27256,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:52.921515+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:21:52.921515+0000","last_peered":"2024-09-11T01:21:52.921515+0000","last_clean":"2024-09-11T01:21:52.921515+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:21:52.921515+0000","last_undegraded":"2024-09-11T01:21:52.921515+0000","last_fullsized":"2024-09-11T01:21:52.921515+0000","mapping_epoch":48,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3010560,"num_objects":735,"num_object_clones":0,"num_object_copies":1470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17209,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19115","reported_seq":19250,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:53.009713+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:21:53.009713+0000","last_peered":"2024-09-11T01:21:53.009713+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:21:53.009713+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3084288,"num_objects":753,"num_object_clones":0,"num_object_copies":1506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15702,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3231}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19378","reported_seq":19507,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:50.192949+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:21:50.192949+0000","last_peered":"2024-09-11T01:21:50.192949+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:21:50.192949+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3395584,"num_objects":829,"num_object_clones":0,"num_object_copies":1658,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":829,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17195,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3195}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003275+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003275+0000","last_peered":"2024-09-11T01:20:22.003275+0000","last_clean":"2024-09-11T01:20:22.003275+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003275+0000","last_undegraded":"2024-09-11T01:20:22.003275+0000","last_fullsized":"2024-09-11T01:20:22.003275+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'18997","reported_seq":26988,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:50.242321+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:21:50.242321+0000","last_peered":"2024-09-11T01:21:50.242321+0000","last_clean":"2024-09-11T01:21:50.242321+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:21:50.242321+0000","last_undegraded":"2024-09-11T01:21:50.242321+0000","last_fullsized":"2024-09-11T01:21:50.242321+0000","mapping_epoch":44,"log_start":"45'8900","ondisk_log_start":"45'8900","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10097,"log_dups_size":0,"ondisk_log_size":10097,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3272704,"num_objects":799,"num_object_clones":0,"num_object_copies":1598,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":799,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16782,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19515","reported_seq":19657,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:53.052698+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:21:53.052698+0000","last_peered":"2024-09-11T01:21:53.052698+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:21:53.052698+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3067904,"num_objects":749,"num_object_clones":0,"num_object_copies":1498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16252,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3201}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007577+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007577+0000","last_peered":"2024-09-11T01:20:22.007577+0000","last_clean":"2024-09-11T01:20:22.007577+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007577+0000","last_undegraded":"2024-09-11T01:20:22.007577+0000","last_fullsized":"2024-09-11T01:20:22.007577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19114","reported_seq":27154,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:52.914779+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:21:52.914779+0000","last_peered":"2024-09-11T01:21:52.914779+0000","last_clean":"2024-09-11T01:21:52.914779+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:21:52.914779+0000","last_undegraded":"2024-09-11T01:21:52.914779+0000","last_fullsized":"2024-09-11T01:21:52.914779+0000","mapping_epoch":46,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3153920,"num_objects":770,"num_object_clones":0,"num_object_copies":1540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15769,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19392","reported_seq":22822,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:53.082156+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:21:53.082156+0000","last_peered":"2024-09-11T01:21:53.082156+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:21:53.082156+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3211264,"num_objects":784,"num_object_clones":0,"num_object_copies":1568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2399,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16001,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1642,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1615}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007616+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007616+0000","last_peered":"2024-09-11T01:20:22.007616+0000","last_clean":"2024-09-11T01:20:22.007616+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007616+0000","last_undegraded":"2024-09-11T01:20:22.007616+0000","last_fullsized":"2024-09-11T01:20:22.007616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008206+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008206+0000","last_peered":"2024-09-11T01:20:22.008206+0000","last_clean":"2024-09-11T01:20:22.008206+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008206+0000","last_undegraded":"2024-09-11T01:20:22.008206+0000","last_fullsized":"2024-09-11T01:20:22.008206+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19262","reported_seq":19406,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:53.030627+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:21:53.030627+0000","last_peered":"2024-09-11T01:21:53.030627+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:21:53.030627+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2826857,"num_objects":691,"num_object_clones":0,"num_object_copies":1382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16332,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3226}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":51217025,"num_objects":12506,"num_object_clones":0,"num_object_copies":25012,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12506,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":269781,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13447,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":296968192,"data_stored":296904384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160831,"ondisk_log_size":160831,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182524,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":555012,"kb_used_data":99956,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93816828,"statfs":{"total":96636764160,"available":96068431872,"internally_reserved":0,"allocated":102354944,"data_stored":102074790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,6,6,4,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986296,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":489176,"kb_used_data":154884,"kb_used_omap":4,"kb_used_meta":334267,"kb_avail":93882664,"statfs":{"total":96636764160,"available":96135847936,"internally_reserved":0,"allocated":158601216,"data_stored":158321067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":342290090},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,0,4,4],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659241,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":434756,"kb_used_data":38004,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93937084,"statfs":{"total":96636764160,"available":96191574016,"internally_reserved":0,"allocated":38916096,"data_stored":38586643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":406253231},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,0,1,4,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":101863424,"data_stored":101859945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":37294080,"data_stored":37241302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":157810688,"data_stored":157803137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:21:55.441 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:21:55.441 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'19511", 'reported_seq': 19599, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:53.005054+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:21:53.005054+0000', 'last_peered': '2024-09-11T01:21:53.005054+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:21:53.005054+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "45'9500", 'ondisk_log_start': "45'9500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10011, 'log_dups_size': 0, 'ondisk_log_size': 10011, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3092480, 'num_objects': 755, 'num_object_clones': 0, 'num_object_copies': 1510, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 755, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19511, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3317}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:55.441 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:21:55.441 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'19290", 'reported_seq': 19429, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:50.158900+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:21:50.158900+0000', 'last_peered': '2024-09-11T01:21:50.158900+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:21:50.158900+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "45'9200", 'ondisk_log_start': "45'9200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10090, 'log_dups_size': 0, 'ondisk_log_size': 10090, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3493912, 'num_objects': 854, 'num_object_clones': 0, 'num_object_copies': 1708, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 854, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17591, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3042}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:55.442 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:21:55.442 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'19458", 'reported_seq': 19637, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:53.010502+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:21:53.010502+0000', 'last_peered': '2024-09-11T01:21:53.010502+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:21:53.010502+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'9400", 'ondisk_log_start': "45'9400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3149824, 'num_objects': 769, 'num_object_clones': 0, 'num_object_copies': 1538, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 769, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15799, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3182}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:55.442 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:21:55.442 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19137", 'reported_seq': 19278, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:53.009920+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:21:53.009920+0000', 'last_peered': '2024-09-11T01:21:53.009920+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:21:53.009920+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9100", 'ondisk_log_start': "45'9100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10037, 'log_dups_size': 0, 'ondisk_log_size': 10037, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3248128, 'num_objects': 793, 'num_object_clones': 0, 'num_object_copies': 1586, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 793, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15994, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3193}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:55.442 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:21:55.442 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19115", 'reported_seq': 19250, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:53.009713+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:21:53.009713+0000', 'last_peered': '2024-09-11T01:21:53.009713+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:21:53.009713+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9100", 'ondisk_log_start': "45'9100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3084288, 'num_objects': 753, 'num_object_clones': 0, 'num_object_copies': 1506, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 753, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15702, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3231}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:55.442 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:21:55.442 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'19378", 'reported_seq': 19507, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:50.192949+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:21:50.192949+0000', 'last_peered': '2024-09-11T01:21:50.192949+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:21:50.192949+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "45'9300", 'ondisk_log_start': "45'9300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3395584, 'num_objects': 829, 'num_object_clones': 0, 'num_object_copies': 1658, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 829, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17195, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3195}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:21:55.442 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:21:55.442 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'19515", 'reported_seq': 19657, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:53.052698+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:21:53.052698+0000', 'last_peered': '2024-09-11T01:21:53.052698+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:21:53.052698+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'9500", 'ondisk_log_start': "45'9500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3067904, 'num_objects': 749, 'num_object_clones': 0, 'num_object_copies': 1498, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 749, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16252, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3201}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:55.443 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:21:55.443 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19392", 'reported_seq': 22822, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:53.082156+0000', 'last_change': '2024-09-11T01:20:20.985862+0000', 'last_active': '2024-09-11T01:21:53.082156+0000', 'last_peered': '2024-09-11T01:21:53.082156+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:21:53.082156+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "45'9300", 'ondisk_log_start': "45'9300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10092, 'log_dups_size': 0, 'ondisk_log_size': 10092, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 3211264, 'num_objects': 784, 'num_object_clones': 0, 'num_object_copies': 1568, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2399, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 784, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16001, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1642, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1615}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:55.443 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:21:55.443 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19262", 'reported_seq': 19406, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:53.030627+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:21:53.030627+0000', 'last_peered': '2024-09-11T01:21:53.030627+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:21:53.030627+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9200", 'ondisk_log_start': "45'9200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10062, 'log_dups_size': 0, 'ondisk_log_size': 10062, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2826857, 'num_objects': 691, 'num_object_clones': 0, 'num_object_copies': 1382, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 691, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16332, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3226}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:21:55.443 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:21:56.400 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:21:56.400 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34192/25016 objects degraded (136.681%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":652,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":12508,"data_bytes":51217044,"bytes_used":1514438656,"bytes_avail":288395853824,"bytes_total":289910292480,"degraded_objects":34192,"degraded_total":25016,"degraded_ratio":1.3668052446434282,"recovering_objects_per_sec":22,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":272,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":526},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:21:56.457 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:21:56.458 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:21:57.242 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:21:57.242 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:21:57.337 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":369,"stamp":"2024-09-11T01:21:56.780403+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":48218772,"num_objects":11776,"num_object_clones":0,"num_object_copies":23552,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11776,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":270515,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13448,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160666,"ondisk_log_size":160666,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1496508,"kb_used_data":288204,"kb_used_omap":12,"kb_used_meta":1208243,"kb_avail":281619012,"statfs":{"total":289910292480,"available":288377868288,"internally_reserved":0,"allocated":295120896,"data_stored":294231140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1237240836},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,6,3,8,11,11,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22302720,"num_objects":-5445,"num_object_clones":0,"num_object_copies":-10890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-184,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5445,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005301"},"pg_stats":[{"pgid":"2.f","version":"49'19511","reported_seq":19599,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:53.005054+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:21:53.005054+0000","last_peered":"2024-09-11T01:21:53.005054+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:21:53.005054+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10011,"log_dups_size":0,"ondisk_log_size":10011,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3092480,"num_objects":755,"num_object_clones":0,"num_object_copies":1510,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":755,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19511,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3317}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19652","reported_seq":19762,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:55.222541+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:21:55.222541+0000","last_peered":"2024-09-11T01:21:55.222541+0000","last_clean":"2024-09-11T01:21:55.222541+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:21:55.222541+0000","last_undegraded":"2024-09-11T01:21:55.222541+0000","last_fullsized":"2024-09-11T01:21:55.222541+0000","mapping_epoch":35,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2867200,"num_objects":700,"num_object_clones":0,"num_object_copies":1400,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":700,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17891,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19435","reported_seq":19574,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:55.169010+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:21:55.169010+0000","last_peered":"2024-09-11T01:21:55.169010+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:21:55.169010+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2899992,"num_objects":709,"num_object_clones":0,"num_object_copies":1418,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":709,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17736,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3187}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'19461","reported_seq":19541,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:55.240376+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:21:55.240376+0000","last_peered":"2024-09-11T01:21:55.240376+0000","last_clean":"2024-09-11T01:21:55.240376+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:21:55.240376+0000","last_undegraded":"2024-09-11T01:21:55.240376+0000","last_fullsized":"2024-09-11T01:21:55.240376+0000","mapping_epoch":35,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2756608,"num_objects":673,"num_object_clones":0,"num_object_copies":1346,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":673,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19461,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19273","reported_seq":19411,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:53.076129+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:21:53.076129+0000","last_peered":"2024-09-11T01:21:53.076129+0000","last_clean":"2024-09-11T01:21:53.076129+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:21:53.076129+0000","last_undegraded":"2024-09-11T01:21:53.076129+0000","last_fullsized":"2024-09-11T01:21:53.076129+0000","mapping_epoch":34,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3014656,"num_objects":736,"num_object_clones":0,"num_object_copies":1472,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":736,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16278,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'19458","reported_seq":19637,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:53.010502+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:21:53.010502+0000","last_peered":"2024-09-11T01:21:53.010502+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:21:53.010502+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3149824,"num_objects":769,"num_object_clones":0,"num_object_copies":1538,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":769,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15799,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3182}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19137","reported_seq":19278,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:53.009920+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:21:53.009920+0000","last_peered":"2024-09-11T01:21:53.009920+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:21:53.009920+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10037,"log_dups_size":0,"ondisk_log_size":10037,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3248128,"num_objects":793,"num_object_clones":0,"num_object_copies":1586,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":793,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15994,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3193}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19458","reported_seq":19598,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:53.048767+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:21:53.048767+0000","last_peered":"2024-09-11T01:21:53.048767+0000","last_clean":"2024-09-11T01:21:53.048767+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:21:53.048767+0000","last_undegraded":"2024-09-11T01:21:53.048767+0000","last_fullsized":"2024-09-11T01:21:53.048767+0000","mapping_epoch":34,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3260416,"num_objects":796,"num_object_clones":0,"num_object_copies":1592,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":796,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16334,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003209+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003209+0000","last_peered":"2024-09-11T01:20:22.003209+0000","last_clean":"2024-09-11T01:20:22.003209+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003209+0000","last_undegraded":"2024-09-11T01:20:22.003209+0000","last_fullsized":"2024-09-11T01:20:22.003209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19238","reported_seq":27256,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:52.921515+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:21:52.921515+0000","last_peered":"2024-09-11T01:21:52.921515+0000","last_clean":"2024-09-11T01:21:52.921515+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:21:52.921515+0000","last_undegraded":"2024-09-11T01:21:52.921515+0000","last_fullsized":"2024-09-11T01:21:52.921515+0000","mapping_epoch":48,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3010560,"num_objects":735,"num_object_clones":0,"num_object_copies":1470,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":735,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17209,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19115","reported_seq":19250,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:53.009713+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:21:53.009713+0000","last_peered":"2024-09-11T01:21:53.009713+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:21:53.009713+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3084288,"num_objects":753,"num_object_clones":0,"num_object_copies":1506,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":753,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15702,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3231}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19514","reported_seq":19643,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:55.094192+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:21:55.094192+0000","last_peered":"2024-09-11T01:21:55.094192+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:21:55.094192+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2838528,"num_objects":693,"num_object_clones":0,"num_object_copies":1386,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":693,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17331,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3331}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003275+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003275+0000","last_peered":"2024-09-11T01:20:22.003275+0000","last_clean":"2024-09-11T01:20:22.003275+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003275+0000","last_undegraded":"2024-09-11T01:20:22.003275+0000","last_fullsized":"2024-09-11T01:20:22.003275+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19128","reported_seq":27119,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:55.234695+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:21:55.234695+0000","last_peered":"2024-09-11T01:21:55.234695+0000","last_clean":"2024-09-11T01:21:55.234695+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:21:55.234695+0000","last_undegraded":"2024-09-11T01:21:55.234695+0000","last_fullsized":"2024-09-11T01:21:55.234695+0000","mapping_epoch":44,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2736128,"num_objects":668,"num_object_clones":0,"num_object_copies":1336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":668,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16913,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19515","reported_seq":19657,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:53.052698+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:21:53.052698+0000","last_peered":"2024-09-11T01:21:53.052698+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:21:53.052698+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3067904,"num_objects":749,"num_object_clones":0,"num_object_copies":1498,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":749,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16252,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3201}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007577+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007577+0000","last_peered":"2024-09-11T01:20:22.007577+0000","last_clean":"2024-09-11T01:20:22.007577+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007577+0000","last_undegraded":"2024-09-11T01:20:22.007577+0000","last_fullsized":"2024-09-11T01:20:22.007577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19114","reported_seq":27154,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:52.914779+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:21:52.914779+0000","last_peered":"2024-09-11T01:21:52.914779+0000","last_clean":"2024-09-11T01:21:52.914779+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:21:52.914779+0000","last_undegraded":"2024-09-11T01:21:52.914779+0000","last_fullsized":"2024-09-11T01:21:52.914779+0000","mapping_epoch":46,"log_start":"45'9100","ondisk_log_start":"45'9100","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10014,"log_dups_size":0,"ondisk_log_size":10014,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3153920,"num_objects":770,"num_object_clones":0,"num_object_copies":1540,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":770,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15769,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19392","reported_seq":22822,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:53.082156+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:21:53.082156+0000","last_peered":"2024-09-11T01:21:53.082156+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:21:53.082156+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10092,"log_dups_size":0,"ondisk_log_size":10092,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3211264,"num_objects":784,"num_object_clones":0,"num_object_copies":1568,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2399,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":784,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16001,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1642,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1615}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007616+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007616+0000","last_peered":"2024-09-11T01:20:22.007616+0000","last_clean":"2024-09-11T01:20:22.007616+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007616+0000","last_undegraded":"2024-09-11T01:20:22.007616+0000","last_fullsized":"2024-09-11T01:20:22.007616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008206+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008206+0000","last_peered":"2024-09-11T01:20:22.008206+0000","last_clean":"2024-09-11T01:20:22.008206+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008206+0000","last_undegraded":"2024-09-11T01:20:22.008206+0000","last_fullsized":"2024-09-11T01:20:22.008206+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19262","reported_seq":19406,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:53.030627+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:21:53.030627+0000","last_peered":"2024-09-11T01:21:53.030627+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:21:53.030627+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2826857,"num_objects":691,"num_object_clones":0,"num_object_copies":1382,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":691,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16332,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3226}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":48218753,"num_objects":11774,"num_object_clones":0,"num_object_copies":23548,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11774,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":270513,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13447,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":292151296,"data_stored":292087488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160663,"ondisk_log_size":160663,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182524,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":555012,"kb_used_data":99956,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93816828,"statfs":{"total":96636764160,"available":96068431872,"internally_reserved":0,"allocated":102354944,"data_stored":102074790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,6,6,4,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986297,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506740,"kb_used_data":150244,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93865100,"statfs":{"total":96636764160,"available":96117862400,"internally_reserved":0,"allocated":153849856,"data_stored":153569707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,1,2,1,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659241,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":434756,"kb_used_data":38004,"kb_used_omap":4,"kb_used_meta":396731,"kb_avail":93937084,"statfs":{"total":96636764160,"available":96191574016,"internally_reserved":0,"allocated":38916096,"data_stored":38586643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":406253231},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,5,0,1,4,6,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":101863424,"data_stored":101859945,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":37294080,"data_stored":37241302,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":152993792,"data_stored":152986241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:00.340 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:01.117 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:01.117 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:22:01.176 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":371,"stamp":"2024-09-11T01:22:00.788875+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":38724244,"num_objects":9458,"num_object_clones":0,"num_object_copies":18916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9458,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":272833,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13534,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160584,"ondisk_log_size":160584,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1495948,"kb_used_data":269564,"kb_used_omap":12,"kb_used_meta":1226291,"kb_avail":281619572,"statfs":{"total":289910292480,"available":288378441728,"internally_reserved":0,"allocated":276033536,"data_stored":275143780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1255721988},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,4,8,9,11,10],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22016000,"num_objects":-5375,"num_object_clones":0,"num_object_copies":-10750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-187,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5375,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.012096"},"pg_stats":[{"pgid":"2.f","version":"49'19639","reported_seq":19727,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:58.061199+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:21:58.061199+0000","last_peered":"2024-09-11T01:21:58.061199+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:21:58.061199+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2568192,"num_objects":627,"num_object_clones":0,"num_object_copies":1254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19639,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3445}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19806","reported_seq":19918,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:00.128741+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:00.128741+0000","last_peered":"2024-09-11T01:22:00.128741+0000","last_clean":"2024-09-11T01:22:00.128741+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:00.128741+0000","last_undegraded":"2024-09-11T01:22:00.128741+0000","last_fullsized":"2024-09-11T01:22:00.128741+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2236416,"num_objects":546,"num_object_clones":0,"num_object_copies":1092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18045,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19615","reported_seq":19753,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:00.130260+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:22:00.130260+0000","last_peered":"2024-09-11T01:22:00.130260+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:22:00.130260+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2162712,"num_objects":529,"num_object_clones":0,"num_object_copies":1058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":529,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17916,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3367}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'19594","reported_seq":19674,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:00.238043+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:00.238043+0000","last_peered":"2024-09-11T01:22:00.238043+0000","last_clean":"2024-09-11T01:22:00.238043+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:00.238043+0000","last_undegraded":"2024-09-11T01:22:00.238043+0000","last_fullsized":"2024-09-11T01:22:00.238043+0000","mapping_epoch":35,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2211840,"num_objects":540,"num_object_clones":0,"num_object_copies":1080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19594,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19408","reported_seq":19546,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:58.016485+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:21:58.016485+0000","last_peered":"2024-09-11T01:21:58.016485+0000","last_clean":"2024-09-11T01:21:58.016485+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:21:58.016485+0000","last_undegraded":"2024-09-11T01:21:58.016485+0000","last_fullsized":"2024-09-11T01:21:58.016485+0000","mapping_epoch":34,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2461696,"num_objects":601,"num_object_clones":0,"num_object_copies":1202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16413,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'19603","reported_seq":19784,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:57.952515+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:21:57.952515+0000","last_peered":"2024-09-11T01:21:57.952515+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:21:57.952515+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2555904,"num_objects":624,"num_object_clones":0,"num_object_copies":1248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15944,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3327}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19303","reported_seq":19444,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:58.064250+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:21:58.064250+0000","last_peered":"2024-09-11T01:21:58.064250+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:21:58.064250+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2568192,"num_objects":627,"num_object_clones":0,"num_object_copies":1254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16160,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3359}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19605","reported_seq":19745,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:58.040915+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:21:58.040915+0000","last_peered":"2024-09-11T01:21:58.040915+0000","last_clean":"2024-09-11T01:21:58.040915+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:21:58.040915+0000","last_undegraded":"2024-09-11T01:21:58.040915+0000","last_fullsized":"2024-09-11T01:21:58.040915+0000","mapping_epoch":34,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2658304,"num_objects":649,"num_object_clones":0,"num_object_copies":1298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16481,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003209+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003209+0000","last_peered":"2024-09-11T01:20:22.003209+0000","last_clean":"2024-09-11T01:20:22.003209+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003209+0000","last_undegraded":"2024-09-11T01:20:22.003209+0000","last_fullsized":"2024-09-11T01:20:22.003209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19381","reported_seq":27399,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:57.920648+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:21:57.920648+0000","last_peered":"2024-09-11T01:21:57.920648+0000","last_clean":"2024-09-11T01:21:57.920648+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:21:57.920648+0000","last_undegraded":"2024-09-11T01:21:57.920648+0000","last_fullsized":"2024-09-11T01:21:57.920648+0000","mapping_epoch":48,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2424832,"num_objects":592,"num_object_clones":0,"num_object_copies":1184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17352,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19272","reported_seq":19411,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:58.107070+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:21:58.107070+0000","last_peered":"2024-09-11T01:21:58.107070+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:21:58.107070+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2441216,"num_objects":596,"num_object_clones":0,"num_object_copies":1192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15859,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3388}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19655","reported_seq":19784,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:00.148585+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:00.148585+0000","last_peered":"2024-09-11T01:22:00.148585+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:00.148585+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2260992,"num_objects":552,"num_object_clones":0,"num_object_copies":1104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17472,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3472}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003275+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003275+0000","last_peered":"2024-09-11T01:20:22.003275+0000","last_clean":"2024-09-11T01:20:22.003275+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003275+0000","last_undegraded":"2024-09-11T01:20:22.003275+0000","last_fullsized":"2024-09-11T01:20:22.003275+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19262","reported_seq":27253,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:00.272963+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:00.272963+0000","last_peered":"2024-09-11T01:22:00.272963+0000","last_clean":"2024-09-11T01:22:00.272963+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:00.272963+0000","last_undegraded":"2024-09-11T01:22:00.272963+0000","last_fullsized":"2024-09-11T01:22:00.272963+0000","mapping_epoch":44,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2187264,"num_objects":534,"num_object_clones":0,"num_object_copies":1068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17047,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19649","reported_seq":19791,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:58.032550+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:21:58.032550+0000","last_peered":"2024-09-11T01:21:58.032550+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:21:58.032550+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2519040,"num_objects":615,"num_object_clones":0,"num_object_copies":1230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16386,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007577+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007577+0000","last_peered":"2024-09-11T01:20:22.007577+0000","last_clean":"2024-09-11T01:20:22.007577+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007577+0000","last_undegraded":"2024-09-11T01:20:22.007577+0000","last_fullsized":"2024-09-11T01:20:22.007577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19244","reported_seq":27284,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:57.918040+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:21:57.918040+0000","last_peered":"2024-09-11T01:21:57.918040+0000","last_clean":"2024-09-11T01:21:57.918040+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:21:57.918040+0000","last_undegraded":"2024-09-11T01:21:57.918040+0000","last_fullsized":"2024-09-11T01:21:57.918040+0000","mapping_epoch":46,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2621440,"num_objects":640,"num_object_clones":0,"num_object_copies":1280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15899,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19538","reported_seq":23140,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:58.091835+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:21:58.091835+0000","last_peered":"2024-09-11T01:21:58.091835+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:21:58.091835+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2613248,"num_objects":638,"num_object_clones":0,"num_object_copies":1276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2313,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16147,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1728,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1675}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007616+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007616+0000","last_peered":"2024-09-11T01:20:22.007616+0000","last_clean":"2024-09-11T01:20:22.007616+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007616+0000","last_undegraded":"2024-09-11T01:20:22.007616+0000","last_fullsized":"2024-09-11T01:20:22.007616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008206+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008206+0000","last_peered":"2024-09-11T01:20:22.008206+0000","last_clean":"2024-09-11T01:20:22.008206+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008206+0000","last_undegraded":"2024-09-11T01:20:22.008206+0000","last_fullsized":"2024-09-11T01:20:22.008206+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19407","reported_seq":19551,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:58.106005+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:21:58.106005+0000","last_peered":"2024-09-11T01:21:58.106005+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:21:58.106005+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2232937,"num_objects":546,"num_object_clones":0,"num_object_copies":1092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16477,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":38724225,"num_objects":9456,"num_object_clones":0,"num_object_copies":18912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9456,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":272831,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13533,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":272977920,"data_stored":272914112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160581,"ondisk_log_size":160581,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182525,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535676,"kb_used_data":93068,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93836164,"statfs":{"total":96636764160,"available":96088231936,"internally_reserved":0,"allocated":95301632,"data_stored":95021478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":453177003},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,6,6,4,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986298,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506140,"kb_used_data":145484,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93865700,"statfs":{"total":96636764160,"available":96118476800,"internally_reserved":0,"allocated":148975616,"data_stored":148695467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659242,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":454132,"kb_used_data":31012,"kb_used_omap":4,"kb_used_meta":423099,"kb_avail":93917708,"statfs":{"total":96636764160,"available":96171732992,"internally_reserved":0,"allocated":31756288,"data_stored":31426835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":433254063},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,1,2,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":94760960,"data_stored":94757481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":30150656,"data_stored":30097878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":148066304,"data_stored":148058753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:01.177 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:01.862 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:01.863 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:22:01.910 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":371,"stamp":"2024-09-11T01:22:00.788875+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":38724244,"num_objects":9458,"num_object_clones":0,"num_object_copies":18916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9458,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":272833,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13534,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160584,"ondisk_log_size":160584,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1495948,"kb_used_data":269564,"kb_used_omap":12,"kb_used_meta":1226291,"kb_avail":281619572,"statfs":{"total":289910292480,"available":288378441728,"internally_reserved":0,"allocated":276033536,"data_stored":275143780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1255721988},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,4,8,9,11,10],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22016000,"num_objects":-5375,"num_object_clones":0,"num_object_copies":-10750,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-187,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5375,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5375,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.012096"},"pg_stats":[{"pgid":"2.f","version":"49'19639","reported_seq":19727,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:58.061199+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:21:58.061199+0000","last_peered":"2024-09-11T01:21:58.061199+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:21:58.061199+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2568192,"num_objects":627,"num_object_clones":0,"num_object_copies":1254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19639,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3445}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19806","reported_seq":19918,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:00.128741+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:00.128741+0000","last_peered":"2024-09-11T01:22:00.128741+0000","last_clean":"2024-09-11T01:22:00.128741+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:00.128741+0000","last_undegraded":"2024-09-11T01:22:00.128741+0000","last_fullsized":"2024-09-11T01:22:00.128741+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2236416,"num_objects":546,"num_object_clones":0,"num_object_copies":1092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18045,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19615","reported_seq":19753,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:00.130260+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:22:00.130260+0000","last_peered":"2024-09-11T01:22:00.130260+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:22:00.130260+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2162712,"num_objects":529,"num_object_clones":0,"num_object_copies":1058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":529,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17916,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3367}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'19594","reported_seq":19674,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:00.238043+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:00.238043+0000","last_peered":"2024-09-11T01:22:00.238043+0000","last_clean":"2024-09-11T01:22:00.238043+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:00.238043+0000","last_undegraded":"2024-09-11T01:22:00.238043+0000","last_fullsized":"2024-09-11T01:22:00.238043+0000","mapping_epoch":35,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2211840,"num_objects":540,"num_object_clones":0,"num_object_copies":1080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19594,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19408","reported_seq":19546,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:58.016485+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:21:58.016485+0000","last_peered":"2024-09-11T01:21:58.016485+0000","last_clean":"2024-09-11T01:21:58.016485+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:21:58.016485+0000","last_undegraded":"2024-09-11T01:21:58.016485+0000","last_fullsized":"2024-09-11T01:21:58.016485+0000","mapping_epoch":34,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2461696,"num_objects":601,"num_object_clones":0,"num_object_copies":1202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16413,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'19603","reported_seq":19784,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:57.952515+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:21:57.952515+0000","last_peered":"2024-09-11T01:21:57.952515+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:21:57.952515+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2555904,"num_objects":624,"num_object_clones":0,"num_object_copies":1248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15944,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3327}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19303","reported_seq":19444,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:58.064250+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:21:58.064250+0000","last_peered":"2024-09-11T01:21:58.064250+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:21:58.064250+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2568192,"num_objects":627,"num_object_clones":0,"num_object_copies":1254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16160,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3359}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19605","reported_seq":19745,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:58.040915+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:21:58.040915+0000","last_peered":"2024-09-11T01:21:58.040915+0000","last_clean":"2024-09-11T01:21:58.040915+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:21:58.040915+0000","last_undegraded":"2024-09-11T01:21:58.040915+0000","last_fullsized":"2024-09-11T01:21:58.040915+0000","mapping_epoch":34,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2658304,"num_objects":649,"num_object_clones":0,"num_object_copies":1298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16481,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003209+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003209+0000","last_peered":"2024-09-11T01:20:22.003209+0000","last_clean":"2024-09-11T01:20:22.003209+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003209+0000","last_undegraded":"2024-09-11T01:20:22.003209+0000","last_fullsized":"2024-09-11T01:20:22.003209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19381","reported_seq":27399,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:57.920648+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:21:57.920648+0000","last_peered":"2024-09-11T01:21:57.920648+0000","last_clean":"2024-09-11T01:21:57.920648+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:21:57.920648+0000","last_undegraded":"2024-09-11T01:21:57.920648+0000","last_fullsized":"2024-09-11T01:21:57.920648+0000","mapping_epoch":48,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2424832,"num_objects":592,"num_object_clones":0,"num_object_copies":1184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17352,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19272","reported_seq":19411,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:58.107070+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:21:58.107070+0000","last_peered":"2024-09-11T01:21:58.107070+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:21:58.107070+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2441216,"num_objects":596,"num_object_clones":0,"num_object_copies":1192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15859,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3388}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19655","reported_seq":19784,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:00.148585+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:00.148585+0000","last_peered":"2024-09-11T01:22:00.148585+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:00.148585+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2260992,"num_objects":552,"num_object_clones":0,"num_object_copies":1104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17472,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3472}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003275+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003275+0000","last_peered":"2024-09-11T01:20:22.003275+0000","last_clean":"2024-09-11T01:20:22.003275+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003275+0000","last_undegraded":"2024-09-11T01:20:22.003275+0000","last_fullsized":"2024-09-11T01:20:22.003275+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19262","reported_seq":27253,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:00.272963+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:00.272963+0000","last_peered":"2024-09-11T01:22:00.272963+0000","last_clean":"2024-09-11T01:22:00.272963+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:00.272963+0000","last_undegraded":"2024-09-11T01:22:00.272963+0000","last_fullsized":"2024-09-11T01:22:00.272963+0000","mapping_epoch":44,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2187264,"num_objects":534,"num_object_clones":0,"num_object_copies":1068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17047,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19649","reported_seq":19791,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:58.032550+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:21:58.032550+0000","last_peered":"2024-09-11T01:21:58.032550+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:21:58.032550+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2519040,"num_objects":615,"num_object_clones":0,"num_object_copies":1230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16386,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007577+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007577+0000","last_peered":"2024-09-11T01:20:22.007577+0000","last_clean":"2024-09-11T01:20:22.007577+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007577+0000","last_undegraded":"2024-09-11T01:20:22.007577+0000","last_fullsized":"2024-09-11T01:20:22.007577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19244","reported_seq":27284,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:57.918040+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:21:57.918040+0000","last_peered":"2024-09-11T01:21:57.918040+0000","last_clean":"2024-09-11T01:21:57.918040+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:21:57.918040+0000","last_undegraded":"2024-09-11T01:21:57.918040+0000","last_fullsized":"2024-09-11T01:21:57.918040+0000","mapping_epoch":46,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2621440,"num_objects":640,"num_object_clones":0,"num_object_copies":1280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15899,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19538","reported_seq":23140,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:58.091835+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:21:58.091835+0000","last_peered":"2024-09-11T01:21:58.091835+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:21:58.091835+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2613248,"num_objects":638,"num_object_clones":0,"num_object_copies":1276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2313,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16147,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1728,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1675}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007616+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007616+0000","last_peered":"2024-09-11T01:20:22.007616+0000","last_clean":"2024-09-11T01:20:22.007616+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007616+0000","last_undegraded":"2024-09-11T01:20:22.007616+0000","last_fullsized":"2024-09-11T01:20:22.007616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008206+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008206+0000","last_peered":"2024-09-11T01:20:22.008206+0000","last_clean":"2024-09-11T01:20:22.008206+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008206+0000","last_undegraded":"2024-09-11T01:20:22.008206+0000","last_fullsized":"2024-09-11T01:20:22.008206+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19407","reported_seq":19551,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:58.106005+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:21:58.106005+0000","last_peered":"2024-09-11T01:21:58.106005+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:21:58.106005+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2232937,"num_objects":546,"num_object_clones":0,"num_object_copies":1092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16477,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":38724225,"num_objects":9456,"num_object_clones":0,"num_object_copies":18912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9456,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":272831,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13533,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":272977920,"data_stored":272914112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160581,"ondisk_log_size":160581,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182525,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535676,"kb_used_data":93068,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93836164,"statfs":{"total":96636764160,"available":96088231936,"internally_reserved":0,"allocated":95301632,"data_stored":95021478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":453177003},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,6,6,4,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986298,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506140,"kb_used_data":145484,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93865700,"statfs":{"total":96636764160,"available":96118476800,"internally_reserved":0,"allocated":148975616,"data_stored":148695467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659242,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":454132,"kb_used_data":31012,"kb_used_omap":4,"kb_used_meta":423099,"kb_avail":93917708,"statfs":{"total":96636764160,"available":96171732992,"internally_reserved":0,"allocated":31756288,"data_stored":31426835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":433254063},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,1,2,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":94760960,"data_stored":94757481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":30150656,"data_stored":30097878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":148066304,"data_stored":148058753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:01.912 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:22:01.912 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'19639", 'reported_seq': 19727, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:58.061199+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:21:58.061199+0000', 'last_peered': '2024-09-11T01:21:58.061199+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:21:58.061199+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10039, 'log_dups_size': 0, 'ondisk_log_size': 10039, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2568192, 'num_objects': 627, 'num_object_clones': 0, 'num_object_copies': 1254, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 627, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19639, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3445}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:01.912 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:22:01.912 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'19615", 'reported_seq': 19753, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:00.130260+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:22:00.130260+0000', 'last_peered': '2024-09-11T01:22:00.130260+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:22:00.130260+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10015, 'log_dups_size': 0, 'ondisk_log_size': 10015, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2162712, 'num_objects': 529, 'num_object_clones': 0, 'num_object_copies': 1058, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 529, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 17916, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3367}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:01.912 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:22:01.912 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'19603", 'reported_seq': 19784, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:57.952515+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:21:57.952515+0000', 'last_peered': '2024-09-11T01:21:57.952515+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:21:57.952515+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2555904, 'num_objects': 624, 'num_object_clones': 0, 'num_object_copies': 1248, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 624, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15944, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3327}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:01.912 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:22:01.912 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19303", 'reported_seq': 19444, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:58.064250+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:21:58.064250+0000', 'last_peered': '2024-09-11T01:21:58.064250+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:21:58.064250+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9300", 'ondisk_log_start': "45'9300", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2568192, 'num_objects': 627, 'num_object_clones': 0, 'num_object_copies': 1254, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 627, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16160, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3359}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:01.913 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:22:01.913 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19272", 'reported_seq': 19411, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:58.107070+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:21:58.107070+0000', 'last_peered': '2024-09-11T01:21:58.107070+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:21:58.107070+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9200", 'ondisk_log_start': "45'9200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10072, 'log_dups_size': 0, 'ondisk_log_size': 10072, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2441216, 'num_objects': 596, 'num_object_clones': 0, 'num_object_copies': 1192, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 596, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15859, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3388}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:01.913 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:22:01.913 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'19655", 'reported_seq': 19784, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:00.148585+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:22:00.148585+0000', 'last_peered': '2024-09-11T01:22:00.148585+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:22:00.148585+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10055, 'log_dups_size': 0, 'ondisk_log_size': 10055, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2260992, 'num_objects': 552, 'num_object_clones': 0, 'num_object_copies': 1104, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 552, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17472, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3472}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:01.913 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:22:01.913 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'19649", 'reported_seq': 19791, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:58.032550+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:21:58.032550+0000', 'last_peered': '2024-09-11T01:21:58.032550+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:21:58.032550+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10049, 'log_dups_size': 0, 'ondisk_log_size': 10049, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2519040, 'num_objects': 615, 'num_object_clones': 0, 'num_object_copies': 1230, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 615, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16386, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3335}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:01.913 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:22:01.913 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19538", 'reported_seq': 23140, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:58.091835+0000', 'last_change': '2024-09-11T01:20:20.985862+0000', 'last_active': '2024-09-11T01:21:58.091835+0000', 'last_peered': '2024-09-11T01:21:58.091835+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:21:58.091835+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "45'9500", 'ondisk_log_start': "45'9500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10038, 'log_dups_size': 0, 'ondisk_log_size': 10038, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2613248, 'num_objects': 638, 'num_object_clones': 0, 'num_object_copies': 1276, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2313, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 638, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16147, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1728, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1675}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:01.913 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:22:01.913 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19407", 'reported_seq': 19551, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:21:58.106005+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:21:58.106005+0000', 'last_peered': '2024-09-11T01:21:58.106005+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:21:58.106005+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9400", 'ondisk_log_start': "45'9400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2232937, 'num_objects': 546, 'num_object_clones': 0, 'num_object_copies': 1092, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 546, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16477, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3371}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:01.914 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:22:02.805 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:02.805 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 34106/18916 objects degraded (180.302%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":659,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":9458,"data_bytes":38724244,"bytes_used":1531850752,"bytes_avail":288378441728,"bytes_total":289910292480,"degraded_objects":34106,"degraded_total":18916,"degraded_ratio":1.8030238951152464,"recovering_objects_per_sec":15,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":187,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":447},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:22:02.852 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:22:02.853 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:03.637 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:03.637 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:22:03.717 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":372,"stamp":"2024-09-11T01:22:02.794349+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":38724244,"num_objects":9458,"num_object_clones":0,"num_object_copies":18916,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9458,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":272833,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13534,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160584,"ondisk_log_size":160584,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1495948,"kb_used_data":269564,"kb_used_omap":12,"kb_used_meta":1226291,"kb_avail":281619572,"statfs":{"total":289910292480,"available":288378441728,"internally_reserved":0,"allocated":276033536,"data_stored":275143780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1255721988},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,4,8,9,11,10],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19124224,"num_objects":-4669,"num_object_clones":0,"num_object_copies":-9338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-187,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4669,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4669,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":187,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.017103"},"pg_stats":[{"pgid":"2.f","version":"49'19639","reported_seq":19727,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:58.061199+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:21:58.061199+0000","last_peered":"2024-09-11T01:21:58.061199+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:21:58.061199+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10039,"log_dups_size":0,"ondisk_log_size":10039,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2568192,"num_objects":627,"num_object_clones":0,"num_object_copies":1254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19639,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3445}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19806","reported_seq":19918,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:00.128741+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:00.128741+0000","last_peered":"2024-09-11T01:22:00.128741+0000","last_clean":"2024-09-11T01:22:00.128741+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:00.128741+0000","last_undegraded":"2024-09-11T01:22:00.128741+0000","last_fullsized":"2024-09-11T01:22:00.128741+0000","mapping_epoch":35,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2236416,"num_objects":546,"num_object_clones":0,"num_object_copies":1092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18045,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19615","reported_seq":19753,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:00.130260+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:22:00.130260+0000","last_peered":"2024-09-11T01:22:00.130260+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:22:00.130260+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10015,"log_dups_size":0,"ondisk_log_size":10015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2162712,"num_objects":529,"num_object_clones":0,"num_object_copies":1058,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":529,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":17916,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3367}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'19594","reported_seq":19674,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:00.238043+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:00.238043+0000","last_peered":"2024-09-11T01:22:00.238043+0000","last_clean":"2024-09-11T01:22:00.238043+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:00.238043+0000","last_undegraded":"2024-09-11T01:22:00.238043+0000","last_fullsized":"2024-09-11T01:22:00.238043+0000","mapping_epoch":35,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2211840,"num_objects":540,"num_object_clones":0,"num_object_copies":1080,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":540,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19594,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19408","reported_seq":19546,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:58.016485+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:21:58.016485+0000","last_peered":"2024-09-11T01:21:58.016485+0000","last_clean":"2024-09-11T01:21:58.016485+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:21:58.016485+0000","last_undegraded":"2024-09-11T01:21:58.016485+0000","last_fullsized":"2024-09-11T01:21:58.016485+0000","mapping_epoch":34,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2461696,"num_objects":601,"num_object_clones":0,"num_object_copies":1202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":601,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16413,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'19603","reported_seq":19784,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:57.952515+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:21:57.952515+0000","last_peered":"2024-09-11T01:21:57.952515+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:21:57.952515+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2555904,"num_objects":624,"num_object_clones":0,"num_object_copies":1248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":624,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15944,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3327}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19303","reported_seq":19444,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:58.064250+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:21:58.064250+0000","last_peered":"2024-09-11T01:21:58.064250+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:21:58.064250+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2568192,"num_objects":627,"num_object_clones":0,"num_object_copies":1254,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":627,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16160,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3359}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19605","reported_seq":19745,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:58.040915+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:21:58.040915+0000","last_peered":"2024-09-11T01:21:58.040915+0000","last_clean":"2024-09-11T01:21:58.040915+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:21:58.040915+0000","last_undegraded":"2024-09-11T01:21:58.040915+0000","last_fullsized":"2024-09-11T01:21:58.040915+0000","mapping_epoch":34,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10005,"log_dups_size":0,"ondisk_log_size":10005,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2658304,"num_objects":649,"num_object_clones":0,"num_object_copies":1298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":649,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16481,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003209+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003209+0000","last_peered":"2024-09-11T01:20:22.003209+0000","last_clean":"2024-09-11T01:20:22.003209+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003209+0000","last_undegraded":"2024-09-11T01:20:22.003209+0000","last_fullsized":"2024-09-11T01:20:22.003209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19381","reported_seq":27399,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:57.920648+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:21:57.920648+0000","last_peered":"2024-09-11T01:21:57.920648+0000","last_clean":"2024-09-11T01:21:57.920648+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:21:57.920648+0000","last_undegraded":"2024-09-11T01:21:57.920648+0000","last_fullsized":"2024-09-11T01:21:57.920648+0000","mapping_epoch":48,"log_start":"45'9300","ondisk_log_start":"45'9300","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10081,"log_dups_size":0,"ondisk_log_size":10081,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2424832,"num_objects":592,"num_object_clones":0,"num_object_copies":1184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":592,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17352,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19272","reported_seq":19411,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:58.107070+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:21:58.107070+0000","last_peered":"2024-09-11T01:21:58.107070+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:21:58.107070+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10072,"log_dups_size":0,"ondisk_log_size":10072,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2441216,"num_objects":596,"num_object_clones":0,"num_object_copies":1192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":596,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15859,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3388}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19655","reported_seq":19784,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:00.148585+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:00.148585+0000","last_peered":"2024-09-11T01:22:00.148585+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:00.148585+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10055,"log_dups_size":0,"ondisk_log_size":10055,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2260992,"num_objects":552,"num_object_clones":0,"num_object_copies":1104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":552,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17472,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3472}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003275+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003275+0000","last_peered":"2024-09-11T01:20:22.003275+0000","last_clean":"2024-09-11T01:20:22.003275+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003275+0000","last_undegraded":"2024-09-11T01:20:22.003275+0000","last_fullsized":"2024-09-11T01:20:22.003275+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19262","reported_seq":27253,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:00.272963+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:00.272963+0000","last_peered":"2024-09-11T01:22:00.272963+0000","last_clean":"2024-09-11T01:22:00.272963+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:00.272963+0000","last_undegraded":"2024-09-11T01:22:00.272963+0000","last_fullsized":"2024-09-11T01:22:00.272963+0000","mapping_epoch":44,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10062,"log_dups_size":0,"ondisk_log_size":10062,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2187264,"num_objects":534,"num_object_clones":0,"num_object_copies":1068,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":534,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17047,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19649","reported_seq":19791,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:58.032550+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:21:58.032550+0000","last_peered":"2024-09-11T01:21:58.032550+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:21:58.032550+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10049,"log_dups_size":0,"ondisk_log_size":10049,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2519040,"num_objects":615,"num_object_clones":0,"num_object_copies":1230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":615,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16386,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3335}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007577+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007577+0000","last_peered":"2024-09-11T01:20:22.007577+0000","last_clean":"2024-09-11T01:20:22.007577+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007577+0000","last_undegraded":"2024-09-11T01:20:22.007577+0000","last_fullsized":"2024-09-11T01:20:22.007577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19244","reported_seq":27284,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:21:57.918040+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:21:57.918040+0000","last_peered":"2024-09-11T01:21:57.918040+0000","last_clean":"2024-09-11T01:21:57.918040+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:21:57.918040+0000","last_undegraded":"2024-09-11T01:21:57.918040+0000","last_fullsized":"2024-09-11T01:21:57.918040+0000","mapping_epoch":46,"log_start":"45'9200","ondisk_log_start":"45'9200","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2621440,"num_objects":640,"num_object_clones":0,"num_object_copies":1280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":640,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15899,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19538","reported_seq":23140,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:58.091835+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:21:58.091835+0000","last_peered":"2024-09-11T01:21:58.091835+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:21:58.091835+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10038,"log_dups_size":0,"ondisk_log_size":10038,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2613248,"num_objects":638,"num_object_clones":0,"num_object_copies":1276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2313,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":638,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16147,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1728,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1675}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007616+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007616+0000","last_peered":"2024-09-11T01:20:22.007616+0000","last_clean":"2024-09-11T01:20:22.007616+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007616+0000","last_undegraded":"2024-09-11T01:20:22.007616+0000","last_fullsized":"2024-09-11T01:20:22.007616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008206+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008206+0000","last_peered":"2024-09-11T01:20:22.008206+0000","last_clean":"2024-09-11T01:20:22.008206+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008206+0000","last_undegraded":"2024-09-11T01:20:22.008206+0000","last_fullsized":"2024-09-11T01:20:22.008206+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19407","reported_seq":19551,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:21:58.106005+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:21:58.106005+0000","last_peered":"2024-09-11T01:21:58.106005+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:21:58.106005+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2232937,"num_objects":546,"num_object_clones":0,"num_object_copies":1092,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":546,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16477,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3371}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":38724225,"num_objects":9456,"num_object_clones":0,"num_object_copies":18912,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34106,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9456,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":272831,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13533,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":272977920,"data_stored":272914112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160581,"ondisk_log_size":160581,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182525,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":535676,"kb_used_data":93068,"kb_used_omap":4,"kb_used_meta":442555,"kb_avail":93836164,"statfs":{"total":96636764160,"available":96088231936,"internally_reserved":0,"allocated":95301632,"data_stored":95021478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":453177003},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,6,6,4,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986298,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":506140,"kb_used_data":145484,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93865700,"statfs":{"total":96636764160,"available":96118476800,"internally_reserved":0,"allocated":148975616,"data_stored":148695467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1,2,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659242,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":454132,"kb_used_data":31012,"kb_used_omap":4,"kb_used_meta":423099,"kb_avail":93917708,"statfs":{"total":96636764160,"available":96171732992,"internally_reserved":0,"allocated":31756288,"data_stored":31426835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":433254063},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,2,0,2,1,2,5,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":94760960,"data_stored":94757481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":30150656,"data_stored":30097878,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":148066304,"data_stored":148058753,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:06.719 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:07.388 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:07.388 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:22:07.432 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":374,"stamp":"2024-09-11T01:22:06.795228+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":29119124,"num_objects":7113,"num_object_clones":0,"num_object_copies":14226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34008,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7113,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":275178,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13632,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160629,"ondisk_log_size":160629,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1495240,"kb_used_data":250836,"kb_used_omap":12,"kb_used_meta":1244339,"kb_avail":281620280,"statfs":{"total":289910292480,"available":288379166720,"internally_reserved":0,"allocated":256856064,"data_stored":255966308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1274203140},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,0,6,3,6,13,3,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22097920,"num_objects":-5395,"num_object_clones":0,"num_object_copies":-10790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-184,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5395,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.016968"},"pg_stats":[{"pgid":"2.f","version":"49'19803","reported_seq":19891,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:03.087851+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:22:03.087851+0000","last_peered":"2024-09-11T01:22:03.087851+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:22:03.087851+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1896448,"num_objects":463,"num_object_clones":0,"num_object_copies":926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19803,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3609}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19954","reported_seq":20065,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:05.265333+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:05.265333+0000","last_peered":"2024-09-11T01:22:05.265333+0000","last_clean":"2024-09-11T01:22:05.265333+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:05.265333+0000","last_undegraded":"2024-09-11T01:22:05.265333+0000","last_fullsized":"2024-09-11T01:22:05.265333+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1630208,"num_objects":398,"num_object_clones":0,"num_object_copies":796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":398,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18193,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19750","reported_seq":19889,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:05.273594+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:22:05.273594+0000","last_peered":"2024-09-11T01:22:05.273594+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:22:05.273594+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1609752,"num_objects":394,"num_object_clones":0,"num_object_copies":788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":394,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18051,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3502}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'19752","reported_seq":19832,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:05.286801+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:05.286801+0000","last_peered":"2024-09-11T01:22:05.286801+0000","last_clean":"2024-09-11T01:22:05.286801+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:05.286801+0000","last_undegraded":"2024-09-11T01:22:05.286801+0000","last_fullsized":"2024-09-11T01:22:05.286801+0000","mapping_epoch":35,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564672,"num_objects":382,"num_object_clones":0,"num_object_copies":764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":382,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19752,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19545","reported_seq":19683,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:03.093574+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:03.093574+0000","last_peered":"2024-09-11T01:22:03.093574+0000","last_clean":"2024-09-11T01:22:03.093574+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:03.093574+0000","last_undegraded":"2024-09-11T01:22:03.093574+0000","last_fullsized":"2024-09-11T01:22:03.093574+0000","mapping_epoch":34,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1900544,"num_objects":464,"num_object_clones":0,"num_object_copies":928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16550,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'19736","reported_seq":19916,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:03.039770+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:03.039770+0000","last_peered":"2024-09-11T01:22:03.039770+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:03.039770+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2011136,"num_objects":491,"num_object_clones":0,"num_object_copies":982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16077,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3460}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19444","reported_seq":19585,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:02.977755+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:02.977755+0000","last_peered":"2024-09-11T01:22:02.977755+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:02.977755+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1990656,"num_objects":486,"num_object_clones":0,"num_object_copies":972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":486,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16301,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3500}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19761","reported_seq":19901,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:03.059589+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:03.059589+0000","last_peered":"2024-09-11T01:22:03.059589+0000","last_clean":"2024-09-11T01:22:03.059589+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:03.059589+0000","last_undegraded":"2024-09-11T01:22:03.059589+0000","last_fullsized":"2024-09-11T01:22:03.059589+0000","mapping_epoch":34,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2019328,"num_objects":493,"num_object_clones":0,"num_object_copies":986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16637,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003209+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003209+0000","last_peered":"2024-09-11T01:20:22.003209+0000","last_clean":"2024-09-11T01:20:22.003209+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003209+0000","last_undegraded":"2024-09-11T01:20:22.003209+0000","last_fullsized":"2024-09-11T01:20:22.003209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19528","reported_seq":27546,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:03.016488+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:03.016488+0000","last_peered":"2024-09-11T01:22:03.016488+0000","last_clean":"2024-09-11T01:22:03.016488+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:03.016488+0000","last_undegraded":"2024-09-11T01:22:03.016488+0000","last_fullsized":"2024-09-11T01:22:03.016488+0000","mapping_epoch":48,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1822720,"num_objects":445,"num_object_clones":0,"num_object_copies":890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19406","reported_seq":19544,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:03.061614+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:03.061614+0000","last_peered":"2024-09-11T01:22:03.061614+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:03.061614+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1892352,"num_objects":462,"num_object_clones":0,"num_object_copies":924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15993,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3522}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19812","reported_seq":19941,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:05.223064+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:05.223064+0000","last_peered":"2024-09-11T01:22:05.223064+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:05.223064+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1617920,"num_objects":395,"num_object_clones":0,"num_object_copies":790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17629,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3629}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003275+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003275+0000","last_peered":"2024-09-11T01:20:22.003275+0000","last_clean":"2024-09-11T01:20:22.003275+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003275+0000","last_undegraded":"2024-09-11T01:20:22.003275+0000","last_fullsized":"2024-09-11T01:20:22.003275+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19431","reported_seq":27422,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:05.303078+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:05.303078+0000","last_peered":"2024-09-11T01:22:05.303078+0000","last_clean":"2024-09-11T01:22:05.303078+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:05.303078+0000","last_undegraded":"2024-09-11T01:22:05.303078+0000","last_fullsized":"2024-09-11T01:22:05.303078+0000","mapping_epoch":44,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1495040,"num_objects":365,"num_object_clones":0,"num_object_copies":730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17216,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19793","reported_seq":19935,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:03.086286+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:03.086286+0000","last_peered":"2024-09-11T01:22:03.086286+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:03.086286+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1929216,"num_objects":471,"num_object_clones":0,"num_object_copies":942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":471,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16530,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3479}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007577+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007577+0000","last_peered":"2024-09-11T01:20:22.007577+0000","last_clean":"2024-09-11T01:20:22.007577+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007577+0000","last_undegraded":"2024-09-11T01:20:22.007577+0000","last_fullsized":"2024-09-11T01:20:22.007577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19412","reported_seq":27453,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:02.971793+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:02.971793+0000","last_peered":"2024-09-11T01:22:02.971793+0000","last_clean":"2024-09-11T01:22:02.971793+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:02.971793+0000","last_undegraded":"2024-09-11T01:22:02.971793+0000","last_fullsized":"2024-09-11T01:22:02.971793+0000","mapping_epoch":46,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1933312,"num_objects":472,"num_object_clones":0,"num_object_copies":944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16067,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19676","reported_seq":23472,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:03.077438+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:22:03.077438+0000","last_peered":"2024-09-11T01:22:03.077438+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:22:03.077438+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2048000,"num_objects":500,"num_object_clones":0,"num_object_copies":1000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2215,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16285,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1826,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1715}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007616+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007616+0000","last_peered":"2024-09-11T01:20:22.007616+0000","last_clean":"2024-09-11T01:20:22.007616+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007616+0000","last_undegraded":"2024-09-11T01:20:22.007616+0000","last_fullsized":"2024-09-11T01:20:22.007616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008206+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008206+0000","last_peered":"2024-09-11T01:20:22.008206+0000","last_clean":"2024-09-11T01:20:22.008206+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008206+0000","last_undegraded":"2024-09-11T01:20:22.008206+0000","last_fullsized":"2024-09-11T01:20:22.008206+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19523","reported_seq":19667,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:02.980017+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:02.980017+0000","last_peered":"2024-09-11T01:22:02.980017+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:02.980017+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1757801,"num_objects":430,"num_object_clones":0,"num_object_copies":860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16593,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3487}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":29119105,"num_objects":7111,"num_object_clones":0,"num_object_copies":14222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34008,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7111,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":275176,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13631,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":253800448,"data_stored":253736640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160626,"ondisk_log_size":160626,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182526,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537052,"kb_used_data":86156,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93834788,"statfs":{"total":96636764160,"available":96086822912,"internally_reserved":0,"allocated":88223744,"data_stored":87943590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":461696683},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,1,4,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986299,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505568,"kb_used_data":140752,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93866272,"statfs":{"total":96636764160,"available":96119062528,"internally_reserved":0,"allocated":144130048,"data_stored":143849899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,0,1,1,2,1,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659243,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":452620,"kb_used_data":23928,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93919220,"statfs":{"total":96636764160,"available":96173281280,"internally_reserved":0,"allocated":24502272,"data_stored":24172819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":438955695},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,1,1,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":87711744,"data_stored":87708265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22962176,"data_stored":22909398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":143126528,"data_stored":143118977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:07.434 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:08.124 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:08.126 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:22:08.169 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":374,"stamp":"2024-09-11T01:22:06.795228+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":29119124,"num_objects":7113,"num_object_clones":0,"num_object_copies":14226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34008,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7113,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":275178,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13632,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160629,"ondisk_log_size":160629,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1495240,"kb_used_data":250836,"kb_used_omap":12,"kb_used_meta":1244339,"kb_avail":281620280,"statfs":{"total":289910292480,"available":288379166720,"internally_reserved":0,"allocated":256856064,"data_stored":255966308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1274203140},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[3,2,0,6,3,6,13,3,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-22097920,"num_objects":-5395,"num_object_clones":0,"num_object_copies":-10790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-184,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5395,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":184,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.016968"},"pg_stats":[{"pgid":"2.f","version":"49'19803","reported_seq":19891,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:03.087851+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:22:03.087851+0000","last_peered":"2024-09-11T01:22:03.087851+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:22:03.087851+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1896448,"num_objects":463,"num_object_clones":0,"num_object_copies":926,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":463,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19803,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3609}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19954","reported_seq":20065,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:05.265333+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:05.265333+0000","last_peered":"2024-09-11T01:22:05.265333+0000","last_clean":"2024-09-11T01:22:05.265333+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:05.265333+0000","last_undegraded":"2024-09-11T01:22:05.265333+0000","last_fullsized":"2024-09-11T01:22:05.265333+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1630208,"num_objects":398,"num_object_clones":0,"num_object_copies":796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":398,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18193,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19750","reported_seq":19889,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:05.273594+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:22:05.273594+0000","last_peered":"2024-09-11T01:22:05.273594+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:22:05.273594+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1609752,"num_objects":394,"num_object_clones":0,"num_object_copies":788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":394,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18051,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3502}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'19752","reported_seq":19832,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:05.286801+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:05.286801+0000","last_peered":"2024-09-11T01:22:05.286801+0000","last_clean":"2024-09-11T01:22:05.286801+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:05.286801+0000","last_undegraded":"2024-09-11T01:22:05.286801+0000","last_fullsized":"2024-09-11T01:22:05.286801+0000","mapping_epoch":35,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564672,"num_objects":382,"num_object_clones":0,"num_object_copies":764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":382,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19752,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19545","reported_seq":19683,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:03.093574+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:03.093574+0000","last_peered":"2024-09-11T01:22:03.093574+0000","last_clean":"2024-09-11T01:22:03.093574+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:03.093574+0000","last_undegraded":"2024-09-11T01:22:03.093574+0000","last_fullsized":"2024-09-11T01:22:03.093574+0000","mapping_epoch":34,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1900544,"num_objects":464,"num_object_clones":0,"num_object_copies":928,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":464,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16550,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'19736","reported_seq":19916,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:03.039770+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:03.039770+0000","last_peered":"2024-09-11T01:22:03.039770+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:03.039770+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10036,"log_dups_size":0,"ondisk_log_size":10036,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2011136,"num_objects":491,"num_object_clones":0,"num_object_copies":982,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":491,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16077,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3460}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19444","reported_seq":19585,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:02.977755+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:02.977755+0000","last_peered":"2024-09-11T01:22:02.977755+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:02.977755+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1990656,"num_objects":486,"num_object_clones":0,"num_object_copies":972,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":486,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16301,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3500}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19761","reported_seq":19901,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:03.059589+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:03.059589+0000","last_peered":"2024-09-11T01:22:03.059589+0000","last_clean":"2024-09-11T01:22:03.059589+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:03.059589+0000","last_undegraded":"2024-09-11T01:22:03.059589+0000","last_fullsized":"2024-09-11T01:22:03.059589+0000","mapping_epoch":34,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10061,"log_dups_size":0,"ondisk_log_size":10061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2019328,"num_objects":493,"num_object_clones":0,"num_object_copies":986,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":493,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16637,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003209+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003209+0000","last_peered":"2024-09-11T01:20:22.003209+0000","last_clean":"2024-09-11T01:20:22.003209+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003209+0000","last_undegraded":"2024-09-11T01:20:22.003209+0000","last_fullsized":"2024-09-11T01:20:22.003209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19528","reported_seq":27546,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:03.016488+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:03.016488+0000","last_peered":"2024-09-11T01:22:03.016488+0000","last_clean":"2024-09-11T01:22:03.016488+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:03.016488+0000","last_undegraded":"2024-09-11T01:22:03.016488+0000","last_fullsized":"2024-09-11T01:22:03.016488+0000","mapping_epoch":48,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10028,"log_dups_size":0,"ondisk_log_size":10028,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1822720,"num_objects":445,"num_object_clones":0,"num_object_copies":890,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":445,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17499,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19406","reported_seq":19544,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:03.061614+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:03.061614+0000","last_peered":"2024-09-11T01:22:03.061614+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:03.061614+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10006,"log_dups_size":0,"ondisk_log_size":10006,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1892352,"num_objects":462,"num_object_clones":0,"num_object_copies":924,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":462,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15993,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3522}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19812","reported_seq":19941,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:05.223064+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:05.223064+0000","last_peered":"2024-09-11T01:22:05.223064+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:05.223064+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1617920,"num_objects":395,"num_object_clones":0,"num_object_copies":790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17629,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3629}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003275+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003275+0000","last_peered":"2024-09-11T01:20:22.003275+0000","last_clean":"2024-09-11T01:20:22.003275+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003275+0000","last_undegraded":"2024-09-11T01:20:22.003275+0000","last_fullsized":"2024-09-11T01:20:22.003275+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19431","reported_seq":27422,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:05.303078+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:05.303078+0000","last_peered":"2024-09-11T01:22:05.303078+0000","last_clean":"2024-09-11T01:22:05.303078+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:05.303078+0000","last_undegraded":"2024-09-11T01:22:05.303078+0000","last_fullsized":"2024-09-11T01:22:05.303078+0000","mapping_epoch":44,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1495040,"num_objects":365,"num_object_clones":0,"num_object_copies":730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17216,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19793","reported_seq":19935,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:03.086286+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:03.086286+0000","last_peered":"2024-09-11T01:22:03.086286+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:03.086286+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1929216,"num_objects":471,"num_object_clones":0,"num_object_copies":942,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":471,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16530,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3479}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007577+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007577+0000","last_peered":"2024-09-11T01:20:22.007577+0000","last_clean":"2024-09-11T01:20:22.007577+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007577+0000","last_undegraded":"2024-09-11T01:20:22.007577+0000","last_fullsized":"2024-09-11T01:20:22.007577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19412","reported_seq":27453,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:02.971793+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:02.971793+0000","last_peered":"2024-09-11T01:22:02.971793+0000","last_clean":"2024-09-11T01:22:02.971793+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:02.971793+0000","last_undegraded":"2024-09-11T01:22:02.971793+0000","last_fullsized":"2024-09-11T01:22:02.971793+0000","mapping_epoch":46,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1933312,"num_objects":472,"num_object_clones":0,"num_object_copies":944,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":472,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16067,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19676","reported_seq":23472,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:03.077438+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:22:03.077438+0000","last_peered":"2024-09-11T01:22:03.077438+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:22:03.077438+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2048000,"num_objects":500,"num_object_clones":0,"num_object_copies":1000,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2215,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":500,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16285,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1826,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1715}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007616+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007616+0000","last_peered":"2024-09-11T01:20:22.007616+0000","last_clean":"2024-09-11T01:20:22.007616+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007616+0000","last_undegraded":"2024-09-11T01:20:22.007616+0000","last_fullsized":"2024-09-11T01:20:22.007616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008206+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008206+0000","last_peered":"2024-09-11T01:20:22.008206+0000","last_clean":"2024-09-11T01:20:22.008206+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008206+0000","last_undegraded":"2024-09-11T01:20:22.008206+0000","last_fullsized":"2024-09-11T01:20:22.008206+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19523","reported_seq":19667,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:02.980017+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:02.980017+0000","last_peered":"2024-09-11T01:22:02.980017+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:02.980017+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10023,"log_dups_size":0,"ondisk_log_size":10023,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1757801,"num_objects":430,"num_object_clones":0,"num_object_copies":860,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":430,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16593,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3487}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":29119105,"num_objects":7111,"num_object_clones":0,"num_object_copies":14222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":34008,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7111,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":275176,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13631,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":253800448,"data_stored":253736640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160626,"ondisk_log_size":160626,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182526,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":537052,"kb_used_data":86156,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93834788,"statfs":{"total":96636764160,"available":96086822912,"internally_reserved":0,"allocated":88223744,"data_stored":87943590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":461696683},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,3,1,4,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986299,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505568,"kb_used_data":140752,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93866272,"statfs":{"total":96636764160,"available":96119062528,"internally_reserved":0,"allocated":144130048,"data_stored":143849899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,0,1,1,2,1,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659243,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":452620,"kb_used_data":23928,"kb_used_omap":4,"kb_used_meta":428667,"kb_avail":93919220,"statfs":{"total":96636764160,"available":96173281280,"internally_reserved":0,"allocated":24502272,"data_stored":24172819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":438955695},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,0,0,3,1,1,4,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":87711744,"data_stored":87708265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":22962176,"data_stored":22909398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":143126528,"data_stored":143118977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:08.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:22:08.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'19803", 'reported_seq': 19891, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:03.087851+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:22:03.087851+0000', 'last_peered': '2024-09-11T01:22:03.087851+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:22:03.087851+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10003, 'log_dups_size': 0, 'ondisk_log_size': 10003, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1896448, 'num_objects': 463, 'num_object_clones': 0, 'num_object_copies': 926, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 463, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 19803, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3609}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:08.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:22:08.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'19750", 'reported_seq': 19889, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:05.273594+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:22:05.273594+0000', 'last_peered': '2024-09-11T01:22:05.273594+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:22:05.273594+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10050, 'log_dups_size': 0, 'ondisk_log_size': 10050, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1609752, 'num_objects': 394, 'num_object_clones': 0, 'num_object_copies': 788, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 394, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18051, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3502}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:08.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:22:08.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'19736", 'reported_seq': 19916, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:03.039770+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:22:03.039770+0000', 'last_peered': '2024-09-11T01:22:03.039770+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:22:03.039770+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10036, 'log_dups_size': 0, 'ondisk_log_size': 10036, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2011136, 'num_objects': 491, 'num_object_clones': 0, 'num_object_copies': 982, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 491, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16077, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3460}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:08.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:22:08.171 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19444", 'reported_seq': 19585, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:02.977755+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:22:02.977755+0000', 'last_peered': '2024-09-11T01:22:02.977755+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:22:02.977755+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9400", 'ondisk_log_start': "45'9400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1990656, 'num_objects': 486, 'num_object_clones': 0, 'num_object_copies': 972, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 486, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16301, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3500}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:08.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:22:08.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19406", 'reported_seq': 19544, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:03.061614+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:22:03.061614+0000', 'last_peered': '2024-09-11T01:22:03.061614+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:22:03.061614+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9400", 'ondisk_log_start': "45'9400", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10006, 'log_dups_size': 0, 'ondisk_log_size': 10006, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1892352, 'num_objects': 462, 'num_object_clones': 0, 'num_object_copies': 924, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 462, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 15993, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3522}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:08.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:22:08.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'19812", 'reported_seq': 19941, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:05.223064+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:22:05.223064+0000', 'last_peered': '2024-09-11T01:22:05.223064+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:22:05.223064+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10012, 'log_dups_size': 0, 'ondisk_log_size': 10012, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1617920, 'num_objects': 395, 'num_object_clones': 0, 'num_object_copies': 790, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 395, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17629, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3629}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:08.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:22:08.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'19793", 'reported_seq': 19935, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:03.086286+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:22:03.086286+0000', 'last_peered': '2024-09-11T01:22:03.086286+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:22:03.086286+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1929216, 'num_objects': 471, 'num_object_clones': 0, 'num_object_copies': 942, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 471, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16530, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3479}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:08.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:22:08.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19676", 'reported_seq': 23472, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:03.077438+0000', 'last_change': '2024-09-11T01:20:20.985862+0000', 'last_active': '2024-09-11T01:22:03.077438+0000', 'last_peered': '2024-09-11T01:22:03.077438+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:22:03.077438+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 2048000, 'num_objects': 500, 'num_object_clones': 0, 'num_object_copies': 1000, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2215, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 500, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16285, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1826, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1715}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:08.172 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:22:08.173 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19523", 'reported_seq': 19667, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:02.980017+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:22:02.980017+0000', 'last_peered': '2024-09-11T01:22:02.980017+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:22:02.980017+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9500", 'ondisk_log_start': "45'9500", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10023, 'log_dups_size': 0, 'ondisk_log_size': 10023, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1757801, 'num_objects': 430, 'num_object_clones': 0, 'num_object_copies': 860, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 430, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16593, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3487}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:08.173 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:22:09.209 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:09.209 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33902/11020 objects degraded (307.641%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":665,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":5510,"data_bytes":22553236,"bytes_used":1548156928,"bytes_avail":288362135552,"bytes_total":289910292480,"degraded_objects":33902,"degraded_total":11020,"degraded_ratio":3.0764065335753177,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":290,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":521},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:22:09.271 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:22:09.271 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:10.034 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:10.036 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:22:10.091 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":375,"stamp":"2024-09-11T01:22:08.795670+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":22553236,"num_objects":5510,"num_object_clones":0,"num_object_copies":11020,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33902,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5510,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":276781,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13738,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160632,"ondisk_log_size":160632,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1511872,"kb_used_data":236944,"kb_used_omap":12,"kb_used_meta":1274867,"kb_avail":281603648,"statfs":{"total":289910292480,"available":288362135552,"internally_reserved":0,"allocated":242630656,"data_stored":241740900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1305463812},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,1,4,4,5,9,4,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25665536,"num_objects":-6266,"num_object_clones":0,"num_object_copies":-12532,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-290,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6266,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6266,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":290,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.015266"},"pg_stats":[{"pgid":"2.f","version":"49'19925","reported_seq":20013,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:08.048639+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:22:08.048639+0000","last_peered":"2024-09-11T01:22:08.048639+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:22:08.048639+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1396736,"num_objects":341,"num_object_clones":0,"num_object_copies":682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19925,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3731}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'19954","reported_seq":20065,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:05.265333+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:05.265333+0000","last_peered":"2024-09-11T01:22:05.265333+0000","last_clean":"2024-09-11T01:22:05.265333+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:05.265333+0000","last_undegraded":"2024-09-11T01:22:05.265333+0000","last_fullsized":"2024-09-11T01:22:05.265333+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1630208,"num_objects":398,"num_object_clones":0,"num_object_copies":796,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":398,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18193,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19750","reported_seq":19889,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:05.273594+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:22:05.273594+0000","last_peered":"2024-09-11T01:22:05.273594+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:22:05.273594+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10050,"log_dups_size":0,"ondisk_log_size":10050,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1609752,"num_objects":394,"num_object_clones":0,"num_object_copies":788,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":394,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18051,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3502}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'19752","reported_seq":19832,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:05.286801+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:05.286801+0000","last_peered":"2024-09-11T01:22:05.286801+0000","last_clean":"2024-09-11T01:22:05.286801+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:05.286801+0000","last_undegraded":"2024-09-11T01:22:05.286801+0000","last_fullsized":"2024-09-11T01:22:05.286801+0000","mapping_epoch":35,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564672,"num_objects":382,"num_object_clones":0,"num_object_copies":764,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":382,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19752,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19703","reported_seq":19841,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:08.059660+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:08.059660+0000","last_peered":"2024-09-11T01:22:08.059660+0000","last_clean":"2024-09-11T01:22:08.059660+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:08.059660+0000","last_undegraded":"2024-09-11T01:22:08.059660+0000","last_fullsized":"2024-09-11T01:22:08.059660+0000","mapping_epoch":34,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1253376,"num_objects":306,"num_object_clones":0,"num_object_copies":612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16708,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'19882","reported_seq":20063,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:08.130722+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:08.130722+0000","last_peered":"2024-09-11T01:22:08.130722+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:08.130722+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1413120,"num_objects":345,"num_object_clones":0,"num_object_copies":690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16223,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3606}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19608","reported_seq":19749,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:08.137590+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:08.137590+0000","last_peered":"2024-09-11T01:22:08.137590+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:08.137590+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1318912,"num_objects":322,"num_object_clones":0,"num_object_copies":644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16465,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3664}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19910","reported_seq":20050,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:07.891068+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:07.891068+0000","last_peered":"2024-09-11T01:22:07.891068+0000","last_clean":"2024-09-11T01:22:07.891068+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:07.891068+0000","last_undegraded":"2024-09-11T01:22:07.891068+0000","last_fullsized":"2024-09-11T01:22:07.891068+0000","mapping_epoch":34,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1409024,"num_objects":344,"num_object_clones":0,"num_object_copies":688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16786,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003209+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003209+0000","last_peered":"2024-09-11T01:20:22.003209+0000","last_clean":"2024-09-11T01:20:22.003209+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003209+0000","last_undegraded":"2024-09-11T01:20:22.003209+0000","last_fullsized":"2024-09-11T01:20:22.003209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19686","reported_seq":27704,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:07.938222+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:07.938222+0000","last_peered":"2024-09-11T01:22:07.938222+0000","last_clean":"2024-09-11T01:22:07.938222+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:07.938222+0000","last_undegraded":"2024-09-11T01:22:07.938222+0000","last_fullsized":"2024-09-11T01:22:07.938222+0000","mapping_epoch":48,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1175552,"num_objects":287,"num_object_clones":0,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17657,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19559","reported_seq":19698,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:08.152826+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:08.152826+0000","last_peered":"2024-09-11T01:22:08.152826+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:08.152826+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1265664,"num_objects":309,"num_object_clones":0,"num_object_copies":618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16146,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19812","reported_seq":19941,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:05.223064+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:05.223064+0000","last_peered":"2024-09-11T01:22:05.223064+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:05.223064+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10012,"log_dups_size":0,"ondisk_log_size":10012,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1617920,"num_objects":395,"num_object_clones":0,"num_object_copies":790,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":395,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17629,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3629}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003275+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003275+0000","last_peered":"2024-09-11T01:20:22.003275+0000","last_clean":"2024-09-11T01:20:22.003275+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003275+0000","last_undegraded":"2024-09-11T01:20:22.003275+0000","last_fullsized":"2024-09-11T01:20:22.003275+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19431","reported_seq":27422,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:05.303078+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:05.303078+0000","last_peered":"2024-09-11T01:22:05.303078+0000","last_clean":"2024-09-11T01:22:05.303078+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:05.303078+0000","last_undegraded":"2024-09-11T01:22:05.303078+0000","last_fullsized":"2024-09-11T01:22:05.303078+0000","mapping_epoch":44,"log_start":"45'9400","ondisk_log_start":"45'9400","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10031,"log_dups_size":0,"ondisk_log_size":10031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1495040,"num_objects":365,"num_object_clones":0,"num_object_copies":730,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":365,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17216,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19948","reported_seq":20090,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:08.129783+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:08.129783+0000","last_peered":"2024-09-11T01:22:08.129783+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:08.129783+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1294336,"num_objects":316,"num_object_clones":0,"num_object_copies":632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16685,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3634}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007577+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007577+0000","last_peered":"2024-09-11T01:20:22.007577+0000","last_clean":"2024-09-11T01:20:22.007577+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007577+0000","last_undegraded":"2024-09-11T01:20:22.007577+0000","last_fullsized":"2024-09-11T01:20:22.007577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19551","reported_seq":27592,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:08.056439+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:08.056439+0000","last_peered":"2024-09-11T01:22:08.056439+0000","last_clean":"2024-09-11T01:22:08.056439+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:08.056439+0000","last_undegraded":"2024-09-11T01:22:08.056439+0000","last_fullsized":"2024-09-11T01:22:08.056439+0000","mapping_epoch":46,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1363968,"num_objects":333,"num_object_clones":0,"num_object_copies":666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16206,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19816","reported_seq":23826,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:08.170619+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:22:08.170619+0000","last_peered":"2024-09-11T01:22:08.170619+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:22:08.170619+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1474560,"num_objects":360,"num_object_clones":0,"num_object_copies":720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2109,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1932,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1749}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007616+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007616+0000","last_peered":"2024-09-11T01:20:22.007616+0000","last_clean":"2024-09-11T01:20:22.007616+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007616+0000","last_undegraded":"2024-09-11T01:20:22.007616+0000","last_fullsized":"2024-09-11T01:20:22.007616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008206+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008206+0000","last_peered":"2024-09-11T01:20:22.008206+0000","last_clean":"2024-09-11T01:20:22.008206+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008206+0000","last_undegraded":"2024-09-11T01:20:22.008206+0000","last_fullsized":"2024-09-11T01:20:22.008206+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19642","reported_seq":19786,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:08.168015+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:08.168015+0000","last_peered":"2024-09-11T01:22:08.168015+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:08.168015+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1270377,"num_objects":311,"num_object_clones":0,"num_object_copies":622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16712,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3606}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":22553217,"num_objects":5508,"num_object_clones":0,"num_object_copies":11016,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33902,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5508,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":276779,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13737,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":239558656,"data_stored":239494848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160629,"ondisk_log_size":160629,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182527,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":534184,"kb_used_data":79128,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93837656,"statfs":{"total":96636764160,"available":96089759744,"internally_reserved":0,"allocated":81027072,"data_stored":80746918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,2,3,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986299,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505568,"kb_used_data":140752,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93866272,"statfs":{"total":96636764160,"available":96119062528,"internally_reserved":0,"allocated":144130048,"data_stored":143849899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[2,1,0,0,1,1,2,1,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659244,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":472120,"kb_used_data":17064,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93899720,"statfs":{"total":96636764160,"available":96153313280,"internally_reserved":0,"allocated":17473536,"data_stored":17144083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":465956527},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":80556032,"data_stored":80552553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15876096,"data_stored":15823318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":143126528,"data_stored":143118977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:13.093 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:14.094 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:14.096 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:22:14.177 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":377,"stamp":"2024-09-11T01:22:12.796831+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19563156,"num_objects":4780,"num_object_clones":0,"num_object_copies":9560,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33902,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4780,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":277511,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13738,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160662,"ondisk_log_size":160662,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1494664,"kb_used_data":232216,"kb_used_omap":12,"kb_used_meta":1262387,"kb_avail":281620856,"statfs":{"total":289910292480,"available":288379756544,"internally_reserved":0,"allocated":237789184,"data_stored":236899428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1292684292},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,7,5,5,8,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19161088,"num_objects":-4678,"num_object_clones":0,"num_object_copies":-9356,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-204,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4678,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4678,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.007956"},"pg_stats":[{"pgid":"2.f","version":"49'19925","reported_seq":20013,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:08.048639+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:22:08.048639+0000","last_peered":"2024-09-11T01:22:08.048639+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:22:08.048639+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10025,"log_dups_size":0,"ondisk_log_size":10025,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1396736,"num_objects":341,"num_object_clones":0,"num_object_copies":682,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":341,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19925,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3731}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20102","reported_seq":20214,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:10.300092+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:10.300092+0000","last_peered":"2024-09-11T01:22:10.300092+0000","last_clean":"2024-09-11T01:22:10.300092+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:10.300092+0000","last_undegraded":"2024-09-11T01:22:10.300092+0000","last_fullsized":"2024-09-11T01:22:10.300092+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1024000,"num_objects":250,"num_object_clones":0,"num_object_copies":500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18341,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19893","reported_seq":20032,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:10.311989+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:22:10.311989+0000","last_peered":"2024-09-11T01:22:10.311989+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:22:10.311989+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1024024,"num_objects":251,"num_object_clones":0,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18194,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3645}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'19904","reported_seq":19984,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:10.296481+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:10.296481+0000","last_peered":"2024-09-11T01:22:10.296481+0000","last_clean":"2024-09-11T01:22:10.296481+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:10.296481+0000","last_undegraded":"2024-09-11T01:22:10.296481+0000","last_fullsized":"2024-09-11T01:22:10.296481+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":942080,"num_objects":230,"num_object_clones":0,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19904,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19703","reported_seq":19841,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:08.059660+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:08.059660+0000","last_peered":"2024-09-11T01:22:08.059660+0000","last_clean":"2024-09-11T01:22:08.059660+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:08.059660+0000","last_undegraded":"2024-09-11T01:22:08.059660+0000","last_fullsized":"2024-09-11T01:22:08.059660+0000","mapping_epoch":34,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1253376,"num_objects":306,"num_object_clones":0,"num_object_copies":612,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":306,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16708,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'19882","reported_seq":20063,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:08.130722+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:08.130722+0000","last_peered":"2024-09-11T01:22:08.130722+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:08.130722+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10082,"log_dups_size":0,"ondisk_log_size":10082,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1413120,"num_objects":345,"num_object_clones":0,"num_object_copies":690,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":345,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16223,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3606}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19608","reported_seq":19749,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:08.137590+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:08.137590+0000","last_peered":"2024-09-11T01:22:08.137590+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:08.137590+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10008,"log_dups_size":0,"ondisk_log_size":10008,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1318912,"num_objects":322,"num_object_clones":0,"num_object_copies":644,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":322,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16465,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3664}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'19910","reported_seq":20050,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:07.891068+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:07.891068+0000","last_peered":"2024-09-11T01:22:07.891068+0000","last_clean":"2024-09-11T01:22:07.891068+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:07.891068+0000","last_undegraded":"2024-09-11T01:22:07.891068+0000","last_fullsized":"2024-09-11T01:22:07.891068+0000","mapping_epoch":34,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10010,"log_dups_size":0,"ondisk_log_size":10010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1409024,"num_objects":344,"num_object_clones":0,"num_object_copies":688,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":344,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16786,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003209+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003209+0000","last_peered":"2024-09-11T01:20:22.003209+0000","last_clean":"2024-09-11T01:20:22.003209+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003209+0000","last_undegraded":"2024-09-11T01:20:22.003209+0000","last_fullsized":"2024-09-11T01:20:22.003209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19686","reported_seq":27704,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:07.938222+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:07.938222+0000","last_peered":"2024-09-11T01:22:07.938222+0000","last_clean":"2024-09-11T01:22:07.938222+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:07.938222+0000","last_undegraded":"2024-09-11T01:22:07.938222+0000","last_fullsized":"2024-09-11T01:22:07.938222+0000","mapping_epoch":48,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10086,"log_dups_size":0,"ondisk_log_size":10086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1175552,"num_objects":287,"num_object_clones":0,"num_object_copies":574,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":287,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17657,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19559","reported_seq":19698,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:08.152826+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:08.152826+0000","last_peered":"2024-09-11T01:22:08.152826+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:08.152826+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10059,"log_dups_size":0,"ondisk_log_size":10059,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1265664,"num_objects":309,"num_object_clones":0,"num_object_copies":618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":309,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16146,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3675}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19946","reported_seq":20075,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:10.260493+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:10.260493+0000","last_peered":"2024-09-11T01:22:10.260493+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:10.260493+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1069056,"num_objects":261,"num_object_clones":0,"num_object_copies":522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17763,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3763}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003275+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003275+0000","last_peered":"2024-09-11T01:20:22.003275+0000","last_clean":"2024-09-11T01:20:22.003275+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003275+0000","last_undegraded":"2024-09-11T01:20:22.003275+0000","last_fullsized":"2024-09-11T01:20:22.003275+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19584","reported_seq":27574,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:10.330851+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:10.330851+0000","last_peered":"2024-09-11T01:22:10.330851+0000","last_clean":"2024-09-11T01:22:10.330851+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:10.330851+0000","last_undegraded":"2024-09-11T01:22:10.330851+0000","last_fullsized":"2024-09-11T01:22:10.330851+0000","mapping_epoch":44,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":868352,"num_objects":212,"num_object_clones":0,"num_object_copies":424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17369,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'19948","reported_seq":20090,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:08.129783+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:08.129783+0000","last_peered":"2024-09-11T01:22:08.129783+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:08.129783+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10048,"log_dups_size":0,"ondisk_log_size":10048,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1294336,"num_objects":316,"num_object_clones":0,"num_object_copies":632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":316,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16685,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3634}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007577+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007577+0000","last_peered":"2024-09-11T01:20:22.007577+0000","last_clean":"2024-09-11T01:20:22.007577+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007577+0000","last_undegraded":"2024-09-11T01:20:22.007577+0000","last_fullsized":"2024-09-11T01:20:22.007577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19551","reported_seq":27592,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:08.056439+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:08.056439+0000","last_peered":"2024-09-11T01:22:08.056439+0000","last_clean":"2024-09-11T01:22:08.056439+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:08.056439+0000","last_undegraded":"2024-09-11T01:22:08.056439+0000","last_fullsized":"2024-09-11T01:22:08.056439+0000","mapping_epoch":46,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10051,"log_dups_size":0,"ondisk_log_size":10051,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1363968,"num_objects":333,"num_object_clones":0,"num_object_copies":666,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":333,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16206,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19816","reported_seq":23826,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:08.170619+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:22:08.170619+0000","last_peered":"2024-09-11T01:22:08.170619+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:22:08.170619+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10016,"log_dups_size":0,"ondisk_log_size":10016,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1474560,"num_objects":360,"num_object_clones":0,"num_object_copies":720,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2109,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":360,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16425,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1932,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1749}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007616+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007616+0000","last_peered":"2024-09-11T01:20:22.007616+0000","last_clean":"2024-09-11T01:20:22.007616+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007616+0000","last_undegraded":"2024-09-11T01:20:22.007616+0000","last_fullsized":"2024-09-11T01:20:22.007616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008206+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008206+0000","last_peered":"2024-09-11T01:20:22.008206+0000","last_clean":"2024-09-11T01:20:22.008206+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008206+0000","last_undegraded":"2024-09-11T01:20:22.008206+0000","last_fullsized":"2024-09-11T01:20:22.008206+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19642","reported_seq":19786,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:08.168015+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:08.168015+0000","last_peered":"2024-09-11T01:22:08.168015+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:08.168015+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1270377,"num_objects":311,"num_object_clones":0,"num_object_copies":622,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":311,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16712,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3606}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":19563137,"num_objects":4778,"num_object_clones":0,"num_object_copies":9556,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33902,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4778,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":277509,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13737,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":234827776,"data_stored":234763968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160659,"ondisk_log_size":160659,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182527,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":534184,"kb_used_data":79128,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93837656,"statfs":{"total":96636764160,"available":96089759744,"internally_reserved":0,"allocated":81027072,"data_stored":80746918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1,4,2,3,4,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986300,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488360,"kb_used_data":136024,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93883480,"statfs":{"total":96636764160,"available":96136683520,"internally_reserved":0,"allocated":139288576,"data_stored":139008427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":360771242},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659244,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":472120,"kb_used_data":17064,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93899720,"statfs":{"total":96636764160,"available":96153313280,"internally_reserved":0,"allocated":17473536,"data_stored":17144083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":465956527},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,1,3],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":80556032,"data_stored":80552553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15876096,"data_stored":15823318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":138395648,"data_stored":138388097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:14.179 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:14.894 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:14.896 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:22:14.940 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":378,"stamp":"2024-09-11T01:22:14.800643+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":12878484,"num_objects":3148,"num_object_clones":0,"num_object_copies":6296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33815,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3148,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":279143,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13825,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160894,"ondisk_log_size":160894,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1463900,"kb_used_data":218092,"kb_used_omap":12,"kb_used_meta":1245747,"kb_avail":281651620,"statfs":{"total":289910292480,"available":288411258880,"internally_reserved":0,"allocated":223326208,"data_stored":222436452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1275644932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,7,5,6,13,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25845760,"num_objects":-6310,"num_object_clones":0,"num_object_copies":-12620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-291,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6310,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006294"},"pg_stats":[{"pgid":"2.f","version":"49'20089","reported_seq":20176,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:13.120097+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:22:13.120097+0000","last_peered":"2024-09-11T01:22:13.120097+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:22:13.120097+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":724992,"num_objects":177,"num_object_clones":0,"num_object_copies":354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20089,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3895}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20102","reported_seq":20214,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:10.300092+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:10.300092+0000","last_peered":"2024-09-11T01:22:10.300092+0000","last_clean":"2024-09-11T01:22:10.300092+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:10.300092+0000","last_undegraded":"2024-09-11T01:22:10.300092+0000","last_fullsized":"2024-09-11T01:22:10.300092+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1024000,"num_objects":250,"num_object_clones":0,"num_object_copies":500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18341,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19893","reported_seq":20032,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:10.311989+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:22:10.311989+0000","last_peered":"2024-09-11T01:22:10.311989+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:22:10.311989+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1024024,"num_objects":251,"num_object_clones":0,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18194,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3645}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'19904","reported_seq":19984,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:10.296481+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:10.296481+0000","last_peered":"2024-09-11T01:22:10.296481+0000","last_clean":"2024-09-11T01:22:10.296481+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:10.296481+0000","last_undegraded":"2024-09-11T01:22:10.296481+0000","last_fullsized":"2024-09-11T01:22:10.296481+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":942080,"num_objects":230,"num_object_clones":0,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19904,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19826","reported_seq":19964,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:13.138492+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:13.138492+0000","last_peered":"2024-09-11T01:22:13.138492+0000","last_clean":"2024-09-11T01:22:13.138492+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:13.138492+0000","last_undegraded":"2024-09-11T01:22:13.138492+0000","last_fullsized":"2024-09-11T01:22:13.138492+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":749568,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16831,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20058","reported_seq":20238,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:13.172957+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:13.172957+0000","last_peered":"2024-09-11T01:22:13.172957+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:13.172957+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":692224,"num_objects":169,"num_object_clones":0,"num_object_copies":338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16399,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3782}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19745","reported_seq":19885,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:13.121521+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:13.121521+0000","last_peered":"2024-09-11T01:22:13.121521+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:13.121521+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":757760,"num_objects":185,"num_object_clones":0,"num_object_copies":370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16602,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3801}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20070","reported_seq":20210,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:13.148200+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:13.148200+0000","last_peered":"2024-09-11T01:22:13.148200+0000","last_clean":"2024-09-11T01:22:13.148200+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:13.148200+0000","last_undegraded":"2024-09-11T01:22:13.148200+0000","last_fullsized":"2024-09-11T01:22:13.148200+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":753664,"num_objects":184,"num_object_clones":0,"num_object_copies":368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16946,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003209+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003209+0000","last_peered":"2024-09-11T01:20:22.003209+0000","last_clean":"2024-09-11T01:20:22.003209+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003209+0000","last_undegraded":"2024-09-11T01:20:22.003209+0000","last_fullsized":"2024-09-11T01:20:22.003209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19821","reported_seq":27839,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:13.147404+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:13.147404+0000","last_peered":"2024-09-11T01:22:13.147404+0000","last_clean":"2024-09-11T01:22:13.147404+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:13.147404+0000","last_undegraded":"2024-09-11T01:22:13.147404+0000","last_fullsized":"2024-09-11T01:22:13.147404+0000","mapping_epoch":48,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622592,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17792,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19700","reported_seq":19839,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:13.177675+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:13.177675+0000","last_peered":"2024-09-11T01:22:13.177675+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:13.177675+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":688128,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16287,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3816}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19946","reported_seq":20075,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:10.260493+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:10.260493+0000","last_peered":"2024-09-11T01:22:10.260493+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:10.260493+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1069056,"num_objects":261,"num_object_clones":0,"num_object_copies":522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17763,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3763}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003275+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003275+0000","last_peered":"2024-09-11T01:20:22.003275+0000","last_clean":"2024-09-11T01:20:22.003275+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003275+0000","last_undegraded":"2024-09-11T01:20:22.003275+0000","last_fullsized":"2024-09-11T01:20:22.003275+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19584","reported_seq":27574,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:10.330851+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:10.330851+0000","last_peered":"2024-09-11T01:22:10.330851+0000","last_clean":"2024-09-11T01:22:10.330851+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:10.330851+0000","last_undegraded":"2024-09-11T01:22:10.330851+0000","last_fullsized":"2024-09-11T01:22:10.330851+0000","mapping_epoch":44,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":868352,"num_objects":212,"num_object_clones":0,"num_object_copies":424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17369,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20078","reported_seq":20220,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:13.011543+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:13.011543+0000","last_peered":"2024-09-11T01:22:13.011543+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:13.011543+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":761856,"num_objects":186,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16815,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3764}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007577+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007577+0000","last_peered":"2024-09-11T01:20:22.007577+0000","last_clean":"2024-09-11T01:20:22.007577+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007577+0000","last_undegraded":"2024-09-11T01:20:22.007577+0000","last_fullsized":"2024-09-11T01:20:22.007577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19709","reported_seq":27751,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:13.138383+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:13.138383+0000","last_peered":"2024-09-11T01:22:13.138383+0000","last_clean":"2024-09-11T01:22:13.138383+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:13.138383+0000","last_undegraded":"2024-09-11T01:22:13.138383+0000","last_fullsized":"2024-09-11T01:22:13.138383+0000","mapping_epoch":46,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":716800,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16364,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19989","reported_seq":24173,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:13.119262+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:22:13.119262+0000","last_peered":"2024-09-11T01:22:13.119262+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:22:13.119262+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":765952,"num_objects":187,"num_object_clones":0,"num_object_copies":374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16598,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2019,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1835}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007616+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007616+0000","last_peered":"2024-09-11T01:20:22.007616+0000","last_clean":"2024-09-11T01:20:22.007616+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007616+0000","last_undegraded":"2024-09-11T01:20:22.007616+0000","last_fullsized":"2024-09-11T01:20:22.007616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008206+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008206+0000","last_peered":"2024-09-11T01:20:22.008206+0000","last_clean":"2024-09-11T01:20:22.008206+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008206+0000","last_undegraded":"2024-09-11T01:20:22.008206+0000","last_fullsized":"2024-09-11T01:20:22.008206+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19777","reported_seq":19921,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:13.027392+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:13.027392+0000","last_peered":"2024-09-11T01:22:13.027392+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:13.027392+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":717417,"num_objects":176,"num_object_clones":0,"num_object_copies":352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16847,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3741}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":12878465,"num_objects":3146,"num_object_clones":0,"num_object_copies":6292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33815,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3146,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":279141,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13824,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":220041216,"data_stored":219977408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160891,"ondisk_log_size":160891,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182528,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518784,"kb_used_data":72048,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93853056,"statfs":{"total":96636764160,"available":96105529344,"internally_reserved":0,"allocated":73777152,"data_stored":73496998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":457436843},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,4,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986300,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488360,"kb_used_data":136024,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93883480,"statfs":{"total":96636764160,"available":96136683520,"internally_reserved":0,"allocated":139288576,"data_stored":139008427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":360771242},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659245,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":456756,"kb_used_data":10020,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93915084,"statfs":{"total":96636764160,"available":96169046016,"internally_reserved":0,"allocated":10260480,"data_stored":9931027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":457436847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":73170944,"data_stored":73167465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8474624,"data_stored":8421846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":138395648,"data_stored":138388097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:14.942 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:22:14.942 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20089", 'reported_seq': 20176, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:13.120097+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:22:13.120097+0000', 'last_peered': '2024-09-11T01:22:13.120097+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:22:13.120097+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 724992, 'num_objects': 177, 'num_object_clones': 0, 'num_object_copies': 354, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 177, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20089, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3895}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:14.942 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:22:14.942 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'19893", 'reported_seq': 20032, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:10.311989+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:22:10.311989+0000', 'last_peered': '2024-09-11T01:22:10.311989+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:22:10.311989+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10093, 'log_dups_size': 0, 'ondisk_log_size': 10093, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1024024, 'num_objects': 251, 'num_object_clones': 0, 'num_object_copies': 502, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 251, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18194, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3645}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:14.942 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:22:14.942 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'20058", 'reported_seq': 20238, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:13.172957+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:22:13.172957+0000', 'last_peered': '2024-09-11T01:22:13.172957+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:22:13.172957+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10058, 'log_dups_size': 0, 'ondisk_log_size': 10058, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 692224, 'num_objects': 169, 'num_object_clones': 0, 'num_object_copies': 338, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 169, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16399, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3782}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:14.942 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:22:14.942 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19745", 'reported_seq': 19885, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:13.121521+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:22:13.121521+0000', 'last_peered': '2024-09-11T01:22:13.121521+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:22:13.121521+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10045, 'log_dups_size': 0, 'ondisk_log_size': 10045, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 757760, 'num_objects': 185, 'num_object_clones': 0, 'num_object_copies': 370, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 185, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16602, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3801}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:14.942 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:22:14.943 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19700", 'reported_seq': 19839, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:13.177675+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:22:13.177675+0000', 'last_peered': '2024-09-11T01:22:13.177675+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:22:13.177675+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9600", 'ondisk_log_start': "45'9600", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 688128, 'num_objects': 168, 'num_object_clones': 0, 'num_object_copies': 336, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 168, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16287, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3816}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:14.943 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:22:14.943 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'19946", 'reported_seq': 20075, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:10.260493+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:22:10.260493+0000', 'last_peered': '2024-09-11T01:22:10.260493+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:22:10.260493+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10046, 'log_dups_size': 0, 'ondisk_log_size': 10046, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 1069056, 'num_objects': 261, 'num_object_clones': 0, 'num_object_copies': 522, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 261, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17763, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3763}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:14.943 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:22:14.943 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20078", 'reported_seq': 20220, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:13.011543+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:22:13.011543+0000', 'last_peered': '2024-09-11T01:22:13.011543+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:22:13.011543+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10078, 'log_dups_size': 0, 'ondisk_log_size': 10078, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 761856, 'num_objects': 186, 'num_object_clones': 0, 'num_object_copies': 372, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 186, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16815, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3764}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:14.943 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:22:14.943 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'19989", 'reported_seq': 24173, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:13.119262+0000', 'last_change': '2024-09-11T01:20:20.985862+0000', 'last_active': '2024-09-11T01:22:13.119262+0000', 'last_peered': '2024-09-11T01:22:13.119262+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:22:13.119262+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10089, 'log_dups_size': 0, 'ondisk_log_size': 10089, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 765952, 'num_objects': 187, 'num_object_clones': 0, 'num_object_copies': 374, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2022, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 187, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16598, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2019, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1835}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:14.943 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:22:14.943 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19777", 'reported_seq': 19921, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:13.027392+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:22:13.027392+0000', 'last_peered': '2024-09-11T01:22:13.027392+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:22:13.027392+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9700", 'ondisk_log_start': "45'9700", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10077, 'log_dups_size': 0, 'ondisk_log_size': 10077, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 717417, 'num_objects': 176, 'num_object_clones': 0, 'num_object_copies': 352, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 176, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16847, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3741}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:14.944 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:22:15.872 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:15.873 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33815/6296 objects degraded (537.087%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":672,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3148,"data_bytes":12878484,"bytes_used":1499033600,"bytes_avail":288411258880,"bytes_total":289910292480,"degraded_objects":33815,"degraded_total":6296,"degraded_ratio":5.3708703939008897,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":525},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:22:15.936 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:22:15.936 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:16.656 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:16.658 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:22:16.717 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":378,"stamp":"2024-09-11T01:22:14.800643+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":12878484,"num_objects":3148,"num_object_clones":0,"num_object_copies":6296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33815,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3148,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":279143,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13825,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160894,"ondisk_log_size":160894,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1463900,"kb_used_data":218092,"kb_used_omap":12,"kb_used_meta":1245747,"kb_avail":281651620,"statfs":{"total":289910292480,"available":288411258880,"internally_reserved":0,"allocated":223326208,"data_stored":222436452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1275644932},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,7,5,6,13,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25845760,"num_objects":-6310,"num_object_clones":0,"num_object_copies":-12620,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-291,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6310,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6310,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":291,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006294"},"pg_stats":[{"pgid":"2.f","version":"49'20089","reported_seq":20176,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:13.120097+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:22:13.120097+0000","last_peered":"2024-09-11T01:22:13.120097+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:22:13.120097+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":724992,"num_objects":177,"num_object_clones":0,"num_object_copies":354,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20089,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3895}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20102","reported_seq":20214,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:10.300092+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:10.300092+0000","last_peered":"2024-09-11T01:22:10.300092+0000","last_clean":"2024-09-11T01:22:10.300092+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:10.300092+0000","last_undegraded":"2024-09-11T01:22:10.300092+0000","last_fullsized":"2024-09-11T01:22:10.300092+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10002,"log_dups_size":0,"ondisk_log_size":10002,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1024000,"num_objects":250,"num_object_clones":0,"num_object_copies":500,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":250,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18341,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'19893","reported_seq":20032,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:10.311989+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:22:10.311989+0000","last_peered":"2024-09-11T01:22:10.311989+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:22:10.311989+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10093,"log_dups_size":0,"ondisk_log_size":10093,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1024024,"num_objects":251,"num_object_clones":0,"num_object_copies":502,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":251,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18194,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3645}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'19904","reported_seq":19984,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:10.296481+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:10.296481+0000","last_peered":"2024-09-11T01:22:10.296481+0000","last_clean":"2024-09-11T01:22:10.296481+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:10.296481+0000","last_undegraded":"2024-09-11T01:22:10.296481+0000","last_fullsized":"2024-09-11T01:22:10.296481+0000","mapping_epoch":35,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10004,"log_dups_size":0,"ondisk_log_size":10004,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":942080,"num_objects":230,"num_object_clones":0,"num_object_copies":460,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":230,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19904,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19826","reported_seq":19964,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:13.138492+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:13.138492+0000","last_peered":"2024-09-11T01:22:13.138492+0000","last_clean":"2024-09-11T01:22:13.138492+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:13.138492+0000","last_undegraded":"2024-09-11T01:22:13.138492+0000","last_fullsized":"2024-09-11T01:22:13.138492+0000","mapping_epoch":34,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":749568,"num_objects":183,"num_object_clones":0,"num_object_copies":366,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":183,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16831,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20058","reported_seq":20238,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:13.172957+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:13.172957+0000","last_peered":"2024-09-11T01:22:13.172957+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:13.172957+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10058,"log_dups_size":0,"ondisk_log_size":10058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":692224,"num_objects":169,"num_object_clones":0,"num_object_copies":338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":169,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16399,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3782}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19745","reported_seq":19885,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:13.121521+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:13.121521+0000","last_peered":"2024-09-11T01:22:13.121521+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:13.121521+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10045,"log_dups_size":0,"ondisk_log_size":10045,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":757760,"num_objects":185,"num_object_clones":0,"num_object_copies":370,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":185,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16602,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3801}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20070","reported_seq":20210,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:13.148200+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:13.148200+0000","last_peered":"2024-09-11T01:22:13.148200+0000","last_clean":"2024-09-11T01:22:13.148200+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:13.148200+0000","last_undegraded":"2024-09-11T01:22:13.148200+0000","last_fullsized":"2024-09-11T01:22:13.148200+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10070,"log_dups_size":0,"ondisk_log_size":10070,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":753664,"num_objects":184,"num_object_clones":0,"num_object_copies":368,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":184,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16946,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003209+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003209+0000","last_peered":"2024-09-11T01:20:22.003209+0000","last_clean":"2024-09-11T01:20:22.003209+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003209+0000","last_undegraded":"2024-09-11T01:20:22.003209+0000","last_fullsized":"2024-09-11T01:20:22.003209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19821","reported_seq":27839,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:13.147404+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:13.147404+0000","last_peered":"2024-09-11T01:22:13.147404+0000","last_clean":"2024-09-11T01:22:13.147404+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:13.147404+0000","last_undegraded":"2024-09-11T01:22:13.147404+0000","last_fullsized":"2024-09-11T01:22:13.147404+0000","mapping_epoch":48,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10021,"log_dups_size":0,"ondisk_log_size":10021,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622592,"num_objects":152,"num_object_clones":0,"num_object_copies":304,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":152,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17792,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19700","reported_seq":19839,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:13.177675+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:13.177675+0000","last_peered":"2024-09-11T01:22:13.177675+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:13.177675+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9600","ondisk_log_start":"45'9600","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":688128,"num_objects":168,"num_object_clones":0,"num_object_copies":336,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":168,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16287,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3816}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'19946","reported_seq":20075,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:10.260493+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:10.260493+0000","last_peered":"2024-09-11T01:22:10.260493+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:10.260493+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10046,"log_dups_size":0,"ondisk_log_size":10046,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1069056,"num_objects":261,"num_object_clones":0,"num_object_copies":522,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":261,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17763,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3763}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003275+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003275+0000","last_peered":"2024-09-11T01:20:22.003275+0000","last_clean":"2024-09-11T01:20:22.003275+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003275+0000","last_undegraded":"2024-09-11T01:20:22.003275+0000","last_fullsized":"2024-09-11T01:20:22.003275+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19584","reported_seq":27574,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:10.330851+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:10.330851+0000","last_peered":"2024-09-11T01:22:10.330851+0000","last_clean":"2024-09-11T01:22:10.330851+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:10.330851+0000","last_undegraded":"2024-09-11T01:22:10.330851+0000","last_fullsized":"2024-09-11T01:22:10.330851+0000","mapping_epoch":44,"log_start":"45'9500","ondisk_log_start":"45'9500","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":868352,"num_objects":212,"num_object_clones":0,"num_object_copies":424,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":212,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17369,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20078","reported_seq":20220,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:13.011543+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:13.011543+0000","last_peered":"2024-09-11T01:22:13.011543+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:13.011543+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10078,"log_dups_size":0,"ondisk_log_size":10078,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":761856,"num_objects":186,"num_object_clones":0,"num_object_copies":372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":186,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16815,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3764}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007577+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007577+0000","last_peered":"2024-09-11T01:20:22.007577+0000","last_clean":"2024-09-11T01:20:22.007577+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007577+0000","last_undegraded":"2024-09-11T01:20:22.007577+0000","last_fullsized":"2024-09-11T01:20:22.007577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19709","reported_seq":27751,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:13.138383+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:13.138383+0000","last_peered":"2024-09-11T01:22:13.138383+0000","last_clean":"2024-09-11T01:22:13.138383+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:13.138383+0000","last_undegraded":"2024-09-11T01:22:13.138383+0000","last_fullsized":"2024-09-11T01:22:13.138383+0000","mapping_epoch":46,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":716800,"num_objects":175,"num_object_clones":0,"num_object_copies":350,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":175,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16364,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'19989","reported_seq":24173,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:13.119262+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:22:13.119262+0000","last_peered":"2024-09-11T01:22:13.119262+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:22:13.119262+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10089,"log_dups_size":0,"ondisk_log_size":10089,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":765952,"num_objects":187,"num_object_clones":0,"num_object_copies":374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2022,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":187,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16598,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2019,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1835}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007616+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007616+0000","last_peered":"2024-09-11T01:20:22.007616+0000","last_clean":"2024-09-11T01:20:22.007616+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007616+0000","last_undegraded":"2024-09-11T01:20:22.007616+0000","last_fullsized":"2024-09-11T01:20:22.007616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008206+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008206+0000","last_peered":"2024-09-11T01:20:22.008206+0000","last_clean":"2024-09-11T01:20:22.008206+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008206+0000","last_undegraded":"2024-09-11T01:20:22.008206+0000","last_fullsized":"2024-09-11T01:20:22.008206+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19777","reported_seq":19921,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:13.027392+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:13.027392+0000","last_peered":"2024-09-11T01:22:13.027392+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:13.027392+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10077,"log_dups_size":0,"ondisk_log_size":10077,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":717417,"num_objects":176,"num_object_clones":0,"num_object_copies":352,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":176,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16847,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3741}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":12878465,"num_objects":3146,"num_object_clones":0,"num_object_copies":6292,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33815,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3146,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":279141,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13824,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":220041216,"data_stored":219977408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160891,"ondisk_log_size":160891,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182528,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518784,"kb_used_data":72048,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93853056,"statfs":{"total":96636764160,"available":96105529344,"internally_reserved":0,"allocated":73777152,"data_stored":73496998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":457436843},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,4,8],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986300,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488360,"kb_used_data":136024,"kb_used_omap":4,"kb_used_meta":352315,"kb_avail":93883480,"statfs":{"total":96636764160,"available":96136683520,"internally_reserved":0,"allocated":139288576,"data_stored":139008427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":360771242},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,3,2,1,1],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659245,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":456756,"kb_used_data":10020,"kb_used_omap":4,"kb_used_meta":446715,"kb_avail":93915084,"statfs":{"total":96636764160,"available":96169046016,"internally_reserved":0,"allocated":10260480,"data_stored":9931027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":457436847},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,1,1,1,4,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":73170944,"data_stored":73167465,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8474624,"data_stored":8421846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":138395648,"data_stored":138388097,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:19.029 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout:Removed 160676 objects 2024-09-11T01:22:19.029 INFO:tasks.rep_lost_unfound_delete.radosbench.client.admin.smithi122.stdout:Clean up completed and total clean up time :354.571 2024-09-11T01:22:19.719 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:20.045 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:20.045 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:22:20.064 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":380,"stamp":"2024-09-11T01:22:18.801549+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3199636,"num_objects":785,"num_object_clones":0,"num_object_copies":1570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33708,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":785,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281506,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13932,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160757,"ondisk_log_size":160757,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1478684,"kb_used_data":198184,"kb_used_omap":12,"kb_used_meta":1280435,"kb_avail":281636836,"statfs":{"total":289910292480,"available":288396120064,"internally_reserved":0,"allocated":202940416,"data_stored":202050660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1311165444},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,4,1,9,7,7,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25919488,"num_objects":-6328,"num_object_clones":0,"num_object_copies":-12656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-300,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6328,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006320"},"pg_stats":[{"pgid":"2.f","version":"49'20242","reported_seq":20329,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.156313+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:22:18.156313+0000","last_peered":"2024-09-11T01:22:18.156313+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:22:18.156313+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":98304,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20242,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20256","reported_seq":20368,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:15.279877+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:15.279877+0000","last_peered":"2024-09-11T01:22:15.279877+0000","last_clean":"2024-09-11T01:22:15.279877+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:15.279877+0000","last_undegraded":"2024-09-11T01:22:15.279877+0000","last_fullsized":"2024-09-11T01:22:15.279877+0000","mapping_epoch":35,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":96,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18495,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20042","reported_seq":20181,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:15.232598+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:22:15.232598+0000","last_peered":"2024-09-11T01:22:15.232598+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:22:15.232598+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":413720,"num_objects":102,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18343,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3794}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'20044","reported_seq":20124,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:15.228090+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:15.228090+0000","last_peered":"2024-09-11T01:22:15.228090+0000","last_clean":"2024-09-11T01:22:15.228090+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:15.228090+0000","last_undegraded":"2024-09-11T01:22:15.228090+0000","last_fullsized":"2024-09-11T01:22:15.228090+0000","mapping_epoch":35,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":368640,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20044,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19975","reported_seq":20113,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.190146+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:18.190146+0000","last_peered":"2024-09-11T01:22:18.190146+0000","last_clean":"2024-09-11T01:22:18.190146+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:18.190146+0000","last_undegraded":"2024-09-11T01:22:18.190146+0000","last_fullsized":"2024-09-11T01:22:18.190146+0000","mapping_epoch":34,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":139264,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16980,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20200","reported_seq":20379,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.201766+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:18.201766+0000","last_peered":"2024-09-11T01:22:18.201766+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:18.201766+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110592,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16541,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3924}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19901","reported_seq":20038,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.184937+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:18.184937+0000","last_peered":"2024-09-11T01:22:18.184937+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:18.184937+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118784,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16758,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3957}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20226","reported_seq":20366,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.169719+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:18.169719+0000","last_peered":"2024-09-11T01:22:18.169719+0000","last_clean":"2024-09-11T01:22:18.169719+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:18.169719+0000","last_undegraded":"2024-09-11T01:22:18.169719+0000","last_fullsized":"2024-09-11T01:22:18.169719+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":114688,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17102,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003209+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003209+0000","last_peered":"2024-09-11T01:20:22.003209+0000","last_clean":"2024-09-11T01:20:22.003209+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003209+0000","last_undegraded":"2024-09-11T01:20:22.003209+0000","last_fullsized":"2024-09-11T01:20:22.003209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19947","reported_seq":27965,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.114019+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:18.114019+0000","last_peered":"2024-09-11T01:22:18.114019+0000","last_clean":"2024-09-11T01:22:18.114019+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:18.114019+0000","last_undegraded":"2024-09-11T01:22:18.114019+0000","last_fullsized":"2024-09-11T01:22:18.114019+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":106496,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17918,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19847","reported_seq":19986,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.151777+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:18.151777+0000","last_peered":"2024-09-11T01:22:18.151777+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:18.151777+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86016,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16434,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3963}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20094","reported_seq":20221,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:15.275965+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:15.275965+0000","last_peered":"2024-09-11T01:22:15.275965+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:15.275965+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":462848,"num_objects":113,"num_object_clones":0,"num_object_copies":226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17911,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3911}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003275+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003275+0000","last_peered":"2024-09-11T01:20:22.003275+0000","last_clean":"2024-09-11T01:20:22.003275+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003275+0000","last_undegraded":"2024-09-11T01:20:22.003275+0000","last_fullsized":"2024-09-11T01:20:22.003275+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19703","reported_seq":27695,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:15.270328+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:15.270328+0000","last_peered":"2024-09-11T01:22:15.270328+0000","last_clean":"2024-09-11T01:22:15.270328+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:15.270328+0000","last_undegraded":"2024-09-11T01:22:15.270328+0000","last_fullsized":"2024-09-11T01:22:15.270328+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":380928,"num_objects":93,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17488,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20235","reported_seq":20377,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.181738+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:18.181738+0000","last_peered":"2024-09-11T01:22:18.181738+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:18.181738+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118784,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16972,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3921}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007577+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007577+0000","last_peered":"2024-09-11T01:20:22.007577+0000","last_clean":"2024-09-11T01:20:22.007577+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007577+0000","last_undegraded":"2024-09-11T01:20:22.007577+0000","last_fullsized":"2024-09-11T01:20:22.007577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19863","reported_seq":27903,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.218551+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:18.218551+0000","last_peered":"2024-09-11T01:22:18.218551+0000","last_clean":"2024-09-11T01:22:18.218551+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:18.218551+0000","last_undegraded":"2024-09-11T01:22:18.218551+0000","last_fullsized":"2024-09-11T01:22:18.218551+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86016,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16518,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20152","reported_seq":24549,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.213808+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:22:18.213808+0000","last_peered":"2024-09-11T01:22:18.213808+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:22:18.213808+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":98304,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1915,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16761,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2126,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1891}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007616+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007616+0000","last_peered":"2024-09-11T01:20:22.007616+0000","last_clean":"2024-09-11T01:20:22.007616+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007616+0000","last_undegraded":"2024-09-11T01:20:22.007616+0000","last_fullsized":"2024-09-11T01:20:22.007616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008206+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008206+0000","last_peered":"2024-09-11T01:20:22.008206+0000","last_clean":"2024-09-11T01:20:22.008206+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008206+0000","last_undegraded":"2024-09-11T01:20:22.008206+0000","last_fullsized":"2024-09-11T01:20:22.008206+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19927","reported_seq":20071,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.172362+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:18.172362+0000","last_peered":"2024-09-11T01:22:18.172362+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:18.172362+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":103017,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16997,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3891}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":3199617,"num_objects":783,"num_object_clones":0,"num_object_copies":1566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33708,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":783,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281504,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13931,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":200912896,"data_stored":200849088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160754,"ondisk_log_size":160754,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182529,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515868,"kb_used_data":64968,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93855972,"statfs":{"total":96636764160,"available":96108515328,"internally_reserved":0,"allocated":66527232,"data_stored":66247078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":461696683},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,2,0,4,3,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986301,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487944,"kb_used_data":131448,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93883896,"statfs":{"total":96636764160,"available":96137109504,"internally_reserved":0,"allocated":134602752,"data_stored":134322603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659246,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":474872,"kb_used_data":1768,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93896968,"statfs":{"total":96636764160,"available":96150495232,"internally_reserved":0,"allocated":1810432,"data_stored":1480979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":484437679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,4,4,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65839104,"data_stored":65835625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1224704,"data_stored":1171926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":133849088,"data_stored":133841537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:20.066 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:20.438 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:20.439 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:22:20.456 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":380,"stamp":"2024-09-11T01:22:18.801549+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":3199636,"num_objects":785,"num_object_clones":0,"num_object_copies":1570,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33708,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":785,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281506,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13932,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160757,"ondisk_log_size":160757,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1478684,"kb_used_data":198184,"kb_used_omap":12,"kb_used_meta":1280435,"kb_avail":281636836,"statfs":{"total":289910292480,"available":288396120064,"internally_reserved":0,"allocated":202940416,"data_stored":202050660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1311165444},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,4,1,9,7,7,8],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-25919488,"num_objects":-6328,"num_object_clones":0,"num_object_copies":-12656,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-300,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-6328,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6328,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006320"},"pg_stats":[{"pgid":"2.f","version":"49'20242","reported_seq":20329,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.156313+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:22:18.156313+0000","last_peered":"2024-09-11T01:22:18.156313+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:22:18.156313+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":98304,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20242,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20256","reported_seq":20368,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:15.279877+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:15.279877+0000","last_peered":"2024-09-11T01:22:15.279877+0000","last_clean":"2024-09-11T01:22:15.279877+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:15.279877+0000","last_undegraded":"2024-09-11T01:22:15.279877+0000","last_fullsized":"2024-09-11T01:22:15.279877+0000","mapping_epoch":35,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10056,"log_dups_size":0,"ondisk_log_size":10056,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":96,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18495,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20042","reported_seq":20181,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:15.232598+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:22:15.232598+0000","last_peered":"2024-09-11T01:22:15.232598+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:22:15.232598+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":413720,"num_objects":102,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18343,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3794}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'20044","reported_seq":20124,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:15.228090+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:15.228090+0000","last_peered":"2024-09-11T01:22:15.228090+0000","last_clean":"2024-09-11T01:22:15.228090+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:15.228090+0000","last_undegraded":"2024-09-11T01:22:15.228090+0000","last_fullsized":"2024-09-11T01:22:15.228090+0000","mapping_epoch":35,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":368640,"num_objects":90,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20044,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19975","reported_seq":20113,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.190146+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:18.190146+0000","last_peered":"2024-09-11T01:22:18.190146+0000","last_clean":"2024-09-11T01:22:18.190146+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:18.190146+0000","last_undegraded":"2024-09-11T01:22:18.190146+0000","last_fullsized":"2024-09-11T01:22:18.190146+0000","mapping_epoch":34,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":139264,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16980,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20200","reported_seq":20379,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.201766+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:18.201766+0000","last_peered":"2024-09-11T01:22:18.201766+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:18.201766+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110592,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16541,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3924}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19901","reported_seq":20038,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.184937+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:18.184937+0000","last_peered":"2024-09-11T01:22:18.184937+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:18.184937+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118784,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16758,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3957}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20226","reported_seq":20366,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.169719+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:18.169719+0000","last_peered":"2024-09-11T01:22:18.169719+0000","last_clean":"2024-09-11T01:22:18.169719+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:18.169719+0000","last_undegraded":"2024-09-11T01:22:18.169719+0000","last_fullsized":"2024-09-11T01:22:18.169719+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":114688,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17102,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003209+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003209+0000","last_peered":"2024-09-11T01:20:22.003209+0000","last_clean":"2024-09-11T01:20:22.003209+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003209+0000","last_undegraded":"2024-09-11T01:20:22.003209+0000","last_fullsized":"2024-09-11T01:20:22.003209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19947","reported_seq":27965,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.114019+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:18.114019+0000","last_peered":"2024-09-11T01:22:18.114019+0000","last_clean":"2024-09-11T01:22:18.114019+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:18.114019+0000","last_undegraded":"2024-09-11T01:22:18.114019+0000","last_fullsized":"2024-09-11T01:22:18.114019+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":106496,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17918,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19847","reported_seq":19986,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.151777+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:18.151777+0000","last_peered":"2024-09-11T01:22:18.151777+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:18.151777+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86016,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16434,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3963}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20094","reported_seq":20221,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:15.275965+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:15.275965+0000","last_peered":"2024-09-11T01:22:15.275965+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:15.275965+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10094,"log_dups_size":0,"ondisk_log_size":10094,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":462848,"num_objects":113,"num_object_clones":0,"num_object_copies":226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":113,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17911,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3911}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003275+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003275+0000","last_peered":"2024-09-11T01:20:22.003275+0000","last_clean":"2024-09-11T01:20:22.003275+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003275+0000","last_undegraded":"2024-09-11T01:20:22.003275+0000","last_fullsized":"2024-09-11T01:20:22.003275+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19703","reported_seq":27695,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:15.270328+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:15.270328+0000","last_peered":"2024-09-11T01:22:15.270328+0000","last_clean":"2024-09-11T01:22:15.270328+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:15.270328+0000","last_undegraded":"2024-09-11T01:22:15.270328+0000","last_fullsized":"2024-09-11T01:22:15.270328+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10003,"log_dups_size":0,"ondisk_log_size":10003,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":380928,"num_objects":93,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":93,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17488,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20235","reported_seq":20377,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.181738+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:18.181738+0000","last_peered":"2024-09-11T01:22:18.181738+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:18.181738+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118784,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16972,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3921}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007577+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007577+0000","last_peered":"2024-09-11T01:20:22.007577+0000","last_clean":"2024-09-11T01:20:22.007577+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007577+0000","last_undegraded":"2024-09-11T01:20:22.007577+0000","last_fullsized":"2024-09-11T01:20:22.007577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19863","reported_seq":27903,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.218551+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:18.218551+0000","last_peered":"2024-09-11T01:22:18.218551+0000","last_clean":"2024-09-11T01:22:18.218551+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:18.218551+0000","last_undegraded":"2024-09-11T01:22:18.218551+0000","last_fullsized":"2024-09-11T01:22:18.218551+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86016,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16518,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20152","reported_seq":24549,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.213808+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:22:18.213808+0000","last_peered":"2024-09-11T01:22:18.213808+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:22:18.213808+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":98304,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1915,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16761,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2126,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1891}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007616+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007616+0000","last_peered":"2024-09-11T01:20:22.007616+0000","last_clean":"2024-09-11T01:20:22.007616+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007616+0000","last_undegraded":"2024-09-11T01:20:22.007616+0000","last_fullsized":"2024-09-11T01:20:22.007616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008206+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008206+0000","last_peered":"2024-09-11T01:20:22.008206+0000","last_clean":"2024-09-11T01:20:22.008206+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008206+0000","last_undegraded":"2024-09-11T01:20:22.008206+0000","last_fullsized":"2024-09-11T01:20:22.008206+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19927","reported_seq":20071,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.172362+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:18.172362+0000","last_peered":"2024-09-11T01:22:18.172362+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:18.172362+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":103017,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16997,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3891}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":3199617,"num_objects":783,"num_object_clones":0,"num_object_copies":1566,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33708,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":783,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281504,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13931,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":200912896,"data_stored":200849088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160754,"ondisk_log_size":160754,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182529,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515868,"kb_used_data":64968,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93855972,"statfs":{"total":96636764160,"available":96108515328,"internally_reserved":0,"allocated":66527232,"data_stored":66247078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":461696683},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,2,0,4,3,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986301,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487944,"kb_used_data":131448,"kb_used_omap":4,"kb_used_meta":356475,"kb_avail":93883896,"statfs":{"total":96636764160,"available":96137109504,"internally_reserved":0,"allocated":134602752,"data_stored":134322603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":365031082},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,2,1,1,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659246,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":474872,"kb_used_data":1768,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93896968,"statfs":{"total":96636764160,"available":96150495232,"internally_reserved":0,"allocated":1810432,"data_stored":1480979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":484437679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,4,4,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65839104,"data_stored":65835625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1224704,"data_stored":1171926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":133849088,"data_stored":133841537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:20.457 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:22:20.457 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20242", 'reported_seq': 20329, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:18.156313+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:22:18.156313+0000', 'last_peered': '2024-09-11T01:22:18.156313+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:22:18.156313+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 98304, 'num_objects': 24, 'num_object_clones': 0, 'num_object_copies': 48, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 24, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20242, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4048}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:20.457 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:22:20.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'20042", 'reported_seq': 20181, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:15.232598+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:22:15.232598+0000', 'last_peered': '2024-09-11T01:22:15.232598+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:22:15.232598+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10042, 'log_dups_size': 0, 'ondisk_log_size': 10042, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 413720, 'num_objects': 102, 'num_object_clones': 0, 'num_object_copies': 204, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 102, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18343, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3794}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:20.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:22:20.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'20200", 'reported_seq': 20379, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:18.201766+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:22:18.201766+0000', 'last_peered': '2024-09-11T01:22:18.201766+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:22:18.201766+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10100, 'log_dups_size': 0, 'ondisk_log_size': 10100, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 110592, 'num_objects': 27, 'num_object_clones': 0, 'num_object_copies': 54, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 27, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16541, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3924}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:20.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:22:20.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19901", 'reported_seq': 20038, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:18.184937+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:22:18.184937+0000', 'last_peered': '2024-09-11T01:22:18.184937+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:22:18.184937+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10001, 'log_dups_size': 0, 'ondisk_log_size': 10001, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 118784, 'num_objects': 29, 'num_object_clones': 0, 'num_object_copies': 58, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 29, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16758, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3957}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:20.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:22:20.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19847", 'reported_seq': 19986, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:18.151777+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:22:18.151777+0000', 'last_peered': '2024-09-11T01:22:18.151777+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:22:18.151777+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10047, 'log_dups_size': 0, 'ondisk_log_size': 10047, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 86016, 'num_objects': 21, 'num_object_clones': 0, 'num_object_copies': 42, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 21, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16434, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3963}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:20.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:22:20.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'20094", 'reported_seq': 20221, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:15.275965+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:22:15.275965+0000', 'last_peered': '2024-09-11T01:22:15.275965+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:22:15.275965+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "45'10000", 'ondisk_log_start': "45'10000", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10094, 'log_dups_size': 0, 'ondisk_log_size': 10094, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 462848, 'num_objects': 113, 'num_object_clones': 0, 'num_object_copies': 226, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 113, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17911, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3911}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:20.458 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:22:20.459 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20235", 'reported_seq': 20377, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:18.181738+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:22:18.181738+0000', 'last_peered': '2024-09-11T01:22:18.181738+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:22:18.181738+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10035, 'log_dups_size': 0, 'ondisk_log_size': 10035, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 118784, 'num_objects': 29, 'num_object_clones': 0, 'num_object_copies': 58, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 29, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16972, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3921}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:20.459 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:22:20.459 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'20152", 'reported_seq': 24549, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:18.213808+0000', 'last_change': '2024-09-11T01:20:20.985862+0000', 'last_active': '2024-09-11T01:22:18.213808+0000', 'last_peered': '2024-09-11T01:22:18.213808+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:22:18.213808+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 98304, 'num_objects': 24, 'num_object_clones': 0, 'num_object_copies': 48, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1915, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 24, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16761, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2126, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1891}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:20.459 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:22:20.459 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19927", 'reported_seq': 20071, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:18.172362+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:22:18.172362+0000', 'last_peered': '2024-09-11T01:22:18.172362+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:22:18.172362+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 103017, 'num_objects': 26, 'num_object_clones': 0, 'num_object_copies': 52, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 26, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16997, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3891}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:20.460 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:22:20.827 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:20.827 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33708/1570 objects degraded (2147.006%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":677,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":785,"data_bytes":3199636,"bytes_used":1514172416,"bytes_avail":288396120064,"bytes_total":289910292480,"degraded_objects":33708,"degraded_total":1570,"degraded_ratio":21.470063694267516,"recovering_objects_per_sec":24,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":300,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":527},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:22:20.845 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:22:20.845 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:21.143 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:21.161 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:22:21.162 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":381,"stamp":"2024-09-11T01:22:20.802072+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1180284,"num_objects":291,"num_object_clones":0,"num_object_copies":582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33708,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":291,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282000,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13932,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160751,"ondisk_log_size":160751,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1479152,"kb_used_data":194492,"kb_used_omap":12,"kb_used_meta":1284595,"kb_avail":281636368,"statfs":{"total":289910292480,"available":288395640832,"internally_reserved":0,"allocated":199159808,"data_stored":198274124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1315425284},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,2,0,8,7,7,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-21372952,"num_objects":-5219,"num_object_clones":0,"num_object_copies":-10438,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-194,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5219,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5219,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006402"},"pg_stats":[{"pgid":"2.f","version":"49'20242","reported_seq":20329,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.156313+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:22:18.156313+0000","last_peered":"2024-09-11T01:22:18.156313+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:22:18.156313+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":98304,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20242,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20464,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.964296+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:18.964296+0000","last_peered":"2024-09-11T01:22:18.964296+0000","last_clean":"2024-09-11T01:22:18.964296+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:18.964296+0000","last_undegraded":"2024-09-11T01:22:18.964296+0000","last_fullsized":"2024-09-11T01:22:18.964296+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20283,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.012455+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:22:19.012455+0000","last_peered":"2024-09-11T01:22:19.012455+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:22:19.012455+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3896}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20214,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.872730+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:18.872730+0000","last_peered":"2024-09-11T01:22:18.872730+0000","last_clean":"2024-09-11T01:22:18.872730+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:18.872730+0000","last_undegraded":"2024-09-11T01:22:18.872730+0000","last_fullsized":"2024-09-11T01:22:18.872730+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19975","reported_seq":20113,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.190146+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:18.190146+0000","last_peered":"2024-09-11T01:22:18.190146+0000","last_clean":"2024-09-11T01:22:18.190146+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:18.190146+0000","last_undegraded":"2024-09-11T01:22:18.190146+0000","last_fullsized":"2024-09-11T01:22:18.190146+0000","mapping_epoch":34,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":139264,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16980,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20200","reported_seq":20379,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.201766+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:18.201766+0000","last_peered":"2024-09-11T01:22:18.201766+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:18.201766+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110592,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16541,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3924}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19901","reported_seq":20038,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.184937+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:18.184937+0000","last_peered":"2024-09-11T01:22:18.184937+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:18.184937+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118784,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16758,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3957}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20226","reported_seq":20366,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.169719+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:18.169719+0000","last_peered":"2024-09-11T01:22:18.169719+0000","last_clean":"2024-09-11T01:22:18.169719+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:18.169719+0000","last_undegraded":"2024-09-11T01:22:18.169719+0000","last_fullsized":"2024-09-11T01:22:18.169719+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":114688,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17102,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003209+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003209+0000","last_peered":"2024-09-11T01:20:22.003209+0000","last_clean":"2024-09-11T01:20:22.003209+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003209+0000","last_undegraded":"2024-09-11T01:20:22.003209+0000","last_fullsized":"2024-09-11T01:20:22.003209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19947","reported_seq":27965,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.114019+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:18.114019+0000","last_peered":"2024-09-11T01:22:18.114019+0000","last_clean":"2024-09-11T01:22:18.114019+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:18.114019+0000","last_undegraded":"2024-09-11T01:22:18.114019+0000","last_fullsized":"2024-09-11T01:22:18.114019+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":106496,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17918,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19847","reported_seq":19986,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.151777+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:18.151777+0000","last_peered":"2024-09-11T01:22:18.151777+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:18.151777+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86016,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16434,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3963}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20336,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.997841+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:18.997841+0000","last_peered":"2024-09-11T01:22:18.997841+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:18.997841+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003275+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003275+0000","last_peered":"2024-09-11T01:20:22.003275+0000","last_clean":"2024-09-11T01:20:22.003275+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003275+0000","last_undegraded":"2024-09-11T01:20:22.003275+0000","last_fullsized":"2024-09-11T01:20:22.003275+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27788,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.973481+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:18.973481+0000","last_peered":"2024-09-11T01:22:18.973481+0000","last_clean":"2024-09-11T01:22:18.973481+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:18.973481+0000","last_undegraded":"2024-09-11T01:22:18.973481+0000","last_fullsized":"2024-09-11T01:22:18.973481+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20235","reported_seq":20377,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.181738+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:18.181738+0000","last_peered":"2024-09-11T01:22:18.181738+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:18.181738+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118784,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16972,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3921}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007577+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007577+0000","last_peered":"2024-09-11T01:20:22.007577+0000","last_clean":"2024-09-11T01:20:22.007577+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007577+0000","last_undegraded":"2024-09-11T01:20:22.007577+0000","last_fullsized":"2024-09-11T01:20:22.007577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19863","reported_seq":27903,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.218551+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:18.218551+0000","last_peered":"2024-09-11T01:22:18.218551+0000","last_clean":"2024-09-11T01:22:18.218551+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:18.218551+0000","last_undegraded":"2024-09-11T01:22:18.218551+0000","last_fullsized":"2024-09-11T01:22:18.218551+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86016,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16518,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20152","reported_seq":24549,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.213808+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:22:18.213808+0000","last_peered":"2024-09-11T01:22:18.213808+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:22:18.213808+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":98304,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1915,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16761,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2126,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1891}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007616+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007616+0000","last_peered":"2024-09-11T01:20:22.007616+0000","last_clean":"2024-09-11T01:20:22.007616+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007616+0000","last_undegraded":"2024-09-11T01:20:22.007616+0000","last_fullsized":"2024-09-11T01:20:22.007616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008206+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008206+0000","last_peered":"2024-09-11T01:20:22.008206+0000","last_clean":"2024-09-11T01:20:22.008206+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008206+0000","last_undegraded":"2024-09-11T01:20:22.008206+0000","last_fullsized":"2024-09-11T01:20:22.008206+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19927","reported_seq":20071,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.172362+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:18.172362+0000","last_peered":"2024-09-11T01:22:18.172362+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:18.172362+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":103017,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16997,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3891}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1180265,"num_objects":289,"num_object_clones":0,"num_object_copies":578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33708,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":289,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281998,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13931,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":197476352,"data_stored":197416616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160748,"ondisk_log_size":160748,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182529,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515868,"kb_used_data":64968,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93855972,"statfs":{"total":96636764160,"available":96108515328,"internally_reserved":0,"allocated":66527232,"data_stored":66247078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":461696683},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,2,0,4,3,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986302,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488412,"kb_used_data":127756,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883428,"statfs":{"total":96636764160,"available":96136630272,"internally_reserved":0,"allocated":130822144,"data_stored":130546067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659246,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":474872,"kb_used_data":1768,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93896968,"statfs":{"total":96636764160,"available":96150495232,"internally_reserved":0,"allocated":1810432,"data_stored":1480979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":484437679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,4,4,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65839104,"data_stored":65835625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1224704,"data_stored":1171926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:24.164 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:24.499 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:24.499 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:22:24.517 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":382,"stamp":"2024-09-11T01:22:22.802761+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1180284,"num_objects":291,"num_object_clones":0,"num_object_copies":582,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33708,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":291,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282000,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13932,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160751,"ondisk_log_size":160751,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1479152,"kb_used_data":194492,"kb_used_omap":12,"kb_used_meta":1284595,"kb_avail":281636368,"statfs":{"total":289910292480,"available":288395640832,"internally_reserved":0,"allocated":199159808,"data_stored":198274124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1315425284},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,4,2,0,8,7,7,7],"upper_bound":256},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-18382872,"num_objects":-4489,"num_object_clones":0,"num_object_copies":-8978,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-194,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4489,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4489,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":194,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006368"},"pg_stats":[{"pgid":"2.f","version":"49'20242","reported_seq":20329,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.156313+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:22:18.156313+0000","last_peered":"2024-09-11T01:22:18.156313+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:22:18.156313+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10042,"log_dups_size":0,"ondisk_log_size":10042,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":98304,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20242,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4048}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20464,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.964296+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:18.964296+0000","last_peered":"2024-09-11T01:22:18.964296+0000","last_clean":"2024-09-11T01:22:18.964296+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:18.964296+0000","last_undegraded":"2024-09-11T01:22:18.964296+0000","last_fullsized":"2024-09-11T01:22:18.964296+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20283,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.012455+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:22:19.012455+0000","last_peered":"2024-09-11T01:22:19.012455+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:22:19.012455+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3896}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20214,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.872730+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:18.872730+0000","last_peered":"2024-09-11T01:22:18.872730+0000","last_clean":"2024-09-11T01:22:18.872730+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:18.872730+0000","last_undegraded":"2024-09-11T01:22:18.872730+0000","last_fullsized":"2024-09-11T01:22:18.872730+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'19975","reported_seq":20113,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.190146+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:18.190146+0000","last_peered":"2024-09-11T01:22:18.190146+0000","last_clean":"2024-09-11T01:22:18.190146+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:18.190146+0000","last_undegraded":"2024-09-11T01:22:18.190146+0000","last_fullsized":"2024-09-11T01:22:18.190146+0000","mapping_epoch":34,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10075,"log_dups_size":0,"ondisk_log_size":10075,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":139264,"num_objects":34,"num_object_clones":0,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16980,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20200","reported_seq":20379,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.201766+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:18.201766+0000","last_peered":"2024-09-11T01:22:18.201766+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:18.201766+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10100,"log_dups_size":0,"ondisk_log_size":10100,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110592,"num_objects":27,"num_object_clones":0,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16541,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3924}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19901","reported_seq":20038,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.184937+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:18.184937+0000","last_peered":"2024-09-11T01:22:18.184937+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:18.184937+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10001,"log_dups_size":0,"ondisk_log_size":10001,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118784,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16758,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3957}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20226","reported_seq":20366,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.169719+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:18.169719+0000","last_peered":"2024-09-11T01:22:18.169719+0000","last_clean":"2024-09-11T01:22:18.169719+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:18.169719+0000","last_undegraded":"2024-09-11T01:22:18.169719+0000","last_fullsized":"2024-09-11T01:22:18.169719+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10026,"log_dups_size":0,"ondisk_log_size":10026,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":114688,"num_objects":28,"num_object_clones":0,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17102,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003209+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003209+0000","last_peered":"2024-09-11T01:20:22.003209+0000","last_clean":"2024-09-11T01:20:22.003209+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003209+0000","last_undegraded":"2024-09-11T01:20:22.003209+0000","last_fullsized":"2024-09-11T01:20:22.003209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19947","reported_seq":27965,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.114019+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:18.114019+0000","last_peered":"2024-09-11T01:22:18.114019+0000","last_clean":"2024-09-11T01:22:18.114019+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:18.114019+0000","last_undegraded":"2024-09-11T01:22:18.114019+0000","last_fullsized":"2024-09-11T01:22:18.114019+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":106496,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17918,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19847","reported_seq":19986,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.151777+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:18.151777+0000","last_peered":"2024-09-11T01:22:18.151777+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:18.151777+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10047,"log_dups_size":0,"ondisk_log_size":10047,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86016,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16434,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3963}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":111,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005199+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005199+0000","last_peered":"2024-09-11T01:20:22.005199+0000","last_clean":"2024-09-11T01:20:22.005199+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005199+0000","last_undegraded":"2024-09-11T01:20:22.005199+0000","last_fullsized":"2024-09-11T01:20:22.005199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20336,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.997841+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:18.997841+0000","last_peered":"2024-09-11T01:22:18.997841+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:18.997841+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003275+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003275+0000","last_peered":"2024-09-11T01:20:22.003275+0000","last_clean":"2024-09-11T01:20:22.003275+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003275+0000","last_undegraded":"2024-09-11T01:20:22.003275+0000","last_fullsized":"2024-09-11T01:20:22.003275+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27788,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.973481+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:18.973481+0000","last_peered":"2024-09-11T01:22:18.973481+0000","last_clean":"2024-09-11T01:22:18.973481+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:18.973481+0000","last_undegraded":"2024-09-11T01:22:18.973481+0000","last_fullsized":"2024-09-11T01:22:18.973481+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20235","reported_seq":20377,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.181738+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:18.181738+0000","last_peered":"2024-09-11T01:22:18.181738+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:18.181738+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10035,"log_dups_size":0,"ondisk_log_size":10035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":118784,"num_objects":29,"num_object_clones":0,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16972,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3921}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007577+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007577+0000","last_peered":"2024-09-11T01:20:22.007577+0000","last_clean":"2024-09-11T01:20:22.007577+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007577+0000","last_undegraded":"2024-09-11T01:20:22.007577+0000","last_fullsized":"2024-09-11T01:20:22.007577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19863","reported_seq":27903,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.218551+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:18.218551+0000","last_peered":"2024-09-11T01:22:18.218551+0000","last_clean":"2024-09-11T01:22:18.218551+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:18.218551+0000","last_undegraded":"2024-09-11T01:22:18.218551+0000","last_fullsized":"2024-09-11T01:22:18.218551+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10063,"log_dups_size":0,"ondisk_log_size":10063,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":86016,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16518,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20152","reported_seq":24549,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.213808+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:22:18.213808+0000","last_peered":"2024-09-11T01:22:18.213808+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:22:18.213808+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":98304,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1915,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16761,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2126,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1891}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007616+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007616+0000","last_peered":"2024-09-11T01:20:22.007616+0000","last_clean":"2024-09-11T01:20:22.007616+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007616+0000","last_undegraded":"2024-09-11T01:20:22.007616+0000","last_fullsized":"2024-09-11T01:20:22.007616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008206+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008206+0000","last_peered":"2024-09-11T01:20:22.008206+0000","last_clean":"2024-09-11T01:20:22.008206+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008206+0000","last_undegraded":"2024-09-11T01:20:22.008206+0000","last_fullsized":"2024-09-11T01:20:22.008206+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19927","reported_seq":20071,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.172362+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:18.172362+0000","last_peered":"2024-09-11T01:22:18.172362+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:18.172362+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":103017,"num_objects":26,"num_object_clones":0,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16997,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3891}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":1180265,"num_objects":289,"num_object_clones":0,"num_object_copies":578,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33708,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":289,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":281998,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13931,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":197476352,"data_stored":197416616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160748,"ondisk_log_size":160748,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182529,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515868,"kb_used_data":64968,"kb_used_omap":4,"kb_used_meta":450875,"kb_avail":93855972,"statfs":{"total":96636764160,"available":96108515328,"internally_reserved":0,"allocated":66527232,"data_stored":66247078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":461696683},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,3,2,0,4,3,5,3],"upper_bound":256},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986302,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488412,"kb_used_data":127756,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883428,"statfs":{"total":96636764160,"available":96136630272,"internally_reserved":0,"allocated":130822144,"data_stored":130546067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659246,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":474872,"kb_used_data":1768,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93896968,"statfs":{"total":96636764160,"available":96150495232,"internally_reserved":0,"allocated":1810432,"data_stored":1480979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":484437679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1,0,0,4,4,2,4],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65839104,"data_stored":65835625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":1224704,"data_stored":1171926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:24.518 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:24.944 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:24.944 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:22:24.962 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":383,"stamp":"2024-09-11T01:22:24.803444+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14357,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1468064,"kb_used_data":191728,"kb_used_omap":12,"kb_used_meta":1276275,"kb_avail":281647456,"statfs":{"total":289910292480,"available":288406994944,"internally_reserved":0,"allocated":196329472,"data_stored":195447860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1306905604},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19562520,"num_objects":-4777,"num_object_clones":0,"num_object_copies":-9554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-619,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4777,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":619,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006613"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":20354,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.943277+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:22:18.943277+0000","last_peered":"2024-09-11T01:22:18.943277+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:22:18.943277+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20464,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.964296+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:18.964296+0000","last_peered":"2024-09-11T01:22:18.964296+0000","last_clean":"2024-09-11T01:22:18.964296+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:18.964296+0000","last_undegraded":"2024-09-11T01:22:18.964296+0000","last_fullsized":"2024-09-11T01:22:18.964296+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20283,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.012455+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:22:19.012455+0000","last_peered":"2024-09-11T01:22:19.012455+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:22:19.012455+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3896}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20214,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.872730+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:18.872730+0000","last_peered":"2024-09-11T01:22:18.872730+0000","last_clean":"2024-09-11T01:22:18.872730+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:18.872730+0000","last_undegraded":"2024-09-11T01:22:18.872730+0000","last_fullsized":"2024-09-11T01:22:18.872730+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20147,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:19.019037+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:19.019037+0000","last_peered":"2024-09-11T01:22:19.019037+0000","last_clean":"2024-09-11T01:22:19.019037+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:19.019037+0000","last_undegraded":"2024-09-11T01:22:19.019037+0000","last_fullsized":"2024-09-11T01:22:19.019037+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20408,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.034150+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:19.034150+0000","last_peered":"2024-09-11T01:22:19.034150+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:19.034150+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20071,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.026055+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:19.026055+0000","last_peered":"2024-09-11T01:22:19.026055+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:19.026055+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20394,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.910833+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:18.910833+0000","last_peered":"2024-09-11T01:22:18.910833+0000","last_clean":"2024-09-11T01:22:18.910833+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:18.910833+0000","last_undegraded":"2024-09-11T01:22:18.910833+0000","last_fullsized":"2024-09-11T01:22:18.910833+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003209+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003209+0000","last_peered":"2024-09-11T01:20:22.003209+0000","last_clean":"2024-09-11T01:20:22.003209+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003209+0000","last_undegraded":"2024-09-11T01:20:22.003209+0000","last_fullsized":"2024-09-11T01:20:22.003209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":27991,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.993850+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:18.993850+0000","last_peered":"2024-09-11T01:22:18.993850+0000","last_clean":"2024-09-11T01:22:18.993850+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:18.993850+0000","last_undegraded":"2024-09-11T01:22:18.993850+0000","last_fullsized":"2024-09-11T01:22:18.993850+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20007,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.022841+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:19.022841+0000","last_peered":"2024-09-11T01:22:19.022841+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:19.022841+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005254+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005254+0000","last_peered":"2024-09-11T01:20:22.005254+0000","last_clean":"2024-09-11T01:20:22.005254+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005254+0000","last_undegraded":"2024-09-11T01:20:22.005254+0000","last_fullsized":"2024-09-11T01:20:22.005254+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20336,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.997841+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:18.997841+0000","last_peered":"2024-09-11T01:22:18.997841+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:18.997841+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003275+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003275+0000","last_peered":"2024-09-11T01:20:22.003275+0000","last_clean":"2024-09-11T01:20:22.003275+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003275+0000","last_undegraded":"2024-09-11T01:20:22.003275+0000","last_fullsized":"2024-09-11T01:20:22.003275+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27788,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.973481+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:18.973481+0000","last_peered":"2024-09-11T01:22:18.973481+0000","last_clean":"2024-09-11T01:22:18.973481+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:18.973481+0000","last_undegraded":"2024-09-11T01:22:18.973481+0000","last_fullsized":"2024-09-11T01:22:18.973481+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20406,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.987858+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:18.987858+0000","last_peered":"2024-09-11T01:22:18.987858+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:18.987858+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007577+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007577+0000","last_peered":"2024-09-11T01:20:22.007577+0000","last_clean":"2024-09-11T01:20:22.007577+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007577+0000","last_undegraded":"2024-09-11T01:20:22.007577+0000","last_fullsized":"2024-09-11T01:20:22.007577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27926,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.950589+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:18.950589+0000","last_peered":"2024-09-11T01:22:18.950589+0000","last_clean":"2024-09-11T01:22:18.950589+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:18.950589+0000","last_undegraded":"2024-09-11T01:22:18.950589+0000","last_fullsized":"2024-09-11T01:22:18.950589+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":25424,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:23.220193+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:22:23.220193+0000","last_peered":"2024-09-11T01:22:23.220193+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:22:23.220193+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1490,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2551,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1490}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007616+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007616+0000","last_peered":"2024-09-11T01:20:22.007616+0000","last_clean":"2024-09-11T01:20:22.007616+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007616+0000","last_undegraded":"2024-09-11T01:20:22.007616+0000","last_fullsized":"2024-09-11T01:20:22.007616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008206+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008206+0000","last_peered":"2024-09-11T01:20:22.008206+0000","last_clean":"2024-09-11T01:20:22.008206+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008206+0000","last_undegraded":"2024-09-11T01:20:22.008206+0000","last_fullsized":"2024-09-11T01:20:22.008206+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20096,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.016874+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:19.016874+0000","last_peered":"2024-09-11T01:22:19.016874+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:19.016874+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14356,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195092480,"data_stored":195036816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182530,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518568,"kb_used_data":63512,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853272,"statfs":{"total":96636764160,"available":96105750528,"internally_reserved":0,"allocated":65036288,"data_stored":64756134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986302,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488412,"kb_used_data":127756,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883428,"statfs":{"total":96636764160,"available":96136630272,"internally_reserved":0,"allocated":130822144,"data_stored":130546067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659247,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461084,"kb_used_data":460,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93910756,"statfs":{"total":96636764160,"available":96164614144,"internally_reserved":0,"allocated":471040,"data_stored":145659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":471658159},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:24.963 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:22:24.963 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20266", 'reported_seq': 20354, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:18.943277+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:22:18.943277+0000', 'last_peered': '2024-09-11T01:22:18.943277+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:22:18.943277+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20266, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4072}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:24.963 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:22:24.963 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'20144", 'reported_seq': 20283, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:19.012455+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:22:19.012455+0000', 'last_peered': '2024-09-11T01:22:19.012455+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:22:19.012455+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18445, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3896}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:24.963 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:22:24.964 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'20227", 'reported_seq': 20408, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:19.034150+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:22:19.034150+0000', 'last_peered': '2024-09-11T01:22:19.034150+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:22:19.034150+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16568, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:24.964 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:22:24.964 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20071, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:19.026055+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:22:19.026055+0000', 'last_peered': '2024-09-11T01:22:19.026055+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:22:19.026055+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:24.964 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:22:24.964 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20007, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:19.022841+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:22:19.022841+0000', 'last_peered': '2024-09-11T01:22:19.022841+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:22:19.022841+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:24.964 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:22:24.964 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'20207", 'reported_seq': 20336, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:18.997841+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:22:18.997841+0000', 'last_peered': '2024-09-11T01:22:18.997841+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:22:18.997841+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18024, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4024}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:24.964 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:22:24.964 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20264", 'reported_seq': 20406, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:18.987858+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:22:18.987858+0000', 'last_peered': '2024-09-11T01:22:18.987858+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:22:18.987858+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17001, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:24.964 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:22:24.964 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'20176", 'reported_seq': 25424, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:23.220193+0000', 'last_change': '2024-09-11T01:20:20.985862+0000', 'last_active': '2024-09-11T01:22:23.220193+0000', 'last_peered': '2024-09-11T01:22:23.220193+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:22:23.220193+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1490, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16785, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2551, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1490}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:24.965 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:22:24.965 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 20096, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:19.016874+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:22:19.016874+0000', 'last_peered': '2024-09-11T01:22:19.016874+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:22:19.016874+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:24.965 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:22:25.415 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:25.415 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 33283/6 objects degraded (554716.667%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":681,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1503297536,"bytes_avail":288406994944,"bytes_total":289910292480,"degraded_objects":33283,"degraded_total":6,"degraded_ratio":5547.1666666666679,"recovering_objects_per_sec":51,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":619,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":397},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:22:25.434 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:22:25.435 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:25.766 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:25.766 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:22:25.784 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":383,"stamp":"2024-09-11T01:22:24.803444+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14357,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1468064,"kb_used_data":191728,"kb_used_omap":12,"kb_used_meta":1276275,"kb_avail":281647456,"statfs":{"total":289910292480,"available":288406994944,"internally_reserved":0,"allocated":196329472,"data_stored":195447860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1306905604},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19562520,"num_objects":-4777,"num_object_clones":0,"num_object_copies":-9554,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-619,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-4777,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4777,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":619,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.006613"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":20354,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.943277+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:22:18.943277+0000","last_peered":"2024-09-11T01:22:18.943277+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:22:18.943277+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20464,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.964296+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:18.964296+0000","last_peered":"2024-09-11T01:22:18.964296+0000","last_clean":"2024-09-11T01:22:18.964296+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:18.964296+0000","last_undegraded":"2024-09-11T01:22:18.964296+0000","last_fullsized":"2024-09-11T01:22:18.964296+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20283,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.012455+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:22:19.012455+0000","last_peered":"2024-09-11T01:22:19.012455+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:22:19.012455+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3896}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20214,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.872730+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:18.872730+0000","last_peered":"2024-09-11T01:22:18.872730+0000","last_clean":"2024-09-11T01:22:18.872730+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:18.872730+0000","last_undegraded":"2024-09-11T01:22:18.872730+0000","last_fullsized":"2024-09-11T01:22:18.872730+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20147,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:19.019037+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:19.019037+0000","last_peered":"2024-09-11T01:22:19.019037+0000","last_clean":"2024-09-11T01:22:19.019037+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:19.019037+0000","last_undegraded":"2024-09-11T01:22:19.019037+0000","last_fullsized":"2024-09-11T01:22:19.019037+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20408,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.034150+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:19.034150+0000","last_peered":"2024-09-11T01:22:19.034150+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:19.034150+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20071,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.026055+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:19.026055+0000","last_peered":"2024-09-11T01:22:19.026055+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:19.026055+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20394,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.910833+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:18.910833+0000","last_peered":"2024-09-11T01:22:18.910833+0000","last_clean":"2024-09-11T01:22:18.910833+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:18.910833+0000","last_undegraded":"2024-09-11T01:22:18.910833+0000","last_fullsized":"2024-09-11T01:22:18.910833+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003209+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003209+0000","last_peered":"2024-09-11T01:20:22.003209+0000","last_clean":"2024-09-11T01:20:22.003209+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003209+0000","last_undegraded":"2024-09-11T01:20:22.003209+0000","last_fullsized":"2024-09-11T01:20:22.003209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":27991,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.993850+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:18.993850+0000","last_peered":"2024-09-11T01:22:18.993850+0000","last_clean":"2024-09-11T01:22:18.993850+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:18.993850+0000","last_undegraded":"2024-09-11T01:22:18.993850+0000","last_fullsized":"2024-09-11T01:22:18.993850+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20007,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.022841+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:19.022841+0000","last_peered":"2024-09-11T01:22:19.022841+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:19.022841+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005254+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005254+0000","last_peered":"2024-09-11T01:20:22.005254+0000","last_clean":"2024-09-11T01:20:22.005254+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005254+0000","last_undegraded":"2024-09-11T01:20:22.005254+0000","last_fullsized":"2024-09-11T01:20:22.005254+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20336,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.997841+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:18.997841+0000","last_peered":"2024-09-11T01:22:18.997841+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:18.997841+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003275+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003275+0000","last_peered":"2024-09-11T01:20:22.003275+0000","last_clean":"2024-09-11T01:20:22.003275+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003275+0000","last_undegraded":"2024-09-11T01:20:22.003275+0000","last_fullsized":"2024-09-11T01:20:22.003275+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27788,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.973481+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:18.973481+0000","last_peered":"2024-09-11T01:22:18.973481+0000","last_clean":"2024-09-11T01:22:18.973481+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:18.973481+0000","last_undegraded":"2024-09-11T01:22:18.973481+0000","last_fullsized":"2024-09-11T01:22:18.973481+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20406,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.987858+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:18.987858+0000","last_peered":"2024-09-11T01:22:18.987858+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:18.987858+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007577+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007577+0000","last_peered":"2024-09-11T01:20:22.007577+0000","last_clean":"2024-09-11T01:20:22.007577+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007577+0000","last_undegraded":"2024-09-11T01:20:22.007577+0000","last_fullsized":"2024-09-11T01:20:22.007577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27926,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.950589+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:18.950589+0000","last_peered":"2024-09-11T01:22:18.950589+0000","last_clean":"2024-09-11T01:22:18.950589+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:18.950589+0000","last_undegraded":"2024-09-11T01:22:18.950589+0000","last_fullsized":"2024-09-11T01:22:18.950589+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":25424,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:23.220193+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:22:23.220193+0000","last_peered":"2024-09-11T01:22:23.220193+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:22:23.220193+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1490,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2551,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1490}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007616+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007616+0000","last_peered":"2024-09-11T01:20:22.007616+0000","last_clean":"2024-09-11T01:20:22.007616+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007616+0000","last_undegraded":"2024-09-11T01:20:22.007616+0000","last_fullsized":"2024-09-11T01:20:22.007616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008206+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008206+0000","last_peered":"2024-09-11T01:20:22.008206+0000","last_clean":"2024-09-11T01:20:22.008206+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008206+0000","last_undegraded":"2024-09-11T01:20:22.008206+0000","last_fullsized":"2024-09-11T01:20:22.008206+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20096,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.016874+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:19.016874+0000","last_peered":"2024-09-11T01:22:19.016874+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:19.016874+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33283,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14356,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195092480,"data_stored":195036816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182530,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518568,"kb_used_data":63512,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853272,"statfs":{"total":96636764160,"available":96105750528,"internally_reserved":0,"allocated":65036288,"data_stored":64756134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986302,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488412,"kb_used_data":127756,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883428,"statfs":{"total":96636764160,"available":96136630272,"internally_reserved":0,"allocated":130822144,"data_stored":130546067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659247,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461084,"kb_used_data":460,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93910756,"statfs":{"total":96636764160,"available":96164614144,"internally_reserved":0,"allocated":471040,"data_stored":145659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":471658159},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:28.786 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:29.128 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:29.128 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:22:29.146 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":385,"stamp":"2024-09-11T01:22:28.804648+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32796,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14844,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472224,"kb_used_data":191728,"kb_used_omap":12,"kb_used_meta":1280435,"kb_avail":281643296,"statfs":{"total":289910292480,"available":288402735104,"internally_reserved":0,"allocated":196329472,"data_stored":195447860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1311165444},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-9969688,"num_objects":-2435,"num_object_clones":0,"num_object_copies":-4870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1019,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2435,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1019,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003530"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":20354,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.943277+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:22:18.943277+0000","last_peered":"2024-09-11T01:22:18.943277+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:22:18.943277+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20464,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.964296+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:18.964296+0000","last_peered":"2024-09-11T01:22:18.964296+0000","last_clean":"2024-09-11T01:22:18.964296+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:18.964296+0000","last_undegraded":"2024-09-11T01:22:18.964296+0000","last_fullsized":"2024-09-11T01:22:18.964296+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20283,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.012455+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:22:19.012455+0000","last_peered":"2024-09-11T01:22:19.012455+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:22:19.012455+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3896}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20214,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.872730+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:18.872730+0000","last_peered":"2024-09-11T01:22:18.872730+0000","last_clean":"2024-09-11T01:22:18.872730+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:18.872730+0000","last_undegraded":"2024-09-11T01:22:18.872730+0000","last_fullsized":"2024-09-11T01:22:18.872730+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20147,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:19.019037+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:19.019037+0000","last_peered":"2024-09-11T01:22:19.019037+0000","last_clean":"2024-09-11T01:22:19.019037+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:19.019037+0000","last_undegraded":"2024-09-11T01:22:19.019037+0000","last_fullsized":"2024-09-11T01:22:19.019037+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20408,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.034150+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:19.034150+0000","last_peered":"2024-09-11T01:22:19.034150+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:19.034150+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20071,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.026055+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:19.026055+0000","last_peered":"2024-09-11T01:22:19.026055+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:19.026055+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20394,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.910833+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:18.910833+0000","last_peered":"2024-09-11T01:22:18.910833+0000","last_clean":"2024-09-11T01:22:18.910833+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:18.910833+0000","last_undegraded":"2024-09-11T01:22:18.910833+0000","last_fullsized":"2024-09-11T01:22:18.910833+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003209+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003209+0000","last_peered":"2024-09-11T01:20:22.003209+0000","last_clean":"2024-09-11T01:20:22.003209+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003209+0000","last_undegraded":"2024-09-11T01:20:22.003209+0000","last_fullsized":"2024-09-11T01:20:22.003209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":27992,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.993959+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:18.993959+0000","last_peered":"2024-09-11T01:22:18.993959+0000","last_clean":"2024-09-11T01:22:18.993959+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:18.993959+0000","last_undegraded":"2024-09-11T01:22:18.993959+0000","last_fullsized":"2024-09-11T01:22:18.993959+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20007,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.022841+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:19.022841+0000","last_peered":"2024-09-11T01:22:19.022841+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:19.022841+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005254+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005254+0000","last_peered":"2024-09-11T01:20:22.005254+0000","last_clean":"2024-09-11T01:20:22.005254+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005254+0000","last_undegraded":"2024-09-11T01:20:22.005254+0000","last_fullsized":"2024-09-11T01:20:22.005254+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20336,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.997841+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:18.997841+0000","last_peered":"2024-09-11T01:22:18.997841+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:18.997841+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003275+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003275+0000","last_peered":"2024-09-11T01:20:22.003275+0000","last_clean":"2024-09-11T01:20:22.003275+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003275+0000","last_undegraded":"2024-09-11T01:20:22.003275+0000","last_fullsized":"2024-09-11T01:20:22.003275+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27788,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.973481+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:18.973481+0000","last_peered":"2024-09-11T01:22:18.973481+0000","last_clean":"2024-09-11T01:22:18.973481+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:18.973481+0000","last_undegraded":"2024-09-11T01:22:18.973481+0000","last_fullsized":"2024-09-11T01:22:18.973481+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20406,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.987858+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:18.987858+0000","last_peered":"2024-09-11T01:22:18.987858+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:18.987858+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007577+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007577+0000","last_peered":"2024-09-11T01:20:22.007577+0000","last_clean":"2024-09-11T01:20:22.007577+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007577+0000","last_undegraded":"2024-09-11T01:20:22.007577+0000","last_fullsized":"2024-09-11T01:20:22.007577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27926,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.950589+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:18.950589+0000","last_peered":"2024-09-11T01:22:18.950589+0000","last_clean":"2024-09-11T01:22:18.950589+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:18.950589+0000","last_undegraded":"2024-09-11T01:22:18.950589+0000","last_fullsized":"2024-09-11T01:22:18.950589+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":26398,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:28.218587+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:22:28.218587+0000","last_peered":"2024-09-11T01:22:28.218587+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:22:28.218587+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1003,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3038,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007616+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007616+0000","last_peered":"2024-09-11T01:20:22.007616+0000","last_clean":"2024-09-11T01:20:22.007616+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007616+0000","last_undegraded":"2024-09-11T01:20:22.007616+0000","last_fullsized":"2024-09-11T01:20:22.007616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008206+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008206+0000","last_peered":"2024-09-11T01:20:22.008206+0000","last_clean":"2024-09-11T01:20:22.008206+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008206+0000","last_undegraded":"2024-09-11T01:20:22.008206+0000","last_fullsized":"2024-09-11T01:20:22.008206+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20096,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.016874+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:19.016874+0000","last_peered":"2024-09-11T01:22:19.016874+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:19.016874+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32796,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14843,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195092480,"data_stored":195036816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182531,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518568,"kb_used_data":63512,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853272,"statfs":{"total":96636764160,"available":96105750528,"internally_reserved":0,"allocated":65036288,"data_stored":64756134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986303,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488412,"kb_used_data":127756,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883428,"statfs":{"total":96636764160,"available":96136630272,"internally_reserved":0,"allocated":130822144,"data_stored":130546067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659248,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465244,"kb_used_data":460,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93906596,"statfs":{"total":96636764160,"available":96160354304,"internally_reserved":0,"allocated":471040,"data_stored":145659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475917999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:29.148 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:29.467 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:29.467 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:22:29.486 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":385,"stamp":"2024-09-11T01:22:28.804648+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32796,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14844,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472224,"kb_used_data":191728,"kb_used_omap":12,"kb_used_meta":1280435,"kb_avail":281643296,"statfs":{"total":289910292480,"available":288402735104,"internally_reserved":0,"allocated":196329472,"data_stored":195447860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1311165444},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-9969688,"num_objects":-2435,"num_object_clones":0,"num_object_copies":-4870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1019,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2435,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1019,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003530"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":20354,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.943277+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:22:18.943277+0000","last_peered":"2024-09-11T01:22:18.943277+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:22:18.943277+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20464,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.964296+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:18.964296+0000","last_peered":"2024-09-11T01:22:18.964296+0000","last_clean":"2024-09-11T01:22:18.964296+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:18.964296+0000","last_undegraded":"2024-09-11T01:22:18.964296+0000","last_fullsized":"2024-09-11T01:22:18.964296+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20283,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.012455+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:22:19.012455+0000","last_peered":"2024-09-11T01:22:19.012455+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:22:19.012455+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3896}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20214,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.872730+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:18.872730+0000","last_peered":"2024-09-11T01:22:18.872730+0000","last_clean":"2024-09-11T01:22:18.872730+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:18.872730+0000","last_undegraded":"2024-09-11T01:22:18.872730+0000","last_fullsized":"2024-09-11T01:22:18.872730+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20147,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:19.019037+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:19.019037+0000","last_peered":"2024-09-11T01:22:19.019037+0000","last_clean":"2024-09-11T01:22:19.019037+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:19.019037+0000","last_undegraded":"2024-09-11T01:22:19.019037+0000","last_fullsized":"2024-09-11T01:22:19.019037+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20408,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.034150+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:19.034150+0000","last_peered":"2024-09-11T01:22:19.034150+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:19.034150+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20071,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.026055+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:19.026055+0000","last_peered":"2024-09-11T01:22:19.026055+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:19.026055+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20394,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.910833+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:18.910833+0000","last_peered":"2024-09-11T01:22:18.910833+0000","last_clean":"2024-09-11T01:22:18.910833+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:18.910833+0000","last_undegraded":"2024-09-11T01:22:18.910833+0000","last_fullsized":"2024-09-11T01:22:18.910833+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003209+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003209+0000","last_peered":"2024-09-11T01:20:22.003209+0000","last_clean":"2024-09-11T01:20:22.003209+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003209+0000","last_undegraded":"2024-09-11T01:20:22.003209+0000","last_fullsized":"2024-09-11T01:20:22.003209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":27992,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.993959+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:18.993959+0000","last_peered":"2024-09-11T01:22:18.993959+0000","last_clean":"2024-09-11T01:22:18.993959+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:18.993959+0000","last_undegraded":"2024-09-11T01:22:18.993959+0000","last_fullsized":"2024-09-11T01:22:18.993959+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20007,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.022841+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:19.022841+0000","last_peered":"2024-09-11T01:22:19.022841+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:19.022841+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005254+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005254+0000","last_peered":"2024-09-11T01:20:22.005254+0000","last_clean":"2024-09-11T01:20:22.005254+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005254+0000","last_undegraded":"2024-09-11T01:20:22.005254+0000","last_fullsized":"2024-09-11T01:20:22.005254+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20336,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.997841+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:18.997841+0000","last_peered":"2024-09-11T01:22:18.997841+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:18.997841+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003275+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003275+0000","last_peered":"2024-09-11T01:20:22.003275+0000","last_clean":"2024-09-11T01:20:22.003275+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003275+0000","last_undegraded":"2024-09-11T01:20:22.003275+0000","last_fullsized":"2024-09-11T01:20:22.003275+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27788,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.973481+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:18.973481+0000","last_peered":"2024-09-11T01:22:18.973481+0000","last_clean":"2024-09-11T01:22:18.973481+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:18.973481+0000","last_undegraded":"2024-09-11T01:22:18.973481+0000","last_fullsized":"2024-09-11T01:22:18.973481+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20406,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.987858+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:18.987858+0000","last_peered":"2024-09-11T01:22:18.987858+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:18.987858+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007577+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007577+0000","last_peered":"2024-09-11T01:20:22.007577+0000","last_clean":"2024-09-11T01:20:22.007577+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007577+0000","last_undegraded":"2024-09-11T01:20:22.007577+0000","last_fullsized":"2024-09-11T01:20:22.007577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27926,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.950589+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:18.950589+0000","last_peered":"2024-09-11T01:22:18.950589+0000","last_clean":"2024-09-11T01:22:18.950589+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:18.950589+0000","last_undegraded":"2024-09-11T01:22:18.950589+0000","last_fullsized":"2024-09-11T01:22:18.950589+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":26398,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:28.218587+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:22:28.218587+0000","last_peered":"2024-09-11T01:22:28.218587+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:22:28.218587+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1003,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3038,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007616+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007616+0000","last_peered":"2024-09-11T01:20:22.007616+0000","last_clean":"2024-09-11T01:20:22.007616+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007616+0000","last_undegraded":"2024-09-11T01:20:22.007616+0000","last_fullsized":"2024-09-11T01:20:22.007616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008206+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008206+0000","last_peered":"2024-09-11T01:20:22.008206+0000","last_clean":"2024-09-11T01:20:22.008206+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008206+0000","last_undegraded":"2024-09-11T01:20:22.008206+0000","last_fullsized":"2024-09-11T01:20:22.008206+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20096,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.016874+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:19.016874+0000","last_peered":"2024-09-11T01:22:19.016874+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:19.016874+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32796,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14843,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195092480,"data_stored":195036816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182531,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518568,"kb_used_data":63512,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853272,"statfs":{"total":96636764160,"available":96105750528,"internally_reserved":0,"allocated":65036288,"data_stored":64756134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986303,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488412,"kb_used_data":127756,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883428,"statfs":{"total":96636764160,"available":96136630272,"internally_reserved":0,"allocated":130822144,"data_stored":130546067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659248,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465244,"kb_used_data":460,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93906596,"statfs":{"total":96636764160,"available":96160354304,"internally_reserved":0,"allocated":471040,"data_stored":145659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475917999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:29.487 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:22:29.487 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20266", 'reported_seq': 20354, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:18.943277+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:22:18.943277+0000', 'last_peered': '2024-09-11T01:22:18.943277+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:22:18.943277+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20266, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4072}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:29.487 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:22:29.487 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'20144", 'reported_seq': 20283, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:19.012455+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:22:19.012455+0000', 'last_peered': '2024-09-11T01:22:19.012455+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:22:19.012455+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18445, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3896}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:29.487 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:22:29.488 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'20227", 'reported_seq': 20408, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:19.034150+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:22:19.034150+0000', 'last_peered': '2024-09-11T01:22:19.034150+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:22:19.034150+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16568, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:29.488 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:22:29.488 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20071, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:19.026055+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:22:19.026055+0000', 'last_peered': '2024-09-11T01:22:19.026055+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:22:19.026055+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:29.488 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:22:29.488 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20007, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:19.022841+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:22:19.022841+0000', 'last_peered': '2024-09-11T01:22:19.022841+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:22:19.022841+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:29.488 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:22:29.488 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'20207", 'reported_seq': 20336, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:18.997841+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:22:18.997841+0000', 'last_peered': '2024-09-11T01:22:18.997841+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:22:18.997841+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18024, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4024}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:29.488 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:22:29.488 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20264", 'reported_seq': 20406, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:18.987858+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:22:18.987858+0000', 'last_peered': '2024-09-11T01:22:18.987858+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:22:18.987858+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17001, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:29.488 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:22:29.489 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'20176", 'reported_seq': 26398, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:28.218587+0000', 'last_change': '2024-09-11T01:20:20.985862+0000', 'last_active': '2024-09-11T01:22:28.218587+0000', 'last_peered': '2024-09-11T01:22:28.218587+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:22:28.218587+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1003, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16785, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3038, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1003}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:29.489 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:22:29.489 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 20096, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:19.016874+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:22:19.016874+0000', 'last_peered': '2024-09-11T01:22:19.016874+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:22:19.016874+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:29.489 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:22:29.854 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:29.855 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 32796/6 objects degraded (546600.000%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":686,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1507557376,"bytes_avail":288402735104,"bytes_total":289910292480,"degraded_objects":32796,"degraded_total":6,"degraded_ratio":5466,"recovering_objects_per_sec":84,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1019,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":202},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:22:29.873 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:22:29.874 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:30.182 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:30.183 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:22:30.200 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":385,"stamp":"2024-09-11T01:22:28.804648+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32796,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14844,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472224,"kb_used_data":191728,"kb_used_omap":12,"kb_used_meta":1280435,"kb_avail":281643296,"statfs":{"total":289910292480,"available":288402735104,"internally_reserved":0,"allocated":196329472,"data_stored":195447860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1311165444},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-9969688,"num_objects":-2435,"num_object_clones":0,"num_object_copies":-4870,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1019,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2435,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2435,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1019,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003530"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":20354,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.943277+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:22:18.943277+0000","last_peered":"2024-09-11T01:22:18.943277+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:22:18.943277+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20464,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.964296+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:18.964296+0000","last_peered":"2024-09-11T01:22:18.964296+0000","last_clean":"2024-09-11T01:22:18.964296+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:18.964296+0000","last_undegraded":"2024-09-11T01:22:18.964296+0000","last_fullsized":"2024-09-11T01:22:18.964296+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20283,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.012455+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:22:19.012455+0000","last_peered":"2024-09-11T01:22:19.012455+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:22:19.012455+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3896}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20214,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.872730+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:18.872730+0000","last_peered":"2024-09-11T01:22:18.872730+0000","last_clean":"2024-09-11T01:22:18.872730+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:18.872730+0000","last_undegraded":"2024-09-11T01:22:18.872730+0000","last_fullsized":"2024-09-11T01:22:18.872730+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20147,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:19.019037+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:19.019037+0000","last_peered":"2024-09-11T01:22:19.019037+0000","last_clean":"2024-09-11T01:22:19.019037+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:19.019037+0000","last_undegraded":"2024-09-11T01:22:19.019037+0000","last_fullsized":"2024-09-11T01:22:19.019037+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20408,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.034150+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:19.034150+0000","last_peered":"2024-09-11T01:22:19.034150+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:19.034150+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20071,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.026055+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:19.026055+0000","last_peered":"2024-09-11T01:22:19.026055+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:19.026055+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20394,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.910833+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:18.910833+0000","last_peered":"2024-09-11T01:22:18.910833+0000","last_clean":"2024-09-11T01:22:18.910833+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:18.910833+0000","last_undegraded":"2024-09-11T01:22:18.910833+0000","last_fullsized":"2024-09-11T01:22:18.910833+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003209+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003209+0000","last_peered":"2024-09-11T01:20:22.003209+0000","last_clean":"2024-09-11T01:20:22.003209+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003209+0000","last_undegraded":"2024-09-11T01:20:22.003209+0000","last_fullsized":"2024-09-11T01:20:22.003209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":27992,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.993959+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:18.993959+0000","last_peered":"2024-09-11T01:22:18.993959+0000","last_clean":"2024-09-11T01:22:18.993959+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:18.993959+0000","last_undegraded":"2024-09-11T01:22:18.993959+0000","last_fullsized":"2024-09-11T01:22:18.993959+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20007,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.022841+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:19.022841+0000","last_peered":"2024-09-11T01:22:19.022841+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:19.022841+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005254+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005254+0000","last_peered":"2024-09-11T01:20:22.005254+0000","last_clean":"2024-09-11T01:20:22.005254+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005254+0000","last_undegraded":"2024-09-11T01:20:22.005254+0000","last_fullsized":"2024-09-11T01:20:22.005254+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20336,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.997841+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:18.997841+0000","last_peered":"2024-09-11T01:22:18.997841+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:18.997841+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003275+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003275+0000","last_peered":"2024-09-11T01:20:22.003275+0000","last_clean":"2024-09-11T01:20:22.003275+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003275+0000","last_undegraded":"2024-09-11T01:20:22.003275+0000","last_fullsized":"2024-09-11T01:20:22.003275+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27788,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.973481+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:18.973481+0000","last_peered":"2024-09-11T01:22:18.973481+0000","last_clean":"2024-09-11T01:22:18.973481+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:18.973481+0000","last_undegraded":"2024-09-11T01:22:18.973481+0000","last_fullsized":"2024-09-11T01:22:18.973481+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20406,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.987858+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:18.987858+0000","last_peered":"2024-09-11T01:22:18.987858+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:18.987858+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007577+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007577+0000","last_peered":"2024-09-11T01:20:22.007577+0000","last_clean":"2024-09-11T01:20:22.007577+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007577+0000","last_undegraded":"2024-09-11T01:20:22.007577+0000","last_fullsized":"2024-09-11T01:20:22.007577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27926,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.950589+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:18.950589+0000","last_peered":"2024-09-11T01:22:18.950589+0000","last_clean":"2024-09-11T01:22:18.950589+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:18.950589+0000","last_undegraded":"2024-09-11T01:22:18.950589+0000","last_fullsized":"2024-09-11T01:22:18.950589+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":26398,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:28.218587+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:22:28.218587+0000","last_peered":"2024-09-11T01:22:28.218587+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:22:28.218587+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1003,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3038,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007616+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007616+0000","last_peered":"2024-09-11T01:20:22.007616+0000","last_clean":"2024-09-11T01:20:22.007616+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007616+0000","last_undegraded":"2024-09-11T01:20:22.007616+0000","last_fullsized":"2024-09-11T01:20:22.007616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008206+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008206+0000","last_peered":"2024-09-11T01:20:22.008206+0000","last_clean":"2024-09-11T01:20:22.008206+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008206+0000","last_undegraded":"2024-09-11T01:20:22.008206+0000","last_fullsized":"2024-09-11T01:20:22.008206+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20096,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.016874+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:19.016874+0000","last_peered":"2024-09-11T01:22:19.016874+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:19.016874+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32796,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14843,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195092480,"data_stored":195036816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182531,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518568,"kb_used_data":63512,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853272,"statfs":{"total":96636764160,"available":96105750528,"internally_reserved":0,"allocated":65036288,"data_stored":64756134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986303,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488412,"kb_used_data":127756,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883428,"statfs":{"total":96636764160,"available":96136630272,"internally_reserved":0,"allocated":130822144,"data_stored":130546067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659248,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465244,"kb_used_data":460,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93906596,"statfs":{"total":96636764160,"available":96160354304,"internally_reserved":0,"allocated":471040,"data_stored":145659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475917999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:33.202 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:33.553 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:33.553 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:22:33.571 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":387,"stamp":"2024-09-11T01:22:32.806381+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32796,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14844,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472224,"kb_used_data":191728,"kb_used_omap":12,"kb_used_meta":1280435,"kb_avail":281643296,"statfs":{"total":289910292480,"available":288402735104,"internally_reserved":0,"allocated":196329472,"data_stored":195447860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1311165444},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1179648,"num_objects":-288,"num_object_clones":0,"num_object_copies":-576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-912,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-288,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":912,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004308"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":20354,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.943277+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:22:18.943277+0000","last_peered":"2024-09-11T01:22:18.943277+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:22:18.943277+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20464,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.964296+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:18.964296+0000","last_peered":"2024-09-11T01:22:18.964296+0000","last_clean":"2024-09-11T01:22:18.964296+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:18.964296+0000","last_undegraded":"2024-09-11T01:22:18.964296+0000","last_fullsized":"2024-09-11T01:22:18.964296+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20283,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.012455+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:22:19.012455+0000","last_peered":"2024-09-11T01:22:19.012455+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:22:19.012455+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3896}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20214,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.872730+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:18.872730+0000","last_peered":"2024-09-11T01:22:18.872730+0000","last_clean":"2024-09-11T01:22:18.872730+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:18.872730+0000","last_undegraded":"2024-09-11T01:22:18.872730+0000","last_fullsized":"2024-09-11T01:22:18.872730+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20147,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:19.019037+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:19.019037+0000","last_peered":"2024-09-11T01:22:19.019037+0000","last_clean":"2024-09-11T01:22:19.019037+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:19.019037+0000","last_undegraded":"2024-09-11T01:22:19.019037+0000","last_fullsized":"2024-09-11T01:22:19.019037+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20408,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.034150+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:19.034150+0000","last_peered":"2024-09-11T01:22:19.034150+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:19.034150+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20071,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.026055+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:19.026055+0000","last_peered":"2024-09-11T01:22:19.026055+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:19.026055+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20394,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.910833+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:18.910833+0000","last_peered":"2024-09-11T01:22:18.910833+0000","last_clean":"2024-09-11T01:22:18.910833+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:18.910833+0000","last_undegraded":"2024-09-11T01:22:18.910833+0000","last_fullsized":"2024-09-11T01:22:18.910833+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003209+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003209+0000","last_peered":"2024-09-11T01:20:22.003209+0000","last_clean":"2024-09-11T01:20:22.003209+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003209+0000","last_undegraded":"2024-09-11T01:20:22.003209+0000","last_fullsized":"2024-09-11T01:20:22.003209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":27992,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.993959+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:18.993959+0000","last_peered":"2024-09-11T01:22:18.993959+0000","last_clean":"2024-09-11T01:22:18.993959+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:18.993959+0000","last_undegraded":"2024-09-11T01:22:18.993959+0000","last_fullsized":"2024-09-11T01:22:18.993959+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20007,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.022841+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:19.022841+0000","last_peered":"2024-09-11T01:22:19.022841+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:19.022841+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005254+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005254+0000","last_peered":"2024-09-11T01:20:22.005254+0000","last_clean":"2024-09-11T01:20:22.005254+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005254+0000","last_undegraded":"2024-09-11T01:20:22.005254+0000","last_fullsized":"2024-09-11T01:20:22.005254+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20336,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.997841+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:18.997841+0000","last_peered":"2024-09-11T01:22:18.997841+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:18.997841+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003275+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003275+0000","last_peered":"2024-09-11T01:20:22.003275+0000","last_clean":"2024-09-11T01:20:22.003275+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003275+0000","last_undegraded":"2024-09-11T01:20:22.003275+0000","last_fullsized":"2024-09-11T01:20:22.003275+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27788,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.973481+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:18.973481+0000","last_peered":"2024-09-11T01:22:18.973481+0000","last_clean":"2024-09-11T01:22:18.973481+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:18.973481+0000","last_undegraded":"2024-09-11T01:22:18.973481+0000","last_fullsized":"2024-09-11T01:22:18.973481+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20406,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.987858+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:18.987858+0000","last_peered":"2024-09-11T01:22:18.987858+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:18.987858+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007577+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007577+0000","last_peered":"2024-09-11T01:20:22.007577+0000","last_clean":"2024-09-11T01:20:22.007577+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007577+0000","last_undegraded":"2024-09-11T01:20:22.007577+0000","last_fullsized":"2024-09-11T01:20:22.007577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27926,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.950589+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:18.950589+0000","last_peered":"2024-09-11T01:22:18.950589+0000","last_clean":"2024-09-11T01:22:18.950589+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:18.950589+0000","last_undegraded":"2024-09-11T01:22:18.950589+0000","last_fullsized":"2024-09-11T01:22:18.950589+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":26398,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:28.218587+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:22:28.218587+0000","last_peered":"2024-09-11T01:22:28.218587+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:22:28.218587+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1003,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3038,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007616+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007616+0000","last_peered":"2024-09-11T01:20:22.007616+0000","last_clean":"2024-09-11T01:20:22.007616+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007616+0000","last_undegraded":"2024-09-11T01:20:22.007616+0000","last_fullsized":"2024-09-11T01:20:22.007616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008206+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008206+0000","last_peered":"2024-09-11T01:20:22.008206+0000","last_clean":"2024-09-11T01:20:22.008206+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008206+0000","last_undegraded":"2024-09-11T01:20:22.008206+0000","last_fullsized":"2024-09-11T01:20:22.008206+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20096,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.016874+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:19.016874+0000","last_peered":"2024-09-11T01:22:19.016874+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:19.016874+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32796,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14843,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195092480,"data_stored":195036816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182531,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518568,"kb_used_data":63512,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853272,"statfs":{"total":96636764160,"available":96105750528,"internally_reserved":0,"allocated":65036288,"data_stored":64756134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986304,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488412,"kb_used_data":127756,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883428,"statfs":{"total":96636764160,"available":96136630272,"internally_reserved":0,"allocated":130822144,"data_stored":130546067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659248,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465244,"kb_used_data":460,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93906596,"statfs":{"total":96636764160,"available":96160354304,"internally_reserved":0,"allocated":471040,"data_stored":145659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475917999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:33.573 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:33.944 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:33.944 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:22:33.961 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":387,"stamp":"2024-09-11T01:22:32.806381+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32796,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14844,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472224,"kb_used_data":191728,"kb_used_omap":12,"kb_used_meta":1280435,"kb_avail":281643296,"statfs":{"total":289910292480,"available":288402735104,"internally_reserved":0,"allocated":196329472,"data_stored":195447860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1311165444},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1179648,"num_objects":-288,"num_object_clones":0,"num_object_copies":-576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-912,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-288,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":912,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004308"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":20354,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.943277+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:22:18.943277+0000","last_peered":"2024-09-11T01:22:18.943277+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:22:18.943277+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20464,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.964296+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:18.964296+0000","last_peered":"2024-09-11T01:22:18.964296+0000","last_clean":"2024-09-11T01:22:18.964296+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:18.964296+0000","last_undegraded":"2024-09-11T01:22:18.964296+0000","last_fullsized":"2024-09-11T01:22:18.964296+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20283,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.012455+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:22:19.012455+0000","last_peered":"2024-09-11T01:22:19.012455+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:22:19.012455+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3896}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20214,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.872730+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:18.872730+0000","last_peered":"2024-09-11T01:22:18.872730+0000","last_clean":"2024-09-11T01:22:18.872730+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:18.872730+0000","last_undegraded":"2024-09-11T01:22:18.872730+0000","last_fullsized":"2024-09-11T01:22:18.872730+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20147,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:19.019037+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:19.019037+0000","last_peered":"2024-09-11T01:22:19.019037+0000","last_clean":"2024-09-11T01:22:19.019037+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:19.019037+0000","last_undegraded":"2024-09-11T01:22:19.019037+0000","last_fullsized":"2024-09-11T01:22:19.019037+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20408,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.034150+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:19.034150+0000","last_peered":"2024-09-11T01:22:19.034150+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:19.034150+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20071,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.026055+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:19.026055+0000","last_peered":"2024-09-11T01:22:19.026055+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:19.026055+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20394,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.910833+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:18.910833+0000","last_peered":"2024-09-11T01:22:18.910833+0000","last_clean":"2024-09-11T01:22:18.910833+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:18.910833+0000","last_undegraded":"2024-09-11T01:22:18.910833+0000","last_fullsized":"2024-09-11T01:22:18.910833+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003209+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003209+0000","last_peered":"2024-09-11T01:20:22.003209+0000","last_clean":"2024-09-11T01:20:22.003209+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003209+0000","last_undegraded":"2024-09-11T01:20:22.003209+0000","last_fullsized":"2024-09-11T01:20:22.003209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":27992,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.993959+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:18.993959+0000","last_peered":"2024-09-11T01:22:18.993959+0000","last_clean":"2024-09-11T01:22:18.993959+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:18.993959+0000","last_undegraded":"2024-09-11T01:22:18.993959+0000","last_fullsized":"2024-09-11T01:22:18.993959+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20007,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.022841+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:19.022841+0000","last_peered":"2024-09-11T01:22:19.022841+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:19.022841+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005254+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005254+0000","last_peered":"2024-09-11T01:20:22.005254+0000","last_clean":"2024-09-11T01:20:22.005254+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005254+0000","last_undegraded":"2024-09-11T01:20:22.005254+0000","last_fullsized":"2024-09-11T01:20:22.005254+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20336,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.997841+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:18.997841+0000","last_peered":"2024-09-11T01:22:18.997841+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:18.997841+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003275+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003275+0000","last_peered":"2024-09-11T01:20:22.003275+0000","last_clean":"2024-09-11T01:20:22.003275+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003275+0000","last_undegraded":"2024-09-11T01:20:22.003275+0000","last_fullsized":"2024-09-11T01:20:22.003275+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27788,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.973481+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:18.973481+0000","last_peered":"2024-09-11T01:22:18.973481+0000","last_clean":"2024-09-11T01:22:18.973481+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:18.973481+0000","last_undegraded":"2024-09-11T01:22:18.973481+0000","last_fullsized":"2024-09-11T01:22:18.973481+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20406,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.987858+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:18.987858+0000","last_peered":"2024-09-11T01:22:18.987858+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:18.987858+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007577+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007577+0000","last_peered":"2024-09-11T01:20:22.007577+0000","last_clean":"2024-09-11T01:20:22.007577+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007577+0000","last_undegraded":"2024-09-11T01:20:22.007577+0000","last_fullsized":"2024-09-11T01:20:22.007577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27926,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.950589+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:18.950589+0000","last_peered":"2024-09-11T01:22:18.950589+0000","last_clean":"2024-09-11T01:22:18.950589+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:18.950589+0000","last_undegraded":"2024-09-11T01:22:18.950589+0000","last_fullsized":"2024-09-11T01:22:18.950589+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":26398,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:28.218587+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:22:28.218587+0000","last_peered":"2024-09-11T01:22:28.218587+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:22:28.218587+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1003,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3038,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007616+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007616+0000","last_peered":"2024-09-11T01:20:22.007616+0000","last_clean":"2024-09-11T01:20:22.007616+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007616+0000","last_undegraded":"2024-09-11T01:20:22.007616+0000","last_fullsized":"2024-09-11T01:20:22.007616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008206+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008206+0000","last_peered":"2024-09-11T01:20:22.008206+0000","last_clean":"2024-09-11T01:20:22.008206+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008206+0000","last_undegraded":"2024-09-11T01:20:22.008206+0000","last_fullsized":"2024-09-11T01:20:22.008206+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20096,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.016874+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:19.016874+0000","last_peered":"2024-09-11T01:22:19.016874+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:19.016874+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32796,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14843,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195092480,"data_stored":195036816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182531,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518568,"kb_used_data":63512,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853272,"statfs":{"total":96636764160,"available":96105750528,"internally_reserved":0,"allocated":65036288,"data_stored":64756134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986304,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488412,"kb_used_data":127756,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883428,"statfs":{"total":96636764160,"available":96136630272,"internally_reserved":0,"allocated":130822144,"data_stored":130546067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659248,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465244,"kb_used_data":460,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93906596,"statfs":{"total":96636764160,"available":96160354304,"internally_reserved":0,"allocated":471040,"data_stored":145659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475917999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:33.962 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:22:33.962 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20266", 'reported_seq': 20354, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:18.943277+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:22:18.943277+0000', 'last_peered': '2024-09-11T01:22:18.943277+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:22:18.943277+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20266, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4072}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:33.962 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:22:33.963 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'20144", 'reported_seq': 20283, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:19.012455+0000', 'last_change': '2024-09-11T01:13:10.202890+0000', 'last_active': '2024-09-11T01:22:19.012455+0000', 'last_peered': '2024-09-11T01:22:19.012455+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:22:19.012455+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3897, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18445, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3896}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:33.963 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:22:33.963 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'20227", 'reported_seq': 20408, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:19.034150+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:22:19.034150+0000', 'last_peered': '2024-09-11T01:22:19.034150+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:22:19.034150+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16568, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:33.963 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:22:33.963 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20071, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:19.026055+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:22:19.026055+0000', 'last_peered': '2024-09-11T01:22:19.026055+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:22:19.026055+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:33.963 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:22:33.963 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20007, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:19.022841+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:22:19.022841+0000', 'last_peered': '2024-09-11T01:22:19.022841+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:22:19.022841+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:33.963 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:22:33.963 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'20207", 'reported_seq': 20336, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:18.997841+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:22:18.997841+0000', 'last_peered': '2024-09-11T01:22:18.997841+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:22:18.997841+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18024, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4024}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:33.963 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:22:33.964 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20264", 'reported_seq': 20406, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:18.987858+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:22:18.987858+0000', 'last_peered': '2024-09-11T01:22:18.987858+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:22:18.987858+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17001, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:33.964 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:22:33.964 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'20176", 'reported_seq': 26398, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:28.218587+0000', 'last_change': '2024-09-11T01:20:20.985862+0000', 'last_active': '2024-09-11T01:22:28.218587+0000', 'last_peered': '2024-09-11T01:22:28.218587+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:22:28.218587+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1003, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16785, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3038, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1003}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:33.964 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:22:33.964 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 20096, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:19.016874+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:22:19.016874+0000', 'last_peered': '2024-09-11T01:22:19.016874+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:22:19.016874+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:33.964 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:22:34.332 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:34.333 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 32796/6 objects degraded (546600.000%), 9 pgs degraded, 9 pgs undersized","count":18},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":690,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":49,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":9},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":15},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":8},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1507557376,"bytes_avail":288402735104,"bytes_total":289910292480,"degraded_objects":32796,"degraded_total":6,"degraded_ratio":5466,"recovering_objects_per_sec":75,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":912,"num_bytes_recovered":0,"num_keys_recovered":0,"write_bytes_sec":0,"read_op_per_sec":0,"write_op_per_sec":23},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:22:34.350 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:22:34.351 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:34.676 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:34.677 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:22:34.698 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":387,"stamp":"2024-09-11T01:22:32.806381+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32796,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14844,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472224,"kb_used_data":191728,"kb_used_omap":12,"kb_used_meta":1280435,"kb_avail":281643296,"statfs":{"total":289910292480,"available":288402735104,"internally_reserved":0,"allocated":196329472,"data_stored":195447860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1311165444},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-1179648,"num_objects":-288,"num_object_clones":0,"num_object_copies":-576,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-912,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-288,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":288,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":912,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004308"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":20354,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.943277+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:22:18.943277+0000","last_peered":"2024-09-11T01:22:18.943277+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:22:18.943277+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20464,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.964296+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:18.964296+0000","last_peered":"2024-09-11T01:22:18.964296+0000","last_clean":"2024-09-11T01:22:18.964296+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:18.964296+0000","last_undegraded":"2024-09-11T01:22:18.964296+0000","last_fullsized":"2024-09-11T01:22:18.964296+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20283,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.012455+0000","last_change":"2024-09-11T01:13:10.202890+0000","last_active":"2024-09-11T01:22:19.012455+0000","last_peered":"2024-09-11T01:22:19.012455+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:22:19.012455+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3897,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3896}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20214,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.872730+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:18.872730+0000","last_peered":"2024-09-11T01:22:18.872730+0000","last_clean":"2024-09-11T01:22:18.872730+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:18.872730+0000","last_undegraded":"2024-09-11T01:22:18.872730+0000","last_fullsized":"2024-09-11T01:22:18.872730+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20147,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:19.019037+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:19.019037+0000","last_peered":"2024-09-11T01:22:19.019037+0000","last_clean":"2024-09-11T01:22:19.019037+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:19.019037+0000","last_undegraded":"2024-09-11T01:22:19.019037+0000","last_fullsized":"2024-09-11T01:22:19.019037+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20408,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.034150+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:19.034150+0000","last_peered":"2024-09-11T01:22:19.034150+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:19.034150+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20071,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.026055+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:19.026055+0000","last_peered":"2024-09-11T01:22:19.026055+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:19.026055+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20394,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.910833+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:18.910833+0000","last_peered":"2024-09-11T01:22:18.910833+0000","last_clean":"2024-09-11T01:22:18.910833+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:18.910833+0000","last_undegraded":"2024-09-11T01:22:18.910833+0000","last_fullsized":"2024-09-11T01:22:18.910833+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003209+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003209+0000","last_peered":"2024-09-11T01:20:22.003209+0000","last_clean":"2024-09-11T01:20:22.003209+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003209+0000","last_undegraded":"2024-09-11T01:20:22.003209+0000","last_fullsized":"2024-09-11T01:20:22.003209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":27992,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.993959+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:18.993959+0000","last_peered":"2024-09-11T01:22:18.993959+0000","last_clean":"2024-09-11T01:22:18.993959+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:18.993959+0000","last_undegraded":"2024-09-11T01:22:18.993959+0000","last_fullsized":"2024-09-11T01:22:18.993959+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017993+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:22.017993+0000","last_peered":"2024-09-11T01:20:22.017993+0000","last_clean":"2024-09-11T01:20:22.017993+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:22.017993+0000","last_undegraded":"2024-09-11T01:20:22.017993+0000","last_fullsized":"2024-09-11T01:20:22.017993+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20007,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.022841+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:19.022841+0000","last_peered":"2024-09-11T01:22:19.022841+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:19.022841+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005254+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005254+0000","last_peered":"2024-09-11T01:20:22.005254+0000","last_clean":"2024-09-11T01:20:22.005254+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005254+0000","last_undegraded":"2024-09-11T01:20:22.005254+0000","last_fullsized":"2024-09-11T01:20:22.005254+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20336,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.997841+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:18.997841+0000","last_peered":"2024-09-11T01:22:18.997841+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:18.997841+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003275+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003275+0000","last_peered":"2024-09-11T01:20:22.003275+0000","last_clean":"2024-09-11T01:20:22.003275+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003275+0000","last_undegraded":"2024-09-11T01:20:22.003275+0000","last_fullsized":"2024-09-11T01:20:22.003275+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27788,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.973481+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:18.973481+0000","last_peered":"2024-09-11T01:22:18.973481+0000","last_clean":"2024-09-11T01:22:18.973481+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:18.973481+0000","last_undegraded":"2024-09-11T01:22:18.973481+0000","last_fullsized":"2024-09-11T01:22:18.973481+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20406,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.987858+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:18.987858+0000","last_peered":"2024-09-11T01:22:18.987858+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:18.987858+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007577+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007577+0000","last_peered":"2024-09-11T01:20:22.007577+0000","last_clean":"2024-09-11T01:20:22.007577+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007577+0000","last_undegraded":"2024-09-11T01:20:22.007577+0000","last_fullsized":"2024-09-11T01:20:22.007577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27926,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.950589+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:18.950589+0000","last_peered":"2024-09-11T01:22:18.950589+0000","last_clean":"2024-09-11T01:22:18.950589+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:18.950589+0000","last_undegraded":"2024-09-11T01:22:18.950589+0000","last_fullsized":"2024-09-11T01:22:18.950589+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.017750+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:22.017750+0000","last_peered":"2024-09-11T01:20:22.017750+0000","last_clean":"2024-09-11T01:20:22.017750+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:22.017750+0000","last_undegraded":"2024-09-11T01:20:22.017750+0000","last_fullsized":"2024-09-11T01:20:22.017750+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":26398,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:28.218587+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:22:28.218587+0000","last_peered":"2024-09-11T01:22:28.218587+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:22:28.218587+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1003,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3038,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1003}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007616+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007616+0000","last_peered":"2024-09-11T01:20:22.007616+0000","last_clean":"2024-09-11T01:20:22.007616+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007616+0000","last_undegraded":"2024-09-11T01:20:22.007616+0000","last_fullsized":"2024-09-11T01:20:22.007616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008206+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008206+0000","last_peered":"2024-09-11T01:20:22.008206+0000","last_clean":"2024-09-11T01:20:22.008206+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008206+0000","last_undegraded":"2024-09-11T01:20:22.008206+0000","last_fullsized":"2024-09-11T01:20:22.008206+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20096,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.016874+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:19.016874+0000","last_peered":"2024-09-11T01:22:19.016874+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:19.016874+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32796,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14843,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195092480,"data_stored":195036816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182531,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518568,"kb_used_data":63512,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853272,"statfs":{"total":96636764160,"available":96105750528,"internally_reserved":0,"allocated":65036288,"data_stored":64756134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986304,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488412,"kb_used_data":127756,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883428,"statfs":{"total":96636764160,"available":96136630272,"internally_reserved":0,"allocated":130822144,"data_stored":130546067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659248,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465244,"kb_used_data":460,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93906596,"statfs":{"total":96636764160,"available":96160354304,"internally_reserved":0,"allocated":471040,"data_stored":145659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475917999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:37.701 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:38.085 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:38.086 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:22:38.106 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":391,"stamp":"2024-09-11T01:22:36.919563+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32185,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15455,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1476384,"kb_used_data":191728,"kb_used_omap":12,"kb_used_meta":1284595,"kb_avail":281639136,"statfs":{"total":289910292480,"available":288398475264,"internally_reserved":0,"allocated":196329472,"data_stored":195447860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1315425284},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-611,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":611,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.114915"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":20354,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.943277+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:22:18.943277+0000","last_peered":"2024-09-11T01:22:18.943277+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:22:18.943277+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20464,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.964296+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:18.964296+0000","last_peered":"2024-09-11T01:22:18.964296+0000","last_clean":"2024-09-11T01:22:18.964296+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:18.964296+0000","last_undegraded":"2024-09-11T01:22:18.964296+0000","last_fullsized":"2024-09-11T01:22:18.964296+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20302,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:35.335864+0000","last_change":"2024-09-11T01:22:35.206502+0000","last_active":"2024-09-11T01:22:35.335864+0000","last_peered":"2024-09-11T01:22:35.335864+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:22:35.335864+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3888,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3888}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20214,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.872730+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:18.872730+0000","last_peered":"2024-09-11T01:22:18.872730+0000","last_clean":"2024-09-11T01:22:18.872730+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:18.872730+0000","last_undegraded":"2024-09-11T01:22:18.872730+0000","last_fullsized":"2024-09-11T01:22:18.872730+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20147,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:19.019037+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:19.019037+0000","last_peered":"2024-09-11T01:22:19.019037+0000","last_clean":"2024-09-11T01:22:19.019037+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:19.019037+0000","last_undegraded":"2024-09-11T01:22:19.019037+0000","last_fullsized":"2024-09-11T01:22:19.019037+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20408,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.034150+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:19.034150+0000","last_peered":"2024-09-11T01:22:19.034150+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:19.034150+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20071,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.026055+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:19.026055+0000","last_peered":"2024-09-11T01:22:19.026055+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:19.026055+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20394,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.910833+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:18.910833+0000","last_peered":"2024-09-11T01:22:18.910833+0000","last_clean":"2024-09-11T01:22:18.910833+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:18.910833+0000","last_undegraded":"2024-09-11T01:22:18.910833+0000","last_fullsized":"2024-09-11T01:22:18.910833+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003209+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003209+0000","last_peered":"2024-09-11T01:20:22.003209+0000","last_clean":"2024-09-11T01:20:22.003209+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003209+0000","last_undegraded":"2024-09-11T01:20:22.003209+0000","last_fullsized":"2024-09-11T01:20:22.003209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":27992,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.993959+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:18.993959+0000","last_peered":"2024-09-11T01:22:18.993959+0000","last_clean":"2024-09-11T01:22:18.993959+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:18.993959+0000","last_undegraded":"2024-09-11T01:22:18.993959+0000","last_fullsized":"2024-09-11T01:22:18.993959+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:30.036418+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:30.036418+0000","last_peered":"2024-09-11T01:20:30.036418+0000","last_clean":"2024-09-11T01:20:30.036418+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:30.036418+0000","last_undegraded":"2024-09-11T01:20:30.036418+0000","last_fullsized":"2024-09-11T01:20:30.036418+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20007,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.022841+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:19.022841+0000","last_peered":"2024-09-11T01:22:19.022841+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:19.022841+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005254+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005254+0000","last_peered":"2024-09-11T01:20:22.005254+0000","last_clean":"2024-09-11T01:20:22.005254+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005254+0000","last_undegraded":"2024-09-11T01:20:22.005254+0000","last_fullsized":"2024-09-11T01:20:22.005254+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20337,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.026024+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:19.026024+0000","last_peered":"2024-09-11T01:22:19.026024+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:19.026024+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003275+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003275+0000","last_peered":"2024-09-11T01:20:22.003275+0000","last_clean":"2024-09-11T01:20:22.003275+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003275+0000","last_undegraded":"2024-09-11T01:20:22.003275+0000","last_fullsized":"2024-09-11T01:20:22.003275+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27788,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.973481+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:18.973481+0000","last_peered":"2024-09-11T01:22:18.973481+0000","last_clean":"2024-09-11T01:22:18.973481+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:18.973481+0000","last_undegraded":"2024-09-11T01:22:18.973481+0000","last_fullsized":"2024-09-11T01:22:18.973481+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20406,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.987858+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:18.987858+0000","last_peered":"2024-09-11T01:22:18.987858+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:18.987858+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007577+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007577+0000","last_peered":"2024-09-11T01:20:22.007577+0000","last_clean":"2024-09-11T01:20:22.007577+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007577+0000","last_undegraded":"2024-09-11T01:20:22.007577+0000","last_fullsized":"2024-09-11T01:20:22.007577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27926,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.950589+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:18.950589+0000","last_peered":"2024-09-11T01:22:18.950589+0000","last_clean":"2024-09-11T01:22:18.950589+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:18.950589+0000","last_undegraded":"2024-09-11T01:22:18.950589+0000","last_fullsized":"2024-09-11T01:22:18.950589+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:30.037495+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:30.037495+0000","last_peered":"2024-09-11T01:20:30.037495+0000","last_clean":"2024-09-11T01:20:30.037495+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:30.037495+0000","last_undegraded":"2024-09-11T01:20:30.037495+0000","last_fullsized":"2024-09-11T01:20:30.037495+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":27603,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:33.224543+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:22:33.224543+0000","last_peered":"2024-09-11T01:22:33.224543+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:22:33.224543+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3640,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-401}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007616+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007616+0000","last_peered":"2024-09-11T01:20:22.007616+0000","last_clean":"2024-09-11T01:20:22.007616+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007616+0000","last_undegraded":"2024-09-11T01:20:22.007616+0000","last_fullsized":"2024-09-11T01:20:22.007616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008206+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008206+0000","last_peered":"2024-09-11T01:20:22.008206+0000","last_clean":"2024-09-11T01:20:22.008206+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008206+0000","last_undegraded":"2024-09-11T01:20:22.008206+0000","last_fullsized":"2024-09-11T01:20:22.008206+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20097,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.025596+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:19.025596+0000","last_peered":"2024-09-11T01:22:19.025596+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:19.025596+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32185,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15454,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195092480,"data_stored":195036816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182532,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518568,"kb_used_data":63512,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853272,"statfs":{"total":96636764160,"available":96105750528,"internally_reserved":0,"allocated":65036288,"data_stored":64756134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986305,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488412,"kb_used_data":127756,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883428,"statfs":{"total":96636764160,"available":96136630272,"internally_reserved":0,"allocated":130822144,"data_stored":130546067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659249,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469404,"kb_used_data":460,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93902436,"statfs":{"total":96636764160,"available":96156094464,"internally_reserved":0,"allocated":471040,"data_stored":145659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":480177839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:38.108 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:38.642 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:38.643 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:22:38.663 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":391,"stamp":"2024-09-11T01:22:36.919563+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32185,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15455,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":39,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1476384,"kb_used_data":191728,"kb_used_omap":12,"kb_used_meta":1284595,"kb_avail":281639136,"statfs":{"total":289910292480,"available":288398475264,"internally_reserved":0,"allocated":196329472,"data_stored":195447860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1315425284},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-611,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":611,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.114915"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":20354,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.943277+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:22:18.943277+0000","last_peered":"2024-09-11T01:22:18.943277+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:22:18.943277+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20464,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.964296+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:18.964296+0000","last_peered":"2024-09-11T01:22:18.964296+0000","last_clean":"2024-09-11T01:22:18.964296+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:18.964296+0000","last_undegraded":"2024-09-11T01:22:18.964296+0000","last_fullsized":"2024-09-11T01:22:18.964296+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20302,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:35.335864+0000","last_change":"2024-09-11T01:22:35.206502+0000","last_active":"2024-09-11T01:22:35.335864+0000","last_peered":"2024-09-11T01:22:35.335864+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:22:35.335864+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3888,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3888}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20214,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.872730+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:18.872730+0000","last_peered":"2024-09-11T01:22:18.872730+0000","last_clean":"2024-09-11T01:22:18.872730+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:18.872730+0000","last_undegraded":"2024-09-11T01:22:18.872730+0000","last_fullsized":"2024-09-11T01:22:18.872730+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20147,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:19.019037+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:19.019037+0000","last_peered":"2024-09-11T01:22:19.019037+0000","last_clean":"2024-09-11T01:22:19.019037+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:19.019037+0000","last_undegraded":"2024-09-11T01:22:19.019037+0000","last_fullsized":"2024-09-11T01:22:19.019037+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20408,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.034150+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:19.034150+0000","last_peered":"2024-09-11T01:22:19.034150+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:19.034150+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20071,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.026055+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:19.026055+0000","last_peered":"2024-09-11T01:22:19.026055+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:19.026055+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20394,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.910833+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:18.910833+0000","last_peered":"2024-09-11T01:22:18.910833+0000","last_clean":"2024-09-11T01:22:18.910833+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:18.910833+0000","last_undegraded":"2024-09-11T01:22:18.910833+0000","last_fullsized":"2024-09-11T01:22:18.910833+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003209+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:20:22.003209+0000","last_peered":"2024-09-11T01:20:22.003209+0000","last_clean":"2024-09-11T01:20:22.003209+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:20:22.003209+0000","last_undegraded":"2024-09-11T01:20:22.003209+0000","last_fullsized":"2024-09-11T01:20:22.003209+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":27992,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.993959+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:18.993959+0000","last_peered":"2024-09-11T01:22:18.993959+0000","last_clean":"2024-09-11T01:22:18.993959+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:18.993959+0000","last_undegraded":"2024-09-11T01:22:18.993959+0000","last_fullsized":"2024-09-11T01:22:18.993959+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:30.036418+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:30.036418+0000","last_peered":"2024-09-11T01:20:30.036418+0000","last_clean":"2024-09-11T01:20:30.036418+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:30.036418+0000","last_undegraded":"2024-09-11T01:20:30.036418+0000","last_fullsized":"2024-09-11T01:20:30.036418+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20007,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.022841+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:19.022841+0000","last_peered":"2024-09-11T01:22:19.022841+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:19.022841+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":112,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.005254+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:20:22.005254+0000","last_peered":"2024-09-11T01:20:22.005254+0000","last_clean":"2024-09-11T01:20:22.005254+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:20:22.005254+0000","last_undegraded":"2024-09-11T01:20:22.005254+0000","last_fullsized":"2024-09-11T01:20:22.005254+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20337,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.026024+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:19.026024+0000","last_peered":"2024-09-11T01:22:19.026024+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:19.026024+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":90,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.003275+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:20:22.003275+0000","last_peered":"2024-09-11T01:20:22.003275+0000","last_clean":"2024-09-11T01:20:22.003275+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:20:22.003275+0000","last_undegraded":"2024-09-11T01:20:22.003275+0000","last_fullsized":"2024-09-11T01:20:22.003275+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27788,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.973481+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:18.973481+0000","last_peered":"2024-09-11T01:22:18.973481+0000","last_clean":"2024-09-11T01:22:18.973481+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:18.973481+0000","last_undegraded":"2024-09-11T01:22:18.973481+0000","last_fullsized":"2024-09-11T01:22:18.973481+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20406,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:18.987858+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:18.987858+0000","last_peered":"2024-09-11T01:22:18.987858+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:18.987858+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007577+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:20:22.007577+0000","last_peered":"2024-09-11T01:20:22.007577+0000","last_clean":"2024-09-11T01:20:22.007577+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:20:22.007577+0000","last_undegraded":"2024-09-11T01:20:22.007577+0000","last_fullsized":"2024-09-11T01:20:22.007577+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27926,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.950589+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:18.950589+0000","last_peered":"2024-09-11T01:22:18.950589+0000","last_clean":"2024-09-11T01:22:18.950589+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:18.950589+0000","last_undegraded":"2024-09-11T01:22:18.950589+0000","last_fullsized":"2024-09-11T01:22:18.950589+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:30.037495+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:30.037495+0000","last_peered":"2024-09-11T01:20:30.037495+0000","last_clean":"2024-09-11T01:20:30.037495+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:30.037495+0000","last_undegraded":"2024-09-11T01:20:30.037495+0000","last_fullsized":"2024-09-11T01:20:30.037495+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":27603,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:33.224543+0000","last_change":"2024-09-11T01:20:20.985862+0000","last_active":"2024-09-11T01:22:33.224543+0000","last_peered":"2024-09-11T01:22:33.224543+0000","last_clean":"2024-09-11T01:13:07.635968+0000","last_became_active":"2024-09-11T01:13:10.221393+0000","last_became_peered":"2024-09-11T01:13:10.221393+0000","last_unstale":"2024-09-11T01:22:33.224543+0000","last_undegraded":"2024-09-11T01:13:09.793386+0000","last_fullsized":"2024-09-11T01:13:09.647461+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":401,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3640,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-401}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":80,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.007616+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:20:22.007616+0000","last_peered":"2024-09-11T01:20:22.007616+0000","last_clean":"2024-09-11T01:20:22.007616+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:20:22.007616+0000","last_undegraded":"2024-09-11T01:20:22.007616+0000","last_fullsized":"2024-09-11T01:20:22.007616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":85,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:22.008206+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:20:22.008206+0000","last_peered":"2024-09-11T01:20:22.008206+0000","last_clean":"2024-09-11T01:20:22.008206+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:20:22.008206+0000","last_undegraded":"2024-09-11T01:20:22.008206+0000","last_fullsized":"2024-09-11T01:20:22.008206+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20097,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.025596+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:19.025596+0000","last_peered":"2024-09-11T01:22:19.025596+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:19.025596+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":32185,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15454,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195092480,"data_stored":195036816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":23,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182532,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518568,"kb_used_data":63512,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853272,"statfs":{"total":96636764160,"available":96105750528,"internally_reserved":0,"allocated":65036288,"data_stored":64756134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986305,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488412,"kb_used_data":127756,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883428,"statfs":{"total":96636764160,"available":96136630272,"internally_reserved":0,"allocated":130822144,"data_stored":130546067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659249,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469404,"kb_used_data":460,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93902436,"statfs":{"total":96636764160,"available":96156094464,"internally_reserved":0,"allocated":471040,"data_stored":145659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":480177839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":8638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:38.664 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:22:38.664 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20266", 'reported_seq': 20354, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:18.943277+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:22:18.943277+0000', 'last_peered': '2024-09-11T01:22:18.943277+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:22:18.943277+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20266, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4072}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:38.664 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:22:38.664 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'20144", 'reported_seq': 20302, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:35.335864+0000', 'last_change': '2024-09-11T01:22:35.206502+0000', 'last_active': '2024-09-11T01:22:35.335864+0000', 'last_peered': '2024-09-11T01:22:35.335864+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:22:35.335864+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3888, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18445, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 10, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3888}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:38.664 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:22:38.664 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'20227", 'reported_seq': 20408, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:19.034150+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:22:19.034150+0000', 'last_peered': '2024-09-11T01:22:19.034150+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:22:19.034150+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16568, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:38.664 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:22:38.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20071, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:19.026055+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:22:19.026055+0000', 'last_peered': '2024-09-11T01:22:19.026055+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:22:19.026055+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:38.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:22:38.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20007, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:19.022841+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:22:19.022841+0000', 'last_peered': '2024-09-11T01:22:19.022841+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:22:19.022841+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:38.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:22:38.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'20207", 'reported_seq': 20337, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:19.026024+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:22:19.026024+0000', 'last_peered': '2024-09-11T01:22:19.026024+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:22:19.026024+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18024, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4024}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:38.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:22:38.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20264", 'reported_seq': 20406, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:18.987858+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:22:18.987858+0000', 'last_peered': '2024-09-11T01:22:18.987858+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:22:18.987858+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17001, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:38.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.5 is not active+clean 2024-09-11T01:22:38.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.5', 'version': "49'20176", 'reported_seq': 27603, 'reported_epoch': 49, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:33.224543+0000', 'last_change': '2024-09-11T01:20:20.985862+0000', 'last_active': '2024-09-11T01:22:33.224543+0000', 'last_peered': '2024-09-11T01:22:33.224543+0000', 'last_clean': '2024-09-11T01:13:07.635968+0000', 'last_became_active': '2024-09-11T01:13:10.221393+0000', 'last_became_peered': '2024-09-11T01:13:10.221393+0000', 'last_unstale': '2024-09-11T01:22:33.224543+0000', 'last_undegraded': '2024-09-11T01:13:09.793386+0000', 'last_fullsized': '2024-09-11T01:13:09.647461+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3438", 'last_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.824758+0000', 'objects_scrubbed': 3387, 'log_size': 10076, 'log_dups_size': 0, 'ondisk_log_size': 10076, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1663, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 401, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16785, 'num_write_kb': 26796, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3640, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -401}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:38.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:22:38.665 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 20097, 'reported_epoch': 49, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:19.025596+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:22:19.025596+0000', 'last_peered': '2024-09-11T01:22:19.025596+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:22:19.025596+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:38.666 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:22:39.154 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:39.154 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 31784/6 objects degraded (529733.333%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":695,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1516105728,"bytes_avail":288394186752,"bytes_total":289910292480,"degraded_objects":31784,"degraded_total":6,"degraded_ratio":5297.3333333333321,"recovering_objects_per_sec":126,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1012,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:22:39.179 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:22:39.180 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:39.496 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:39.496 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:22:39.515 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":392,"stamp":"2024-09-11T01:22:38.807910+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31784,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15856,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1480572,"kb_used_data":191756,"kb_used_omap":12,"kb_used_meta":1288755,"kb_avail":281634948,"statfs":{"total":289910292480,"available":288394186752,"internally_reserved":0,"allocated":196358144,"data_stored":195458921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1319685124},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1012,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1012,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002354"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":20358,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.924493+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:22:36.924493+0000","last_peered":"2024-09-11T01:22:36.924493+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:22:36.924493+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20464,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.964296+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:18.964296+0000","last_peered":"2024-09-11T01:22:18.964296+0000","last_clean":"2024-09-11T01:22:18.964296+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:18.964296+0000","last_undegraded":"2024-09-11T01:22:18.964296+0000","last_fullsized":"2024-09-11T01:22:18.964296+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20302,"reported_epoch":49,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:35.335864+0000","last_change":"2024-09-11T01:22:35.206502+0000","last_active":"2024-09-11T01:22:35.335864+0000","last_peered":"2024-09-11T01:22:35.335864+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:22:35.335864+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3888,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3888}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20214,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.872730+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:18.872730+0000","last_peered":"2024-09-11T01:22:18.872730+0000","last_clean":"2024-09-11T01:22:18.872730+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:18.872730+0000","last_undegraded":"2024-09-11T01:22:18.872730+0000","last_fullsized":"2024-09-11T01:22:18.872730+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20151,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.924815+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:36.924815+0000","last_peered":"2024-09-11T01:22:36.924815+0000","last_clean":"2024-09-11T01:22:36.924815+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:36.924815+0000","last_undegraded":"2024-09-11T01:22:36.924815+0000","last_fullsized":"2024-09-11T01:22:36.924815+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20412,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923407+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:36.923407+0000","last_peered":"2024-09-11T01:22:36.923407+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:36.923407+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20075,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923142+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:36.923142+0000","last_peered":"2024-09-11T01:22:36.923142+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:36.923142+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20398,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.923036+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:36.923036+0000","last_peered":"2024-09-11T01:22:36.923036+0000","last_clean":"2024-09-11T01:22:36.923036+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:36.923036+0000","last_undegraded":"2024-09-11T01:22:36.923036+0000","last_fullsized":"2024-09-11T01:22:36.923036+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.924438+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:36.924438+0000","last_peered":"2024-09-11T01:22:36.924438+0000","last_clean":"2024-09-11T01:22:36.924438+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:36.924438+0000","last_undegraded":"2024-09-11T01:22:36.924438+0000","last_fullsized":"2024-09-11T01:22:36.924438+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":27996,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.925734+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:36.925734+0000","last_peered":"2024-09-11T01:22:36.925734+0000","last_clean":"2024-09-11T01:22:36.925734+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:36.925734+0000","last_undegraded":"2024-09-11T01:22:36.925734+0000","last_fullsized":"2024-09-11T01:22:36.925734+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:30.036418+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:20:30.036418+0000","last_peered":"2024-09-11T01:20:30.036418+0000","last_clean":"2024-09-11T01:20:30.036418+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:20:30.036418+0000","last_undegraded":"2024-09-11T01:20:30.036418+0000","last_fullsized":"2024-09-11T01:20:30.036418+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20011,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.924926+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:36.924926+0000","last_peered":"2024-09-11T01:22:36.924926+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:36.924926+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.923268+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:36.923268+0000","last_peered":"2024-09-11T01:22:36.923268+0000","last_clean":"2024-09-11T01:22:36.923268+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:36.923268+0000","last_undegraded":"2024-09-11T01:22:36.923268+0000","last_fullsized":"2024-09-11T01:22:36.923268+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20337,"reported_epoch":49,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:19.026024+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:19.026024+0000","last_peered":"2024-09-11T01:22:19.026024+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:19.026024+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.925633+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:36.925633+0000","last_peered":"2024-09-11T01:22:36.925633+0000","last_clean":"2024-09-11T01:22:36.925633+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:36.925633+0000","last_undegraded":"2024-09-11T01:22:36.925633+0000","last_fullsized":"2024-09-11T01:22:36.925633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27788,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:22:18.973481+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:18.973481+0000","last_peered":"2024-09-11T01:22:18.973481+0000","last_clean":"2024-09-11T01:22:18.973481+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:18.973481+0000","last_undegraded":"2024-09-11T01:22:18.973481+0000","last_fullsized":"2024-09-11T01:22:18.973481+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20410,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923998+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:36.923998+0000","last_peered":"2024-09-11T01:22:36.923998+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:36.923998+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.923557+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:36.923557+0000","last_peered":"2024-09-11T01:22:36.923557+0000","last_clean":"2024-09-11T01:22:36.923557+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:36.923557+0000","last_undegraded":"2024-09-11T01:22:36.923557+0000","last_fullsized":"2024-09-11T01:22:36.923557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27930,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.927032+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:36.927032+0000","last_peered":"2024-09-11T01:22:36.927032+0000","last_clean":"2024-09-11T01:22:36.927032+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:36.927032+0000","last_undegraded":"2024-09-11T01:22:36.927032+0000","last_fullsized":"2024-09-11T01:22:36.927032+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":81,"reported_epoch":49,"state":"active+clean","last_fresh":"2024-09-11T01:20:30.037495+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:20:30.037495+0000","last_peered":"2024-09-11T01:20:30.037495+0000","last_clean":"2024-09-11T01:20:30.037495+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:20:30.037495+0000","last_undegraded":"2024-09-11T01:20:30.037495+0000","last_fullsized":"2024-09-11T01:20:30.037495+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24609,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:37.033944+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:37.033944+0000","last_peered":"2024-09-11T01:22:37.033944+0000","last_clean":"2024-09-11T01:22:37.033944+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:37.033944+0000","last_undegraded":"2024-09-11T01:22:37.033944+0000","last_fullsized":"2024-09-11T01:22:37.033944+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.926911+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:36.926911+0000","last_peered":"2024-09-11T01:22:36.926911+0000","last_clean":"2024-09-11T01:22:36.926911+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:36.926911+0000","last_undegraded":"2024-09-11T01:22:36.926911+0000","last_fullsized":"2024-09-11T01:22:36.926911+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.926929+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:36.926929+0000","last_peered":"2024-09-11T01:22:36.926929+0000","last_clean":"2024-09-11T01:22:36.926929+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:36.926929+0000","last_undegraded":"2024-09-11T01:22:36.926929+0000","last_fullsized":"2024-09-11T01:22:36.926929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20101,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923445+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:36.923445+0000","last_peered":"2024-09-11T01:22:36.923445+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:36.923445+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31784,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":15855,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195088384,"data_stored":195036199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182533,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518584,"kb_used_data":63528,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853256,"statfs":{"total":96636764160,"available":96105734144,"internally_reserved":0,"allocated":65052672,"data_stored":64761973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986305,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488412,"kb_used_data":127756,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883428,"statfs":{"total":96636764160,"available":96136630272,"internally_reserved":0,"allocated":130822144,"data_stored":130546067,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659250,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473576,"kb_used_data":472,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93898264,"statfs":{"total":96636764160,"available":96151822336,"internally_reserved":0,"allocated":483328,"data_stored":150881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":484437679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":8021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:42.517 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:42.895 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:42.895 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:22:42.914 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":394,"stamp":"2024-09-11T01:22:42.809322+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31141,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16499,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1480588,"kb_used_data":191772,"kb_used_omap":12,"kb_used_meta":1288755,"kb_avail":281634932,"statfs":{"total":289910292480,"available":288394170368,"internally_reserved":0,"allocated":196374528,"data_stored":195464760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1319685124},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1053,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1053,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002183"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":20358,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.924493+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:22:36.924493+0000","last_peered":"2024-09-11T01:22:36.924493+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:22:36.924493+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20468,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.929677+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:36.929677+0000","last_peered":"2024-09-11T01:22:36.929677+0000","last_clean":"2024-09-11T01:22:36.929677+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:36.929677+0000","last_undegraded":"2024-09-11T01:22:36.929677+0000","last_fullsized":"2024-09-11T01:22:36.929677+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":21592,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:40.342940+0000","last_change":"2024-09-11T01:22:35.206502+0000","last_active":"2024-09-11T01:22:40.342940+0000","last_peered":"2024-09-11T01:22:40.342940+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:22:40.342940+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3245,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":653,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3245}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20218,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.929591+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:36.929591+0000","last_peered":"2024-09-11T01:22:36.929591+0000","last_clean":"2024-09-11T01:22:36.929591+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:36.929591+0000","last_undegraded":"2024-09-11T01:22:36.929591+0000","last_fullsized":"2024-09-11T01:22:36.929591+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20151,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.924815+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:36.924815+0000","last_peered":"2024-09-11T01:22:36.924815+0000","last_clean":"2024-09-11T01:22:36.924815+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:36.924815+0000","last_undegraded":"2024-09-11T01:22:36.924815+0000","last_fullsized":"2024-09-11T01:22:36.924815+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20412,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923407+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:36.923407+0000","last_peered":"2024-09-11T01:22:36.923407+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:36.923407+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20075,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923142+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:36.923142+0000","last_peered":"2024-09-11T01:22:36.923142+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:36.923142+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20398,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.923036+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:36.923036+0000","last_peered":"2024-09-11T01:22:36.923036+0000","last_clean":"2024-09-11T01:22:36.923036+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:36.923036+0000","last_undegraded":"2024-09-11T01:22:36.923036+0000","last_fullsized":"2024-09-11T01:22:36.923036+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.924438+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:36.924438+0000","last_peered":"2024-09-11T01:22:36.924438+0000","last_clean":"2024-09-11T01:22:36.924438+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:36.924438+0000","last_undegraded":"2024-09-11T01:22:36.924438+0000","last_fullsized":"2024-09-11T01:22:36.924438+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":27996,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.925734+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:36.925734+0000","last_peered":"2024-09-11T01:22:36.925734+0000","last_clean":"2024-09-11T01:22:36.925734+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:36.925734+0000","last_undegraded":"2024-09-11T01:22:36.925734+0000","last_fullsized":"2024-09-11T01:22:36.925734+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.929215+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:36.929215+0000","last_peered":"2024-09-11T01:22:36.929215+0000","last_clean":"2024-09-11T01:22:36.929215+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:36.929215+0000","last_undegraded":"2024-09-11T01:22:36.929215+0000","last_fullsized":"2024-09-11T01:22:36.929215+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20011,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.924926+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:36.924926+0000","last_peered":"2024-09-11T01:22:36.924926+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:36.924926+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.923268+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:36.923268+0000","last_peered":"2024-09-11T01:22:36.923268+0000","last_clean":"2024-09-11T01:22:36.923268+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:36.923268+0000","last_undegraded":"2024-09-11T01:22:36.923268+0000","last_fullsized":"2024-09-11T01:22:36.923268+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20341,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.929381+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:36.929381+0000","last_peered":"2024-09-11T01:22:36.929381+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:36.929381+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.925633+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:36.925633+0000","last_peered":"2024-09-11T01:22:36.925633+0000","last_clean":"2024-09-11T01:22:36.925633+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:36.925633+0000","last_undegraded":"2024-09-11T01:22:36.925633+0000","last_fullsized":"2024-09-11T01:22:36.925633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27792,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.929150+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:36.929150+0000","last_peered":"2024-09-11T01:22:36.929150+0000","last_clean":"2024-09-11T01:22:36.929150+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:36.929150+0000","last_undegraded":"2024-09-11T01:22:36.929150+0000","last_fullsized":"2024-09-11T01:22:36.929150+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20410,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923998+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:36.923998+0000","last_peered":"2024-09-11T01:22:36.923998+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:36.923998+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.923557+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:36.923557+0000","last_peered":"2024-09-11T01:22:36.923557+0000","last_clean":"2024-09-11T01:22:36.923557+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:36.923557+0000","last_undegraded":"2024-09-11T01:22:36.923557+0000","last_fullsized":"2024-09-11T01:22:36.923557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27930,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.927032+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:36.927032+0000","last_peered":"2024-09-11T01:22:36.927032+0000","last_clean":"2024-09-11T01:22:36.927032+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:36.927032+0000","last_undegraded":"2024-09-11T01:22:36.927032+0000","last_fullsized":"2024-09-11T01:22:36.927032+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.933836+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:36.933836+0000","last_peered":"2024-09-11T01:22:36.933836+0000","last_clean":"2024-09-11T01:22:36.933836+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:36.933836+0000","last_undegraded":"2024-09-11T01:22:36.933836+0000","last_fullsized":"2024-09-11T01:22:36.933836+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24609,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:37.033944+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:37.033944+0000","last_peered":"2024-09-11T01:22:37.033944+0000","last_clean":"2024-09-11T01:22:37.033944+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:37.033944+0000","last_undegraded":"2024-09-11T01:22:37.033944+0000","last_fullsized":"2024-09-11T01:22:37.033944+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.926911+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:36.926911+0000","last_peered":"2024-09-11T01:22:36.926911+0000","last_clean":"2024-09-11T01:22:36.926911+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:36.926911+0000","last_undegraded":"2024-09-11T01:22:36.926911+0000","last_fullsized":"2024-09-11T01:22:36.926911+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.926929+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:36.926929+0000","last_peered":"2024-09-11T01:22:36.926929+0000","last_clean":"2024-09-11T01:22:36.926929+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:36.926929+0000","last_undegraded":"2024-09-11T01:22:36.926929+0000","last_fullsized":"2024-09-11T01:22:36.926929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20101,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923445+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:36.923445+0000","last_peered":"2024-09-11T01:22:36.923445+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:36.923445+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31141,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16498,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195088384,"data_stored":195036199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182533,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518584,"kb_used_data":63528,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853256,"statfs":{"total":96636764160,"available":96105734144,"internally_reserved":0,"allocated":65052672,"data_stored":64761973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986306,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488428,"kb_used_data":127772,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883412,"statfs":{"total":96636764160,"available":96136613888,"internally_reserved":0,"allocated":130838528,"data_stored":130551906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659250,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473576,"kb_used_data":472,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93898264,"statfs":{"total":96636764160,"available":96151822336,"internally_reserved":0,"allocated":483328,"data_stored":150881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":484437679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":8021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:42.915 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:43.255 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:43.255 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:22:43.275 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":394,"stamp":"2024-09-11T01:22:42.809322+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31141,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16499,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1480588,"kb_used_data":191772,"kb_used_omap":12,"kb_used_meta":1288755,"kb_avail":281634932,"statfs":{"total":289910292480,"available":288394170368,"internally_reserved":0,"allocated":196374528,"data_stored":195464760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1319685124},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1053,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1053,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002183"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":20358,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.924493+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:22:36.924493+0000","last_peered":"2024-09-11T01:22:36.924493+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:22:36.924493+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20468,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.929677+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:36.929677+0000","last_peered":"2024-09-11T01:22:36.929677+0000","last_clean":"2024-09-11T01:22:36.929677+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:36.929677+0000","last_undegraded":"2024-09-11T01:22:36.929677+0000","last_fullsized":"2024-09-11T01:22:36.929677+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":21592,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:40.342940+0000","last_change":"2024-09-11T01:22:35.206502+0000","last_active":"2024-09-11T01:22:40.342940+0000","last_peered":"2024-09-11T01:22:40.342940+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:22:40.342940+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3245,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":653,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3245}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20218,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.929591+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:36.929591+0000","last_peered":"2024-09-11T01:22:36.929591+0000","last_clean":"2024-09-11T01:22:36.929591+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:36.929591+0000","last_undegraded":"2024-09-11T01:22:36.929591+0000","last_fullsized":"2024-09-11T01:22:36.929591+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20151,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.924815+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:36.924815+0000","last_peered":"2024-09-11T01:22:36.924815+0000","last_clean":"2024-09-11T01:22:36.924815+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:36.924815+0000","last_undegraded":"2024-09-11T01:22:36.924815+0000","last_fullsized":"2024-09-11T01:22:36.924815+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20412,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923407+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:36.923407+0000","last_peered":"2024-09-11T01:22:36.923407+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:36.923407+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20075,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923142+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:36.923142+0000","last_peered":"2024-09-11T01:22:36.923142+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:36.923142+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20398,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.923036+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:36.923036+0000","last_peered":"2024-09-11T01:22:36.923036+0000","last_clean":"2024-09-11T01:22:36.923036+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:36.923036+0000","last_undegraded":"2024-09-11T01:22:36.923036+0000","last_fullsized":"2024-09-11T01:22:36.923036+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.924438+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:36.924438+0000","last_peered":"2024-09-11T01:22:36.924438+0000","last_clean":"2024-09-11T01:22:36.924438+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:36.924438+0000","last_undegraded":"2024-09-11T01:22:36.924438+0000","last_fullsized":"2024-09-11T01:22:36.924438+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":27996,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.925734+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:36.925734+0000","last_peered":"2024-09-11T01:22:36.925734+0000","last_clean":"2024-09-11T01:22:36.925734+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:36.925734+0000","last_undegraded":"2024-09-11T01:22:36.925734+0000","last_fullsized":"2024-09-11T01:22:36.925734+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.929215+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:36.929215+0000","last_peered":"2024-09-11T01:22:36.929215+0000","last_clean":"2024-09-11T01:22:36.929215+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:36.929215+0000","last_undegraded":"2024-09-11T01:22:36.929215+0000","last_fullsized":"2024-09-11T01:22:36.929215+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20011,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.924926+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:36.924926+0000","last_peered":"2024-09-11T01:22:36.924926+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:36.924926+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.923268+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:36.923268+0000","last_peered":"2024-09-11T01:22:36.923268+0000","last_clean":"2024-09-11T01:22:36.923268+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:36.923268+0000","last_undegraded":"2024-09-11T01:22:36.923268+0000","last_fullsized":"2024-09-11T01:22:36.923268+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20341,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.929381+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:36.929381+0000","last_peered":"2024-09-11T01:22:36.929381+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:36.929381+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.925633+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:36.925633+0000","last_peered":"2024-09-11T01:22:36.925633+0000","last_clean":"2024-09-11T01:22:36.925633+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:36.925633+0000","last_undegraded":"2024-09-11T01:22:36.925633+0000","last_fullsized":"2024-09-11T01:22:36.925633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27792,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.929150+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:36.929150+0000","last_peered":"2024-09-11T01:22:36.929150+0000","last_clean":"2024-09-11T01:22:36.929150+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:36.929150+0000","last_undegraded":"2024-09-11T01:22:36.929150+0000","last_fullsized":"2024-09-11T01:22:36.929150+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20410,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923998+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:36.923998+0000","last_peered":"2024-09-11T01:22:36.923998+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:36.923998+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.923557+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:36.923557+0000","last_peered":"2024-09-11T01:22:36.923557+0000","last_clean":"2024-09-11T01:22:36.923557+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:36.923557+0000","last_undegraded":"2024-09-11T01:22:36.923557+0000","last_fullsized":"2024-09-11T01:22:36.923557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27930,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.927032+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:36.927032+0000","last_peered":"2024-09-11T01:22:36.927032+0000","last_clean":"2024-09-11T01:22:36.927032+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:36.927032+0000","last_undegraded":"2024-09-11T01:22:36.927032+0000","last_fullsized":"2024-09-11T01:22:36.927032+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.933836+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:36.933836+0000","last_peered":"2024-09-11T01:22:36.933836+0000","last_clean":"2024-09-11T01:22:36.933836+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:36.933836+0000","last_undegraded":"2024-09-11T01:22:36.933836+0000","last_fullsized":"2024-09-11T01:22:36.933836+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24609,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:37.033944+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:37.033944+0000","last_peered":"2024-09-11T01:22:37.033944+0000","last_clean":"2024-09-11T01:22:37.033944+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:37.033944+0000","last_undegraded":"2024-09-11T01:22:37.033944+0000","last_fullsized":"2024-09-11T01:22:37.033944+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.926911+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:36.926911+0000","last_peered":"2024-09-11T01:22:36.926911+0000","last_clean":"2024-09-11T01:22:36.926911+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:36.926911+0000","last_undegraded":"2024-09-11T01:22:36.926911+0000","last_fullsized":"2024-09-11T01:22:36.926911+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.926929+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:36.926929+0000","last_peered":"2024-09-11T01:22:36.926929+0000","last_clean":"2024-09-11T01:22:36.926929+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:36.926929+0000","last_undegraded":"2024-09-11T01:22:36.926929+0000","last_fullsized":"2024-09-11T01:22:36.926929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20101,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923445+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:36.923445+0000","last_peered":"2024-09-11T01:22:36.923445+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:36.923445+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31141,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16498,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195088384,"data_stored":195036199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182533,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518584,"kb_used_data":63528,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853256,"statfs":{"total":96636764160,"available":96105734144,"internally_reserved":0,"allocated":65052672,"data_stored":64761973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986306,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488428,"kb_used_data":127772,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883412,"statfs":{"total":96636764160,"available":96136613888,"internally_reserved":0,"allocated":130838528,"data_stored":130551906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659250,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473576,"kb_used_data":472,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93898264,"statfs":{"total":96636764160,"available":96151822336,"internally_reserved":0,"allocated":483328,"data_stored":150881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":484437679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":8021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:43.276 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:22:43.276 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20266", 'reported_seq': 20358, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:36.924493+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:22:36.924493+0000', 'last_peered': '2024-09-11T01:22:36.924493+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:22:36.924493+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20266, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4072}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:43.276 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:22:43.276 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'20144", 'reported_seq': 21592, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:40.342940+0000', 'last_change': '2024-09-11T01:22:35.206502+0000', 'last_active': '2024-09-11T01:22:40.342940+0000', 'last_peered': '2024-09-11T01:22:40.342940+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:22:40.342940+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3245, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18445, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 653, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3245}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:43.277 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:22:43.277 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'20227", 'reported_seq': 20412, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:36.923407+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:22:36.923407+0000', 'last_peered': '2024-09-11T01:22:36.923407+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:22:36.923407+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16568, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:43.277 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:22:43.277 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20075, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:36.923142+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:22:36.923142+0000', 'last_peered': '2024-09-11T01:22:36.923142+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:22:36.923142+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:43.277 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:22:43.277 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20011, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:36.924926+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:22:36.924926+0000', 'last_peered': '2024-09-11T01:22:36.924926+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:22:36.924926+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:43.277 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:22:43.277 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'20207", 'reported_seq': 20341, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:36.929381+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:22:36.929381+0000', 'last_peered': '2024-09-11T01:22:36.929381+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:22:36.929381+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18024, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4024}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:43.277 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:22:43.277 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20264", 'reported_seq': 20410, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:36.923998+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:22:36.923998+0000', 'last_peered': '2024-09-11T01:22:36.923998+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:22:36.923998+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17001, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:43.277 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:22:43.278 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 20101, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:36.923445+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:22:36.923445+0000', 'last_peered': '2024-09-11T01:22:36.923445+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:22:36.923445+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:43.278 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:22:43.706 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:43.706 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 31141/6 objects degraded (519016.667%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":700,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1516122112,"bytes_avail":288394170368,"bytes_total":289910292480,"degraded_objects":31141,"degraded_total":6,"degraded_ratio":5190.166666666667,"recovering_objects_per_sec":131,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1053,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:22:43.726 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:22:43.726 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:44.083 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:44.083 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:22:44.101 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":394,"stamp":"2024-09-11T01:22:42.809322+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31141,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16499,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1480588,"kb_used_data":191772,"kb_used_omap":12,"kb_used_meta":1288755,"kb_avail":281634932,"statfs":{"total":289910292480,"available":288394170368,"internally_reserved":0,"allocated":196374528,"data_stored":195464760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1319685124},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1053,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1053,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002183"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":20358,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.924493+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:22:36.924493+0000","last_peered":"2024-09-11T01:22:36.924493+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:22:36.924493+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20468,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.929677+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:36.929677+0000","last_peered":"2024-09-11T01:22:36.929677+0000","last_clean":"2024-09-11T01:22:36.929677+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:36.929677+0000","last_undegraded":"2024-09-11T01:22:36.929677+0000","last_fullsized":"2024-09-11T01:22:36.929677+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":21592,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:40.342940+0000","last_change":"2024-09-11T01:22:35.206502+0000","last_active":"2024-09-11T01:22:40.342940+0000","last_peered":"2024-09-11T01:22:40.342940+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:22:40.342940+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3245,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":653,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3245}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20218,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.929591+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:36.929591+0000","last_peered":"2024-09-11T01:22:36.929591+0000","last_clean":"2024-09-11T01:22:36.929591+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:36.929591+0000","last_undegraded":"2024-09-11T01:22:36.929591+0000","last_fullsized":"2024-09-11T01:22:36.929591+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20151,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.924815+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:36.924815+0000","last_peered":"2024-09-11T01:22:36.924815+0000","last_clean":"2024-09-11T01:22:36.924815+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:36.924815+0000","last_undegraded":"2024-09-11T01:22:36.924815+0000","last_fullsized":"2024-09-11T01:22:36.924815+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20412,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923407+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:36.923407+0000","last_peered":"2024-09-11T01:22:36.923407+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:36.923407+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20075,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923142+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:36.923142+0000","last_peered":"2024-09-11T01:22:36.923142+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:36.923142+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20398,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.923036+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:36.923036+0000","last_peered":"2024-09-11T01:22:36.923036+0000","last_clean":"2024-09-11T01:22:36.923036+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:36.923036+0000","last_undegraded":"2024-09-11T01:22:36.923036+0000","last_fullsized":"2024-09-11T01:22:36.923036+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.924438+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:36.924438+0000","last_peered":"2024-09-11T01:22:36.924438+0000","last_clean":"2024-09-11T01:22:36.924438+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:36.924438+0000","last_undegraded":"2024-09-11T01:22:36.924438+0000","last_fullsized":"2024-09-11T01:22:36.924438+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":27996,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.925734+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:36.925734+0000","last_peered":"2024-09-11T01:22:36.925734+0000","last_clean":"2024-09-11T01:22:36.925734+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:36.925734+0000","last_undegraded":"2024-09-11T01:22:36.925734+0000","last_fullsized":"2024-09-11T01:22:36.925734+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.929215+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:36.929215+0000","last_peered":"2024-09-11T01:22:36.929215+0000","last_clean":"2024-09-11T01:22:36.929215+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:36.929215+0000","last_undegraded":"2024-09-11T01:22:36.929215+0000","last_fullsized":"2024-09-11T01:22:36.929215+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20011,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.924926+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:36.924926+0000","last_peered":"2024-09-11T01:22:36.924926+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:36.924926+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.923268+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:36.923268+0000","last_peered":"2024-09-11T01:22:36.923268+0000","last_clean":"2024-09-11T01:22:36.923268+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:36.923268+0000","last_undegraded":"2024-09-11T01:22:36.923268+0000","last_fullsized":"2024-09-11T01:22:36.923268+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20341,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.929381+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:36.929381+0000","last_peered":"2024-09-11T01:22:36.929381+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:36.929381+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.925633+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:36.925633+0000","last_peered":"2024-09-11T01:22:36.925633+0000","last_clean":"2024-09-11T01:22:36.925633+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:36.925633+0000","last_undegraded":"2024-09-11T01:22:36.925633+0000","last_fullsized":"2024-09-11T01:22:36.925633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27792,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.929150+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:36.929150+0000","last_peered":"2024-09-11T01:22:36.929150+0000","last_clean":"2024-09-11T01:22:36.929150+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:36.929150+0000","last_undegraded":"2024-09-11T01:22:36.929150+0000","last_fullsized":"2024-09-11T01:22:36.929150+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20410,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923998+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:36.923998+0000","last_peered":"2024-09-11T01:22:36.923998+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:36.923998+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.923557+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:36.923557+0000","last_peered":"2024-09-11T01:22:36.923557+0000","last_clean":"2024-09-11T01:22:36.923557+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:36.923557+0000","last_undegraded":"2024-09-11T01:22:36.923557+0000","last_fullsized":"2024-09-11T01:22:36.923557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27930,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.927032+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:36.927032+0000","last_peered":"2024-09-11T01:22:36.927032+0000","last_clean":"2024-09-11T01:22:36.927032+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:36.927032+0000","last_undegraded":"2024-09-11T01:22:36.927032+0000","last_fullsized":"2024-09-11T01:22:36.927032+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.933836+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:36.933836+0000","last_peered":"2024-09-11T01:22:36.933836+0000","last_clean":"2024-09-11T01:22:36.933836+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:36.933836+0000","last_undegraded":"2024-09-11T01:22:36.933836+0000","last_fullsized":"2024-09-11T01:22:36.933836+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24609,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:37.033944+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:37.033944+0000","last_peered":"2024-09-11T01:22:37.033944+0000","last_clean":"2024-09-11T01:22:37.033944+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:37.033944+0000","last_undegraded":"2024-09-11T01:22:37.033944+0000","last_fullsized":"2024-09-11T01:22:37.033944+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.926911+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:36.926911+0000","last_peered":"2024-09-11T01:22:36.926911+0000","last_clean":"2024-09-11T01:22:36.926911+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:36.926911+0000","last_undegraded":"2024-09-11T01:22:36.926911+0000","last_fullsized":"2024-09-11T01:22:36.926911+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.926929+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:36.926929+0000","last_peered":"2024-09-11T01:22:36.926929+0000","last_clean":"2024-09-11T01:22:36.926929+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:36.926929+0000","last_undegraded":"2024-09-11T01:22:36.926929+0000","last_fullsized":"2024-09-11T01:22:36.926929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20101,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923445+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:36.923445+0000","last_peered":"2024-09-11T01:22:36.923445+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:36.923445+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":31141,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":16498,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195088384,"data_stored":195036199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182533,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518584,"kb_used_data":63528,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853256,"statfs":{"total":96636764160,"available":96105734144,"internally_reserved":0,"allocated":65052672,"data_stored":64761973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986306,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488428,"kb_used_data":127772,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883412,"statfs":{"total":96636764160,"available":96136613888,"internally_reserved":0,"allocated":130838528,"data_stored":130551906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659250,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473576,"kb_used_data":472,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93898264,"statfs":{"total":96636764160,"available":96151822336,"internally_reserved":0,"allocated":483328,"data_stored":150881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":484437679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":8021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:47.103 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:47.456 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:47.456 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:22:47.475 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":396,"stamp":"2024-09-11T01:22:46.810561+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30223,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17417,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472268,"kb_used_data":191772,"kb_used_omap":12,"kb_used_meta":1280435,"kb_avail":281643252,"statfs":{"total":289910292480,"available":288402690048,"internally_reserved":0,"allocated":196374528,"data_stored":195464760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1311165444},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1962,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1962,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003065"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":20358,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.924493+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:22:36.924493+0000","last_peered":"2024-09-11T01:22:36.924493+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:22:36.924493+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20468,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.929677+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:36.929677+0000","last_peered":"2024-09-11T01:22:36.929677+0000","last_clean":"2024-09-11T01:22:36.929677+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:36.929677+0000","last_undegraded":"2024-09-11T01:22:36.929677+0000","last_fullsized":"2024-09-11T01:22:36.929677+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":23428,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:45.345874+0000","last_change":"2024-09-11T01:22:35.206502+0000","last_active":"2024-09-11T01:22:45.345874+0000","last_peered":"2024-09-11T01:22:45.345874+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:22:45.345874+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1571,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2327}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20218,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.929591+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:36.929591+0000","last_peered":"2024-09-11T01:22:36.929591+0000","last_clean":"2024-09-11T01:22:36.929591+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:36.929591+0000","last_undegraded":"2024-09-11T01:22:36.929591+0000","last_fullsized":"2024-09-11T01:22:36.929591+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20151,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.924815+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:36.924815+0000","last_peered":"2024-09-11T01:22:36.924815+0000","last_clean":"2024-09-11T01:22:36.924815+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:36.924815+0000","last_undegraded":"2024-09-11T01:22:36.924815+0000","last_fullsized":"2024-09-11T01:22:36.924815+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20412,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923407+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:36.923407+0000","last_peered":"2024-09-11T01:22:36.923407+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:36.923407+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20075,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923142+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:36.923142+0000","last_peered":"2024-09-11T01:22:36.923142+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:36.923142+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20398,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.923036+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:36.923036+0000","last_peered":"2024-09-11T01:22:36.923036+0000","last_clean":"2024-09-11T01:22:36.923036+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:36.923036+0000","last_undegraded":"2024-09-11T01:22:36.923036+0000","last_fullsized":"2024-09-11T01:22:36.923036+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.924438+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:36.924438+0000","last_peered":"2024-09-11T01:22:36.924438+0000","last_clean":"2024-09-11T01:22:36.924438+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:36.924438+0000","last_undegraded":"2024-09-11T01:22:36.924438+0000","last_fullsized":"2024-09-11T01:22:36.924438+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":27996,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.925734+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:36.925734+0000","last_peered":"2024-09-11T01:22:36.925734+0000","last_clean":"2024-09-11T01:22:36.925734+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:36.925734+0000","last_undegraded":"2024-09-11T01:22:36.925734+0000","last_fullsized":"2024-09-11T01:22:36.925734+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.929215+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:36.929215+0000","last_peered":"2024-09-11T01:22:36.929215+0000","last_clean":"2024-09-11T01:22:36.929215+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:36.929215+0000","last_undegraded":"2024-09-11T01:22:36.929215+0000","last_fullsized":"2024-09-11T01:22:36.929215+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20011,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.924926+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:36.924926+0000","last_peered":"2024-09-11T01:22:36.924926+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:36.924926+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.923268+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:36.923268+0000","last_peered":"2024-09-11T01:22:36.923268+0000","last_clean":"2024-09-11T01:22:36.923268+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:36.923268+0000","last_undegraded":"2024-09-11T01:22:36.923268+0000","last_fullsized":"2024-09-11T01:22:36.923268+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20341,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.929381+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:36.929381+0000","last_peered":"2024-09-11T01:22:36.929381+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:36.929381+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.925633+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:36.925633+0000","last_peered":"2024-09-11T01:22:36.925633+0000","last_clean":"2024-09-11T01:22:36.925633+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:36.925633+0000","last_undegraded":"2024-09-11T01:22:36.925633+0000","last_fullsized":"2024-09-11T01:22:36.925633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27792,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.929150+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:36.929150+0000","last_peered":"2024-09-11T01:22:36.929150+0000","last_clean":"2024-09-11T01:22:36.929150+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:36.929150+0000","last_undegraded":"2024-09-11T01:22:36.929150+0000","last_fullsized":"2024-09-11T01:22:36.929150+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20410,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923998+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:36.923998+0000","last_peered":"2024-09-11T01:22:36.923998+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:36.923998+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.923557+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:36.923557+0000","last_peered":"2024-09-11T01:22:36.923557+0000","last_clean":"2024-09-11T01:22:36.923557+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:36.923557+0000","last_undegraded":"2024-09-11T01:22:36.923557+0000","last_fullsized":"2024-09-11T01:22:36.923557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27930,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.927032+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:36.927032+0000","last_peered":"2024-09-11T01:22:36.927032+0000","last_clean":"2024-09-11T01:22:36.927032+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:36.927032+0000","last_undegraded":"2024-09-11T01:22:36.927032+0000","last_fullsized":"2024-09-11T01:22:36.927032+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.933836+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:36.933836+0000","last_peered":"2024-09-11T01:22:36.933836+0000","last_clean":"2024-09-11T01:22:36.933836+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:36.933836+0000","last_undegraded":"2024-09-11T01:22:36.933836+0000","last_fullsized":"2024-09-11T01:22:36.933836+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24609,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:37.033944+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:37.033944+0000","last_peered":"2024-09-11T01:22:37.033944+0000","last_clean":"2024-09-11T01:22:37.033944+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:37.033944+0000","last_undegraded":"2024-09-11T01:22:37.033944+0000","last_fullsized":"2024-09-11T01:22:37.033944+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.926911+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:36.926911+0000","last_peered":"2024-09-11T01:22:36.926911+0000","last_clean":"2024-09-11T01:22:36.926911+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:36.926911+0000","last_undegraded":"2024-09-11T01:22:36.926911+0000","last_fullsized":"2024-09-11T01:22:36.926911+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.926929+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:36.926929+0000","last_peered":"2024-09-11T01:22:36.926929+0000","last_clean":"2024-09-11T01:22:36.926929+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:36.926929+0000","last_undegraded":"2024-09-11T01:22:36.926929+0000","last_fullsized":"2024-09-11T01:22:36.926929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20101,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923445+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:36.923445+0000","last_peered":"2024-09-11T01:22:36.923445+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:36.923445+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30223,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17416,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195088384,"data_stored":195036199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182534,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518584,"kb_used_data":63528,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853256,"statfs":{"total":96636764160,"available":96105734144,"internally_reserved":0,"allocated":65052672,"data_stored":64761973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986307,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488428,"kb_used_data":127772,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883412,"statfs":{"total":96636764160,"available":96136613888,"internally_reserved":0,"allocated":130838528,"data_stored":130551906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659251,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465256,"kb_used_data":472,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93906584,"statfs":{"total":96636764160,"available":96160342016,"internally_reserved":0,"allocated":483328,"data_stored":150881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475917999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":8021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:47.477 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:47.815 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:47.815 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:22:47.834 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":396,"stamp":"2024-09-11T01:22:46.810561+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30223,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17417,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472268,"kb_used_data":191772,"kb_used_omap":12,"kb_used_meta":1280435,"kb_avail":281643252,"statfs":{"total":289910292480,"available":288402690048,"internally_reserved":0,"allocated":196374528,"data_stored":195464760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1311165444},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1962,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1962,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003065"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":20358,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.924493+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:22:36.924493+0000","last_peered":"2024-09-11T01:22:36.924493+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:22:36.924493+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20468,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.929677+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:36.929677+0000","last_peered":"2024-09-11T01:22:36.929677+0000","last_clean":"2024-09-11T01:22:36.929677+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:36.929677+0000","last_undegraded":"2024-09-11T01:22:36.929677+0000","last_fullsized":"2024-09-11T01:22:36.929677+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":23428,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:45.345874+0000","last_change":"2024-09-11T01:22:35.206502+0000","last_active":"2024-09-11T01:22:45.345874+0000","last_peered":"2024-09-11T01:22:45.345874+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:22:45.345874+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1571,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2327}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20218,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.929591+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:36.929591+0000","last_peered":"2024-09-11T01:22:36.929591+0000","last_clean":"2024-09-11T01:22:36.929591+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:36.929591+0000","last_undegraded":"2024-09-11T01:22:36.929591+0000","last_fullsized":"2024-09-11T01:22:36.929591+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20151,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.924815+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:36.924815+0000","last_peered":"2024-09-11T01:22:36.924815+0000","last_clean":"2024-09-11T01:22:36.924815+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:36.924815+0000","last_undegraded":"2024-09-11T01:22:36.924815+0000","last_fullsized":"2024-09-11T01:22:36.924815+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20412,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923407+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:36.923407+0000","last_peered":"2024-09-11T01:22:36.923407+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:36.923407+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20075,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923142+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:36.923142+0000","last_peered":"2024-09-11T01:22:36.923142+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:36.923142+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20398,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.923036+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:36.923036+0000","last_peered":"2024-09-11T01:22:36.923036+0000","last_clean":"2024-09-11T01:22:36.923036+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:36.923036+0000","last_undegraded":"2024-09-11T01:22:36.923036+0000","last_fullsized":"2024-09-11T01:22:36.923036+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.924438+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:36.924438+0000","last_peered":"2024-09-11T01:22:36.924438+0000","last_clean":"2024-09-11T01:22:36.924438+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:36.924438+0000","last_undegraded":"2024-09-11T01:22:36.924438+0000","last_fullsized":"2024-09-11T01:22:36.924438+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":27996,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.925734+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:36.925734+0000","last_peered":"2024-09-11T01:22:36.925734+0000","last_clean":"2024-09-11T01:22:36.925734+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:36.925734+0000","last_undegraded":"2024-09-11T01:22:36.925734+0000","last_fullsized":"2024-09-11T01:22:36.925734+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.929215+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:36.929215+0000","last_peered":"2024-09-11T01:22:36.929215+0000","last_clean":"2024-09-11T01:22:36.929215+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:36.929215+0000","last_undegraded":"2024-09-11T01:22:36.929215+0000","last_fullsized":"2024-09-11T01:22:36.929215+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20011,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.924926+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:36.924926+0000","last_peered":"2024-09-11T01:22:36.924926+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:36.924926+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.923268+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:36.923268+0000","last_peered":"2024-09-11T01:22:36.923268+0000","last_clean":"2024-09-11T01:22:36.923268+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:36.923268+0000","last_undegraded":"2024-09-11T01:22:36.923268+0000","last_fullsized":"2024-09-11T01:22:36.923268+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20341,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.929381+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:36.929381+0000","last_peered":"2024-09-11T01:22:36.929381+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:36.929381+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.925633+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:36.925633+0000","last_peered":"2024-09-11T01:22:36.925633+0000","last_clean":"2024-09-11T01:22:36.925633+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:36.925633+0000","last_undegraded":"2024-09-11T01:22:36.925633+0000","last_fullsized":"2024-09-11T01:22:36.925633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27792,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.929150+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:36.929150+0000","last_peered":"2024-09-11T01:22:36.929150+0000","last_clean":"2024-09-11T01:22:36.929150+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:36.929150+0000","last_undegraded":"2024-09-11T01:22:36.929150+0000","last_fullsized":"2024-09-11T01:22:36.929150+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20410,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923998+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:36.923998+0000","last_peered":"2024-09-11T01:22:36.923998+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:36.923998+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.923557+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:36.923557+0000","last_peered":"2024-09-11T01:22:36.923557+0000","last_clean":"2024-09-11T01:22:36.923557+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:36.923557+0000","last_undegraded":"2024-09-11T01:22:36.923557+0000","last_fullsized":"2024-09-11T01:22:36.923557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27930,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.927032+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:36.927032+0000","last_peered":"2024-09-11T01:22:36.927032+0000","last_clean":"2024-09-11T01:22:36.927032+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:36.927032+0000","last_undegraded":"2024-09-11T01:22:36.927032+0000","last_fullsized":"2024-09-11T01:22:36.927032+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.933836+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:36.933836+0000","last_peered":"2024-09-11T01:22:36.933836+0000","last_clean":"2024-09-11T01:22:36.933836+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:36.933836+0000","last_undegraded":"2024-09-11T01:22:36.933836+0000","last_fullsized":"2024-09-11T01:22:36.933836+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24609,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:37.033944+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:37.033944+0000","last_peered":"2024-09-11T01:22:37.033944+0000","last_clean":"2024-09-11T01:22:37.033944+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:37.033944+0000","last_undegraded":"2024-09-11T01:22:37.033944+0000","last_fullsized":"2024-09-11T01:22:37.033944+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.926911+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:36.926911+0000","last_peered":"2024-09-11T01:22:36.926911+0000","last_clean":"2024-09-11T01:22:36.926911+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:36.926911+0000","last_undegraded":"2024-09-11T01:22:36.926911+0000","last_fullsized":"2024-09-11T01:22:36.926911+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.926929+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:36.926929+0000","last_peered":"2024-09-11T01:22:36.926929+0000","last_clean":"2024-09-11T01:22:36.926929+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:36.926929+0000","last_undegraded":"2024-09-11T01:22:36.926929+0000","last_fullsized":"2024-09-11T01:22:36.926929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20101,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923445+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:36.923445+0000","last_peered":"2024-09-11T01:22:36.923445+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:36.923445+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30223,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17416,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195088384,"data_stored":195036199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182534,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518584,"kb_used_data":63528,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853256,"statfs":{"total":96636764160,"available":96105734144,"internally_reserved":0,"allocated":65052672,"data_stored":64761973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986307,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488428,"kb_used_data":127772,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883412,"statfs":{"total":96636764160,"available":96136613888,"internally_reserved":0,"allocated":130838528,"data_stored":130551906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659251,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465256,"kb_used_data":472,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93906584,"statfs":{"total":96636764160,"available":96160342016,"internally_reserved":0,"allocated":483328,"data_stored":150881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475917999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":8021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:47.835 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:22:47.836 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20266", 'reported_seq': 20358, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:36.924493+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:22:36.924493+0000', 'last_peered': '2024-09-11T01:22:36.924493+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:22:36.924493+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20266, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4072}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:47.836 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:22:47.836 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'20144", 'reported_seq': 23428, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:45.345874+0000', 'last_change': '2024-09-11T01:22:35.206502+0000', 'last_active': '2024-09-11T01:22:45.345874+0000', 'last_peered': '2024-09-11T01:22:45.345874+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:22:45.345874+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2327, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18445, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1571, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2327}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:47.836 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:22:47.836 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'20227", 'reported_seq': 20412, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:36.923407+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:22:36.923407+0000', 'last_peered': '2024-09-11T01:22:36.923407+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:22:36.923407+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16568, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:47.836 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:22:47.836 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20075, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:36.923142+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:22:36.923142+0000', 'last_peered': '2024-09-11T01:22:36.923142+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:22:36.923142+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:47.836 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:22:47.836 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20011, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:36.924926+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:22:36.924926+0000', 'last_peered': '2024-09-11T01:22:36.924926+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:22:36.924926+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:47.837 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:22:47.837 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'20207", 'reported_seq': 20341, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:36.929381+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:22:36.929381+0000', 'last_peered': '2024-09-11T01:22:36.929381+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:22:36.929381+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18024, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4024}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:47.837 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:22:47.837 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20264", 'reported_seq': 20410, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:36.923998+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:22:36.923998+0000', 'last_peered': '2024-09-11T01:22:36.923998+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:22:36.923998+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17001, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:47.837 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:22:47.837 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 20101, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:36.923445+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:22:36.923445+0000', 'last_peered': '2024-09-11T01:22:36.923445+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:22:36.923445+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:47.837 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:22:48.229 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:48.229 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 30223/6 objects degraded (503716.667%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":704,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1507602432,"bytes_avail":288402690048,"bytes_total":289910292480,"degraded_objects":30223,"degraded_total":6,"degraded_ratio":5037.166666666667,"recovering_objects_per_sec":196,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1962,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:22:48.250 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:22:48.250 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:48.583 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:48.584 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:22:48.603 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":396,"stamp":"2024-09-11T01:22:46.810561+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30223,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17417,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472268,"kb_used_data":191772,"kb_used_omap":12,"kb_used_meta":1280435,"kb_avail":281643252,"statfs":{"total":289910292480,"available":288402690048,"internally_reserved":0,"allocated":196374528,"data_stored":195464760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1311165444},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1962,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1962,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003065"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":20358,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.924493+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:22:36.924493+0000","last_peered":"2024-09-11T01:22:36.924493+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:22:36.924493+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20468,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.929677+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:36.929677+0000","last_peered":"2024-09-11T01:22:36.929677+0000","last_clean":"2024-09-11T01:22:36.929677+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:36.929677+0000","last_undegraded":"2024-09-11T01:22:36.929677+0000","last_fullsized":"2024-09-11T01:22:36.929677+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":23428,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:45.345874+0000","last_change":"2024-09-11T01:22:35.206502+0000","last_active":"2024-09-11T01:22:45.345874+0000","last_peered":"2024-09-11T01:22:45.345874+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:22:45.345874+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2327,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1571,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2327}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20218,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.929591+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:36.929591+0000","last_peered":"2024-09-11T01:22:36.929591+0000","last_clean":"2024-09-11T01:22:36.929591+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:36.929591+0000","last_undegraded":"2024-09-11T01:22:36.929591+0000","last_fullsized":"2024-09-11T01:22:36.929591+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20151,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.924815+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:36.924815+0000","last_peered":"2024-09-11T01:22:36.924815+0000","last_clean":"2024-09-11T01:22:36.924815+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:36.924815+0000","last_undegraded":"2024-09-11T01:22:36.924815+0000","last_fullsized":"2024-09-11T01:22:36.924815+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20412,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923407+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:36.923407+0000","last_peered":"2024-09-11T01:22:36.923407+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:36.923407+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20075,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923142+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:36.923142+0000","last_peered":"2024-09-11T01:22:36.923142+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:36.923142+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20398,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.923036+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:36.923036+0000","last_peered":"2024-09-11T01:22:36.923036+0000","last_clean":"2024-09-11T01:22:36.923036+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:36.923036+0000","last_undegraded":"2024-09-11T01:22:36.923036+0000","last_fullsized":"2024-09-11T01:22:36.923036+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.924438+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:36.924438+0000","last_peered":"2024-09-11T01:22:36.924438+0000","last_clean":"2024-09-11T01:22:36.924438+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:36.924438+0000","last_undegraded":"2024-09-11T01:22:36.924438+0000","last_fullsized":"2024-09-11T01:22:36.924438+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":27996,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.925734+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:36.925734+0000","last_peered":"2024-09-11T01:22:36.925734+0000","last_clean":"2024-09-11T01:22:36.925734+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:36.925734+0000","last_undegraded":"2024-09-11T01:22:36.925734+0000","last_fullsized":"2024-09-11T01:22:36.925734+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.929215+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:36.929215+0000","last_peered":"2024-09-11T01:22:36.929215+0000","last_clean":"2024-09-11T01:22:36.929215+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:36.929215+0000","last_undegraded":"2024-09-11T01:22:36.929215+0000","last_fullsized":"2024-09-11T01:22:36.929215+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20011,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.924926+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:36.924926+0000","last_peered":"2024-09-11T01:22:36.924926+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:36.924926+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.923268+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:36.923268+0000","last_peered":"2024-09-11T01:22:36.923268+0000","last_clean":"2024-09-11T01:22:36.923268+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:36.923268+0000","last_undegraded":"2024-09-11T01:22:36.923268+0000","last_fullsized":"2024-09-11T01:22:36.923268+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20341,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.929381+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:36.929381+0000","last_peered":"2024-09-11T01:22:36.929381+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:36.929381+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.925633+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:36.925633+0000","last_peered":"2024-09-11T01:22:36.925633+0000","last_clean":"2024-09-11T01:22:36.925633+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:36.925633+0000","last_undegraded":"2024-09-11T01:22:36.925633+0000","last_fullsized":"2024-09-11T01:22:36.925633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27792,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.929150+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:36.929150+0000","last_peered":"2024-09-11T01:22:36.929150+0000","last_clean":"2024-09-11T01:22:36.929150+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:36.929150+0000","last_undegraded":"2024-09-11T01:22:36.929150+0000","last_fullsized":"2024-09-11T01:22:36.929150+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20410,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923998+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:36.923998+0000","last_peered":"2024-09-11T01:22:36.923998+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:36.923998+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.923557+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:36.923557+0000","last_peered":"2024-09-11T01:22:36.923557+0000","last_clean":"2024-09-11T01:22:36.923557+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:36.923557+0000","last_undegraded":"2024-09-11T01:22:36.923557+0000","last_fullsized":"2024-09-11T01:22:36.923557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27930,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.927032+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:36.927032+0000","last_peered":"2024-09-11T01:22:36.927032+0000","last_clean":"2024-09-11T01:22:36.927032+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:36.927032+0000","last_undegraded":"2024-09-11T01:22:36.927032+0000","last_fullsized":"2024-09-11T01:22:36.927032+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.933836+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:36.933836+0000","last_peered":"2024-09-11T01:22:36.933836+0000","last_clean":"2024-09-11T01:22:36.933836+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:36.933836+0000","last_undegraded":"2024-09-11T01:22:36.933836+0000","last_fullsized":"2024-09-11T01:22:36.933836+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24609,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:37.033944+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:37.033944+0000","last_peered":"2024-09-11T01:22:37.033944+0000","last_clean":"2024-09-11T01:22:37.033944+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:37.033944+0000","last_undegraded":"2024-09-11T01:22:37.033944+0000","last_fullsized":"2024-09-11T01:22:37.033944+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.926911+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:36.926911+0000","last_peered":"2024-09-11T01:22:36.926911+0000","last_clean":"2024-09-11T01:22:36.926911+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:36.926911+0000","last_undegraded":"2024-09-11T01:22:36.926911+0000","last_fullsized":"2024-09-11T01:22:36.926911+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.926929+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:36.926929+0000","last_peered":"2024-09-11T01:22:36.926929+0000","last_clean":"2024-09-11T01:22:36.926929+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:36.926929+0000","last_undegraded":"2024-09-11T01:22:36.926929+0000","last_fullsized":"2024-09-11T01:22:36.926929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20101,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923445+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:36.923445+0000","last_peered":"2024-09-11T01:22:36.923445+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:36.923445+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":30223,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17416,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195088384,"data_stored":195036199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182534,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518584,"kb_used_data":63528,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853256,"statfs":{"total":96636764160,"available":96105734144,"internally_reserved":0,"allocated":65052672,"data_stored":64761973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986307,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488428,"kb_used_data":127772,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883412,"statfs":{"total":96636764160,"available":96136613888,"internally_reserved":0,"allocated":130838528,"data_stored":130551906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659251,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465256,"kb_used_data":472,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93906584,"statfs":{"total":96636764160,"available":96160342016,"internally_reserved":0,"allocated":483328,"data_stored":150881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475917999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":8021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:51.605 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:51.951 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:51.952 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:22:51.972 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":398,"stamp":"2024-09-11T01:22:50.811981+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29159,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18481,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1480588,"kb_used_data":191772,"kb_used_omap":12,"kb_used_meta":1288755,"kb_avail":281634932,"statfs":{"total":289910292480,"available":288394170368,"internally_reserved":0,"allocated":196374528,"data_stored":195464760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1319685124},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2625,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2625,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004071"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":20358,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.924493+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:22:36.924493+0000","last_peered":"2024-09-11T01:22:36.924493+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:22:36.924493+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20468,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.929677+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:36.929677+0000","last_peered":"2024-09-11T01:22:36.929677+0000","last_clean":"2024-09-11T01:22:36.929677+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:36.929677+0000","last_undegraded":"2024-09-11T01:22:36.929677+0000","last_fullsized":"2024-09-11T01:22:36.929677+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":25556,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:50.348774+0000","last_change":"2024-09-11T01:22:35.206502+0000","last_active":"2024-09-11T01:22:50.348774+0000","last_peered":"2024-09-11T01:22:50.348774+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:22:50.348774+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1263,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2635,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1263}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20218,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.929591+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:36.929591+0000","last_peered":"2024-09-11T01:22:36.929591+0000","last_clean":"2024-09-11T01:22:36.929591+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:36.929591+0000","last_undegraded":"2024-09-11T01:22:36.929591+0000","last_fullsized":"2024-09-11T01:22:36.929591+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20151,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.924815+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:36.924815+0000","last_peered":"2024-09-11T01:22:36.924815+0000","last_clean":"2024-09-11T01:22:36.924815+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:36.924815+0000","last_undegraded":"2024-09-11T01:22:36.924815+0000","last_fullsized":"2024-09-11T01:22:36.924815+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20412,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923407+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:36.923407+0000","last_peered":"2024-09-11T01:22:36.923407+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:36.923407+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20075,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923142+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:36.923142+0000","last_peered":"2024-09-11T01:22:36.923142+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:36.923142+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20399,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.923128+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:36.923128+0000","last_peered":"2024-09-11T01:22:36.923128+0000","last_clean":"2024-09-11T01:22:36.923128+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:36.923128+0000","last_undegraded":"2024-09-11T01:22:36.923128+0000","last_fullsized":"2024-09-11T01:22:36.923128+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.924438+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:36.924438+0000","last_peered":"2024-09-11T01:22:36.924438+0000","last_clean":"2024-09-11T01:22:36.924438+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:36.924438+0000","last_undegraded":"2024-09-11T01:22:36.924438+0000","last_fullsized":"2024-09-11T01:22:36.924438+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":27996,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.925734+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:36.925734+0000","last_peered":"2024-09-11T01:22:36.925734+0000","last_clean":"2024-09-11T01:22:36.925734+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:36.925734+0000","last_undegraded":"2024-09-11T01:22:36.925734+0000","last_fullsized":"2024-09-11T01:22:36.925734+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.929215+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:36.929215+0000","last_peered":"2024-09-11T01:22:36.929215+0000","last_clean":"2024-09-11T01:22:36.929215+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:36.929215+0000","last_undegraded":"2024-09-11T01:22:36.929215+0000","last_fullsized":"2024-09-11T01:22:36.929215+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20011,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.924926+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:36.924926+0000","last_peered":"2024-09-11T01:22:36.924926+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:36.924926+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.923268+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:36.923268+0000","last_peered":"2024-09-11T01:22:36.923268+0000","last_clean":"2024-09-11T01:22:36.923268+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:36.923268+0000","last_undegraded":"2024-09-11T01:22:36.923268+0000","last_fullsized":"2024-09-11T01:22:36.923268+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20341,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.929381+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:36.929381+0000","last_peered":"2024-09-11T01:22:36.929381+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:36.929381+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.925633+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:36.925633+0000","last_peered":"2024-09-11T01:22:36.925633+0000","last_clean":"2024-09-11T01:22:36.925633+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:36.925633+0000","last_undegraded":"2024-09-11T01:22:36.925633+0000","last_fullsized":"2024-09-11T01:22:36.925633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27792,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.929150+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:36.929150+0000","last_peered":"2024-09-11T01:22:36.929150+0000","last_clean":"2024-09-11T01:22:36.929150+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:36.929150+0000","last_undegraded":"2024-09-11T01:22:36.929150+0000","last_fullsized":"2024-09-11T01:22:36.929150+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20410,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923998+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:36.923998+0000","last_peered":"2024-09-11T01:22:36.923998+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:36.923998+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.923557+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:36.923557+0000","last_peered":"2024-09-11T01:22:36.923557+0000","last_clean":"2024-09-11T01:22:36.923557+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:36.923557+0000","last_undegraded":"2024-09-11T01:22:36.923557+0000","last_fullsized":"2024-09-11T01:22:36.923557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27930,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.927032+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:36.927032+0000","last_peered":"2024-09-11T01:22:36.927032+0000","last_clean":"2024-09-11T01:22:36.927032+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:36.927032+0000","last_undegraded":"2024-09-11T01:22:36.927032+0000","last_fullsized":"2024-09-11T01:22:36.927032+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.933836+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:36.933836+0000","last_peered":"2024-09-11T01:22:36.933836+0000","last_clean":"2024-09-11T01:22:36.933836+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:36.933836+0000","last_undegraded":"2024-09-11T01:22:36.933836+0000","last_fullsized":"2024-09-11T01:22:36.933836+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24609,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:37.033944+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:37.033944+0000","last_peered":"2024-09-11T01:22:37.033944+0000","last_clean":"2024-09-11T01:22:37.033944+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:37.033944+0000","last_undegraded":"2024-09-11T01:22:37.033944+0000","last_fullsized":"2024-09-11T01:22:37.033944+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.926911+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:36.926911+0000","last_peered":"2024-09-11T01:22:36.926911+0000","last_clean":"2024-09-11T01:22:36.926911+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:36.926911+0000","last_undegraded":"2024-09-11T01:22:36.926911+0000","last_fullsized":"2024-09-11T01:22:36.926911+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.926929+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:36.926929+0000","last_peered":"2024-09-11T01:22:36.926929+0000","last_clean":"2024-09-11T01:22:36.926929+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:36.926929+0000","last_undegraded":"2024-09-11T01:22:36.926929+0000","last_fullsized":"2024-09-11T01:22:36.926929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20101,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923445+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:36.923445+0000","last_peered":"2024-09-11T01:22:36.923445+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:36.923445+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29159,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18480,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195088384,"data_stored":195036199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182535,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518584,"kb_used_data":63528,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853256,"statfs":{"total":96636764160,"available":96105734144,"internally_reserved":0,"allocated":65052672,"data_stored":64761973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986308,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488428,"kb_used_data":127772,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883412,"statfs":{"total":96636764160,"available":96136613888,"internally_reserved":0,"allocated":130838528,"data_stored":130551906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659252,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473576,"kb_used_data":472,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93898264,"statfs":{"total":96636764160,"available":96151822336,"internally_reserved":0,"allocated":483328,"data_stored":150881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":484437679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":8021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:51.973 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:52.326 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:52.326 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:22:52.344 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":398,"stamp":"2024-09-11T01:22:50.811981+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29159,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18481,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1480588,"kb_used_data":191772,"kb_used_omap":12,"kb_used_meta":1288755,"kb_avail":281634932,"statfs":{"total":289910292480,"available":288394170368,"internally_reserved":0,"allocated":196374528,"data_stored":195464760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1319685124},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2625,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2625,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004071"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":20358,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.924493+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:22:36.924493+0000","last_peered":"2024-09-11T01:22:36.924493+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:22:36.924493+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20468,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.929677+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:36.929677+0000","last_peered":"2024-09-11T01:22:36.929677+0000","last_clean":"2024-09-11T01:22:36.929677+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:36.929677+0000","last_undegraded":"2024-09-11T01:22:36.929677+0000","last_fullsized":"2024-09-11T01:22:36.929677+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":25556,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:50.348774+0000","last_change":"2024-09-11T01:22:35.206502+0000","last_active":"2024-09-11T01:22:50.348774+0000","last_peered":"2024-09-11T01:22:50.348774+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:22:50.348774+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1263,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2635,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1263}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20218,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.929591+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:36.929591+0000","last_peered":"2024-09-11T01:22:36.929591+0000","last_clean":"2024-09-11T01:22:36.929591+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:36.929591+0000","last_undegraded":"2024-09-11T01:22:36.929591+0000","last_fullsized":"2024-09-11T01:22:36.929591+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20151,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.924815+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:36.924815+0000","last_peered":"2024-09-11T01:22:36.924815+0000","last_clean":"2024-09-11T01:22:36.924815+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:36.924815+0000","last_undegraded":"2024-09-11T01:22:36.924815+0000","last_fullsized":"2024-09-11T01:22:36.924815+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20412,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923407+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:36.923407+0000","last_peered":"2024-09-11T01:22:36.923407+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:36.923407+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20075,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923142+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:36.923142+0000","last_peered":"2024-09-11T01:22:36.923142+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:36.923142+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20399,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.923128+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:36.923128+0000","last_peered":"2024-09-11T01:22:36.923128+0000","last_clean":"2024-09-11T01:22:36.923128+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:36.923128+0000","last_undegraded":"2024-09-11T01:22:36.923128+0000","last_fullsized":"2024-09-11T01:22:36.923128+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.924438+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:36.924438+0000","last_peered":"2024-09-11T01:22:36.924438+0000","last_clean":"2024-09-11T01:22:36.924438+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:36.924438+0000","last_undegraded":"2024-09-11T01:22:36.924438+0000","last_fullsized":"2024-09-11T01:22:36.924438+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":27996,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.925734+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:36.925734+0000","last_peered":"2024-09-11T01:22:36.925734+0000","last_clean":"2024-09-11T01:22:36.925734+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:36.925734+0000","last_undegraded":"2024-09-11T01:22:36.925734+0000","last_fullsized":"2024-09-11T01:22:36.925734+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.929215+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:36.929215+0000","last_peered":"2024-09-11T01:22:36.929215+0000","last_clean":"2024-09-11T01:22:36.929215+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:36.929215+0000","last_undegraded":"2024-09-11T01:22:36.929215+0000","last_fullsized":"2024-09-11T01:22:36.929215+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20011,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.924926+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:36.924926+0000","last_peered":"2024-09-11T01:22:36.924926+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:36.924926+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.923268+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:36.923268+0000","last_peered":"2024-09-11T01:22:36.923268+0000","last_clean":"2024-09-11T01:22:36.923268+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:36.923268+0000","last_undegraded":"2024-09-11T01:22:36.923268+0000","last_fullsized":"2024-09-11T01:22:36.923268+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20341,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.929381+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:36.929381+0000","last_peered":"2024-09-11T01:22:36.929381+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:36.929381+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.925633+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:36.925633+0000","last_peered":"2024-09-11T01:22:36.925633+0000","last_clean":"2024-09-11T01:22:36.925633+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:36.925633+0000","last_undegraded":"2024-09-11T01:22:36.925633+0000","last_fullsized":"2024-09-11T01:22:36.925633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27792,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.929150+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:36.929150+0000","last_peered":"2024-09-11T01:22:36.929150+0000","last_clean":"2024-09-11T01:22:36.929150+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:36.929150+0000","last_undegraded":"2024-09-11T01:22:36.929150+0000","last_fullsized":"2024-09-11T01:22:36.929150+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20410,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923998+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:36.923998+0000","last_peered":"2024-09-11T01:22:36.923998+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:36.923998+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.923557+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:36.923557+0000","last_peered":"2024-09-11T01:22:36.923557+0000","last_clean":"2024-09-11T01:22:36.923557+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:36.923557+0000","last_undegraded":"2024-09-11T01:22:36.923557+0000","last_fullsized":"2024-09-11T01:22:36.923557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27930,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.927032+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:36.927032+0000","last_peered":"2024-09-11T01:22:36.927032+0000","last_clean":"2024-09-11T01:22:36.927032+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:36.927032+0000","last_undegraded":"2024-09-11T01:22:36.927032+0000","last_fullsized":"2024-09-11T01:22:36.927032+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.933836+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:36.933836+0000","last_peered":"2024-09-11T01:22:36.933836+0000","last_clean":"2024-09-11T01:22:36.933836+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:36.933836+0000","last_undegraded":"2024-09-11T01:22:36.933836+0000","last_fullsized":"2024-09-11T01:22:36.933836+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24609,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:37.033944+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:37.033944+0000","last_peered":"2024-09-11T01:22:37.033944+0000","last_clean":"2024-09-11T01:22:37.033944+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:37.033944+0000","last_undegraded":"2024-09-11T01:22:37.033944+0000","last_fullsized":"2024-09-11T01:22:37.033944+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.926911+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:36.926911+0000","last_peered":"2024-09-11T01:22:36.926911+0000","last_clean":"2024-09-11T01:22:36.926911+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:36.926911+0000","last_undegraded":"2024-09-11T01:22:36.926911+0000","last_fullsized":"2024-09-11T01:22:36.926911+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.926929+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:36.926929+0000","last_peered":"2024-09-11T01:22:36.926929+0000","last_clean":"2024-09-11T01:22:36.926929+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:36.926929+0000","last_undegraded":"2024-09-11T01:22:36.926929+0000","last_fullsized":"2024-09-11T01:22:36.926929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20101,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923445+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:36.923445+0000","last_peered":"2024-09-11T01:22:36.923445+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:36.923445+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29159,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18480,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195088384,"data_stored":195036199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182535,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518584,"kb_used_data":63528,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853256,"statfs":{"total":96636764160,"available":96105734144,"internally_reserved":0,"allocated":65052672,"data_stored":64761973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986308,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488428,"kb_used_data":127772,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883412,"statfs":{"total":96636764160,"available":96136613888,"internally_reserved":0,"allocated":130838528,"data_stored":130551906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659252,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473576,"kb_used_data":472,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93898264,"statfs":{"total":96636764160,"available":96151822336,"internally_reserved":0,"allocated":483328,"data_stored":150881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":484437679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":8021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:52.346 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:22:52.346 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20266", 'reported_seq': 20358, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:36.924493+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:22:36.924493+0000', 'last_peered': '2024-09-11T01:22:36.924493+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:22:36.924493+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20266, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4072}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:52.346 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:22:52.346 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'20144", 'reported_seq': 25556, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:50.348774+0000', 'last_change': '2024-09-11T01:22:35.206502+0000', 'last_active': '2024-09-11T01:22:50.348774+0000', 'last_peered': '2024-09-11T01:22:50.348774+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:22:50.348774+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1263, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18445, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2635, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1263}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:52.346 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:22:52.346 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'20227", 'reported_seq': 20412, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:36.923407+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:22:36.923407+0000', 'last_peered': '2024-09-11T01:22:36.923407+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:22:36.923407+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16568, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:52.346 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:22:52.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20075, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:36.923142+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:22:36.923142+0000', 'last_peered': '2024-09-11T01:22:36.923142+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:22:36.923142+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:52.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:22:52.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20011, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:36.924926+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:22:36.924926+0000', 'last_peered': '2024-09-11T01:22:36.924926+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:22:36.924926+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:52.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:22:52.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'20207", 'reported_seq': 20341, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:36.929381+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:22:36.929381+0000', 'last_peered': '2024-09-11T01:22:36.929381+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:22:36.929381+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18024, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4024}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:52.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:22:52.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20264", 'reported_seq': 20410, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:36.923998+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:22:36.923998+0000', 'last_peered': '2024-09-11T01:22:36.923998+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:22:36.923998+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17001, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:52.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:22:52.347 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 20101, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:36.923445+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:22:36.923445+0000', 'last_peered': '2024-09-11T01:22:36.923445+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:22:36.923445+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:52.348 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:22:52.740 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:52.740 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 29159/6 objects degraded (485983.333%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":709,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":51,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":8},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1516122112,"bytes_avail":288394170368,"bytes_total":289910292480,"degraded_objects":29159,"degraded_total":6,"degraded_ratio":4859.833333333333,"recovering_objects_per_sec":218,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2625,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:22:52.757 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:22:52.758 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:53.074 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:53.075 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:22:53.091 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":399,"stamp":"2024-09-11T01:22:52.812351+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29159,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18481,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1480588,"kb_used_data":191772,"kb_used_omap":12,"kb_used_meta":1288755,"kb_avail":281634932,"statfs":{"total":289910292480,"available":288394170368,"internally_reserved":0,"allocated":196374528,"data_stored":195464760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1319685124},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1982,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003708"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":20358,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.924493+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:22:36.924493+0000","last_peered":"2024-09-11T01:22:36.924493+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:22:36.924493+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20468,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.929677+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:36.929677+0000","last_peered":"2024-09-11T01:22:36.929677+0000","last_clean":"2024-09-11T01:22:36.929677+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:36.929677+0000","last_undegraded":"2024-09-11T01:22:36.929677+0000","last_fullsized":"2024-09-11T01:22:36.929677+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":25556,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:50.348774+0000","last_change":"2024-09-11T01:22:35.206502+0000","last_active":"2024-09-11T01:22:50.348774+0000","last_peered":"2024-09-11T01:22:50.348774+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:22:50.348774+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1263,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2635,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1263}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20218,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.929591+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:36.929591+0000","last_peered":"2024-09-11T01:22:36.929591+0000","last_clean":"2024-09-11T01:22:36.929591+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:36.929591+0000","last_undegraded":"2024-09-11T01:22:36.929591+0000","last_fullsized":"2024-09-11T01:22:36.929591+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20151,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.924815+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:36.924815+0000","last_peered":"2024-09-11T01:22:36.924815+0000","last_clean":"2024-09-11T01:22:36.924815+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:36.924815+0000","last_undegraded":"2024-09-11T01:22:36.924815+0000","last_fullsized":"2024-09-11T01:22:36.924815+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20412,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923407+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:36.923407+0000","last_peered":"2024-09-11T01:22:36.923407+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:36.923407+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20075,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923142+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:36.923142+0000","last_peered":"2024-09-11T01:22:36.923142+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:36.923142+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20399,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.923128+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:36.923128+0000","last_peered":"2024-09-11T01:22:36.923128+0000","last_clean":"2024-09-11T01:22:36.923128+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:36.923128+0000","last_undegraded":"2024-09-11T01:22:36.923128+0000","last_fullsized":"2024-09-11T01:22:36.923128+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.924438+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:36.924438+0000","last_peered":"2024-09-11T01:22:36.924438+0000","last_clean":"2024-09-11T01:22:36.924438+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:36.924438+0000","last_undegraded":"2024-09-11T01:22:36.924438+0000","last_fullsized":"2024-09-11T01:22:36.924438+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":27996,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.925734+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:36.925734+0000","last_peered":"2024-09-11T01:22:36.925734+0000","last_clean":"2024-09-11T01:22:36.925734+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:36.925734+0000","last_undegraded":"2024-09-11T01:22:36.925734+0000","last_fullsized":"2024-09-11T01:22:36.925734+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.929215+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:36.929215+0000","last_peered":"2024-09-11T01:22:36.929215+0000","last_clean":"2024-09-11T01:22:36.929215+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:36.929215+0000","last_undegraded":"2024-09-11T01:22:36.929215+0000","last_fullsized":"2024-09-11T01:22:36.929215+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20011,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.924926+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:36.924926+0000","last_peered":"2024-09-11T01:22:36.924926+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:36.924926+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.923268+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:36.923268+0000","last_peered":"2024-09-11T01:22:36.923268+0000","last_clean":"2024-09-11T01:22:36.923268+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:36.923268+0000","last_undegraded":"2024-09-11T01:22:36.923268+0000","last_fullsized":"2024-09-11T01:22:36.923268+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20341,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.929381+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:36.929381+0000","last_peered":"2024-09-11T01:22:36.929381+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:36.929381+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.925633+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:36.925633+0000","last_peered":"2024-09-11T01:22:36.925633+0000","last_clean":"2024-09-11T01:22:36.925633+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:36.925633+0000","last_undegraded":"2024-09-11T01:22:36.925633+0000","last_fullsized":"2024-09-11T01:22:36.925633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27792,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.929150+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:36.929150+0000","last_peered":"2024-09-11T01:22:36.929150+0000","last_clean":"2024-09-11T01:22:36.929150+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:36.929150+0000","last_undegraded":"2024-09-11T01:22:36.929150+0000","last_fullsized":"2024-09-11T01:22:36.929150+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20410,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923998+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:36.923998+0000","last_peered":"2024-09-11T01:22:36.923998+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:36.923998+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.923557+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:36.923557+0000","last_peered":"2024-09-11T01:22:36.923557+0000","last_clean":"2024-09-11T01:22:36.923557+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:36.923557+0000","last_undegraded":"2024-09-11T01:22:36.923557+0000","last_fullsized":"2024-09-11T01:22:36.923557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27930,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.927032+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:36.927032+0000","last_peered":"2024-09-11T01:22:36.927032+0000","last_clean":"2024-09-11T01:22:36.927032+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:36.927032+0000","last_undegraded":"2024-09-11T01:22:36.927032+0000","last_fullsized":"2024-09-11T01:22:36.927032+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.933836+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:36.933836+0000","last_peered":"2024-09-11T01:22:36.933836+0000","last_clean":"2024-09-11T01:22:36.933836+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:36.933836+0000","last_undegraded":"2024-09-11T01:22:36.933836+0000","last_fullsized":"2024-09-11T01:22:36.933836+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24609,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:37.033944+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:37.033944+0000","last_peered":"2024-09-11T01:22:37.033944+0000","last_clean":"2024-09-11T01:22:37.033944+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:37.033944+0000","last_undegraded":"2024-09-11T01:22:37.033944+0000","last_fullsized":"2024-09-11T01:22:37.033944+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.926911+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:36.926911+0000","last_peered":"2024-09-11T01:22:36.926911+0000","last_clean":"2024-09-11T01:22:36.926911+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:36.926911+0000","last_undegraded":"2024-09-11T01:22:36.926911+0000","last_fullsized":"2024-09-11T01:22:36.926911+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.926929+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:36.926929+0000","last_peered":"2024-09-11T01:22:36.926929+0000","last_clean":"2024-09-11T01:22:36.926929+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:36.926929+0000","last_undegraded":"2024-09-11T01:22:36.926929+0000","last_fullsized":"2024-09-11T01:22:36.926929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20101,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923445+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:36.923445+0000","last_peered":"2024-09-11T01:22:36.923445+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:36.923445+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29159,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18480,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195088384,"data_stored":195036199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182535,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518584,"kb_used_data":63528,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853256,"statfs":{"total":96636764160,"available":96105734144,"internally_reserved":0,"allocated":65052672,"data_stored":64761973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986308,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488428,"kb_used_data":127772,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883412,"statfs":{"total":96636764160,"available":96136613888,"internally_reserved":0,"allocated":130838528,"data_stored":130551906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659252,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473576,"kb_used_data":472,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93898264,"statfs":{"total":96636764160,"available":96151822336,"internally_reserved":0,"allocated":483328,"data_stored":150881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":484437679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":8021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:56.094 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:56.470 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:56.470 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:22:56.492 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":402,"stamp":"2024-09-11T01:22:55.998137+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29159,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18481,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472268,"kb_used_data":191772,"kb_used_omap":12,"kb_used_meta":1280435,"kb_avail":281643252,"statfs":{"total":289910292480,"available":288402690048,"internally_reserved":0,"allocated":196374528,"data_stored":195464760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1311165444},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1064,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1064,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.187575"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":20358,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.924493+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:22:36.924493+0000","last_peered":"2024-09-11T01:22:36.924493+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:22:36.924493+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20470,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.003205+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:55.003205+0000","last_peered":"2024-09-11T01:22:55.003205+0000","last_clean":"2024-09-11T01:22:55.003205+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:55.003205+0000","last_undegraded":"2024-09-11T01:22:55.003205+0000","last_fullsized":"2024-09-11T01:22:55.003205+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":25556,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:50.348774+0000","last_change":"2024-09-11T01:22:35.206502+0000","last_active":"2024-09-11T01:22:50.348774+0000","last_peered":"2024-09-11T01:22:50.348774+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:22:50.348774+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1263,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2635,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1263}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20220,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.002502+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:55.002502+0000","last_peered":"2024-09-11T01:22:55.002502+0000","last_clean":"2024-09-11T01:22:55.002502+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:55.002502+0000","last_undegraded":"2024-09-11T01:22:55.002502+0000","last_fullsized":"2024-09-11T01:22:55.002502+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20151,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.924815+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:36.924815+0000","last_peered":"2024-09-11T01:22:36.924815+0000","last_clean":"2024-09-11T01:22:36.924815+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:36.924815+0000","last_undegraded":"2024-09-11T01:22:36.924815+0000","last_fullsized":"2024-09-11T01:22:36.924815+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20412,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923407+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:36.923407+0000","last_peered":"2024-09-11T01:22:36.923407+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:36.923407+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20075,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923142+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:36.923142+0000","last_peered":"2024-09-11T01:22:36.923142+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:36.923142+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20399,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.923128+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:36.923128+0000","last_peered":"2024-09-11T01:22:36.923128+0000","last_clean":"2024-09-11T01:22:36.923128+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:36.923128+0000","last_undegraded":"2024-09-11T01:22:36.923128+0000","last_fullsized":"2024-09-11T01:22:36.923128+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.924438+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:36.924438+0000","last_peered":"2024-09-11T01:22:36.924438+0000","last_clean":"2024-09-11T01:22:36.924438+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:36.924438+0000","last_undegraded":"2024-09-11T01:22:36.924438+0000","last_fullsized":"2024-09-11T01:22:36.924438+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":27996,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.925734+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:36.925734+0000","last_peered":"2024-09-11T01:22:36.925734+0000","last_clean":"2024-09-11T01:22:36.925734+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:36.925734+0000","last_undegraded":"2024-09-11T01:22:36.925734+0000","last_fullsized":"2024-09-11T01:22:36.925734+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":87,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-11T01:22:54.999727+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:54.999727+0000","last_peered":"2024-09-11T01:22:54.999727+0000","last_clean":"2024-09-11T01:22:54.999727+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:54.999727+0000","last_undegraded":"2024-09-11T01:22:54.999727+0000","last_fullsized":"2024-09-11T01:22:54.999727+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20011,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.924926+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:36.924926+0000","last_peered":"2024-09-11T01:22:36.924926+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:36.924926+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.923268+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:36.923268+0000","last_peered":"2024-09-11T01:22:36.923268+0000","last_clean":"2024-09-11T01:22:36.923268+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:36.923268+0000","last_undegraded":"2024-09-11T01:22:36.923268+0000","last_fullsized":"2024-09-11T01:22:36.923268+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20343,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:54.999830+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:54.999830+0000","last_peered":"2024-09-11T01:22:54.999830+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:54.999830+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.925633+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:36.925633+0000","last_peered":"2024-09-11T01:22:36.925633+0000","last_clean":"2024-09-11T01:22:36.925633+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:36.925633+0000","last_undegraded":"2024-09-11T01:22:36.925633+0000","last_fullsized":"2024-09-11T01:22:36.925633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27794,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-11T01:22:54.999524+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:54.999524+0000","last_peered":"2024-09-11T01:22:54.999524+0000","last_clean":"2024-09-11T01:22:54.999524+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:54.999524+0000","last_undegraded":"2024-09-11T01:22:54.999524+0000","last_fullsized":"2024-09-11T01:22:54.999524+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20410,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923998+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:36.923998+0000","last_peered":"2024-09-11T01:22:36.923998+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:36.923998+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.923557+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:36.923557+0000","last_peered":"2024-09-11T01:22:36.923557+0000","last_clean":"2024-09-11T01:22:36.923557+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:36.923557+0000","last_undegraded":"2024-09-11T01:22:36.923557+0000","last_fullsized":"2024-09-11T01:22:36.923557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27930,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.927032+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:36.927032+0000","last_peered":"2024-09-11T01:22:36.927032+0000","last_clean":"2024-09-11T01:22:36.927032+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:36.927032+0000","last_undegraded":"2024-09-11T01:22:36.927032+0000","last_fullsized":"2024-09-11T01:22:36.927032+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":87,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.008459+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:55.008459+0000","last_peered":"2024-09-11T01:22:55.008459+0000","last_clean":"2024-09-11T01:22:55.008459+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:55.008459+0000","last_undegraded":"2024-09-11T01:22:55.008459+0000","last_fullsized":"2024-09-11T01:22:55.008459+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24609,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:37.033944+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:37.033944+0000","last_peered":"2024-09-11T01:22:37.033944+0000","last_clean":"2024-09-11T01:22:37.033944+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:37.033944+0000","last_undegraded":"2024-09-11T01:22:37.033944+0000","last_fullsized":"2024-09-11T01:22:37.033944+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.926911+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:36.926911+0000","last_peered":"2024-09-11T01:22:36.926911+0000","last_clean":"2024-09-11T01:22:36.926911+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:36.926911+0000","last_undegraded":"2024-09-11T01:22:36.926911+0000","last_fullsized":"2024-09-11T01:22:36.926911+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.926929+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:36.926929+0000","last_peered":"2024-09-11T01:22:36.926929+0000","last_clean":"2024-09-11T01:22:36.926929+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:36.926929+0000","last_undegraded":"2024-09-11T01:22:36.926929+0000","last_fullsized":"2024-09-11T01:22:36.926929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20101,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923445+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:36.923445+0000","last_peered":"2024-09-11T01:22:36.923445+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:36.923445+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29159,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18480,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195088384,"data_stored":195036199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182536,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518584,"kb_used_data":63528,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853256,"statfs":{"total":96636764160,"available":96105734144,"internally_reserved":0,"allocated":65052672,"data_stored":64761973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986309,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488428,"kb_used_data":127772,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883412,"statfs":{"total":96636764160,"available":96136613888,"internally_reserved":0,"allocated":130838528,"data_stored":130551906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659253,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465256,"kb_used_data":472,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93906584,"statfs":{"total":96636764160,"available":96160342016,"internally_reserved":0,"allocated":483328,"data_stored":150881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475917999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":8021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:56.494 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:56.843 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:56.844 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:22:56.867 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":403,"stamp":"2024-09-11T01:22:56.813416+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29159,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18481,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472268,"kb_used_data":191772,"kb_used_omap":12,"kb_used_meta":1280435,"kb_avail":281643252,"statfs":{"total":289910292480,"available":288402690048,"internally_reserved":0,"allocated":196374528,"data_stored":195464760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1311165444},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1064,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1064,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002083"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":20358,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.924493+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:22:36.924493+0000","last_peered":"2024-09-11T01:22:36.924493+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:22:36.924493+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20470,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.003205+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:55.003205+0000","last_peered":"2024-09-11T01:22:55.003205+0000","last_clean":"2024-09-11T01:22:55.003205+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:55.003205+0000","last_undegraded":"2024-09-11T01:22:55.003205+0000","last_fullsized":"2024-09-11T01:22:55.003205+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":25556,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:50.348774+0000","last_change":"2024-09-11T01:22:35.206502+0000","last_active":"2024-09-11T01:22:50.348774+0000","last_peered":"2024-09-11T01:22:50.348774+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:22:50.348774+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1263,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2635,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1263}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20220,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.002502+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:55.002502+0000","last_peered":"2024-09-11T01:22:55.002502+0000","last_clean":"2024-09-11T01:22:55.002502+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:55.002502+0000","last_undegraded":"2024-09-11T01:22:55.002502+0000","last_fullsized":"2024-09-11T01:22:55.002502+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20151,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.924815+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:36.924815+0000","last_peered":"2024-09-11T01:22:36.924815+0000","last_clean":"2024-09-11T01:22:36.924815+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:36.924815+0000","last_undegraded":"2024-09-11T01:22:36.924815+0000","last_fullsized":"2024-09-11T01:22:36.924815+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20412,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923407+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:36.923407+0000","last_peered":"2024-09-11T01:22:36.923407+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:36.923407+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20075,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923142+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:36.923142+0000","last_peered":"2024-09-11T01:22:36.923142+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:36.923142+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20399,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.923128+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:36.923128+0000","last_peered":"2024-09-11T01:22:36.923128+0000","last_clean":"2024-09-11T01:22:36.923128+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:36.923128+0000","last_undegraded":"2024-09-11T01:22:36.923128+0000","last_fullsized":"2024-09-11T01:22:36.923128+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.924438+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:36.924438+0000","last_peered":"2024-09-11T01:22:36.924438+0000","last_clean":"2024-09-11T01:22:36.924438+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:36.924438+0000","last_undegraded":"2024-09-11T01:22:36.924438+0000","last_fullsized":"2024-09-11T01:22:36.924438+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":27996,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.925734+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:36.925734+0000","last_peered":"2024-09-11T01:22:36.925734+0000","last_clean":"2024-09-11T01:22:36.925734+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:36.925734+0000","last_undegraded":"2024-09-11T01:22:36.925734+0000","last_fullsized":"2024-09-11T01:22:36.925734+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":87,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-11T01:22:54.999727+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:54.999727+0000","last_peered":"2024-09-11T01:22:54.999727+0000","last_clean":"2024-09-11T01:22:54.999727+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:54.999727+0000","last_undegraded":"2024-09-11T01:22:54.999727+0000","last_fullsized":"2024-09-11T01:22:54.999727+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20011,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.924926+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:36.924926+0000","last_peered":"2024-09-11T01:22:36.924926+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:36.924926+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.923268+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:36.923268+0000","last_peered":"2024-09-11T01:22:36.923268+0000","last_clean":"2024-09-11T01:22:36.923268+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:36.923268+0000","last_undegraded":"2024-09-11T01:22:36.923268+0000","last_fullsized":"2024-09-11T01:22:36.923268+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20343,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:54.999830+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:54.999830+0000","last_peered":"2024-09-11T01:22:54.999830+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:54.999830+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.925633+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:36.925633+0000","last_peered":"2024-09-11T01:22:36.925633+0000","last_clean":"2024-09-11T01:22:36.925633+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:36.925633+0000","last_undegraded":"2024-09-11T01:22:36.925633+0000","last_fullsized":"2024-09-11T01:22:36.925633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27794,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-11T01:22:54.999524+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:54.999524+0000","last_peered":"2024-09-11T01:22:54.999524+0000","last_clean":"2024-09-11T01:22:54.999524+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:54.999524+0000","last_undegraded":"2024-09-11T01:22:54.999524+0000","last_fullsized":"2024-09-11T01:22:54.999524+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20410,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923998+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:36.923998+0000","last_peered":"2024-09-11T01:22:36.923998+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:36.923998+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.923557+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:36.923557+0000","last_peered":"2024-09-11T01:22:36.923557+0000","last_clean":"2024-09-11T01:22:36.923557+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:36.923557+0000","last_undegraded":"2024-09-11T01:22:36.923557+0000","last_fullsized":"2024-09-11T01:22:36.923557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27930,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.927032+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:36.927032+0000","last_peered":"2024-09-11T01:22:36.927032+0000","last_clean":"2024-09-11T01:22:36.927032+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:36.927032+0000","last_undegraded":"2024-09-11T01:22:36.927032+0000","last_fullsized":"2024-09-11T01:22:36.927032+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":87,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.008459+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:55.008459+0000","last_peered":"2024-09-11T01:22:55.008459+0000","last_clean":"2024-09-11T01:22:55.008459+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:55.008459+0000","last_undegraded":"2024-09-11T01:22:55.008459+0000","last_fullsized":"2024-09-11T01:22:55.008459+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24609,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:37.033944+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:37.033944+0000","last_peered":"2024-09-11T01:22:37.033944+0000","last_clean":"2024-09-11T01:22:37.033944+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:37.033944+0000","last_undegraded":"2024-09-11T01:22:37.033944+0000","last_fullsized":"2024-09-11T01:22:37.033944+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.926911+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:36.926911+0000","last_peered":"2024-09-11T01:22:36.926911+0000","last_clean":"2024-09-11T01:22:36.926911+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:36.926911+0000","last_undegraded":"2024-09-11T01:22:36.926911+0000","last_fullsized":"2024-09-11T01:22:36.926911+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.926929+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:36.926929+0000","last_peered":"2024-09-11T01:22:36.926929+0000","last_clean":"2024-09-11T01:22:36.926929+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:36.926929+0000","last_undegraded":"2024-09-11T01:22:36.926929+0000","last_fullsized":"2024-09-11T01:22:36.926929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20101,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923445+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:36.923445+0000","last_peered":"2024-09-11T01:22:36.923445+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:36.923445+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29159,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18480,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195088384,"data_stored":195036199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182536,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518584,"kb_used_data":63528,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853256,"statfs":{"total":96636764160,"available":96105734144,"internally_reserved":0,"allocated":65052672,"data_stored":64761973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986309,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488428,"kb_used_data":127772,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883412,"statfs":{"total":96636764160,"available":96136613888,"internally_reserved":0,"allocated":130838528,"data_stored":130551906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659253,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465256,"kb_used_data":472,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93906584,"statfs":{"total":96636764160,"available":96160342016,"internally_reserved":0,"allocated":483328,"data_stored":150881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475917999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":8021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:22:56.869 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:22:56.869 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20266", 'reported_seq': 20358, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:36.924493+0000', 'last_change': '2024-09-11T01:13:10.277595+0000', 'last_active': '2024-09-11T01:22:36.924493+0000', 'last_peered': '2024-09-11T01:22:36.924493+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:22:36.924493+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4072, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20266, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -4072}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:56.869 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.d is not active+clean 2024-09-11T01:22:56.869 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.d', 'version': "49'20144", 'reported_seq': 25556, 'reported_epoch': 51, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:50.348774+0000', 'last_change': '2024-09-11T01:22:35.206502+0000', 'last_active': '2024-09-11T01:22:50.348774+0000', 'last_peered': '2024-09-11T01:22:50.348774+0000', 'last_clean': '2024-09-11T01:13:07.633706+0000', 'last_became_active': '2024-09-11T01:13:10.196679+0000', 'last_became_peered': '2024-09-11T01:13:10.196679+0000', 'last_unstale': '2024-09-11T01:22:50.348774+0000', 'last_undegraded': '2024-09-11T01:13:09.750434+0000', 'last_fullsized': '2024-09-11T01:13:09.658079+0000', 'mapping_epoch': 42, 'log_start': "45'10100", 'ondisk_log_start': "45'10100", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'1723", 'last_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:37.612157+0000', 'objects_scrubbed': 1695, 'log_size': 10044, 'log_dups_size': 0, 'ondisk_log_size': 10044, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 675, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1263, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 1, 'num_read_kb': 1, 'num_write': 18445, 'num_write_kb': 33497, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2635, 'num_bytes_recovered': 4096, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1263}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:56.869 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:22:56.869 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'20227", 'reported_seq': 20412, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:36.923407+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:22:36.923407+0000', 'last_peered': '2024-09-11T01:22:36.923407+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:22:36.923407+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16568, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:56.869 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:22:56.869 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20075, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:36.923142+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:22:36.923142+0000', 'last_peered': '2024-09-11T01:22:36.923142+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:22:36.923142+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:56.870 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:22:56.870 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20011, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:36.924926+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:22:36.924926+0000', 'last_peered': '2024-09-11T01:22:36.924926+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:22:36.924926+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:56.870 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:22:56.870 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'20207", 'reported_seq': 20343, 'reported_epoch': 52, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:54.999830+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:22:54.999830+0000', 'last_peered': '2024-09-11T01:22:54.999830+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:22:54.999830+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18024, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4024}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:22:56.870 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:22:56.870 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20264", 'reported_seq': 20410, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:36.923998+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:22:36.923998+0000', 'last_peered': '2024-09-11T01:22:36.923998+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:22:36.923998+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17001, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:56.870 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:22:56.870 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 20101, 'reported_epoch': 51, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:36.923445+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:22:36.923445+0000', 'last_peered': '2024-09-11T01:22:36.923445+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:22:36.923445+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:22:56.870 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:22:57.283 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:57.284 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 29159/6 objects degraded (485983.333%), 8 pgs degraded, 8 pgs undersized","count":16},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":713,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":16},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":7},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1507602432,"bytes_avail":288402690048,"bytes_total":289910292480,"degraded_objects":29159,"degraded_total":6,"degraded_ratio":4859.833333333333,"recovering_objects_per_sec":132,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1064,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:22:57.303 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:22:57.303 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:22:57.615 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:22:57.616 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:22:57.634 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":403,"stamp":"2024-09-11T01:22:56.813416+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29159,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18481,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":40,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472268,"kb_used_data":191772,"kb_used_omap":12,"kb_used_meta":1280435,"kb_avail":281643252,"statfs":{"total":289910292480,"available":288402690048,"internally_reserved":0,"allocated":196374528,"data_stored":195464760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1311165444},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1064,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1064,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002083"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":20358,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.924493+0000","last_change":"2024-09-11T01:13:10.277595+0000","last_active":"2024-09-11T01:22:36.924493+0000","last_peered":"2024-09-11T01:22:36.924493+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:22:36.924493+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4072,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-4072}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20470,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.003205+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:55.003205+0000","last_peered":"2024-09-11T01:22:55.003205+0000","last_clean":"2024-09-11T01:22:55.003205+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:55.003205+0000","last_undegraded":"2024-09-11T01:22:55.003205+0000","last_fullsized":"2024-09-11T01:22:55.003205+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":25556,"reported_epoch":51,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:50.348774+0000","last_change":"2024-09-11T01:22:35.206502+0000","last_active":"2024-09-11T01:22:50.348774+0000","last_peered":"2024-09-11T01:22:50.348774+0000","last_clean":"2024-09-11T01:13:07.633706+0000","last_became_active":"2024-09-11T01:13:10.196679+0000","last_became_peered":"2024-09-11T01:13:10.196679+0000","last_unstale":"2024-09-11T01:22:50.348774+0000","last_undegraded":"2024-09-11T01:13:09.750434+0000","last_fullsized":"2024-09-11T01:13:09.658079+0000","mapping_epoch":42,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1263,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2635,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1263}],"blocked_by":[],"up_primary":1,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20220,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.002502+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:55.002502+0000","last_peered":"2024-09-11T01:22:55.002502+0000","last_clean":"2024-09-11T01:22:55.002502+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:55.002502+0000","last_undegraded":"2024-09-11T01:22:55.002502+0000","last_fullsized":"2024-09-11T01:22:55.002502+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20151,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.924815+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:36.924815+0000","last_peered":"2024-09-11T01:22:36.924815+0000","last_clean":"2024-09-11T01:22:36.924815+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:36.924815+0000","last_undegraded":"2024-09-11T01:22:36.924815+0000","last_fullsized":"2024-09-11T01:22:36.924815+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20412,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923407+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:36.923407+0000","last_peered":"2024-09-11T01:22:36.923407+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:36.923407+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20075,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923142+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:36.923142+0000","last_peered":"2024-09-11T01:22:36.923142+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:36.923142+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20399,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.923128+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:36.923128+0000","last_peered":"2024-09-11T01:22:36.923128+0000","last_clean":"2024-09-11T01:22:36.923128+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:36.923128+0000","last_undegraded":"2024-09-11T01:22:36.923128+0000","last_fullsized":"2024-09-11T01:22:36.923128+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.924438+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:36.924438+0000","last_peered":"2024-09-11T01:22:36.924438+0000","last_clean":"2024-09-11T01:22:36.924438+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:36.924438+0000","last_undegraded":"2024-09-11T01:22:36.924438+0000","last_fullsized":"2024-09-11T01:22:36.924438+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":27996,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.925734+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:36.925734+0000","last_peered":"2024-09-11T01:22:36.925734+0000","last_clean":"2024-09-11T01:22:36.925734+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:36.925734+0000","last_undegraded":"2024-09-11T01:22:36.925734+0000","last_fullsized":"2024-09-11T01:22:36.925734+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":87,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-11T01:22:54.999727+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:54.999727+0000","last_peered":"2024-09-11T01:22:54.999727+0000","last_clean":"2024-09-11T01:22:54.999727+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:54.999727+0000","last_undegraded":"2024-09-11T01:22:54.999727+0000","last_fullsized":"2024-09-11T01:22:54.999727+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20011,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.924926+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:36.924926+0000","last_peered":"2024-09-11T01:22:36.924926+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:36.924926+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":116,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.923268+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:36.923268+0000","last_peered":"2024-09-11T01:22:36.923268+0000","last_clean":"2024-09-11T01:22:36.923268+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:36.923268+0000","last_undegraded":"2024-09-11T01:22:36.923268+0000","last_fullsized":"2024-09-11T01:22:36.923268+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20343,"reported_epoch":52,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:54.999830+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:54.999830+0000","last_peered":"2024-09-11T01:22:54.999830+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:54.999830+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":94,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.925633+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:36.925633+0000","last_peered":"2024-09-11T01:22:36.925633+0000","last_clean":"2024-09-11T01:22:36.925633+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:36.925633+0000","last_undegraded":"2024-09-11T01:22:36.925633+0000","last_fullsized":"2024-09-11T01:22:36.925633+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27794,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-11T01:22:54.999524+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:54.999524+0000","last_peered":"2024-09-11T01:22:54.999524+0000","last_clean":"2024-09-11T01:22:54.999524+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:54.999524+0000","last_undegraded":"2024-09-11T01:22:54.999524+0000","last_fullsized":"2024-09-11T01:22:54.999524+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20410,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923998+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:36.923998+0000","last_peered":"2024-09-11T01:22:36.923998+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:36.923998+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.923557+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:36.923557+0000","last_peered":"2024-09-11T01:22:36.923557+0000","last_clean":"2024-09-11T01:22:36.923557+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:36.923557+0000","last_undegraded":"2024-09-11T01:22:36.923557+0000","last_fullsized":"2024-09-11T01:22:36.923557+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27930,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.927032+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:36.927032+0000","last_peered":"2024-09-11T01:22:36.927032+0000","last_clean":"2024-09-11T01:22:36.927032+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:36.927032+0000","last_undegraded":"2024-09-11T01:22:36.927032+0000","last_fullsized":"2024-09-11T01:22:36.927032+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":87,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.008459+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:55.008459+0000","last_peered":"2024-09-11T01:22:55.008459+0000","last_clean":"2024-09-11T01:22:55.008459+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:55.008459+0000","last_undegraded":"2024-09-11T01:22:55.008459+0000","last_fullsized":"2024-09-11T01:22:55.008459+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24609,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:37.033944+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:37.033944+0000","last_peered":"2024-09-11T01:22:37.033944+0000","last_clean":"2024-09-11T01:22:37.033944+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:37.033944+0000","last_undegraded":"2024-09-11T01:22:37.033944+0000","last_fullsized":"2024-09-11T01:22:37.033944+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":84,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.926911+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:36.926911+0000","last_peered":"2024-09-11T01:22:36.926911+0000","last_clean":"2024-09-11T01:22:36.926911+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:36.926911+0000","last_undegraded":"2024-09-11T01:22:36.926911+0000","last_fullsized":"2024-09-11T01:22:36.926911+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":89,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-09-11T01:22:36.926929+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:36.926929+0000","last_peered":"2024-09-11T01:22:36.926929+0000","last_clean":"2024-09-11T01:22:36.926929+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:36.926929+0000","last_undegraded":"2024-09-11T01:22:36.926929+0000","last_fullsized":"2024-09-11T01:22:36.926929+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20101,"reported_epoch":51,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:36.923445+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:36.923445+0000","last_peered":"2024-09-11T01:22:36.923445+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:36.923445+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":29159,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18480,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195088384,"data_stored":195036199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":24,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182536,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518584,"kb_used_data":63528,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853256,"statfs":{"total":96636764160,"available":96105734144,"internally_reserved":0,"allocated":65052672,"data_stored":64761973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986309,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488428,"kb_used_data":127772,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883412,"statfs":{"total":96636764160,"available":96136613888,"internally_reserved":0,"allocated":130838528,"data_stored":130551906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659253,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465256,"kb_used_data":472,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93906584,"statfs":{"total":96636764160,"available":96160342016,"internally_reserved":0,"allocated":483328,"data_stored":150881,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475917999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":8021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:00.637 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:00.963 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:00.964 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:23:00.983 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":405,"stamp":"2024-09-11T01:23:00.814530+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27540,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20100,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1480636,"kb_used_data":191820,"kb_used_omap":12,"kb_used_meta":1288755,"kb_avail":281634884,"statfs":{"total":289910292480,"available":288394121216,"internally_reserved":0,"allocated":196423680,"data_stored":195482127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1319685124},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1619,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1619,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002179"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":21075,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:58.218571+0000","last_change":"2024-09-11T01:22:54.587427+0000","last_active":"2024-09-11T01:22:58.218571+0000","last_peered":"2024-09-11T01:22:58.218571+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:22:58.218571+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3716,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":356,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3716}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20472,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996478+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:55.996478+0000","last_peered":"2024-09-11T01:22:55.996478+0000","last_clean":"2024-09-11T01:22:55.996478+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:55.996478+0000","last_undegraded":"2024-09-11T01:22:55.996478+0000","last_fullsized":"2024-09-11T01:22:55.996478+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20296,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.149418+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:22:56.149418+0000","last_peered":"2024-09-11T01:22:56.149418+0000","last_clean":"2024-09-11T01:22:56.149418+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:22:56.149418+0000","last_undegraded":"2024-09-11T01:22:56.149418+0000","last_fullsized":"2024-09-11T01:22:56.149418+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20222,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996767+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:55.996767+0000","last_peered":"2024-09-11T01:22:55.996767+0000","last_clean":"2024-09-11T01:22:55.996767+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:55.996767+0000","last_undegraded":"2024-09-11T01:22:55.996767+0000","last_fullsized":"2024-09-11T01:22:55.996767+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20155,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001027+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:56.001027+0000","last_peered":"2024-09-11T01:22:56.001027+0000","last_clean":"2024-09-11T01:22:56.001027+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:56.001027+0000","last_undegraded":"2024-09-11T01:22:56.001027+0000","last_fullsized":"2024-09-11T01:22:56.001027+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20416,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000893+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:56.000893+0000","last_peered":"2024-09-11T01:22:56.000893+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:56.000893+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20079,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000461+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:56.000461+0000","last_peered":"2024-09-11T01:22:56.000461+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:56.000461+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20403,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000263+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:56.000263+0000","last_peered":"2024-09-11T01:22:56.000263+0000","last_clean":"2024-09-11T01:22:56.000263+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:56.000263+0000","last_undegraded":"2024-09-11T01:22:56.000263+0000","last_fullsized":"2024-09-11T01:22:56.000263+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999147+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:55.999147+0000","last_peered":"2024-09-11T01:22:55.999147+0000","last_clean":"2024-09-11T01:22:55.999147+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:55.999147+0000","last_undegraded":"2024-09-11T01:22:55.999147+0000","last_fullsized":"2024-09-11T01:22:55.999147+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28000,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999439+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:55.999439+0000","last_peered":"2024-09-11T01:22:55.999439+0000","last_clean":"2024-09-11T01:22:55.999439+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:55.999439+0000","last_undegraded":"2024-09-11T01:22:55.999439+0000","last_fullsized":"2024-09-11T01:22:55.999439+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:55.994373+0000","last_peered":"2024-09-11T01:22:55.994373+0000","last_clean":"2024-09-11T01:22:55.994373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:55.994373+0000","last_undegraded":"2024-09-11T01:22:55.994373+0000","last_fullsized":"2024-09-11T01:22:55.994373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20015,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001237+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:56.001237+0000","last_peered":"2024-09-11T01:22:56.001237+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:56.001237+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000522+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:56.000522+0000","last_peered":"2024-09-11T01:22:56.000522+0000","last_clean":"2024-09-11T01:22:56.000522+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:56.000522+0000","last_undegraded":"2024-09-11T01:22:56.000522+0000","last_fullsized":"2024-09-11T01:22:56.000522+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20345,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:55.994507+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:55.994507+0000","last_peered":"2024-09-11T01:22:55.994507+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:55.994507+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999168+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:55.999168+0000","last_peered":"2024-09-11T01:22:55.999168+0000","last_clean":"2024-09-11T01:22:55.999168+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:55.999168+0000","last_undegraded":"2024-09-11T01:22:55.999168+0000","last_fullsized":"2024-09-11T01:22:55.999168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27796,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994249+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:55.994249+0000","last_peered":"2024-09-11T01:22:55.994249+0000","last_clean":"2024-09-11T01:22:55.994249+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:55.994249+0000","last_undegraded":"2024-09-11T01:22:55.994249+0000","last_fullsized":"2024-09-11T01:22:55.994249+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20415,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.006380+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:56.006380+0000","last_peered":"2024-09-11T01:22:56.006380+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:56.006380+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003870+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:56.003870+0000","last_peered":"2024-09-11T01:22:56.003870+0000","last_clean":"2024-09-11T01:22:56.003870+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:56.003870+0000","last_undegraded":"2024-09-11T01:22:56.003870+0000","last_fullsized":"2024-09-11T01:22:56.003870+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27934,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003846+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:56.003846+0000","last_peered":"2024-09-11T01:22:56.003846+0000","last_clean":"2024-09-11T01:22:56.003846+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:56.003846+0000","last_undegraded":"2024-09-11T01:22:56.003846+0000","last_fullsized":"2024-09-11T01:22:56.003846+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996759+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:55.996759+0000","last_peered":"2024-09-11T01:22:55.996759+0000","last_clean":"2024-09-11T01:22:55.996759+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:55.996759+0000","last_undegraded":"2024-09-11T01:22:55.996759+0000","last_fullsized":"2024-09-11T01:22:55.996759+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24613,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001573+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:56.001573+0000","last_peered":"2024-09-11T01:22:56.001573+0000","last_clean":"2024-09-11T01:22:56.001573+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:56.001573+0000","last_undegraded":"2024-09-11T01:22:56.001573+0000","last_fullsized":"2024-09-11T01:22:56.001573+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999413+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:55.999413+0000","last_peered":"2024-09-11T01:22:55.999413+0000","last_clean":"2024-09-11T01:22:55.999413+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:55.999413+0000","last_undegraded":"2024-09-11T01:22:55.999413+0000","last_fullsized":"2024-09-11T01:22:55.999413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999502+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:55.999502+0000","last_peered":"2024-09-11T01:22:55.999502+0000","last_clean":"2024-09-11T01:22:55.999502+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:55.999502+0000","last_undegraded":"2024-09-11T01:22:55.999502+0000","last_fullsized":"2024-09-11T01:22:55.999502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20105,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001118+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:56.001118+0000","last_peered":"2024-09-11T01:22:56.001118+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:56.001118+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27540,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20099,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195088384,"data_stored":195036199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182537,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518600,"kb_used_data":63544,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853240,"statfs":{"total":96636764160,"available":96105717760,"internally_reserved":0,"allocated":65069056,"data_stored":64767762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986310,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488444,"kb_used_data":127788,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883396,"statfs":{"total":96636764160,"available":96136597504,"internally_reserved":0,"allocated":130854912,"data_stored":130557695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659254,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473592,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93898248,"statfs":{"total":96636764160,"available":96151805952,"internally_reserved":0,"allocated":499712,"data_stored":156670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":484437679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":8021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:00.985 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:01.305 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:01.306 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:23:01.324 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":405,"stamp":"2024-09-11T01:23:00.814530+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27540,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20100,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1480636,"kb_used_data":191820,"kb_used_omap":12,"kb_used_meta":1288755,"kb_avail":281634884,"statfs":{"total":289910292480,"available":288394121216,"internally_reserved":0,"allocated":196423680,"data_stored":195482127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1319685124},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1619,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1619,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002179"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":21075,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:58.218571+0000","last_change":"2024-09-11T01:22:54.587427+0000","last_active":"2024-09-11T01:22:58.218571+0000","last_peered":"2024-09-11T01:22:58.218571+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:22:58.218571+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3716,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":356,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3716}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20472,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996478+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:55.996478+0000","last_peered":"2024-09-11T01:22:55.996478+0000","last_clean":"2024-09-11T01:22:55.996478+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:55.996478+0000","last_undegraded":"2024-09-11T01:22:55.996478+0000","last_fullsized":"2024-09-11T01:22:55.996478+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20296,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.149418+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:22:56.149418+0000","last_peered":"2024-09-11T01:22:56.149418+0000","last_clean":"2024-09-11T01:22:56.149418+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:22:56.149418+0000","last_undegraded":"2024-09-11T01:22:56.149418+0000","last_fullsized":"2024-09-11T01:22:56.149418+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20222,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996767+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:55.996767+0000","last_peered":"2024-09-11T01:22:55.996767+0000","last_clean":"2024-09-11T01:22:55.996767+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:55.996767+0000","last_undegraded":"2024-09-11T01:22:55.996767+0000","last_fullsized":"2024-09-11T01:22:55.996767+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20155,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001027+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:56.001027+0000","last_peered":"2024-09-11T01:22:56.001027+0000","last_clean":"2024-09-11T01:22:56.001027+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:56.001027+0000","last_undegraded":"2024-09-11T01:22:56.001027+0000","last_fullsized":"2024-09-11T01:22:56.001027+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20416,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000893+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:56.000893+0000","last_peered":"2024-09-11T01:22:56.000893+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:56.000893+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20079,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000461+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:56.000461+0000","last_peered":"2024-09-11T01:22:56.000461+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:56.000461+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20403,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000263+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:56.000263+0000","last_peered":"2024-09-11T01:22:56.000263+0000","last_clean":"2024-09-11T01:22:56.000263+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:56.000263+0000","last_undegraded":"2024-09-11T01:22:56.000263+0000","last_fullsized":"2024-09-11T01:22:56.000263+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999147+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:55.999147+0000","last_peered":"2024-09-11T01:22:55.999147+0000","last_clean":"2024-09-11T01:22:55.999147+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:55.999147+0000","last_undegraded":"2024-09-11T01:22:55.999147+0000","last_fullsized":"2024-09-11T01:22:55.999147+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28000,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999439+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:55.999439+0000","last_peered":"2024-09-11T01:22:55.999439+0000","last_clean":"2024-09-11T01:22:55.999439+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:55.999439+0000","last_undegraded":"2024-09-11T01:22:55.999439+0000","last_fullsized":"2024-09-11T01:22:55.999439+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:55.994373+0000","last_peered":"2024-09-11T01:22:55.994373+0000","last_clean":"2024-09-11T01:22:55.994373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:55.994373+0000","last_undegraded":"2024-09-11T01:22:55.994373+0000","last_fullsized":"2024-09-11T01:22:55.994373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20015,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001237+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:56.001237+0000","last_peered":"2024-09-11T01:22:56.001237+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:56.001237+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000522+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:56.000522+0000","last_peered":"2024-09-11T01:22:56.000522+0000","last_clean":"2024-09-11T01:22:56.000522+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:56.000522+0000","last_undegraded":"2024-09-11T01:22:56.000522+0000","last_fullsized":"2024-09-11T01:22:56.000522+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20345,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:55.994507+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:55.994507+0000","last_peered":"2024-09-11T01:22:55.994507+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:55.994507+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999168+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:55.999168+0000","last_peered":"2024-09-11T01:22:55.999168+0000","last_clean":"2024-09-11T01:22:55.999168+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:55.999168+0000","last_undegraded":"2024-09-11T01:22:55.999168+0000","last_fullsized":"2024-09-11T01:22:55.999168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27796,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994249+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:55.994249+0000","last_peered":"2024-09-11T01:22:55.994249+0000","last_clean":"2024-09-11T01:22:55.994249+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:55.994249+0000","last_undegraded":"2024-09-11T01:22:55.994249+0000","last_fullsized":"2024-09-11T01:22:55.994249+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20415,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.006380+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:56.006380+0000","last_peered":"2024-09-11T01:22:56.006380+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:56.006380+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003870+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:56.003870+0000","last_peered":"2024-09-11T01:22:56.003870+0000","last_clean":"2024-09-11T01:22:56.003870+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:56.003870+0000","last_undegraded":"2024-09-11T01:22:56.003870+0000","last_fullsized":"2024-09-11T01:22:56.003870+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27934,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003846+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:56.003846+0000","last_peered":"2024-09-11T01:22:56.003846+0000","last_clean":"2024-09-11T01:22:56.003846+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:56.003846+0000","last_undegraded":"2024-09-11T01:22:56.003846+0000","last_fullsized":"2024-09-11T01:22:56.003846+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996759+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:55.996759+0000","last_peered":"2024-09-11T01:22:55.996759+0000","last_clean":"2024-09-11T01:22:55.996759+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:55.996759+0000","last_undegraded":"2024-09-11T01:22:55.996759+0000","last_fullsized":"2024-09-11T01:22:55.996759+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24613,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001573+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:56.001573+0000","last_peered":"2024-09-11T01:22:56.001573+0000","last_clean":"2024-09-11T01:22:56.001573+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:56.001573+0000","last_undegraded":"2024-09-11T01:22:56.001573+0000","last_fullsized":"2024-09-11T01:22:56.001573+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999413+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:55.999413+0000","last_peered":"2024-09-11T01:22:55.999413+0000","last_clean":"2024-09-11T01:22:55.999413+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:55.999413+0000","last_undegraded":"2024-09-11T01:22:55.999413+0000","last_fullsized":"2024-09-11T01:22:55.999413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999502+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:55.999502+0000","last_peered":"2024-09-11T01:22:55.999502+0000","last_clean":"2024-09-11T01:22:55.999502+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:55.999502+0000","last_undegraded":"2024-09-11T01:22:55.999502+0000","last_fullsized":"2024-09-11T01:22:55.999502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20105,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001118+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:56.001118+0000","last_peered":"2024-09-11T01:22:56.001118+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:56.001118+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27540,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20099,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195088384,"data_stored":195036199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182537,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518600,"kb_used_data":63544,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853240,"statfs":{"total":96636764160,"available":96105717760,"internally_reserved":0,"allocated":65069056,"data_stored":64767762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986310,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488444,"kb_used_data":127788,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883396,"statfs":{"total":96636764160,"available":96136597504,"internally_reserved":0,"allocated":130854912,"data_stored":130557695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659254,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473592,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93898248,"statfs":{"total":96636764160,"available":96151805952,"internally_reserved":0,"allocated":499712,"data_stored":156670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":484437679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":8021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:01.325 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:23:01.325 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20266", 'reported_seq': 21075, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:58.218571+0000', 'last_change': '2024-09-11T01:22:54.587427+0000', 'last_active': '2024-09-11T01:22:58.218571+0000', 'last_peered': '2024-09-11T01:22:58.218571+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:22:58.218571+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3716, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20266, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 356, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3716}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:01.326 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:23:01.326 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'20227", 'reported_seq': 20416, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.000893+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:22:56.000893+0000', 'last_peered': '2024-09-11T01:22:56.000893+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:22:56.000893+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16568, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:01.326 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:23:01.326 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20079, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.000461+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:22:56.000461+0000', 'last_peered': '2024-09-11T01:22:56.000461+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:22:56.000461+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:01.326 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:23:01.326 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20015, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.001237+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:22:56.001237+0000', 'last_peered': '2024-09-11T01:22:56.001237+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:22:56.001237+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:01.326 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:23:01.326 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'20207", 'reported_seq': 20345, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:55.994507+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:22:55.994507+0000', 'last_peered': '2024-09-11T01:22:55.994507+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:22:55.994507+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18024, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4024}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:01.326 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:23:01.326 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20264", 'reported_seq': 20415, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.006380+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:22:56.006380+0000', 'last_peered': '2024-09-11T01:22:56.006380+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:22:56.006380+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17001, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:01.327 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:23:01.327 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 20105, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.001118+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:22:56.001118+0000', 'last_peered': '2024-09-11T01:22:56.001118+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:22:56.001118+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:01.327 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:23:01.718 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:01.718 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 27540/6 objects degraded (459000.000%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":718,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1516171264,"bytes_avail":288394121216,"bytes_total":289910292480,"degraded_objects":27540,"degraded_total":6,"degraded_ratio":4590,"recovering_objects_per_sec":202,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1619,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:23:01.740 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:23:01.741 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:02.057 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:02.058 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:23:02.075 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":405,"stamp":"2024-09-11T01:23:00.814530+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27540,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20100,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1480636,"kb_used_data":191820,"kb_used_omap":12,"kb_used_meta":1288755,"kb_avail":281634884,"statfs":{"total":289910292480,"available":288394121216,"internally_reserved":0,"allocated":196423680,"data_stored":195482127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1319685124},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1619,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1619,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002179"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":21075,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:58.218571+0000","last_change":"2024-09-11T01:22:54.587427+0000","last_active":"2024-09-11T01:22:58.218571+0000","last_peered":"2024-09-11T01:22:58.218571+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:22:58.218571+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3716,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":356,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3716}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20472,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996478+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:55.996478+0000","last_peered":"2024-09-11T01:22:55.996478+0000","last_clean":"2024-09-11T01:22:55.996478+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:55.996478+0000","last_undegraded":"2024-09-11T01:22:55.996478+0000","last_fullsized":"2024-09-11T01:22:55.996478+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20296,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.149418+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:22:56.149418+0000","last_peered":"2024-09-11T01:22:56.149418+0000","last_clean":"2024-09-11T01:22:56.149418+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:22:56.149418+0000","last_undegraded":"2024-09-11T01:22:56.149418+0000","last_fullsized":"2024-09-11T01:22:56.149418+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20222,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996767+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:55.996767+0000","last_peered":"2024-09-11T01:22:55.996767+0000","last_clean":"2024-09-11T01:22:55.996767+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:55.996767+0000","last_undegraded":"2024-09-11T01:22:55.996767+0000","last_fullsized":"2024-09-11T01:22:55.996767+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20155,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001027+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:56.001027+0000","last_peered":"2024-09-11T01:22:56.001027+0000","last_clean":"2024-09-11T01:22:56.001027+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:56.001027+0000","last_undegraded":"2024-09-11T01:22:56.001027+0000","last_fullsized":"2024-09-11T01:22:56.001027+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20416,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000893+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:56.000893+0000","last_peered":"2024-09-11T01:22:56.000893+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:56.000893+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20079,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000461+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:56.000461+0000","last_peered":"2024-09-11T01:22:56.000461+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:56.000461+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20403,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000263+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:56.000263+0000","last_peered":"2024-09-11T01:22:56.000263+0000","last_clean":"2024-09-11T01:22:56.000263+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:56.000263+0000","last_undegraded":"2024-09-11T01:22:56.000263+0000","last_fullsized":"2024-09-11T01:22:56.000263+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999147+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:55.999147+0000","last_peered":"2024-09-11T01:22:55.999147+0000","last_clean":"2024-09-11T01:22:55.999147+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:55.999147+0000","last_undegraded":"2024-09-11T01:22:55.999147+0000","last_fullsized":"2024-09-11T01:22:55.999147+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28000,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999439+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:55.999439+0000","last_peered":"2024-09-11T01:22:55.999439+0000","last_clean":"2024-09-11T01:22:55.999439+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:55.999439+0000","last_undegraded":"2024-09-11T01:22:55.999439+0000","last_fullsized":"2024-09-11T01:22:55.999439+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:55.994373+0000","last_peered":"2024-09-11T01:22:55.994373+0000","last_clean":"2024-09-11T01:22:55.994373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:55.994373+0000","last_undegraded":"2024-09-11T01:22:55.994373+0000","last_fullsized":"2024-09-11T01:22:55.994373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20015,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001237+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:56.001237+0000","last_peered":"2024-09-11T01:22:56.001237+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:56.001237+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000522+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:56.000522+0000","last_peered":"2024-09-11T01:22:56.000522+0000","last_clean":"2024-09-11T01:22:56.000522+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:56.000522+0000","last_undegraded":"2024-09-11T01:22:56.000522+0000","last_fullsized":"2024-09-11T01:22:56.000522+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20345,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:55.994507+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:55.994507+0000","last_peered":"2024-09-11T01:22:55.994507+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:55.994507+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999168+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:55.999168+0000","last_peered":"2024-09-11T01:22:55.999168+0000","last_clean":"2024-09-11T01:22:55.999168+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:55.999168+0000","last_undegraded":"2024-09-11T01:22:55.999168+0000","last_fullsized":"2024-09-11T01:22:55.999168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27796,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994249+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:55.994249+0000","last_peered":"2024-09-11T01:22:55.994249+0000","last_clean":"2024-09-11T01:22:55.994249+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:55.994249+0000","last_undegraded":"2024-09-11T01:22:55.994249+0000","last_fullsized":"2024-09-11T01:22:55.994249+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20415,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.006380+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:56.006380+0000","last_peered":"2024-09-11T01:22:56.006380+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:56.006380+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003870+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:56.003870+0000","last_peered":"2024-09-11T01:22:56.003870+0000","last_clean":"2024-09-11T01:22:56.003870+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:56.003870+0000","last_undegraded":"2024-09-11T01:22:56.003870+0000","last_fullsized":"2024-09-11T01:22:56.003870+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27934,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003846+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:56.003846+0000","last_peered":"2024-09-11T01:22:56.003846+0000","last_clean":"2024-09-11T01:22:56.003846+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:56.003846+0000","last_undegraded":"2024-09-11T01:22:56.003846+0000","last_fullsized":"2024-09-11T01:22:56.003846+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996759+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:55.996759+0000","last_peered":"2024-09-11T01:22:55.996759+0000","last_clean":"2024-09-11T01:22:55.996759+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:55.996759+0000","last_undegraded":"2024-09-11T01:22:55.996759+0000","last_fullsized":"2024-09-11T01:22:55.996759+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24613,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001573+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:56.001573+0000","last_peered":"2024-09-11T01:22:56.001573+0000","last_clean":"2024-09-11T01:22:56.001573+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:56.001573+0000","last_undegraded":"2024-09-11T01:22:56.001573+0000","last_fullsized":"2024-09-11T01:22:56.001573+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999413+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:55.999413+0000","last_peered":"2024-09-11T01:22:55.999413+0000","last_clean":"2024-09-11T01:22:55.999413+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:55.999413+0000","last_undegraded":"2024-09-11T01:22:55.999413+0000","last_fullsized":"2024-09-11T01:22:55.999413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999502+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:55.999502+0000","last_peered":"2024-09-11T01:22:55.999502+0000","last_clean":"2024-09-11T01:22:55.999502+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:55.999502+0000","last_undegraded":"2024-09-11T01:22:55.999502+0000","last_fullsized":"2024-09-11T01:22:55.999502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20105,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001118+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:56.001118+0000","last_peered":"2024-09-11T01:22:56.001118+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:56.001118+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27540,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20099,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195088384,"data_stored":195036199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182537,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518600,"kb_used_data":63544,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853240,"statfs":{"total":96636764160,"available":96105717760,"internally_reserved":0,"allocated":65069056,"data_stored":64767762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986310,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488444,"kb_used_data":127788,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883396,"statfs":{"total":96636764160,"available":96136597504,"internally_reserved":0,"allocated":130854912,"data_stored":130557695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659254,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473592,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93898248,"statfs":{"total":96636764160,"available":96151805952,"internally_reserved":0,"allocated":499712,"data_stored":156670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":484437679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":8021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:05.078 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:05.410 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:05.411 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:23:05.430 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":407,"stamp":"2024-09-11T01:23:04.815831+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27250,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20390,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1484796,"kb_used_data":191820,"kb_used_omap":12,"kb_used_meta":1292915,"kb_avail":281630724,"statfs":{"total":289910292480,"available":288389861376,"internally_reserved":0,"allocated":196423680,"data_stored":195482127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1323944964},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1909,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1909,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.827993"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":21655,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:03.224675+0000","last_change":"2024-09-11T01:22:54.587427+0000","last_active":"2024-09-11T01:23:03.224675+0000","last_peered":"2024-09-11T01:23:03.224675+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:23:03.224675+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3426,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":646,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3426}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20472,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996478+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:55.996478+0000","last_peered":"2024-09-11T01:22:55.996478+0000","last_clean":"2024-09-11T01:22:55.996478+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:55.996478+0000","last_undegraded":"2024-09-11T01:22:55.996478+0000","last_fullsized":"2024-09-11T01:22:55.996478+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20296,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.149418+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:22:56.149418+0000","last_peered":"2024-09-11T01:22:56.149418+0000","last_clean":"2024-09-11T01:22:56.149418+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:22:56.149418+0000","last_undegraded":"2024-09-11T01:22:56.149418+0000","last_fullsized":"2024-09-11T01:22:56.149418+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20222,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996767+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:55.996767+0000","last_peered":"2024-09-11T01:22:55.996767+0000","last_clean":"2024-09-11T01:22:55.996767+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:55.996767+0000","last_undegraded":"2024-09-11T01:22:55.996767+0000","last_fullsized":"2024-09-11T01:22:55.996767+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20156,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001082+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:56.001082+0000","last_peered":"2024-09-11T01:22:56.001082+0000","last_clean":"2024-09-11T01:22:56.001082+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:56.001082+0000","last_undegraded":"2024-09-11T01:22:56.001082+0000","last_fullsized":"2024-09-11T01:22:56.001082+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20416,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000893+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:56.000893+0000","last_peered":"2024-09-11T01:22:56.000893+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:56.000893+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20079,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000461+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:56.000461+0000","last_peered":"2024-09-11T01:22:56.000461+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:56.000461+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20403,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000263+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:56.000263+0000","last_peered":"2024-09-11T01:22:56.000263+0000","last_clean":"2024-09-11T01:22:56.000263+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:56.000263+0000","last_undegraded":"2024-09-11T01:22:56.000263+0000","last_fullsized":"2024-09-11T01:22:56.000263+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999147+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:55.999147+0000","last_peered":"2024-09-11T01:22:55.999147+0000","last_clean":"2024-09-11T01:22:55.999147+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:55.999147+0000","last_undegraded":"2024-09-11T01:22:55.999147+0000","last_fullsized":"2024-09-11T01:22:55.999147+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28000,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999439+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:55.999439+0000","last_peered":"2024-09-11T01:22:55.999439+0000","last_clean":"2024-09-11T01:22:55.999439+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:55.999439+0000","last_undegraded":"2024-09-11T01:22:55.999439+0000","last_fullsized":"2024-09-11T01:22:55.999439+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:55.994373+0000","last_peered":"2024-09-11T01:22:55.994373+0000","last_clean":"2024-09-11T01:22:55.994373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:55.994373+0000","last_undegraded":"2024-09-11T01:22:55.994373+0000","last_fullsized":"2024-09-11T01:22:55.994373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20015,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001237+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:56.001237+0000","last_peered":"2024-09-11T01:22:56.001237+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:56.001237+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000522+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:56.000522+0000","last_peered":"2024-09-11T01:22:56.000522+0000","last_clean":"2024-09-11T01:22:56.000522+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:56.000522+0000","last_undegraded":"2024-09-11T01:22:56.000522+0000","last_fullsized":"2024-09-11T01:22:56.000522+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20345,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:55.994507+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:55.994507+0000","last_peered":"2024-09-11T01:22:55.994507+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:55.994507+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999168+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:55.999168+0000","last_peered":"2024-09-11T01:22:55.999168+0000","last_clean":"2024-09-11T01:22:55.999168+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:55.999168+0000","last_undegraded":"2024-09-11T01:22:55.999168+0000","last_fullsized":"2024-09-11T01:22:55.999168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27796,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994249+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:55.994249+0000","last_peered":"2024-09-11T01:22:55.994249+0000","last_clean":"2024-09-11T01:22:55.994249+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:55.994249+0000","last_undegraded":"2024-09-11T01:22:55.994249+0000","last_fullsized":"2024-09-11T01:22:55.994249+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20415,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.006380+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:56.006380+0000","last_peered":"2024-09-11T01:22:56.006380+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:56.006380+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003870+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:56.003870+0000","last_peered":"2024-09-11T01:22:56.003870+0000","last_clean":"2024-09-11T01:22:56.003870+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:56.003870+0000","last_undegraded":"2024-09-11T01:22:56.003870+0000","last_fullsized":"2024-09-11T01:22:56.003870+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27934,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003846+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:56.003846+0000","last_peered":"2024-09-11T01:22:56.003846+0000","last_clean":"2024-09-11T01:22:56.003846+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:56.003846+0000","last_undegraded":"2024-09-11T01:22:56.003846+0000","last_fullsized":"2024-09-11T01:22:56.003846+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996759+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:55.996759+0000","last_peered":"2024-09-11T01:22:55.996759+0000","last_clean":"2024-09-11T01:22:55.996759+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:55.996759+0000","last_undegraded":"2024-09-11T01:22:55.996759+0000","last_fullsized":"2024-09-11T01:22:55.996759+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24613,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001573+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:56.001573+0000","last_peered":"2024-09-11T01:22:56.001573+0000","last_clean":"2024-09-11T01:22:56.001573+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:56.001573+0000","last_undegraded":"2024-09-11T01:22:56.001573+0000","last_fullsized":"2024-09-11T01:22:56.001573+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999413+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:55.999413+0000","last_peered":"2024-09-11T01:22:55.999413+0000","last_clean":"2024-09-11T01:22:55.999413+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:55.999413+0000","last_undegraded":"2024-09-11T01:22:55.999413+0000","last_fullsized":"2024-09-11T01:22:55.999413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999502+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:55.999502+0000","last_peered":"2024-09-11T01:22:55.999502+0000","last_clean":"2024-09-11T01:22:55.999502+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:55.999502+0000","last_undegraded":"2024-09-11T01:22:55.999502+0000","last_fullsized":"2024-09-11T01:22:55.999502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20105,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001118+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:56.001118+0000","last_peered":"2024-09-11T01:22:56.001118+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:56.001118+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27250,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20389,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195088384,"data_stored":195036199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182538,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518600,"kb_used_data":63544,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853240,"statfs":{"total":96636764160,"available":96105717760,"internally_reserved":0,"allocated":65069056,"data_stored":64767762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986310,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488444,"kb_used_data":127788,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883396,"statfs":{"total":96636764160,"available":96136597504,"internally_reserved":0,"allocated":130854912,"data_stored":130557695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659255,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":477752,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":477243,"kb_avail":93894088,"statfs":{"total":96636764160,"available":96147546112,"internally_reserved":0,"allocated":499712,"data_stored":156670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":488697519},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":8021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:05.431 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:05.850 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:05.850 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:23:05.869 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":407,"stamp":"2024-09-11T01:23:04.815831+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27250,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20390,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1484796,"kb_used_data":191820,"kb_used_omap":12,"kb_used_meta":1292915,"kb_avail":281630724,"statfs":{"total":289910292480,"available":288389861376,"internally_reserved":0,"allocated":196423680,"data_stored":195482127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1323944964},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1909,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1909,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.827993"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":21655,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:03.224675+0000","last_change":"2024-09-11T01:22:54.587427+0000","last_active":"2024-09-11T01:23:03.224675+0000","last_peered":"2024-09-11T01:23:03.224675+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:23:03.224675+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3426,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":646,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3426}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20472,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996478+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:55.996478+0000","last_peered":"2024-09-11T01:22:55.996478+0000","last_clean":"2024-09-11T01:22:55.996478+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:55.996478+0000","last_undegraded":"2024-09-11T01:22:55.996478+0000","last_fullsized":"2024-09-11T01:22:55.996478+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20296,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.149418+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:22:56.149418+0000","last_peered":"2024-09-11T01:22:56.149418+0000","last_clean":"2024-09-11T01:22:56.149418+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:22:56.149418+0000","last_undegraded":"2024-09-11T01:22:56.149418+0000","last_fullsized":"2024-09-11T01:22:56.149418+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20222,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996767+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:55.996767+0000","last_peered":"2024-09-11T01:22:55.996767+0000","last_clean":"2024-09-11T01:22:55.996767+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:55.996767+0000","last_undegraded":"2024-09-11T01:22:55.996767+0000","last_fullsized":"2024-09-11T01:22:55.996767+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20156,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001082+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:56.001082+0000","last_peered":"2024-09-11T01:22:56.001082+0000","last_clean":"2024-09-11T01:22:56.001082+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:56.001082+0000","last_undegraded":"2024-09-11T01:22:56.001082+0000","last_fullsized":"2024-09-11T01:22:56.001082+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20416,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000893+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:56.000893+0000","last_peered":"2024-09-11T01:22:56.000893+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:56.000893+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20079,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000461+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:56.000461+0000","last_peered":"2024-09-11T01:22:56.000461+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:56.000461+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20403,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000263+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:56.000263+0000","last_peered":"2024-09-11T01:22:56.000263+0000","last_clean":"2024-09-11T01:22:56.000263+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:56.000263+0000","last_undegraded":"2024-09-11T01:22:56.000263+0000","last_fullsized":"2024-09-11T01:22:56.000263+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999147+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:55.999147+0000","last_peered":"2024-09-11T01:22:55.999147+0000","last_clean":"2024-09-11T01:22:55.999147+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:55.999147+0000","last_undegraded":"2024-09-11T01:22:55.999147+0000","last_fullsized":"2024-09-11T01:22:55.999147+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28000,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999439+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:55.999439+0000","last_peered":"2024-09-11T01:22:55.999439+0000","last_clean":"2024-09-11T01:22:55.999439+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:55.999439+0000","last_undegraded":"2024-09-11T01:22:55.999439+0000","last_fullsized":"2024-09-11T01:22:55.999439+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:55.994373+0000","last_peered":"2024-09-11T01:22:55.994373+0000","last_clean":"2024-09-11T01:22:55.994373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:55.994373+0000","last_undegraded":"2024-09-11T01:22:55.994373+0000","last_fullsized":"2024-09-11T01:22:55.994373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20015,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001237+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:56.001237+0000","last_peered":"2024-09-11T01:22:56.001237+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:56.001237+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000522+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:56.000522+0000","last_peered":"2024-09-11T01:22:56.000522+0000","last_clean":"2024-09-11T01:22:56.000522+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:56.000522+0000","last_undegraded":"2024-09-11T01:22:56.000522+0000","last_fullsized":"2024-09-11T01:22:56.000522+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20345,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:55.994507+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:55.994507+0000","last_peered":"2024-09-11T01:22:55.994507+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:55.994507+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999168+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:55.999168+0000","last_peered":"2024-09-11T01:22:55.999168+0000","last_clean":"2024-09-11T01:22:55.999168+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:55.999168+0000","last_undegraded":"2024-09-11T01:22:55.999168+0000","last_fullsized":"2024-09-11T01:22:55.999168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27796,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994249+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:55.994249+0000","last_peered":"2024-09-11T01:22:55.994249+0000","last_clean":"2024-09-11T01:22:55.994249+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:55.994249+0000","last_undegraded":"2024-09-11T01:22:55.994249+0000","last_fullsized":"2024-09-11T01:22:55.994249+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20415,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.006380+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:56.006380+0000","last_peered":"2024-09-11T01:22:56.006380+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:56.006380+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003870+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:56.003870+0000","last_peered":"2024-09-11T01:22:56.003870+0000","last_clean":"2024-09-11T01:22:56.003870+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:56.003870+0000","last_undegraded":"2024-09-11T01:22:56.003870+0000","last_fullsized":"2024-09-11T01:22:56.003870+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27934,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003846+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:56.003846+0000","last_peered":"2024-09-11T01:22:56.003846+0000","last_clean":"2024-09-11T01:22:56.003846+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:56.003846+0000","last_undegraded":"2024-09-11T01:22:56.003846+0000","last_fullsized":"2024-09-11T01:22:56.003846+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996759+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:55.996759+0000","last_peered":"2024-09-11T01:22:55.996759+0000","last_clean":"2024-09-11T01:22:55.996759+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:55.996759+0000","last_undegraded":"2024-09-11T01:22:55.996759+0000","last_fullsized":"2024-09-11T01:22:55.996759+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24613,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001573+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:56.001573+0000","last_peered":"2024-09-11T01:22:56.001573+0000","last_clean":"2024-09-11T01:22:56.001573+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:56.001573+0000","last_undegraded":"2024-09-11T01:22:56.001573+0000","last_fullsized":"2024-09-11T01:22:56.001573+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999413+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:55.999413+0000","last_peered":"2024-09-11T01:22:55.999413+0000","last_clean":"2024-09-11T01:22:55.999413+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:55.999413+0000","last_undegraded":"2024-09-11T01:22:55.999413+0000","last_fullsized":"2024-09-11T01:22:55.999413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999502+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:55.999502+0000","last_peered":"2024-09-11T01:22:55.999502+0000","last_clean":"2024-09-11T01:22:55.999502+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:55.999502+0000","last_undegraded":"2024-09-11T01:22:55.999502+0000","last_fullsized":"2024-09-11T01:22:55.999502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20105,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001118+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:56.001118+0000","last_peered":"2024-09-11T01:22:56.001118+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:56.001118+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27250,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20389,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195088384,"data_stored":195036199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182538,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518600,"kb_used_data":63544,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853240,"statfs":{"total":96636764160,"available":96105717760,"internally_reserved":0,"allocated":65069056,"data_stored":64767762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986310,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488444,"kb_used_data":127788,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883396,"statfs":{"total":96636764160,"available":96136597504,"internally_reserved":0,"allocated":130854912,"data_stored":130557695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659255,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":477752,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":477243,"kb_avail":93894088,"statfs":{"total":96636764160,"available":96147546112,"internally_reserved":0,"allocated":499712,"data_stored":156670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":488697519},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":8021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:05.870 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:23:05.870 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20266", 'reported_seq': 21655, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:03.224675+0000', 'last_change': '2024-09-11T01:22:54.587427+0000', 'last_active': '2024-09-11T01:23:03.224675+0000', 'last_peered': '2024-09-11T01:23:03.224675+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:23:03.224675+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3426, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20266, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 646, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3426}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:05.871 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:23:05.871 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'20227", 'reported_seq': 20416, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.000893+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:22:56.000893+0000', 'last_peered': '2024-09-11T01:22:56.000893+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:22:56.000893+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16568, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:05.871 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:23:05.871 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20079, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.000461+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:22:56.000461+0000', 'last_peered': '2024-09-11T01:22:56.000461+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:22:56.000461+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:05.871 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:23:05.871 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20015, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.001237+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:22:56.001237+0000', 'last_peered': '2024-09-11T01:22:56.001237+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:22:56.001237+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:05.871 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:23:05.871 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'20207", 'reported_seq': 20345, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:55.994507+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:22:55.994507+0000', 'last_peered': '2024-09-11T01:22:55.994507+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:22:55.994507+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18024, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4024}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:05.871 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:23:05.871 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20264", 'reported_seq': 20415, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.006380+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:22:56.006380+0000', 'last_peered': '2024-09-11T01:22:56.006380+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:22:56.006380+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17001, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:05.871 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:23:05.871 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 20105, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.001118+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:22:56.001118+0000', 'last_peered': '2024-09-11T01:22:56.001118+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:22:56.001118+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:05.872 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:23:06.243 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:06.243 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 27250/6 objects degraded (454166.667%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":722,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1520431104,"bytes_avail":288389861376,"bytes_total":289910292480,"degraded_objects":27250,"degraded_total":6,"degraded_ratio":4541.666666666667,"recovering_objects_per_sec":194,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1909,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:23:06.262 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:23:06.262 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:06.574 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:06.575 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:23:06.593 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":407,"stamp":"2024-09-11T01:23:04.815831+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27250,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20390,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1484796,"kb_used_data":191820,"kb_used_omap":12,"kb_used_meta":1292915,"kb_avail":281630724,"statfs":{"total":289910292480,"available":288389861376,"internally_reserved":0,"allocated":196423680,"data_stored":195482127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1323944964},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1909,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1909,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.827993"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":21655,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:03.224675+0000","last_change":"2024-09-11T01:22:54.587427+0000","last_active":"2024-09-11T01:23:03.224675+0000","last_peered":"2024-09-11T01:23:03.224675+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:23:03.224675+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3426,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":646,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3426}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20472,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996478+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:55.996478+0000","last_peered":"2024-09-11T01:22:55.996478+0000","last_clean":"2024-09-11T01:22:55.996478+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:55.996478+0000","last_undegraded":"2024-09-11T01:22:55.996478+0000","last_fullsized":"2024-09-11T01:22:55.996478+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20296,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.149418+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:22:56.149418+0000","last_peered":"2024-09-11T01:22:56.149418+0000","last_clean":"2024-09-11T01:22:56.149418+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:22:56.149418+0000","last_undegraded":"2024-09-11T01:22:56.149418+0000","last_fullsized":"2024-09-11T01:22:56.149418+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20222,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996767+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:55.996767+0000","last_peered":"2024-09-11T01:22:55.996767+0000","last_clean":"2024-09-11T01:22:55.996767+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:55.996767+0000","last_undegraded":"2024-09-11T01:22:55.996767+0000","last_fullsized":"2024-09-11T01:22:55.996767+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20156,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001082+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:56.001082+0000","last_peered":"2024-09-11T01:22:56.001082+0000","last_clean":"2024-09-11T01:22:56.001082+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:56.001082+0000","last_undegraded":"2024-09-11T01:22:56.001082+0000","last_fullsized":"2024-09-11T01:22:56.001082+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20416,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000893+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:56.000893+0000","last_peered":"2024-09-11T01:22:56.000893+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:56.000893+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20079,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000461+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:56.000461+0000","last_peered":"2024-09-11T01:22:56.000461+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:56.000461+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20403,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000263+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:56.000263+0000","last_peered":"2024-09-11T01:22:56.000263+0000","last_clean":"2024-09-11T01:22:56.000263+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:56.000263+0000","last_undegraded":"2024-09-11T01:22:56.000263+0000","last_fullsized":"2024-09-11T01:22:56.000263+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999147+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:55.999147+0000","last_peered":"2024-09-11T01:22:55.999147+0000","last_clean":"2024-09-11T01:22:55.999147+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:55.999147+0000","last_undegraded":"2024-09-11T01:22:55.999147+0000","last_fullsized":"2024-09-11T01:22:55.999147+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28000,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999439+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:55.999439+0000","last_peered":"2024-09-11T01:22:55.999439+0000","last_clean":"2024-09-11T01:22:55.999439+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:55.999439+0000","last_undegraded":"2024-09-11T01:22:55.999439+0000","last_fullsized":"2024-09-11T01:22:55.999439+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:55.994373+0000","last_peered":"2024-09-11T01:22:55.994373+0000","last_clean":"2024-09-11T01:22:55.994373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:55.994373+0000","last_undegraded":"2024-09-11T01:22:55.994373+0000","last_fullsized":"2024-09-11T01:22:55.994373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20015,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001237+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:56.001237+0000","last_peered":"2024-09-11T01:22:56.001237+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:56.001237+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000522+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:56.000522+0000","last_peered":"2024-09-11T01:22:56.000522+0000","last_clean":"2024-09-11T01:22:56.000522+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:56.000522+0000","last_undegraded":"2024-09-11T01:22:56.000522+0000","last_fullsized":"2024-09-11T01:22:56.000522+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20345,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:55.994507+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:55.994507+0000","last_peered":"2024-09-11T01:22:55.994507+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:55.994507+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999168+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:55.999168+0000","last_peered":"2024-09-11T01:22:55.999168+0000","last_clean":"2024-09-11T01:22:55.999168+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:55.999168+0000","last_undegraded":"2024-09-11T01:22:55.999168+0000","last_fullsized":"2024-09-11T01:22:55.999168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27796,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994249+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:55.994249+0000","last_peered":"2024-09-11T01:22:55.994249+0000","last_clean":"2024-09-11T01:22:55.994249+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:55.994249+0000","last_undegraded":"2024-09-11T01:22:55.994249+0000","last_fullsized":"2024-09-11T01:22:55.994249+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20415,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.006380+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:56.006380+0000","last_peered":"2024-09-11T01:22:56.006380+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:56.006380+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003870+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:56.003870+0000","last_peered":"2024-09-11T01:22:56.003870+0000","last_clean":"2024-09-11T01:22:56.003870+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:56.003870+0000","last_undegraded":"2024-09-11T01:22:56.003870+0000","last_fullsized":"2024-09-11T01:22:56.003870+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27934,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003846+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:56.003846+0000","last_peered":"2024-09-11T01:22:56.003846+0000","last_clean":"2024-09-11T01:22:56.003846+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:56.003846+0000","last_undegraded":"2024-09-11T01:22:56.003846+0000","last_fullsized":"2024-09-11T01:22:56.003846+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996759+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:55.996759+0000","last_peered":"2024-09-11T01:22:55.996759+0000","last_clean":"2024-09-11T01:22:55.996759+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:55.996759+0000","last_undegraded":"2024-09-11T01:22:55.996759+0000","last_fullsized":"2024-09-11T01:22:55.996759+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24613,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001573+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:56.001573+0000","last_peered":"2024-09-11T01:22:56.001573+0000","last_clean":"2024-09-11T01:22:56.001573+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:56.001573+0000","last_undegraded":"2024-09-11T01:22:56.001573+0000","last_fullsized":"2024-09-11T01:22:56.001573+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999413+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:55.999413+0000","last_peered":"2024-09-11T01:22:55.999413+0000","last_clean":"2024-09-11T01:22:55.999413+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:55.999413+0000","last_undegraded":"2024-09-11T01:22:55.999413+0000","last_fullsized":"2024-09-11T01:22:55.999413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999502+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:55.999502+0000","last_peered":"2024-09-11T01:22:55.999502+0000","last_clean":"2024-09-11T01:22:55.999502+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:55.999502+0000","last_undegraded":"2024-09-11T01:22:55.999502+0000","last_fullsized":"2024-09-11T01:22:55.999502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20105,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001118+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:56.001118+0000","last_peered":"2024-09-11T01:22:56.001118+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:56.001118+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":27250,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20389,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195088384,"data_stored":195036199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182538,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518600,"kb_used_data":63544,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853240,"statfs":{"total":96636764160,"available":96105717760,"internally_reserved":0,"allocated":65069056,"data_stored":64767762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986310,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488444,"kb_used_data":127788,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883396,"statfs":{"total":96636764160,"available":96136597504,"internally_reserved":0,"allocated":130854912,"data_stored":130557695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659255,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":477752,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":477243,"kb_avail":93894088,"statfs":{"total":96636764160,"available":96147546112,"internally_reserved":0,"allocated":499712,"data_stored":156670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":488697519},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":8021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:09.596 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:09.921 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:09.921 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:23:09.942 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":409,"stamp":"2024-09-11T01:23:08.817450+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20687,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1468156,"kb_used_data":191820,"kb_used_omap":12,"kb_used_meta":1276275,"kb_avail":281647364,"statfs":{"total":289910292480,"available":288406900736,"internally_reserved":0,"allocated":196423680,"data_stored":195482127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1306905604},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2206,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004033"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":22249,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:08.222096+0000","last_change":"2024-09-11T01:22:54.587427+0000","last_active":"2024-09-11T01:23:08.222096+0000","last_peered":"2024-09-11T01:23:08.222096+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:23:08.222096+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3129,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":943,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3129}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20472,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996478+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:55.996478+0000","last_peered":"2024-09-11T01:22:55.996478+0000","last_clean":"2024-09-11T01:22:55.996478+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:55.996478+0000","last_undegraded":"2024-09-11T01:22:55.996478+0000","last_fullsized":"2024-09-11T01:22:55.996478+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20296,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.149418+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:22:56.149418+0000","last_peered":"2024-09-11T01:22:56.149418+0000","last_clean":"2024-09-11T01:22:56.149418+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:22:56.149418+0000","last_undegraded":"2024-09-11T01:22:56.149418+0000","last_fullsized":"2024-09-11T01:22:56.149418+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20222,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996767+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:55.996767+0000","last_peered":"2024-09-11T01:22:55.996767+0000","last_clean":"2024-09-11T01:22:55.996767+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:55.996767+0000","last_undegraded":"2024-09-11T01:22:55.996767+0000","last_fullsized":"2024-09-11T01:22:55.996767+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20156,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001082+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:56.001082+0000","last_peered":"2024-09-11T01:22:56.001082+0000","last_clean":"2024-09-11T01:22:56.001082+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:56.001082+0000","last_undegraded":"2024-09-11T01:22:56.001082+0000","last_fullsized":"2024-09-11T01:22:56.001082+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20416,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000893+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:56.000893+0000","last_peered":"2024-09-11T01:22:56.000893+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:56.000893+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20079,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000461+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:56.000461+0000","last_peered":"2024-09-11T01:22:56.000461+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:56.000461+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20403,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000263+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:56.000263+0000","last_peered":"2024-09-11T01:22:56.000263+0000","last_clean":"2024-09-11T01:22:56.000263+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:56.000263+0000","last_undegraded":"2024-09-11T01:22:56.000263+0000","last_fullsized":"2024-09-11T01:22:56.000263+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999147+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:55.999147+0000","last_peered":"2024-09-11T01:22:55.999147+0000","last_clean":"2024-09-11T01:22:55.999147+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:55.999147+0000","last_undegraded":"2024-09-11T01:22:55.999147+0000","last_fullsized":"2024-09-11T01:22:55.999147+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28000,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999439+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:55.999439+0000","last_peered":"2024-09-11T01:22:55.999439+0000","last_clean":"2024-09-11T01:22:55.999439+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:55.999439+0000","last_undegraded":"2024-09-11T01:22:55.999439+0000","last_fullsized":"2024-09-11T01:22:55.999439+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:55.994373+0000","last_peered":"2024-09-11T01:22:55.994373+0000","last_clean":"2024-09-11T01:22:55.994373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:55.994373+0000","last_undegraded":"2024-09-11T01:22:55.994373+0000","last_fullsized":"2024-09-11T01:22:55.994373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20015,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001237+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:56.001237+0000","last_peered":"2024-09-11T01:22:56.001237+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:56.001237+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000522+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:56.000522+0000","last_peered":"2024-09-11T01:22:56.000522+0000","last_clean":"2024-09-11T01:22:56.000522+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:56.000522+0000","last_undegraded":"2024-09-11T01:22:56.000522+0000","last_fullsized":"2024-09-11T01:22:56.000522+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20345,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:55.994507+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:55.994507+0000","last_peered":"2024-09-11T01:22:55.994507+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:55.994507+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999168+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:55.999168+0000","last_peered":"2024-09-11T01:22:55.999168+0000","last_clean":"2024-09-11T01:22:55.999168+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:55.999168+0000","last_undegraded":"2024-09-11T01:22:55.999168+0000","last_fullsized":"2024-09-11T01:22:55.999168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27796,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994249+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:55.994249+0000","last_peered":"2024-09-11T01:22:55.994249+0000","last_clean":"2024-09-11T01:22:55.994249+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:55.994249+0000","last_undegraded":"2024-09-11T01:22:55.994249+0000","last_fullsized":"2024-09-11T01:22:55.994249+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20415,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.006380+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:56.006380+0000","last_peered":"2024-09-11T01:22:56.006380+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:56.006380+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003870+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:56.003870+0000","last_peered":"2024-09-11T01:22:56.003870+0000","last_clean":"2024-09-11T01:22:56.003870+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:56.003870+0000","last_undegraded":"2024-09-11T01:22:56.003870+0000","last_fullsized":"2024-09-11T01:22:56.003870+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27934,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003846+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:56.003846+0000","last_peered":"2024-09-11T01:22:56.003846+0000","last_clean":"2024-09-11T01:22:56.003846+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:56.003846+0000","last_undegraded":"2024-09-11T01:22:56.003846+0000","last_fullsized":"2024-09-11T01:22:56.003846+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996759+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:55.996759+0000","last_peered":"2024-09-11T01:22:55.996759+0000","last_clean":"2024-09-11T01:22:55.996759+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:55.996759+0000","last_undegraded":"2024-09-11T01:22:55.996759+0000","last_fullsized":"2024-09-11T01:22:55.996759+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24613,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001573+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:56.001573+0000","last_peered":"2024-09-11T01:22:56.001573+0000","last_clean":"2024-09-11T01:22:56.001573+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:56.001573+0000","last_undegraded":"2024-09-11T01:22:56.001573+0000","last_fullsized":"2024-09-11T01:22:56.001573+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999413+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:55.999413+0000","last_peered":"2024-09-11T01:22:55.999413+0000","last_clean":"2024-09-11T01:22:55.999413+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:55.999413+0000","last_undegraded":"2024-09-11T01:22:55.999413+0000","last_fullsized":"2024-09-11T01:22:55.999413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999502+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:55.999502+0000","last_peered":"2024-09-11T01:22:55.999502+0000","last_clean":"2024-09-11T01:22:55.999502+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:55.999502+0000","last_undegraded":"2024-09-11T01:22:55.999502+0000","last_fullsized":"2024-09-11T01:22:55.999502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20105,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001118+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:56.001118+0000","last_peered":"2024-09-11T01:22:56.001118+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:56.001118+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20686,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195088384,"data_stored":195036199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182539,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518600,"kb_used_data":63544,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853240,"statfs":{"total":96636764160,"available":96105717760,"internally_reserved":0,"allocated":65069056,"data_stored":64767762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986311,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488444,"kb_used_data":127788,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883396,"statfs":{"total":96636764160,"available":96136597504,"internally_reserved":0,"allocated":130854912,"data_stored":130557695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659256,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461112,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93910728,"statfs":{"total":96636764160,"available":96164585472,"internally_reserved":0,"allocated":499712,"data_stored":156670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":471658159},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":8021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:09.943 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:10.257 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:10.257 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:23:10.276 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":409,"stamp":"2024-09-11T01:23:08.817450+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20687,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1468156,"kb_used_data":191820,"kb_used_omap":12,"kb_used_meta":1276275,"kb_avail":281647364,"statfs":{"total":289910292480,"available":288406900736,"internally_reserved":0,"allocated":196423680,"data_stored":195482127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1306905604},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2206,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2206,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004033"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":22249,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:08.222096+0000","last_change":"2024-09-11T01:22:54.587427+0000","last_active":"2024-09-11T01:23:08.222096+0000","last_peered":"2024-09-11T01:23:08.222096+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:23:08.222096+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3129,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":943,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3129}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20472,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996478+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:55.996478+0000","last_peered":"2024-09-11T01:22:55.996478+0000","last_clean":"2024-09-11T01:22:55.996478+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:55.996478+0000","last_undegraded":"2024-09-11T01:22:55.996478+0000","last_fullsized":"2024-09-11T01:22:55.996478+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20296,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.149418+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:22:56.149418+0000","last_peered":"2024-09-11T01:22:56.149418+0000","last_clean":"2024-09-11T01:22:56.149418+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:22:56.149418+0000","last_undegraded":"2024-09-11T01:22:56.149418+0000","last_fullsized":"2024-09-11T01:22:56.149418+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20222,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996767+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:55.996767+0000","last_peered":"2024-09-11T01:22:55.996767+0000","last_clean":"2024-09-11T01:22:55.996767+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:55.996767+0000","last_undegraded":"2024-09-11T01:22:55.996767+0000","last_fullsized":"2024-09-11T01:22:55.996767+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20156,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001082+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:56.001082+0000","last_peered":"2024-09-11T01:22:56.001082+0000","last_clean":"2024-09-11T01:22:56.001082+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:56.001082+0000","last_undegraded":"2024-09-11T01:22:56.001082+0000","last_fullsized":"2024-09-11T01:22:56.001082+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20416,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000893+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:56.000893+0000","last_peered":"2024-09-11T01:22:56.000893+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:56.000893+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20079,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000461+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:56.000461+0000","last_peered":"2024-09-11T01:22:56.000461+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:56.000461+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20403,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000263+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:56.000263+0000","last_peered":"2024-09-11T01:22:56.000263+0000","last_clean":"2024-09-11T01:22:56.000263+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:56.000263+0000","last_undegraded":"2024-09-11T01:22:56.000263+0000","last_fullsized":"2024-09-11T01:22:56.000263+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999147+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:55.999147+0000","last_peered":"2024-09-11T01:22:55.999147+0000","last_clean":"2024-09-11T01:22:55.999147+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:55.999147+0000","last_undegraded":"2024-09-11T01:22:55.999147+0000","last_fullsized":"2024-09-11T01:22:55.999147+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28000,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999439+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:55.999439+0000","last_peered":"2024-09-11T01:22:55.999439+0000","last_clean":"2024-09-11T01:22:55.999439+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:55.999439+0000","last_undegraded":"2024-09-11T01:22:55.999439+0000","last_fullsized":"2024-09-11T01:22:55.999439+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:55.994373+0000","last_peered":"2024-09-11T01:22:55.994373+0000","last_clean":"2024-09-11T01:22:55.994373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:55.994373+0000","last_undegraded":"2024-09-11T01:22:55.994373+0000","last_fullsized":"2024-09-11T01:22:55.994373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20015,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001237+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:56.001237+0000","last_peered":"2024-09-11T01:22:56.001237+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:56.001237+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000522+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:56.000522+0000","last_peered":"2024-09-11T01:22:56.000522+0000","last_clean":"2024-09-11T01:22:56.000522+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:56.000522+0000","last_undegraded":"2024-09-11T01:22:56.000522+0000","last_fullsized":"2024-09-11T01:22:56.000522+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20345,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:55.994507+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:55.994507+0000","last_peered":"2024-09-11T01:22:55.994507+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:55.994507+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999168+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:55.999168+0000","last_peered":"2024-09-11T01:22:55.999168+0000","last_clean":"2024-09-11T01:22:55.999168+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:55.999168+0000","last_undegraded":"2024-09-11T01:22:55.999168+0000","last_fullsized":"2024-09-11T01:22:55.999168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27796,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994249+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:55.994249+0000","last_peered":"2024-09-11T01:22:55.994249+0000","last_clean":"2024-09-11T01:22:55.994249+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:55.994249+0000","last_undegraded":"2024-09-11T01:22:55.994249+0000","last_fullsized":"2024-09-11T01:22:55.994249+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20415,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.006380+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:56.006380+0000","last_peered":"2024-09-11T01:22:56.006380+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:56.006380+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003870+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:56.003870+0000","last_peered":"2024-09-11T01:22:56.003870+0000","last_clean":"2024-09-11T01:22:56.003870+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:56.003870+0000","last_undegraded":"2024-09-11T01:22:56.003870+0000","last_fullsized":"2024-09-11T01:22:56.003870+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27934,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003846+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:56.003846+0000","last_peered":"2024-09-11T01:22:56.003846+0000","last_clean":"2024-09-11T01:22:56.003846+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:56.003846+0000","last_undegraded":"2024-09-11T01:22:56.003846+0000","last_fullsized":"2024-09-11T01:22:56.003846+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996759+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:55.996759+0000","last_peered":"2024-09-11T01:22:55.996759+0000","last_clean":"2024-09-11T01:22:55.996759+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:55.996759+0000","last_undegraded":"2024-09-11T01:22:55.996759+0000","last_fullsized":"2024-09-11T01:22:55.996759+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24613,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001573+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:56.001573+0000","last_peered":"2024-09-11T01:22:56.001573+0000","last_clean":"2024-09-11T01:22:56.001573+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:56.001573+0000","last_undegraded":"2024-09-11T01:22:56.001573+0000","last_fullsized":"2024-09-11T01:22:56.001573+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999413+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:55.999413+0000","last_peered":"2024-09-11T01:22:55.999413+0000","last_clean":"2024-09-11T01:22:55.999413+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:55.999413+0000","last_undegraded":"2024-09-11T01:22:55.999413+0000","last_fullsized":"2024-09-11T01:22:55.999413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999502+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:55.999502+0000","last_peered":"2024-09-11T01:22:55.999502+0000","last_clean":"2024-09-11T01:22:55.999502+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:55.999502+0000","last_undegraded":"2024-09-11T01:22:55.999502+0000","last_fullsized":"2024-09-11T01:22:55.999502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20105,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001118+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:56.001118+0000","last_peered":"2024-09-11T01:22:56.001118+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:56.001118+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20686,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195088384,"data_stored":195036199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182539,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518600,"kb_used_data":63544,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853240,"statfs":{"total":96636764160,"available":96105717760,"internally_reserved":0,"allocated":65069056,"data_stored":64767762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986311,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488444,"kb_used_data":127788,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883396,"statfs":{"total":96636764160,"available":96136597504,"internally_reserved":0,"allocated":130854912,"data_stored":130557695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659256,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461112,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93910728,"statfs":{"total":96636764160,"available":96164585472,"internally_reserved":0,"allocated":499712,"data_stored":156670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":471658159},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":8021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:10.277 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:23:10.278 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20266", 'reported_seq': 22249, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:08.222096+0000', 'last_change': '2024-09-11T01:22:54.587427+0000', 'last_active': '2024-09-11T01:23:08.222096+0000', 'last_peered': '2024-09-11T01:23:08.222096+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:23:08.222096+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3129, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20266, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 943, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3129}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:10.278 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:23:10.278 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'20227", 'reported_seq': 20416, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.000893+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:22:56.000893+0000', 'last_peered': '2024-09-11T01:22:56.000893+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:22:56.000893+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16568, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:10.278 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:23:10.278 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20079, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.000461+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:22:56.000461+0000', 'last_peered': '2024-09-11T01:22:56.000461+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:22:56.000461+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:10.278 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:23:10.278 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20015, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.001237+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:22:56.001237+0000', 'last_peered': '2024-09-11T01:22:56.001237+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:22:56.001237+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:10.278 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:23:10.278 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'20207", 'reported_seq': 20345, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:55.994507+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:22:55.994507+0000', 'last_peered': '2024-09-11T01:22:55.994507+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:22:55.994507+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18024, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4024}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:10.279 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:23:10.279 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20264", 'reported_seq': 20415, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.006380+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:22:56.006380+0000', 'last_peered': '2024-09-11T01:22:56.006380+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:22:56.006380+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17001, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:10.279 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:23:10.279 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 20105, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.001118+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:22:56.001118+0000', 'last_peered': '2024-09-11T01:22:56.001118+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:22:56.001118+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:10.279 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:23:10.656 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:10.656 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 26953/6 objects degraded (449216.667%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":727,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1503391744,"bytes_avail":288406900736,"bytes_total":289910292480,"degraded_objects":26953,"degraded_total":6,"degraded_ratio":4492.166666666667,"recovering_objects_per_sec":183,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2206,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:23:10.677 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:23:10.678 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:10.985 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:10.986 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:23:11.004 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":410,"stamp":"2024-09-11T01:23:10.818112+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20687,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1468156,"kb_used_data":191820,"kb_used_omap":12,"kb_used_meta":1276275,"kb_avail":281647364,"statfs":{"total":289910292480,"available":288406900736,"internally_reserved":0,"allocated":196423680,"data_stored":195482127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1306905604},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-587,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":587,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004311"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":22249,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:08.222096+0000","last_change":"2024-09-11T01:22:54.587427+0000","last_active":"2024-09-11T01:23:08.222096+0000","last_peered":"2024-09-11T01:23:08.222096+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:23:08.222096+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3129,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":943,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3129}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20472,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996478+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:55.996478+0000","last_peered":"2024-09-11T01:22:55.996478+0000","last_clean":"2024-09-11T01:22:55.996478+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:55.996478+0000","last_undegraded":"2024-09-11T01:22:55.996478+0000","last_fullsized":"2024-09-11T01:22:55.996478+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20296,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.149418+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:22:56.149418+0000","last_peered":"2024-09-11T01:22:56.149418+0000","last_clean":"2024-09-11T01:22:56.149418+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:22:56.149418+0000","last_undegraded":"2024-09-11T01:22:56.149418+0000","last_fullsized":"2024-09-11T01:22:56.149418+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20222,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996767+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:55.996767+0000","last_peered":"2024-09-11T01:22:55.996767+0000","last_clean":"2024-09-11T01:22:55.996767+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:55.996767+0000","last_undegraded":"2024-09-11T01:22:55.996767+0000","last_fullsized":"2024-09-11T01:22:55.996767+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20156,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001082+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:56.001082+0000","last_peered":"2024-09-11T01:22:56.001082+0000","last_clean":"2024-09-11T01:22:56.001082+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:56.001082+0000","last_undegraded":"2024-09-11T01:22:56.001082+0000","last_fullsized":"2024-09-11T01:22:56.001082+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20416,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000893+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:56.000893+0000","last_peered":"2024-09-11T01:22:56.000893+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:56.000893+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20079,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000461+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:56.000461+0000","last_peered":"2024-09-11T01:22:56.000461+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:56.000461+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20403,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000263+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:56.000263+0000","last_peered":"2024-09-11T01:22:56.000263+0000","last_clean":"2024-09-11T01:22:56.000263+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:56.000263+0000","last_undegraded":"2024-09-11T01:22:56.000263+0000","last_fullsized":"2024-09-11T01:22:56.000263+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999147+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:55.999147+0000","last_peered":"2024-09-11T01:22:55.999147+0000","last_clean":"2024-09-11T01:22:55.999147+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:55.999147+0000","last_undegraded":"2024-09-11T01:22:55.999147+0000","last_fullsized":"2024-09-11T01:22:55.999147+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28000,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999439+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:55.999439+0000","last_peered":"2024-09-11T01:22:55.999439+0000","last_clean":"2024-09-11T01:22:55.999439+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:55.999439+0000","last_undegraded":"2024-09-11T01:22:55.999439+0000","last_fullsized":"2024-09-11T01:22:55.999439+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:55.994373+0000","last_peered":"2024-09-11T01:22:55.994373+0000","last_clean":"2024-09-11T01:22:55.994373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:55.994373+0000","last_undegraded":"2024-09-11T01:22:55.994373+0000","last_fullsized":"2024-09-11T01:22:55.994373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20015,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001237+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:56.001237+0000","last_peered":"2024-09-11T01:22:56.001237+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:56.001237+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000522+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:56.000522+0000","last_peered":"2024-09-11T01:22:56.000522+0000","last_clean":"2024-09-11T01:22:56.000522+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:56.000522+0000","last_undegraded":"2024-09-11T01:22:56.000522+0000","last_fullsized":"2024-09-11T01:22:56.000522+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20345,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:55.994507+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:55.994507+0000","last_peered":"2024-09-11T01:22:55.994507+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:55.994507+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999168+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:55.999168+0000","last_peered":"2024-09-11T01:22:55.999168+0000","last_clean":"2024-09-11T01:22:55.999168+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:55.999168+0000","last_undegraded":"2024-09-11T01:22:55.999168+0000","last_fullsized":"2024-09-11T01:22:55.999168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27796,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994249+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:55.994249+0000","last_peered":"2024-09-11T01:22:55.994249+0000","last_clean":"2024-09-11T01:22:55.994249+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:55.994249+0000","last_undegraded":"2024-09-11T01:22:55.994249+0000","last_fullsized":"2024-09-11T01:22:55.994249+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20415,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.006380+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:56.006380+0000","last_peered":"2024-09-11T01:22:56.006380+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:56.006380+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003870+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:56.003870+0000","last_peered":"2024-09-11T01:22:56.003870+0000","last_clean":"2024-09-11T01:22:56.003870+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:56.003870+0000","last_undegraded":"2024-09-11T01:22:56.003870+0000","last_fullsized":"2024-09-11T01:22:56.003870+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27934,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003846+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:56.003846+0000","last_peered":"2024-09-11T01:22:56.003846+0000","last_clean":"2024-09-11T01:22:56.003846+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:56.003846+0000","last_undegraded":"2024-09-11T01:22:56.003846+0000","last_fullsized":"2024-09-11T01:22:56.003846+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996759+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:55.996759+0000","last_peered":"2024-09-11T01:22:55.996759+0000","last_clean":"2024-09-11T01:22:55.996759+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:55.996759+0000","last_undegraded":"2024-09-11T01:22:55.996759+0000","last_fullsized":"2024-09-11T01:22:55.996759+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24613,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001573+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:56.001573+0000","last_peered":"2024-09-11T01:22:56.001573+0000","last_clean":"2024-09-11T01:22:56.001573+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:56.001573+0000","last_undegraded":"2024-09-11T01:22:56.001573+0000","last_fullsized":"2024-09-11T01:22:56.001573+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999413+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:55.999413+0000","last_peered":"2024-09-11T01:22:55.999413+0000","last_clean":"2024-09-11T01:22:55.999413+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:55.999413+0000","last_undegraded":"2024-09-11T01:22:55.999413+0000","last_fullsized":"2024-09-11T01:22:55.999413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999502+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:55.999502+0000","last_peered":"2024-09-11T01:22:55.999502+0000","last_clean":"2024-09-11T01:22:55.999502+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:55.999502+0000","last_undegraded":"2024-09-11T01:22:55.999502+0000","last_fullsized":"2024-09-11T01:22:55.999502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20105,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001118+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:56.001118+0000","last_peered":"2024-09-11T01:22:56.001118+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:56.001118+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20686,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195088384,"data_stored":195036199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182539,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518600,"kb_used_data":63544,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853240,"statfs":{"total":96636764160,"available":96105717760,"internally_reserved":0,"allocated":65069056,"data_stored":64767762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986312,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488444,"kb_used_data":127788,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883396,"statfs":{"total":96636764160,"available":96136597504,"internally_reserved":0,"allocated":130854912,"data_stored":130557695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659256,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461112,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93910728,"statfs":{"total":96636764160,"available":96164585472,"internally_reserved":0,"allocated":499712,"data_stored":156670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":471658159},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":8021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:14.005 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:14.329 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:14.329 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:23:14.348 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":411,"stamp":"2024-09-11T01:23:12.818535+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20687,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1468156,"kb_used_data":191820,"kb_used_omap":12,"kb_used_meta":1276275,"kb_avail":281647364,"statfs":{"total":289910292480,"available":288406900736,"internally_reserved":0,"allocated":196423680,"data_stored":195482127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1306905604},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-587,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":587,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004005"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":22249,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:08.222096+0000","last_change":"2024-09-11T01:22:54.587427+0000","last_active":"2024-09-11T01:23:08.222096+0000","last_peered":"2024-09-11T01:23:08.222096+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:23:08.222096+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3129,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":943,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3129}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20472,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996478+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:55.996478+0000","last_peered":"2024-09-11T01:22:55.996478+0000","last_clean":"2024-09-11T01:22:55.996478+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:55.996478+0000","last_undegraded":"2024-09-11T01:22:55.996478+0000","last_fullsized":"2024-09-11T01:22:55.996478+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20296,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.149418+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:22:56.149418+0000","last_peered":"2024-09-11T01:22:56.149418+0000","last_clean":"2024-09-11T01:22:56.149418+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:22:56.149418+0000","last_undegraded":"2024-09-11T01:22:56.149418+0000","last_fullsized":"2024-09-11T01:22:56.149418+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20222,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996767+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:55.996767+0000","last_peered":"2024-09-11T01:22:55.996767+0000","last_clean":"2024-09-11T01:22:55.996767+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:55.996767+0000","last_undegraded":"2024-09-11T01:22:55.996767+0000","last_fullsized":"2024-09-11T01:22:55.996767+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20156,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001082+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:56.001082+0000","last_peered":"2024-09-11T01:22:56.001082+0000","last_clean":"2024-09-11T01:22:56.001082+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:56.001082+0000","last_undegraded":"2024-09-11T01:22:56.001082+0000","last_fullsized":"2024-09-11T01:22:56.001082+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20416,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000893+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:56.000893+0000","last_peered":"2024-09-11T01:22:56.000893+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:56.000893+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20079,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000461+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:56.000461+0000","last_peered":"2024-09-11T01:22:56.000461+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:56.000461+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20403,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000263+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:56.000263+0000","last_peered":"2024-09-11T01:22:56.000263+0000","last_clean":"2024-09-11T01:22:56.000263+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:56.000263+0000","last_undegraded":"2024-09-11T01:22:56.000263+0000","last_fullsized":"2024-09-11T01:22:56.000263+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999147+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:55.999147+0000","last_peered":"2024-09-11T01:22:55.999147+0000","last_clean":"2024-09-11T01:22:55.999147+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:55.999147+0000","last_undegraded":"2024-09-11T01:22:55.999147+0000","last_fullsized":"2024-09-11T01:22:55.999147+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28000,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999439+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:55.999439+0000","last_peered":"2024-09-11T01:22:55.999439+0000","last_clean":"2024-09-11T01:22:55.999439+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:55.999439+0000","last_undegraded":"2024-09-11T01:22:55.999439+0000","last_fullsized":"2024-09-11T01:22:55.999439+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:55.994373+0000","last_peered":"2024-09-11T01:22:55.994373+0000","last_clean":"2024-09-11T01:22:55.994373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:55.994373+0000","last_undegraded":"2024-09-11T01:22:55.994373+0000","last_fullsized":"2024-09-11T01:22:55.994373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20015,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001237+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:56.001237+0000","last_peered":"2024-09-11T01:22:56.001237+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:56.001237+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000522+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:56.000522+0000","last_peered":"2024-09-11T01:22:56.000522+0000","last_clean":"2024-09-11T01:22:56.000522+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:56.000522+0000","last_undegraded":"2024-09-11T01:22:56.000522+0000","last_fullsized":"2024-09-11T01:22:56.000522+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20345,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:55.994507+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:55.994507+0000","last_peered":"2024-09-11T01:22:55.994507+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:55.994507+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999168+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:55.999168+0000","last_peered":"2024-09-11T01:22:55.999168+0000","last_clean":"2024-09-11T01:22:55.999168+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:55.999168+0000","last_undegraded":"2024-09-11T01:22:55.999168+0000","last_fullsized":"2024-09-11T01:22:55.999168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27796,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994249+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:55.994249+0000","last_peered":"2024-09-11T01:22:55.994249+0000","last_clean":"2024-09-11T01:22:55.994249+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:55.994249+0000","last_undegraded":"2024-09-11T01:22:55.994249+0000","last_fullsized":"2024-09-11T01:22:55.994249+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20415,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.006380+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:56.006380+0000","last_peered":"2024-09-11T01:22:56.006380+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:56.006380+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003870+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:56.003870+0000","last_peered":"2024-09-11T01:22:56.003870+0000","last_clean":"2024-09-11T01:22:56.003870+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:56.003870+0000","last_undegraded":"2024-09-11T01:22:56.003870+0000","last_fullsized":"2024-09-11T01:22:56.003870+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27934,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003846+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:56.003846+0000","last_peered":"2024-09-11T01:22:56.003846+0000","last_clean":"2024-09-11T01:22:56.003846+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:56.003846+0000","last_undegraded":"2024-09-11T01:22:56.003846+0000","last_fullsized":"2024-09-11T01:22:56.003846+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996759+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:55.996759+0000","last_peered":"2024-09-11T01:22:55.996759+0000","last_clean":"2024-09-11T01:22:55.996759+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:55.996759+0000","last_undegraded":"2024-09-11T01:22:55.996759+0000","last_fullsized":"2024-09-11T01:22:55.996759+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24613,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001573+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:56.001573+0000","last_peered":"2024-09-11T01:22:56.001573+0000","last_clean":"2024-09-11T01:22:56.001573+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:56.001573+0000","last_undegraded":"2024-09-11T01:22:56.001573+0000","last_fullsized":"2024-09-11T01:22:56.001573+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999413+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:55.999413+0000","last_peered":"2024-09-11T01:22:55.999413+0000","last_clean":"2024-09-11T01:22:55.999413+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:55.999413+0000","last_undegraded":"2024-09-11T01:22:55.999413+0000","last_fullsized":"2024-09-11T01:22:55.999413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999502+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:55.999502+0000","last_peered":"2024-09-11T01:22:55.999502+0000","last_clean":"2024-09-11T01:22:55.999502+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:55.999502+0000","last_undegraded":"2024-09-11T01:22:55.999502+0000","last_fullsized":"2024-09-11T01:22:55.999502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20105,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001118+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:56.001118+0000","last_peered":"2024-09-11T01:22:56.001118+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:56.001118+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20686,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195088384,"data_stored":195036199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182539,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518600,"kb_used_data":63544,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853240,"statfs":{"total":96636764160,"available":96105717760,"internally_reserved":0,"allocated":65069056,"data_stored":64767762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986312,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488444,"kb_used_data":127788,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883396,"statfs":{"total":96636764160,"available":96136597504,"internally_reserved":0,"allocated":130854912,"data_stored":130557695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659256,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461112,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93910728,"statfs":{"total":96636764160,"available":96164585472,"internally_reserved":0,"allocated":499712,"data_stored":156670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":471658159},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":8021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:14.349 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:14.665 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:14.666 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:23:14.684 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":411,"stamp":"2024-09-11T01:23:12.818535+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20687,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1468156,"kb_used_data":191820,"kb_used_omap":12,"kb_used_meta":1276275,"kb_avail":281647364,"statfs":{"total":289910292480,"available":288406900736,"internally_reserved":0,"allocated":196423680,"data_stored":195482127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1306905604},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-587,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":587,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004005"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":22249,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:08.222096+0000","last_change":"2024-09-11T01:22:54.587427+0000","last_active":"2024-09-11T01:23:08.222096+0000","last_peered":"2024-09-11T01:23:08.222096+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:23:08.222096+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3129,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":943,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3129}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20472,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996478+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:55.996478+0000","last_peered":"2024-09-11T01:22:55.996478+0000","last_clean":"2024-09-11T01:22:55.996478+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:55.996478+0000","last_undegraded":"2024-09-11T01:22:55.996478+0000","last_fullsized":"2024-09-11T01:22:55.996478+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20296,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.149418+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:22:56.149418+0000","last_peered":"2024-09-11T01:22:56.149418+0000","last_clean":"2024-09-11T01:22:56.149418+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:22:56.149418+0000","last_undegraded":"2024-09-11T01:22:56.149418+0000","last_fullsized":"2024-09-11T01:22:56.149418+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20222,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996767+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:55.996767+0000","last_peered":"2024-09-11T01:22:55.996767+0000","last_clean":"2024-09-11T01:22:55.996767+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:55.996767+0000","last_undegraded":"2024-09-11T01:22:55.996767+0000","last_fullsized":"2024-09-11T01:22:55.996767+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20156,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001082+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:56.001082+0000","last_peered":"2024-09-11T01:22:56.001082+0000","last_clean":"2024-09-11T01:22:56.001082+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:56.001082+0000","last_undegraded":"2024-09-11T01:22:56.001082+0000","last_fullsized":"2024-09-11T01:22:56.001082+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20416,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000893+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:56.000893+0000","last_peered":"2024-09-11T01:22:56.000893+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:56.000893+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20079,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000461+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:56.000461+0000","last_peered":"2024-09-11T01:22:56.000461+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:56.000461+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20403,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000263+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:56.000263+0000","last_peered":"2024-09-11T01:22:56.000263+0000","last_clean":"2024-09-11T01:22:56.000263+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:56.000263+0000","last_undegraded":"2024-09-11T01:22:56.000263+0000","last_fullsized":"2024-09-11T01:22:56.000263+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999147+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:55.999147+0000","last_peered":"2024-09-11T01:22:55.999147+0000","last_clean":"2024-09-11T01:22:55.999147+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:55.999147+0000","last_undegraded":"2024-09-11T01:22:55.999147+0000","last_fullsized":"2024-09-11T01:22:55.999147+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28000,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999439+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:55.999439+0000","last_peered":"2024-09-11T01:22:55.999439+0000","last_clean":"2024-09-11T01:22:55.999439+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:55.999439+0000","last_undegraded":"2024-09-11T01:22:55.999439+0000","last_fullsized":"2024-09-11T01:22:55.999439+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:55.994373+0000","last_peered":"2024-09-11T01:22:55.994373+0000","last_clean":"2024-09-11T01:22:55.994373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:55.994373+0000","last_undegraded":"2024-09-11T01:22:55.994373+0000","last_fullsized":"2024-09-11T01:22:55.994373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20015,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001237+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:56.001237+0000","last_peered":"2024-09-11T01:22:56.001237+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:56.001237+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000522+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:56.000522+0000","last_peered":"2024-09-11T01:22:56.000522+0000","last_clean":"2024-09-11T01:22:56.000522+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:56.000522+0000","last_undegraded":"2024-09-11T01:22:56.000522+0000","last_fullsized":"2024-09-11T01:22:56.000522+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20345,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:55.994507+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:55.994507+0000","last_peered":"2024-09-11T01:22:55.994507+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:55.994507+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999168+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:55.999168+0000","last_peered":"2024-09-11T01:22:55.999168+0000","last_clean":"2024-09-11T01:22:55.999168+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:55.999168+0000","last_undegraded":"2024-09-11T01:22:55.999168+0000","last_fullsized":"2024-09-11T01:22:55.999168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27796,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994249+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:55.994249+0000","last_peered":"2024-09-11T01:22:55.994249+0000","last_clean":"2024-09-11T01:22:55.994249+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:55.994249+0000","last_undegraded":"2024-09-11T01:22:55.994249+0000","last_fullsized":"2024-09-11T01:22:55.994249+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20415,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.006380+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:56.006380+0000","last_peered":"2024-09-11T01:22:56.006380+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:56.006380+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003870+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:56.003870+0000","last_peered":"2024-09-11T01:22:56.003870+0000","last_clean":"2024-09-11T01:22:56.003870+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:56.003870+0000","last_undegraded":"2024-09-11T01:22:56.003870+0000","last_fullsized":"2024-09-11T01:22:56.003870+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27934,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003846+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:56.003846+0000","last_peered":"2024-09-11T01:22:56.003846+0000","last_clean":"2024-09-11T01:22:56.003846+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:56.003846+0000","last_undegraded":"2024-09-11T01:22:56.003846+0000","last_fullsized":"2024-09-11T01:22:56.003846+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996759+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:55.996759+0000","last_peered":"2024-09-11T01:22:55.996759+0000","last_clean":"2024-09-11T01:22:55.996759+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:55.996759+0000","last_undegraded":"2024-09-11T01:22:55.996759+0000","last_fullsized":"2024-09-11T01:22:55.996759+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24613,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001573+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:56.001573+0000","last_peered":"2024-09-11T01:22:56.001573+0000","last_clean":"2024-09-11T01:22:56.001573+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:56.001573+0000","last_undegraded":"2024-09-11T01:22:56.001573+0000","last_fullsized":"2024-09-11T01:22:56.001573+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999413+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:55.999413+0000","last_peered":"2024-09-11T01:22:55.999413+0000","last_clean":"2024-09-11T01:22:55.999413+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:55.999413+0000","last_undegraded":"2024-09-11T01:22:55.999413+0000","last_fullsized":"2024-09-11T01:22:55.999413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999502+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:55.999502+0000","last_peered":"2024-09-11T01:22:55.999502+0000","last_clean":"2024-09-11T01:22:55.999502+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:55.999502+0000","last_undegraded":"2024-09-11T01:22:55.999502+0000","last_fullsized":"2024-09-11T01:22:55.999502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20105,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001118+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:56.001118+0000","last_peered":"2024-09-11T01:22:56.001118+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:56.001118+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20686,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195088384,"data_stored":195036199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182539,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518600,"kb_used_data":63544,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853240,"statfs":{"total":96636764160,"available":96105717760,"internally_reserved":0,"allocated":65069056,"data_stored":64767762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986312,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488444,"kb_used_data":127788,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883396,"statfs":{"total":96636764160,"available":96136597504,"internally_reserved":0,"allocated":130854912,"data_stored":130557695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659256,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461112,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93910728,"statfs":{"total":96636764160,"available":96164585472,"internally_reserved":0,"allocated":499712,"data_stored":156670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":471658159},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":8021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:14.685 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:23:14.685 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20266", 'reported_seq': 22249, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:08.222096+0000', 'last_change': '2024-09-11T01:22:54.587427+0000', 'last_active': '2024-09-11T01:23:08.222096+0000', 'last_peered': '2024-09-11T01:23:08.222096+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:23:08.222096+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3129, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20266, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 943, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3129}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:14.685 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:23:14.686 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'20227", 'reported_seq': 20416, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.000893+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:22:56.000893+0000', 'last_peered': '2024-09-11T01:22:56.000893+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:22:56.000893+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16568, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:14.686 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:23:14.686 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20079, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.000461+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:22:56.000461+0000', 'last_peered': '2024-09-11T01:22:56.000461+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:22:56.000461+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:14.686 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:23:14.686 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20015, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.001237+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:22:56.001237+0000', 'last_peered': '2024-09-11T01:22:56.001237+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:22:56.001237+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:14.686 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:23:14.686 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'20207", 'reported_seq': 20345, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:55.994507+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:22:55.994507+0000', 'last_peered': '2024-09-11T01:22:55.994507+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:22:55.994507+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18024, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4024}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:14.686 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:23:14.686 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20264", 'reported_seq': 20415, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.006380+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:22:56.006380+0000', 'last_peered': '2024-09-11T01:22:56.006380+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:22:56.006380+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17001, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:14.687 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:23:14.687 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 20105, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.001118+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:22:56.001118+0000', 'last_peered': '2024-09-11T01:22:56.001118+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:22:56.001118+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:14.687 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:23:15.068 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:15.069 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 26953/6 objects degraded (449216.667%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":731,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1503391744,"bytes_avail":288406900736,"bytes_total":289910292480,"degraded_objects":26953,"degraded_total":6,"degraded_ratio":4492.166666666667,"recovering_objects_per_sec":48,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":587,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:23:15.088 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:23:15.088 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:15.498 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:15.498 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:23:15.516 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":412,"stamp":"2024-09-11T01:23:14.819228+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26642,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20998,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472316,"kb_used_data":191820,"kb_used_omap":12,"kb_used_meta":1280435,"kb_avail":281643204,"statfs":{"total":289910292480,"available":288402640896,"internally_reserved":0,"allocated":196423680,"data_stored":195482127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1311165444},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-898,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":898,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004056"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":22872,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:13.235680+0000","last_change":"2024-09-11T01:22:54.587427+0000","last_active":"2024-09-11T01:23:13.235680+0000","last_peered":"2024-09-11T01:23:13.235680+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:23:13.235680+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2818,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1254,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2818}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20472,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996478+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:55.996478+0000","last_peered":"2024-09-11T01:22:55.996478+0000","last_clean":"2024-09-11T01:22:55.996478+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:55.996478+0000","last_undegraded":"2024-09-11T01:22:55.996478+0000","last_fullsized":"2024-09-11T01:22:55.996478+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20296,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.149418+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:22:56.149418+0000","last_peered":"2024-09-11T01:22:56.149418+0000","last_clean":"2024-09-11T01:22:56.149418+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:22:56.149418+0000","last_undegraded":"2024-09-11T01:22:56.149418+0000","last_fullsized":"2024-09-11T01:22:56.149418+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20222,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996767+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:55.996767+0000","last_peered":"2024-09-11T01:22:55.996767+0000","last_clean":"2024-09-11T01:22:55.996767+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:55.996767+0000","last_undegraded":"2024-09-11T01:22:55.996767+0000","last_fullsized":"2024-09-11T01:22:55.996767+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20156,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001082+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:56.001082+0000","last_peered":"2024-09-11T01:22:56.001082+0000","last_clean":"2024-09-11T01:22:56.001082+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:56.001082+0000","last_undegraded":"2024-09-11T01:22:56.001082+0000","last_fullsized":"2024-09-11T01:22:56.001082+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20416,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000893+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:56.000893+0000","last_peered":"2024-09-11T01:22:56.000893+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:56.000893+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20079,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000461+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:56.000461+0000","last_peered":"2024-09-11T01:22:56.000461+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:56.000461+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20403,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000263+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:56.000263+0000","last_peered":"2024-09-11T01:22:56.000263+0000","last_clean":"2024-09-11T01:22:56.000263+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:56.000263+0000","last_undegraded":"2024-09-11T01:22:56.000263+0000","last_fullsized":"2024-09-11T01:22:56.000263+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999267+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:55.999267+0000","last_peered":"2024-09-11T01:22:55.999267+0000","last_clean":"2024-09-11T01:22:55.999267+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:55.999267+0000","last_undegraded":"2024-09-11T01:22:55.999267+0000","last_fullsized":"2024-09-11T01:22:55.999267+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28000,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999439+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:55.999439+0000","last_peered":"2024-09-11T01:22:55.999439+0000","last_clean":"2024-09-11T01:22:55.999439+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:55.999439+0000","last_undegraded":"2024-09-11T01:22:55.999439+0000","last_fullsized":"2024-09-11T01:22:55.999439+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:55.994373+0000","last_peered":"2024-09-11T01:22:55.994373+0000","last_clean":"2024-09-11T01:22:55.994373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:55.994373+0000","last_undegraded":"2024-09-11T01:22:55.994373+0000","last_fullsized":"2024-09-11T01:22:55.994373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20015,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001237+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:56.001237+0000","last_peered":"2024-09-11T01:22:56.001237+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:56.001237+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000522+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:56.000522+0000","last_peered":"2024-09-11T01:22:56.000522+0000","last_clean":"2024-09-11T01:22:56.000522+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:56.000522+0000","last_undegraded":"2024-09-11T01:22:56.000522+0000","last_fullsized":"2024-09-11T01:22:56.000522+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20345,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:55.994507+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:55.994507+0000","last_peered":"2024-09-11T01:22:55.994507+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:55.994507+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999168+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:55.999168+0000","last_peered":"2024-09-11T01:22:55.999168+0000","last_clean":"2024-09-11T01:22:55.999168+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:55.999168+0000","last_undegraded":"2024-09-11T01:22:55.999168+0000","last_fullsized":"2024-09-11T01:22:55.999168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27796,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994249+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:55.994249+0000","last_peered":"2024-09-11T01:22:55.994249+0000","last_clean":"2024-09-11T01:22:55.994249+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:55.994249+0000","last_undegraded":"2024-09-11T01:22:55.994249+0000","last_fullsized":"2024-09-11T01:22:55.994249+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20415,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.006380+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:56.006380+0000","last_peered":"2024-09-11T01:22:56.006380+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:56.006380+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003870+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:56.003870+0000","last_peered":"2024-09-11T01:22:56.003870+0000","last_clean":"2024-09-11T01:22:56.003870+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:56.003870+0000","last_undegraded":"2024-09-11T01:22:56.003870+0000","last_fullsized":"2024-09-11T01:22:56.003870+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27934,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003846+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:56.003846+0000","last_peered":"2024-09-11T01:22:56.003846+0000","last_clean":"2024-09-11T01:22:56.003846+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:56.003846+0000","last_undegraded":"2024-09-11T01:22:56.003846+0000","last_fullsized":"2024-09-11T01:22:56.003846+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996759+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:55.996759+0000","last_peered":"2024-09-11T01:22:55.996759+0000","last_clean":"2024-09-11T01:22:55.996759+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:55.996759+0000","last_undegraded":"2024-09-11T01:22:55.996759+0000","last_fullsized":"2024-09-11T01:22:55.996759+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24613,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001573+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:56.001573+0000","last_peered":"2024-09-11T01:22:56.001573+0000","last_clean":"2024-09-11T01:22:56.001573+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:56.001573+0000","last_undegraded":"2024-09-11T01:22:56.001573+0000","last_fullsized":"2024-09-11T01:22:56.001573+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999413+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:55.999413+0000","last_peered":"2024-09-11T01:22:55.999413+0000","last_clean":"2024-09-11T01:22:55.999413+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:55.999413+0000","last_undegraded":"2024-09-11T01:22:55.999413+0000","last_fullsized":"2024-09-11T01:22:55.999413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999502+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:55.999502+0000","last_peered":"2024-09-11T01:22:55.999502+0000","last_clean":"2024-09-11T01:22:55.999502+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:55.999502+0000","last_undegraded":"2024-09-11T01:22:55.999502+0000","last_fullsized":"2024-09-11T01:22:55.999502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20105,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001118+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:56.001118+0000","last_peered":"2024-09-11T01:22:56.001118+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:56.001118+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26642,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20997,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195088384,"data_stored":195036199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182540,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518600,"kb_used_data":63544,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853240,"statfs":{"total":96636764160,"available":96105717760,"internally_reserved":0,"allocated":65069056,"data_stored":64767762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986312,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488444,"kb_used_data":127788,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883396,"statfs":{"total":96636764160,"available":96136597504,"internally_reserved":0,"allocated":130854912,"data_stored":130557695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659257,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465272,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93906568,"statfs":{"total":96636764160,"available":96160325632,"internally_reserved":0,"allocated":499712,"data_stored":156670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475917999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":8021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:18.518 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:18.851 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:18.851 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:23:18.869 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":414,"stamp":"2024-09-11T01:23:18.820398+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21334,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472316,"kb_used_data":191820,"kb_used_omap":12,"kb_used_meta":1280435,"kb_avail":281643204,"statfs":{"total":289910292480,"available":288402640896,"internally_reserved":0,"allocated":196423680,"data_stored":195482127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1311165444},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-944,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":944,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003832"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":23544,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:18.244252+0000","last_change":"2024-09-11T01:22:54.587427+0000","last_active":"2024-09-11T01:23:18.244252+0000","last_peered":"2024-09-11T01:23:18.244252+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:23:18.244252+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2482,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1590,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2482}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20472,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996478+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:55.996478+0000","last_peered":"2024-09-11T01:22:55.996478+0000","last_clean":"2024-09-11T01:22:55.996478+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:55.996478+0000","last_undegraded":"2024-09-11T01:22:55.996478+0000","last_fullsized":"2024-09-11T01:22:55.996478+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20296,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.149418+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:22:56.149418+0000","last_peered":"2024-09-11T01:22:56.149418+0000","last_clean":"2024-09-11T01:22:56.149418+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:22:56.149418+0000","last_undegraded":"2024-09-11T01:22:56.149418+0000","last_fullsized":"2024-09-11T01:22:56.149418+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20222,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996767+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:55.996767+0000","last_peered":"2024-09-11T01:22:55.996767+0000","last_clean":"2024-09-11T01:22:55.996767+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:55.996767+0000","last_undegraded":"2024-09-11T01:22:55.996767+0000","last_fullsized":"2024-09-11T01:22:55.996767+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20156,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001082+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:56.001082+0000","last_peered":"2024-09-11T01:22:56.001082+0000","last_clean":"2024-09-11T01:22:56.001082+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:56.001082+0000","last_undegraded":"2024-09-11T01:22:56.001082+0000","last_fullsized":"2024-09-11T01:22:56.001082+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20416,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000893+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:56.000893+0000","last_peered":"2024-09-11T01:22:56.000893+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:56.000893+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20079,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000461+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:56.000461+0000","last_peered":"2024-09-11T01:22:56.000461+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:56.000461+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20403,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000263+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:56.000263+0000","last_peered":"2024-09-11T01:22:56.000263+0000","last_clean":"2024-09-11T01:22:56.000263+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:56.000263+0000","last_undegraded":"2024-09-11T01:22:56.000263+0000","last_fullsized":"2024-09-11T01:22:56.000263+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999267+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:55.999267+0000","last_peered":"2024-09-11T01:22:55.999267+0000","last_clean":"2024-09-11T01:22:55.999267+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:55.999267+0000","last_undegraded":"2024-09-11T01:22:55.999267+0000","last_fullsized":"2024-09-11T01:22:55.999267+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28000,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999439+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:55.999439+0000","last_peered":"2024-09-11T01:22:55.999439+0000","last_clean":"2024-09-11T01:22:55.999439+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:55.999439+0000","last_undegraded":"2024-09-11T01:22:55.999439+0000","last_fullsized":"2024-09-11T01:22:55.999439+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:55.994373+0000","last_peered":"2024-09-11T01:22:55.994373+0000","last_clean":"2024-09-11T01:22:55.994373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:55.994373+0000","last_undegraded":"2024-09-11T01:22:55.994373+0000","last_fullsized":"2024-09-11T01:22:55.994373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20015,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001237+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:56.001237+0000","last_peered":"2024-09-11T01:22:56.001237+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:56.001237+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000522+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:56.000522+0000","last_peered":"2024-09-11T01:22:56.000522+0000","last_clean":"2024-09-11T01:22:56.000522+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:56.000522+0000","last_undegraded":"2024-09-11T01:22:56.000522+0000","last_fullsized":"2024-09-11T01:22:56.000522+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20345,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:55.994507+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:55.994507+0000","last_peered":"2024-09-11T01:22:55.994507+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:55.994507+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999168+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:55.999168+0000","last_peered":"2024-09-11T01:22:55.999168+0000","last_clean":"2024-09-11T01:22:55.999168+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:55.999168+0000","last_undegraded":"2024-09-11T01:22:55.999168+0000","last_fullsized":"2024-09-11T01:22:55.999168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27796,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994249+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:55.994249+0000","last_peered":"2024-09-11T01:22:55.994249+0000","last_clean":"2024-09-11T01:22:55.994249+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:55.994249+0000","last_undegraded":"2024-09-11T01:22:55.994249+0000","last_fullsized":"2024-09-11T01:22:55.994249+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20415,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.006380+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:56.006380+0000","last_peered":"2024-09-11T01:22:56.006380+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:56.006380+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003870+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:56.003870+0000","last_peered":"2024-09-11T01:22:56.003870+0000","last_clean":"2024-09-11T01:22:56.003870+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:56.003870+0000","last_undegraded":"2024-09-11T01:22:56.003870+0000","last_fullsized":"2024-09-11T01:22:56.003870+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27934,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003846+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:56.003846+0000","last_peered":"2024-09-11T01:22:56.003846+0000","last_clean":"2024-09-11T01:22:56.003846+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:56.003846+0000","last_undegraded":"2024-09-11T01:22:56.003846+0000","last_fullsized":"2024-09-11T01:22:56.003846+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996759+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:55.996759+0000","last_peered":"2024-09-11T01:22:55.996759+0000","last_clean":"2024-09-11T01:22:55.996759+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:55.996759+0000","last_undegraded":"2024-09-11T01:22:55.996759+0000","last_fullsized":"2024-09-11T01:22:55.996759+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24613,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001573+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:56.001573+0000","last_peered":"2024-09-11T01:22:56.001573+0000","last_clean":"2024-09-11T01:22:56.001573+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:56.001573+0000","last_undegraded":"2024-09-11T01:22:56.001573+0000","last_fullsized":"2024-09-11T01:22:56.001573+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999413+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:55.999413+0000","last_peered":"2024-09-11T01:22:55.999413+0000","last_clean":"2024-09-11T01:22:55.999413+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:55.999413+0000","last_undegraded":"2024-09-11T01:22:55.999413+0000","last_fullsized":"2024-09-11T01:22:55.999413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999502+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:55.999502+0000","last_peered":"2024-09-11T01:22:55.999502+0000","last_clean":"2024-09-11T01:22:55.999502+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:55.999502+0000","last_undegraded":"2024-09-11T01:22:55.999502+0000","last_fullsized":"2024-09-11T01:22:55.999502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20105,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001118+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:56.001118+0000","last_peered":"2024-09-11T01:22:56.001118+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:56.001118+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21333,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195088384,"data_stored":195036199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182541,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518600,"kb_used_data":63544,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853240,"statfs":{"total":96636764160,"available":96105717760,"internally_reserved":0,"allocated":65069056,"data_stored":64767762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986313,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488444,"kb_used_data":127788,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883396,"statfs":{"total":96636764160,"available":96136597504,"internally_reserved":0,"allocated":130854912,"data_stored":130557695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659258,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465272,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93906568,"statfs":{"total":96636764160,"available":96160325632,"internally_reserved":0,"allocated":499712,"data_stored":156670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475917999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":8021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:18.871 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:19.217 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:19.218 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:23:19.240 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":414,"stamp":"2024-09-11T01:23:18.820398+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21334,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472316,"kb_used_data":191820,"kb_used_omap":12,"kb_used_meta":1280435,"kb_avail":281643204,"statfs":{"total":289910292480,"available":288402640896,"internally_reserved":0,"allocated":196423680,"data_stored":195482127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1311165444},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-944,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":944,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003832"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":23544,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:18.244252+0000","last_change":"2024-09-11T01:22:54.587427+0000","last_active":"2024-09-11T01:23:18.244252+0000","last_peered":"2024-09-11T01:23:18.244252+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:23:18.244252+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2482,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1590,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2482}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20472,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996478+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:55.996478+0000","last_peered":"2024-09-11T01:22:55.996478+0000","last_clean":"2024-09-11T01:22:55.996478+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:55.996478+0000","last_undegraded":"2024-09-11T01:22:55.996478+0000","last_fullsized":"2024-09-11T01:22:55.996478+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20296,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.149418+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:22:56.149418+0000","last_peered":"2024-09-11T01:22:56.149418+0000","last_clean":"2024-09-11T01:22:56.149418+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:22:56.149418+0000","last_undegraded":"2024-09-11T01:22:56.149418+0000","last_fullsized":"2024-09-11T01:22:56.149418+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20222,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996767+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:55.996767+0000","last_peered":"2024-09-11T01:22:55.996767+0000","last_clean":"2024-09-11T01:22:55.996767+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:55.996767+0000","last_undegraded":"2024-09-11T01:22:55.996767+0000","last_fullsized":"2024-09-11T01:22:55.996767+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20156,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001082+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:56.001082+0000","last_peered":"2024-09-11T01:22:56.001082+0000","last_clean":"2024-09-11T01:22:56.001082+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:56.001082+0000","last_undegraded":"2024-09-11T01:22:56.001082+0000","last_fullsized":"2024-09-11T01:22:56.001082+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20416,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000893+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:56.000893+0000","last_peered":"2024-09-11T01:22:56.000893+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:56.000893+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20079,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000461+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:56.000461+0000","last_peered":"2024-09-11T01:22:56.000461+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:56.000461+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20403,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000263+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:56.000263+0000","last_peered":"2024-09-11T01:22:56.000263+0000","last_clean":"2024-09-11T01:22:56.000263+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:56.000263+0000","last_undegraded":"2024-09-11T01:22:56.000263+0000","last_fullsized":"2024-09-11T01:22:56.000263+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999267+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:55.999267+0000","last_peered":"2024-09-11T01:22:55.999267+0000","last_clean":"2024-09-11T01:22:55.999267+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:55.999267+0000","last_undegraded":"2024-09-11T01:22:55.999267+0000","last_fullsized":"2024-09-11T01:22:55.999267+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28000,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999439+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:55.999439+0000","last_peered":"2024-09-11T01:22:55.999439+0000","last_clean":"2024-09-11T01:22:55.999439+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:55.999439+0000","last_undegraded":"2024-09-11T01:22:55.999439+0000","last_fullsized":"2024-09-11T01:22:55.999439+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:55.994373+0000","last_peered":"2024-09-11T01:22:55.994373+0000","last_clean":"2024-09-11T01:22:55.994373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:55.994373+0000","last_undegraded":"2024-09-11T01:22:55.994373+0000","last_fullsized":"2024-09-11T01:22:55.994373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20015,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001237+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:56.001237+0000","last_peered":"2024-09-11T01:22:56.001237+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:56.001237+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000522+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:56.000522+0000","last_peered":"2024-09-11T01:22:56.000522+0000","last_clean":"2024-09-11T01:22:56.000522+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:56.000522+0000","last_undegraded":"2024-09-11T01:22:56.000522+0000","last_fullsized":"2024-09-11T01:22:56.000522+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20345,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:55.994507+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:55.994507+0000","last_peered":"2024-09-11T01:22:55.994507+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:55.994507+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999168+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:55.999168+0000","last_peered":"2024-09-11T01:22:55.999168+0000","last_clean":"2024-09-11T01:22:55.999168+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:55.999168+0000","last_undegraded":"2024-09-11T01:22:55.999168+0000","last_fullsized":"2024-09-11T01:22:55.999168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27796,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994249+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:55.994249+0000","last_peered":"2024-09-11T01:22:55.994249+0000","last_clean":"2024-09-11T01:22:55.994249+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:55.994249+0000","last_undegraded":"2024-09-11T01:22:55.994249+0000","last_fullsized":"2024-09-11T01:22:55.994249+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20415,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.006380+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:56.006380+0000","last_peered":"2024-09-11T01:22:56.006380+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:56.006380+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003870+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:56.003870+0000","last_peered":"2024-09-11T01:22:56.003870+0000","last_clean":"2024-09-11T01:22:56.003870+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:56.003870+0000","last_undegraded":"2024-09-11T01:22:56.003870+0000","last_fullsized":"2024-09-11T01:22:56.003870+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27934,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003846+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:56.003846+0000","last_peered":"2024-09-11T01:22:56.003846+0000","last_clean":"2024-09-11T01:22:56.003846+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:56.003846+0000","last_undegraded":"2024-09-11T01:22:56.003846+0000","last_fullsized":"2024-09-11T01:22:56.003846+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996759+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:55.996759+0000","last_peered":"2024-09-11T01:22:55.996759+0000","last_clean":"2024-09-11T01:22:55.996759+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:55.996759+0000","last_undegraded":"2024-09-11T01:22:55.996759+0000","last_fullsized":"2024-09-11T01:22:55.996759+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24613,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001573+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:56.001573+0000","last_peered":"2024-09-11T01:22:56.001573+0000","last_clean":"2024-09-11T01:22:56.001573+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:56.001573+0000","last_undegraded":"2024-09-11T01:22:56.001573+0000","last_fullsized":"2024-09-11T01:22:56.001573+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999413+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:55.999413+0000","last_peered":"2024-09-11T01:22:55.999413+0000","last_clean":"2024-09-11T01:22:55.999413+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:55.999413+0000","last_undegraded":"2024-09-11T01:22:55.999413+0000","last_fullsized":"2024-09-11T01:22:55.999413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999502+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:55.999502+0000","last_peered":"2024-09-11T01:22:55.999502+0000","last_clean":"2024-09-11T01:22:55.999502+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:55.999502+0000","last_undegraded":"2024-09-11T01:22:55.999502+0000","last_fullsized":"2024-09-11T01:22:55.999502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20105,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001118+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:56.001118+0000","last_peered":"2024-09-11T01:22:56.001118+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:56.001118+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21333,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195088384,"data_stored":195036199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182541,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518600,"kb_used_data":63544,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853240,"statfs":{"total":96636764160,"available":96105717760,"internally_reserved":0,"allocated":65069056,"data_stored":64767762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986313,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488444,"kb_used_data":127788,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883396,"statfs":{"total":96636764160,"available":96136597504,"internally_reserved":0,"allocated":130854912,"data_stored":130557695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659258,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465272,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93906568,"statfs":{"total":96636764160,"available":96160325632,"internally_reserved":0,"allocated":499712,"data_stored":156670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475917999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":8021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:19.241 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:23:19.241 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20266", 'reported_seq': 23544, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:18.244252+0000', 'last_change': '2024-09-11T01:22:54.587427+0000', 'last_active': '2024-09-11T01:23:18.244252+0000', 'last_peered': '2024-09-11T01:23:18.244252+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:23:18.244252+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2482, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20266, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1590, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2482}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:19.241 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:23:19.241 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'20227", 'reported_seq': 20416, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.000893+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:22:56.000893+0000', 'last_peered': '2024-09-11T01:22:56.000893+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:22:56.000893+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16568, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:19.242 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:23:19.242 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20079, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.000461+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:22:56.000461+0000', 'last_peered': '2024-09-11T01:22:56.000461+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:22:56.000461+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:19.242 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:23:19.242 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20015, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.001237+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:22:56.001237+0000', 'last_peered': '2024-09-11T01:22:56.001237+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:22:56.001237+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:19.242 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:23:19.242 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'20207", 'reported_seq': 20345, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:55.994507+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:22:55.994507+0000', 'last_peered': '2024-09-11T01:22:55.994507+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:22:55.994507+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18024, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4024}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:19.242 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:23:19.242 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20264", 'reported_seq': 20415, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.006380+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:22:56.006380+0000', 'last_peered': '2024-09-11T01:22:56.006380+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:22:56.006380+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17001, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:19.242 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:23:19.242 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 20105, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.001118+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:22:56.001118+0000', 'last_peered': '2024-09-11T01:22:56.001118+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:22:56.001118+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:19.243 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:23:19.627 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:19.627 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 26306/6 objects degraded (438433.333%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":736,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1507651584,"bytes_avail":288402640896,"bytes_total":289910292480,"degraded_objects":26306,"degraded_total":6,"degraded_ratio":4384.333333333333,"recovering_objects_per_sec":78,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":944,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:23:19.647 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:23:19.647 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:19.993 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:19.994 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:23:20.012 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":414,"stamp":"2024-09-11T01:23:18.820398+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21334,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472316,"kb_used_data":191820,"kb_used_omap":12,"kb_used_meta":1280435,"kb_avail":281643204,"statfs":{"total":289910292480,"available":288402640896,"internally_reserved":0,"allocated":196423680,"data_stored":195482127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1311165444},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-944,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":944,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003832"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":23544,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:18.244252+0000","last_change":"2024-09-11T01:22:54.587427+0000","last_active":"2024-09-11T01:23:18.244252+0000","last_peered":"2024-09-11T01:23:18.244252+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:23:18.244252+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2482,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1590,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2482}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20472,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996478+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:55.996478+0000","last_peered":"2024-09-11T01:22:55.996478+0000","last_clean":"2024-09-11T01:22:55.996478+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:55.996478+0000","last_undegraded":"2024-09-11T01:22:55.996478+0000","last_fullsized":"2024-09-11T01:22:55.996478+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20296,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.149418+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:22:56.149418+0000","last_peered":"2024-09-11T01:22:56.149418+0000","last_clean":"2024-09-11T01:22:56.149418+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:22:56.149418+0000","last_undegraded":"2024-09-11T01:22:56.149418+0000","last_fullsized":"2024-09-11T01:22:56.149418+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20222,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996767+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:55.996767+0000","last_peered":"2024-09-11T01:22:55.996767+0000","last_clean":"2024-09-11T01:22:55.996767+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:55.996767+0000","last_undegraded":"2024-09-11T01:22:55.996767+0000","last_fullsized":"2024-09-11T01:22:55.996767+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20156,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001082+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:56.001082+0000","last_peered":"2024-09-11T01:22:56.001082+0000","last_clean":"2024-09-11T01:22:56.001082+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:56.001082+0000","last_undegraded":"2024-09-11T01:22:56.001082+0000","last_fullsized":"2024-09-11T01:22:56.001082+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20416,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000893+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:56.000893+0000","last_peered":"2024-09-11T01:22:56.000893+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:56.000893+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20079,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000461+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:56.000461+0000","last_peered":"2024-09-11T01:22:56.000461+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:56.000461+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20403,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000263+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:56.000263+0000","last_peered":"2024-09-11T01:22:56.000263+0000","last_clean":"2024-09-11T01:22:56.000263+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:56.000263+0000","last_undegraded":"2024-09-11T01:22:56.000263+0000","last_fullsized":"2024-09-11T01:22:56.000263+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999267+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:55.999267+0000","last_peered":"2024-09-11T01:22:55.999267+0000","last_clean":"2024-09-11T01:22:55.999267+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:55.999267+0000","last_undegraded":"2024-09-11T01:22:55.999267+0000","last_fullsized":"2024-09-11T01:22:55.999267+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28000,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999439+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:55.999439+0000","last_peered":"2024-09-11T01:22:55.999439+0000","last_clean":"2024-09-11T01:22:55.999439+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:55.999439+0000","last_undegraded":"2024-09-11T01:22:55.999439+0000","last_fullsized":"2024-09-11T01:22:55.999439+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:55.994373+0000","last_peered":"2024-09-11T01:22:55.994373+0000","last_clean":"2024-09-11T01:22:55.994373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:55.994373+0000","last_undegraded":"2024-09-11T01:22:55.994373+0000","last_fullsized":"2024-09-11T01:22:55.994373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20015,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001237+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:56.001237+0000","last_peered":"2024-09-11T01:22:56.001237+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:56.001237+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000522+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:56.000522+0000","last_peered":"2024-09-11T01:22:56.000522+0000","last_clean":"2024-09-11T01:22:56.000522+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:56.000522+0000","last_undegraded":"2024-09-11T01:22:56.000522+0000","last_fullsized":"2024-09-11T01:22:56.000522+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20345,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:55.994507+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:55.994507+0000","last_peered":"2024-09-11T01:22:55.994507+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:55.994507+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999168+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:55.999168+0000","last_peered":"2024-09-11T01:22:55.999168+0000","last_clean":"2024-09-11T01:22:55.999168+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:55.999168+0000","last_undegraded":"2024-09-11T01:22:55.999168+0000","last_fullsized":"2024-09-11T01:22:55.999168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27796,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994249+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:55.994249+0000","last_peered":"2024-09-11T01:22:55.994249+0000","last_clean":"2024-09-11T01:22:55.994249+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:55.994249+0000","last_undegraded":"2024-09-11T01:22:55.994249+0000","last_fullsized":"2024-09-11T01:22:55.994249+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20415,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.006380+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:56.006380+0000","last_peered":"2024-09-11T01:22:56.006380+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:56.006380+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003870+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:56.003870+0000","last_peered":"2024-09-11T01:22:56.003870+0000","last_clean":"2024-09-11T01:22:56.003870+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:56.003870+0000","last_undegraded":"2024-09-11T01:22:56.003870+0000","last_fullsized":"2024-09-11T01:22:56.003870+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27934,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003846+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:56.003846+0000","last_peered":"2024-09-11T01:22:56.003846+0000","last_clean":"2024-09-11T01:22:56.003846+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:56.003846+0000","last_undegraded":"2024-09-11T01:22:56.003846+0000","last_fullsized":"2024-09-11T01:22:56.003846+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996759+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:55.996759+0000","last_peered":"2024-09-11T01:22:55.996759+0000","last_clean":"2024-09-11T01:22:55.996759+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:55.996759+0000","last_undegraded":"2024-09-11T01:22:55.996759+0000","last_fullsized":"2024-09-11T01:22:55.996759+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24613,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001573+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:56.001573+0000","last_peered":"2024-09-11T01:22:56.001573+0000","last_clean":"2024-09-11T01:22:56.001573+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:56.001573+0000","last_undegraded":"2024-09-11T01:22:56.001573+0000","last_fullsized":"2024-09-11T01:22:56.001573+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999413+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:55.999413+0000","last_peered":"2024-09-11T01:22:55.999413+0000","last_clean":"2024-09-11T01:22:55.999413+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:55.999413+0000","last_undegraded":"2024-09-11T01:22:55.999413+0000","last_fullsized":"2024-09-11T01:22:55.999413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999502+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:55.999502+0000","last_peered":"2024-09-11T01:22:55.999502+0000","last_clean":"2024-09-11T01:22:55.999502+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:55.999502+0000","last_undegraded":"2024-09-11T01:22:55.999502+0000","last_fullsized":"2024-09-11T01:22:55.999502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20105,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001118+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:56.001118+0000","last_peered":"2024-09-11T01:22:56.001118+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:56.001118+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21333,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195088384,"data_stored":195036199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182541,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518600,"kb_used_data":63544,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853240,"statfs":{"total":96636764160,"available":96105717760,"internally_reserved":0,"allocated":65069056,"data_stored":64767762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986313,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488444,"kb_used_data":127788,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883396,"statfs":{"total":96636764160,"available":96136597504,"internally_reserved":0,"allocated":130854912,"data_stored":130557695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659258,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465272,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93906568,"statfs":{"total":96636764160,"available":96160325632,"internally_reserved":0,"allocated":499712,"data_stored":156670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475917999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":8021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:23.015 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:23.338 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:23.339 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:23:23.356 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":416,"stamp":"2024-09-11T01:23:22.821991+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21334,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472316,"kb_used_data":191820,"kb_used_omap":12,"kb_used_meta":1280435,"kb_avail":281643204,"statfs":{"total":289910292480,"available":288402640896,"internally_reserved":0,"allocated":196423680,"data_stored":195482127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1311165444},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-647,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":647,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003879"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":23544,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:18.244252+0000","last_change":"2024-09-11T01:22:54.587427+0000","last_active":"2024-09-11T01:23:18.244252+0000","last_peered":"2024-09-11T01:23:18.244252+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:23:18.244252+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2482,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1590,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2482}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20472,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996478+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:55.996478+0000","last_peered":"2024-09-11T01:22:55.996478+0000","last_clean":"2024-09-11T01:22:55.996478+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:55.996478+0000","last_undegraded":"2024-09-11T01:22:55.996478+0000","last_fullsized":"2024-09-11T01:22:55.996478+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20296,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.149418+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:22:56.149418+0000","last_peered":"2024-09-11T01:22:56.149418+0000","last_clean":"2024-09-11T01:22:56.149418+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:22:56.149418+0000","last_undegraded":"2024-09-11T01:22:56.149418+0000","last_fullsized":"2024-09-11T01:22:56.149418+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20222,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996767+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:55.996767+0000","last_peered":"2024-09-11T01:22:55.996767+0000","last_clean":"2024-09-11T01:22:55.996767+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:55.996767+0000","last_undegraded":"2024-09-11T01:22:55.996767+0000","last_fullsized":"2024-09-11T01:22:55.996767+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20156,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001082+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:56.001082+0000","last_peered":"2024-09-11T01:22:56.001082+0000","last_clean":"2024-09-11T01:22:56.001082+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:56.001082+0000","last_undegraded":"2024-09-11T01:22:56.001082+0000","last_fullsized":"2024-09-11T01:22:56.001082+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20416,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000893+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:56.000893+0000","last_peered":"2024-09-11T01:22:56.000893+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:56.000893+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20079,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000461+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:56.000461+0000","last_peered":"2024-09-11T01:22:56.000461+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:56.000461+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20403,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000263+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:56.000263+0000","last_peered":"2024-09-11T01:22:56.000263+0000","last_clean":"2024-09-11T01:22:56.000263+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:56.000263+0000","last_undegraded":"2024-09-11T01:22:56.000263+0000","last_fullsized":"2024-09-11T01:22:56.000263+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999267+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:55.999267+0000","last_peered":"2024-09-11T01:22:55.999267+0000","last_clean":"2024-09-11T01:22:55.999267+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:55.999267+0000","last_undegraded":"2024-09-11T01:22:55.999267+0000","last_fullsized":"2024-09-11T01:22:55.999267+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28000,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999439+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:55.999439+0000","last_peered":"2024-09-11T01:22:55.999439+0000","last_clean":"2024-09-11T01:22:55.999439+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:55.999439+0000","last_undegraded":"2024-09-11T01:22:55.999439+0000","last_fullsized":"2024-09-11T01:22:55.999439+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:55.994373+0000","last_peered":"2024-09-11T01:22:55.994373+0000","last_clean":"2024-09-11T01:22:55.994373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:55.994373+0000","last_undegraded":"2024-09-11T01:22:55.994373+0000","last_fullsized":"2024-09-11T01:22:55.994373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20015,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001237+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:56.001237+0000","last_peered":"2024-09-11T01:22:56.001237+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:56.001237+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000522+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:56.000522+0000","last_peered":"2024-09-11T01:22:56.000522+0000","last_clean":"2024-09-11T01:22:56.000522+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:56.000522+0000","last_undegraded":"2024-09-11T01:22:56.000522+0000","last_fullsized":"2024-09-11T01:22:56.000522+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20345,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:55.994507+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:55.994507+0000","last_peered":"2024-09-11T01:22:55.994507+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:55.994507+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999168+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:55.999168+0000","last_peered":"2024-09-11T01:22:55.999168+0000","last_clean":"2024-09-11T01:22:55.999168+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:55.999168+0000","last_undegraded":"2024-09-11T01:22:55.999168+0000","last_fullsized":"2024-09-11T01:22:55.999168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27796,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994249+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:55.994249+0000","last_peered":"2024-09-11T01:22:55.994249+0000","last_clean":"2024-09-11T01:22:55.994249+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:55.994249+0000","last_undegraded":"2024-09-11T01:22:55.994249+0000","last_fullsized":"2024-09-11T01:22:55.994249+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20415,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.006380+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:56.006380+0000","last_peered":"2024-09-11T01:22:56.006380+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:56.006380+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003870+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:56.003870+0000","last_peered":"2024-09-11T01:22:56.003870+0000","last_clean":"2024-09-11T01:22:56.003870+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:56.003870+0000","last_undegraded":"2024-09-11T01:22:56.003870+0000","last_fullsized":"2024-09-11T01:22:56.003870+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27934,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003846+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:56.003846+0000","last_peered":"2024-09-11T01:22:56.003846+0000","last_clean":"2024-09-11T01:22:56.003846+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:56.003846+0000","last_undegraded":"2024-09-11T01:22:56.003846+0000","last_fullsized":"2024-09-11T01:22:56.003846+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996759+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:55.996759+0000","last_peered":"2024-09-11T01:22:55.996759+0000","last_clean":"2024-09-11T01:22:55.996759+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:55.996759+0000","last_undegraded":"2024-09-11T01:22:55.996759+0000","last_fullsized":"2024-09-11T01:22:55.996759+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24613,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001573+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:56.001573+0000","last_peered":"2024-09-11T01:22:56.001573+0000","last_clean":"2024-09-11T01:22:56.001573+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:56.001573+0000","last_undegraded":"2024-09-11T01:22:56.001573+0000","last_fullsized":"2024-09-11T01:22:56.001573+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999413+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:55.999413+0000","last_peered":"2024-09-11T01:22:55.999413+0000","last_clean":"2024-09-11T01:22:55.999413+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:55.999413+0000","last_undegraded":"2024-09-11T01:22:55.999413+0000","last_fullsized":"2024-09-11T01:22:55.999413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999502+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:55.999502+0000","last_peered":"2024-09-11T01:22:55.999502+0000","last_clean":"2024-09-11T01:22:55.999502+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:55.999502+0000","last_undegraded":"2024-09-11T01:22:55.999502+0000","last_fullsized":"2024-09-11T01:22:55.999502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20105,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001118+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:56.001118+0000","last_peered":"2024-09-11T01:22:56.001118+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:56.001118+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21333,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195088384,"data_stored":195036199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182541,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518600,"kb_used_data":63544,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853240,"statfs":{"total":96636764160,"available":96105717760,"internally_reserved":0,"allocated":65069056,"data_stored":64767762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986314,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488444,"kb_used_data":127788,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883396,"statfs":{"total":96636764160,"available":96136597504,"internally_reserved":0,"allocated":130854912,"data_stored":130557695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659258,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465272,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93906568,"statfs":{"total":96636764160,"available":96160325632,"internally_reserved":0,"allocated":499712,"data_stored":156670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475917999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":8021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:23.357 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:23.678 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:23.678 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:23:23.692 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":416,"stamp":"2024-09-11T01:23:22.821991+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21334,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472316,"kb_used_data":191820,"kb_used_omap":12,"kb_used_meta":1280435,"kb_avail":281643204,"statfs":{"total":289910292480,"available":288402640896,"internally_reserved":0,"allocated":196423680,"data_stored":195482127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1311165444},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-647,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":647,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003879"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":23544,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:18.244252+0000","last_change":"2024-09-11T01:22:54.587427+0000","last_active":"2024-09-11T01:23:18.244252+0000","last_peered":"2024-09-11T01:23:18.244252+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:23:18.244252+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2482,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1590,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2482}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20472,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996478+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:55.996478+0000","last_peered":"2024-09-11T01:22:55.996478+0000","last_clean":"2024-09-11T01:22:55.996478+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:55.996478+0000","last_undegraded":"2024-09-11T01:22:55.996478+0000","last_fullsized":"2024-09-11T01:22:55.996478+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20296,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.149418+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:22:56.149418+0000","last_peered":"2024-09-11T01:22:56.149418+0000","last_clean":"2024-09-11T01:22:56.149418+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:22:56.149418+0000","last_undegraded":"2024-09-11T01:22:56.149418+0000","last_fullsized":"2024-09-11T01:22:56.149418+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20222,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996767+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:55.996767+0000","last_peered":"2024-09-11T01:22:55.996767+0000","last_clean":"2024-09-11T01:22:55.996767+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:55.996767+0000","last_undegraded":"2024-09-11T01:22:55.996767+0000","last_fullsized":"2024-09-11T01:22:55.996767+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20156,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001082+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:56.001082+0000","last_peered":"2024-09-11T01:22:56.001082+0000","last_clean":"2024-09-11T01:22:56.001082+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:56.001082+0000","last_undegraded":"2024-09-11T01:22:56.001082+0000","last_fullsized":"2024-09-11T01:22:56.001082+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20416,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000893+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:56.000893+0000","last_peered":"2024-09-11T01:22:56.000893+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:56.000893+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20079,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000461+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:56.000461+0000","last_peered":"2024-09-11T01:22:56.000461+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:56.000461+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20403,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000263+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:56.000263+0000","last_peered":"2024-09-11T01:22:56.000263+0000","last_clean":"2024-09-11T01:22:56.000263+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:56.000263+0000","last_undegraded":"2024-09-11T01:22:56.000263+0000","last_fullsized":"2024-09-11T01:22:56.000263+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999267+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:55.999267+0000","last_peered":"2024-09-11T01:22:55.999267+0000","last_clean":"2024-09-11T01:22:55.999267+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:55.999267+0000","last_undegraded":"2024-09-11T01:22:55.999267+0000","last_fullsized":"2024-09-11T01:22:55.999267+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28000,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999439+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:55.999439+0000","last_peered":"2024-09-11T01:22:55.999439+0000","last_clean":"2024-09-11T01:22:55.999439+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:55.999439+0000","last_undegraded":"2024-09-11T01:22:55.999439+0000","last_fullsized":"2024-09-11T01:22:55.999439+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:55.994373+0000","last_peered":"2024-09-11T01:22:55.994373+0000","last_clean":"2024-09-11T01:22:55.994373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:55.994373+0000","last_undegraded":"2024-09-11T01:22:55.994373+0000","last_fullsized":"2024-09-11T01:22:55.994373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20015,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001237+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:56.001237+0000","last_peered":"2024-09-11T01:22:56.001237+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:56.001237+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000522+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:56.000522+0000","last_peered":"2024-09-11T01:22:56.000522+0000","last_clean":"2024-09-11T01:22:56.000522+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:56.000522+0000","last_undegraded":"2024-09-11T01:22:56.000522+0000","last_fullsized":"2024-09-11T01:22:56.000522+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20345,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:55.994507+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:55.994507+0000","last_peered":"2024-09-11T01:22:55.994507+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:55.994507+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999168+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:55.999168+0000","last_peered":"2024-09-11T01:22:55.999168+0000","last_clean":"2024-09-11T01:22:55.999168+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:55.999168+0000","last_undegraded":"2024-09-11T01:22:55.999168+0000","last_fullsized":"2024-09-11T01:22:55.999168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27796,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994249+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:55.994249+0000","last_peered":"2024-09-11T01:22:55.994249+0000","last_clean":"2024-09-11T01:22:55.994249+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:55.994249+0000","last_undegraded":"2024-09-11T01:22:55.994249+0000","last_fullsized":"2024-09-11T01:22:55.994249+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20415,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.006380+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:56.006380+0000","last_peered":"2024-09-11T01:22:56.006380+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:56.006380+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003870+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:56.003870+0000","last_peered":"2024-09-11T01:22:56.003870+0000","last_clean":"2024-09-11T01:22:56.003870+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:56.003870+0000","last_undegraded":"2024-09-11T01:22:56.003870+0000","last_fullsized":"2024-09-11T01:22:56.003870+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27934,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003846+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:56.003846+0000","last_peered":"2024-09-11T01:22:56.003846+0000","last_clean":"2024-09-11T01:22:56.003846+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:56.003846+0000","last_undegraded":"2024-09-11T01:22:56.003846+0000","last_fullsized":"2024-09-11T01:22:56.003846+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996759+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:55.996759+0000","last_peered":"2024-09-11T01:22:55.996759+0000","last_clean":"2024-09-11T01:22:55.996759+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:55.996759+0000","last_undegraded":"2024-09-11T01:22:55.996759+0000","last_fullsized":"2024-09-11T01:22:55.996759+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24613,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001573+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:56.001573+0000","last_peered":"2024-09-11T01:22:56.001573+0000","last_clean":"2024-09-11T01:22:56.001573+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:56.001573+0000","last_undegraded":"2024-09-11T01:22:56.001573+0000","last_fullsized":"2024-09-11T01:22:56.001573+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999413+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:55.999413+0000","last_peered":"2024-09-11T01:22:55.999413+0000","last_clean":"2024-09-11T01:22:55.999413+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:55.999413+0000","last_undegraded":"2024-09-11T01:22:55.999413+0000","last_fullsized":"2024-09-11T01:22:55.999413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999502+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:55.999502+0000","last_peered":"2024-09-11T01:22:55.999502+0000","last_clean":"2024-09-11T01:22:55.999502+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:55.999502+0000","last_undegraded":"2024-09-11T01:22:55.999502+0000","last_fullsized":"2024-09-11T01:22:55.999502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20105,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001118+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:56.001118+0000","last_peered":"2024-09-11T01:22:56.001118+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:56.001118+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21333,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195088384,"data_stored":195036199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182541,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518600,"kb_used_data":63544,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853240,"statfs":{"total":96636764160,"available":96105717760,"internally_reserved":0,"allocated":65069056,"data_stored":64767762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986314,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488444,"kb_used_data":127788,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883396,"statfs":{"total":96636764160,"available":96136597504,"internally_reserved":0,"allocated":130854912,"data_stored":130557695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659258,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465272,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93906568,"statfs":{"total":96636764160,"available":96160325632,"internally_reserved":0,"allocated":499712,"data_stored":156670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475917999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":8021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:23.694 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:23:23.694 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20266", 'reported_seq': 23544, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:18.244252+0000', 'last_change': '2024-09-11T01:22:54.587427+0000', 'last_active': '2024-09-11T01:23:18.244252+0000', 'last_peered': '2024-09-11T01:23:18.244252+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:23:18.244252+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2482, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20266, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1590, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2482}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:23.694 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:23:23.694 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'20227", 'reported_seq': 20416, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.000893+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:22:56.000893+0000', 'last_peered': '2024-09-11T01:22:56.000893+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:22:56.000893+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16568, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:23.694 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:23:23.694 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20079, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.000461+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:22:56.000461+0000', 'last_peered': '2024-09-11T01:22:56.000461+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:22:56.000461+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:23.694 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:23:23.694 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20015, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.001237+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:22:56.001237+0000', 'last_peered': '2024-09-11T01:22:56.001237+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:22:56.001237+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:23.694 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:23:23.695 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'20207", 'reported_seq': 20345, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:55.994507+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:22:55.994507+0000', 'last_peered': '2024-09-11T01:22:55.994507+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:22:55.994507+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18024, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4024}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:23.695 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:23:23.695 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20264", 'reported_seq': 20415, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.006380+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:22:56.006380+0000', 'last_peered': '2024-09-11T01:22:56.006380+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:22:56.006380+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17001, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:23.695 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:23:23.695 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 20105, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.001118+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:22:56.001118+0000', 'last_peered': '2024-09-11T01:22:56.001118+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:22:56.001118+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:23.695 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:23:24.123 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:24.123 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 26306/6 objects degraded (438433.333%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":740,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1507651584,"bytes_avail":288402640896,"bytes_total":289910292480,"degraded_objects":26306,"degraded_total":6,"degraded_ratio":4384.333333333333,"recovering_objects_per_sec":53,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":647,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:23:24.142 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:23:24.142 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:24.444 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:24.444 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:23:24.461 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":416,"stamp":"2024-09-11T01:23:22.821991+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21334,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472316,"kb_used_data":191820,"kb_used_omap":12,"kb_used_meta":1280435,"kb_avail":281643204,"statfs":{"total":289910292480,"available":288402640896,"internally_reserved":0,"allocated":196423680,"data_stored":195482127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1311165444},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-647,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":647,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003879"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":23544,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:18.244252+0000","last_change":"2024-09-11T01:22:54.587427+0000","last_active":"2024-09-11T01:23:18.244252+0000","last_peered":"2024-09-11T01:23:18.244252+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:23:18.244252+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2482,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1590,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2482}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20472,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996478+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:55.996478+0000","last_peered":"2024-09-11T01:22:55.996478+0000","last_clean":"2024-09-11T01:22:55.996478+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:55.996478+0000","last_undegraded":"2024-09-11T01:22:55.996478+0000","last_fullsized":"2024-09-11T01:22:55.996478+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20296,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.149418+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:22:56.149418+0000","last_peered":"2024-09-11T01:22:56.149418+0000","last_clean":"2024-09-11T01:22:56.149418+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:22:56.149418+0000","last_undegraded":"2024-09-11T01:22:56.149418+0000","last_fullsized":"2024-09-11T01:22:56.149418+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20222,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996767+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:55.996767+0000","last_peered":"2024-09-11T01:22:55.996767+0000","last_clean":"2024-09-11T01:22:55.996767+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:55.996767+0000","last_undegraded":"2024-09-11T01:22:55.996767+0000","last_fullsized":"2024-09-11T01:22:55.996767+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20156,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001082+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:56.001082+0000","last_peered":"2024-09-11T01:22:56.001082+0000","last_clean":"2024-09-11T01:22:56.001082+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:56.001082+0000","last_undegraded":"2024-09-11T01:22:56.001082+0000","last_fullsized":"2024-09-11T01:22:56.001082+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20416,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000893+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:56.000893+0000","last_peered":"2024-09-11T01:22:56.000893+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:56.000893+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20079,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000461+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:56.000461+0000","last_peered":"2024-09-11T01:22:56.000461+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:56.000461+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20403,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000263+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:56.000263+0000","last_peered":"2024-09-11T01:22:56.000263+0000","last_clean":"2024-09-11T01:22:56.000263+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:56.000263+0000","last_undegraded":"2024-09-11T01:22:56.000263+0000","last_fullsized":"2024-09-11T01:22:56.000263+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999267+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:55.999267+0000","last_peered":"2024-09-11T01:22:55.999267+0000","last_clean":"2024-09-11T01:22:55.999267+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:55.999267+0000","last_undegraded":"2024-09-11T01:22:55.999267+0000","last_fullsized":"2024-09-11T01:22:55.999267+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28000,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999439+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:55.999439+0000","last_peered":"2024-09-11T01:22:55.999439+0000","last_clean":"2024-09-11T01:22:55.999439+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:55.999439+0000","last_undegraded":"2024-09-11T01:22:55.999439+0000","last_fullsized":"2024-09-11T01:22:55.999439+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:55.994373+0000","last_peered":"2024-09-11T01:22:55.994373+0000","last_clean":"2024-09-11T01:22:55.994373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:55.994373+0000","last_undegraded":"2024-09-11T01:22:55.994373+0000","last_fullsized":"2024-09-11T01:22:55.994373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20015,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001237+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:56.001237+0000","last_peered":"2024-09-11T01:22:56.001237+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:56.001237+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000522+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:56.000522+0000","last_peered":"2024-09-11T01:22:56.000522+0000","last_clean":"2024-09-11T01:22:56.000522+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:56.000522+0000","last_undegraded":"2024-09-11T01:22:56.000522+0000","last_fullsized":"2024-09-11T01:22:56.000522+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20345,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:55.994507+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:55.994507+0000","last_peered":"2024-09-11T01:22:55.994507+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:55.994507+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999168+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:55.999168+0000","last_peered":"2024-09-11T01:22:55.999168+0000","last_clean":"2024-09-11T01:22:55.999168+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:55.999168+0000","last_undegraded":"2024-09-11T01:22:55.999168+0000","last_fullsized":"2024-09-11T01:22:55.999168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27796,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994249+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:55.994249+0000","last_peered":"2024-09-11T01:22:55.994249+0000","last_clean":"2024-09-11T01:22:55.994249+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:55.994249+0000","last_undegraded":"2024-09-11T01:22:55.994249+0000","last_fullsized":"2024-09-11T01:22:55.994249+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20415,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.006380+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:56.006380+0000","last_peered":"2024-09-11T01:22:56.006380+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:56.006380+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003870+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:56.003870+0000","last_peered":"2024-09-11T01:22:56.003870+0000","last_clean":"2024-09-11T01:22:56.003870+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:56.003870+0000","last_undegraded":"2024-09-11T01:22:56.003870+0000","last_fullsized":"2024-09-11T01:22:56.003870+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27934,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003846+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:56.003846+0000","last_peered":"2024-09-11T01:22:56.003846+0000","last_clean":"2024-09-11T01:22:56.003846+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:56.003846+0000","last_undegraded":"2024-09-11T01:22:56.003846+0000","last_fullsized":"2024-09-11T01:22:56.003846+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996759+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:55.996759+0000","last_peered":"2024-09-11T01:22:55.996759+0000","last_clean":"2024-09-11T01:22:55.996759+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:55.996759+0000","last_undegraded":"2024-09-11T01:22:55.996759+0000","last_fullsized":"2024-09-11T01:22:55.996759+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24613,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001573+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:56.001573+0000","last_peered":"2024-09-11T01:22:56.001573+0000","last_clean":"2024-09-11T01:22:56.001573+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:56.001573+0000","last_undegraded":"2024-09-11T01:22:56.001573+0000","last_fullsized":"2024-09-11T01:22:56.001573+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999413+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:55.999413+0000","last_peered":"2024-09-11T01:22:55.999413+0000","last_clean":"2024-09-11T01:22:55.999413+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:55.999413+0000","last_undegraded":"2024-09-11T01:22:55.999413+0000","last_fullsized":"2024-09-11T01:22:55.999413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999502+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:55.999502+0000","last_peered":"2024-09-11T01:22:55.999502+0000","last_clean":"2024-09-11T01:22:55.999502+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:55.999502+0000","last_undegraded":"2024-09-11T01:22:55.999502+0000","last_fullsized":"2024-09-11T01:22:55.999502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20105,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001118+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:56.001118+0000","last_peered":"2024-09-11T01:22:56.001118+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:56.001118+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":26306,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21333,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195088384,"data_stored":195036199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182541,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518600,"kb_used_data":63544,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853240,"statfs":{"total":96636764160,"available":96105717760,"internally_reserved":0,"allocated":65069056,"data_stored":64767762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986314,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488444,"kb_used_data":127788,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883396,"statfs":{"total":96636764160,"available":96136597504,"internally_reserved":0,"allocated":130854912,"data_stored":130557695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659258,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465272,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93906568,"statfs":{"total":96636764160,"available":96160325632,"internally_reserved":0,"allocated":499712,"data_stored":156670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475917999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":8021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:27.462 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:27.880 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:27.881 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:23:27.900 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":418,"stamp":"2024-09-11T01:23:26.823515+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25947,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21693,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1476476,"kb_used_data":191820,"kb_used_omap":12,"kb_used_meta":1284595,"kb_avail":281639044,"statfs":{"total":289910292480,"available":288398381056,"internally_reserved":0,"allocated":196423680,"data_stored":195482127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1315425284},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-695,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":695,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004286"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":24262,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:23.241372+0000","last_change":"2024-09-11T01:22:54.587427+0000","last_active":"2024-09-11T01:23:23.241372+0000","last_peered":"2024-09-11T01:23:23.241372+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:23:23.241372+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1949,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20473,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996631+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:55.996631+0000","last_peered":"2024-09-11T01:22:55.996631+0000","last_clean":"2024-09-11T01:22:55.996631+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:55.996631+0000","last_undegraded":"2024-09-11T01:22:55.996631+0000","last_fullsized":"2024-09-11T01:22:55.996631+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20296,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.149418+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:22:56.149418+0000","last_peered":"2024-09-11T01:22:56.149418+0000","last_clean":"2024-09-11T01:22:56.149418+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:22:56.149418+0000","last_undegraded":"2024-09-11T01:22:56.149418+0000","last_fullsized":"2024-09-11T01:22:56.149418+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20223,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996887+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:55.996887+0000","last_peered":"2024-09-11T01:22:55.996887+0000","last_clean":"2024-09-11T01:22:55.996887+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:55.996887+0000","last_undegraded":"2024-09-11T01:22:55.996887+0000","last_fullsized":"2024-09-11T01:22:55.996887+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20156,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001082+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:56.001082+0000","last_peered":"2024-09-11T01:22:56.001082+0000","last_clean":"2024-09-11T01:22:56.001082+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:56.001082+0000","last_undegraded":"2024-09-11T01:22:56.001082+0000","last_fullsized":"2024-09-11T01:22:56.001082+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20416,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000893+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:56.000893+0000","last_peered":"2024-09-11T01:22:56.000893+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:56.000893+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20079,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000461+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:56.000461+0000","last_peered":"2024-09-11T01:22:56.000461+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:56.000461+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20403,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000263+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:56.000263+0000","last_peered":"2024-09-11T01:22:56.000263+0000","last_clean":"2024-09-11T01:22:56.000263+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:56.000263+0000","last_undegraded":"2024-09-11T01:22:56.000263+0000","last_fullsized":"2024-09-11T01:22:56.000263+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999267+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:55.999267+0000","last_peered":"2024-09-11T01:22:55.999267+0000","last_clean":"2024-09-11T01:22:55.999267+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:55.999267+0000","last_undegraded":"2024-09-11T01:22:55.999267+0000","last_fullsized":"2024-09-11T01:22:55.999267+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28000,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999439+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:55.999439+0000","last_peered":"2024-09-11T01:22:55.999439+0000","last_clean":"2024-09-11T01:22:55.999439+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:55.999439+0000","last_undegraded":"2024-09-11T01:22:55.999439+0000","last_fullsized":"2024-09-11T01:22:55.999439+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:55.994373+0000","last_peered":"2024-09-11T01:22:55.994373+0000","last_clean":"2024-09-11T01:22:55.994373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:55.994373+0000","last_undegraded":"2024-09-11T01:22:55.994373+0000","last_fullsized":"2024-09-11T01:22:55.994373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20015,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001237+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:56.001237+0000","last_peered":"2024-09-11T01:22:56.001237+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:56.001237+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000522+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:56.000522+0000","last_peered":"2024-09-11T01:22:56.000522+0000","last_clean":"2024-09-11T01:22:56.000522+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:56.000522+0000","last_undegraded":"2024-09-11T01:22:56.000522+0000","last_fullsized":"2024-09-11T01:22:56.000522+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20345,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:55.994507+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:55.994507+0000","last_peered":"2024-09-11T01:22:55.994507+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:55.994507+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999168+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:55.999168+0000","last_peered":"2024-09-11T01:22:55.999168+0000","last_clean":"2024-09-11T01:22:55.999168+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:55.999168+0000","last_undegraded":"2024-09-11T01:22:55.999168+0000","last_fullsized":"2024-09-11T01:22:55.999168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27796,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994249+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:55.994249+0000","last_peered":"2024-09-11T01:22:55.994249+0000","last_clean":"2024-09-11T01:22:55.994249+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:55.994249+0000","last_undegraded":"2024-09-11T01:22:55.994249+0000","last_fullsized":"2024-09-11T01:22:55.994249+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20415,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.006380+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:56.006380+0000","last_peered":"2024-09-11T01:22:56.006380+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:56.006380+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003870+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:56.003870+0000","last_peered":"2024-09-11T01:22:56.003870+0000","last_clean":"2024-09-11T01:22:56.003870+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:56.003870+0000","last_undegraded":"2024-09-11T01:22:56.003870+0000","last_fullsized":"2024-09-11T01:22:56.003870+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27934,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003846+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:56.003846+0000","last_peered":"2024-09-11T01:22:56.003846+0000","last_clean":"2024-09-11T01:22:56.003846+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:56.003846+0000","last_undegraded":"2024-09-11T01:22:56.003846+0000","last_fullsized":"2024-09-11T01:22:56.003846+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996759+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:55.996759+0000","last_peered":"2024-09-11T01:22:55.996759+0000","last_clean":"2024-09-11T01:22:55.996759+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:55.996759+0000","last_undegraded":"2024-09-11T01:22:55.996759+0000","last_fullsized":"2024-09-11T01:22:55.996759+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24613,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001573+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:56.001573+0000","last_peered":"2024-09-11T01:22:56.001573+0000","last_clean":"2024-09-11T01:22:56.001573+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:56.001573+0000","last_undegraded":"2024-09-11T01:22:56.001573+0000","last_fullsized":"2024-09-11T01:22:56.001573+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999413+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:55.999413+0000","last_peered":"2024-09-11T01:22:55.999413+0000","last_clean":"2024-09-11T01:22:55.999413+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:55.999413+0000","last_undegraded":"2024-09-11T01:22:55.999413+0000","last_fullsized":"2024-09-11T01:22:55.999413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999502+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:55.999502+0000","last_peered":"2024-09-11T01:22:55.999502+0000","last_clean":"2024-09-11T01:22:55.999502+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:55.999502+0000","last_undegraded":"2024-09-11T01:22:55.999502+0000","last_fullsized":"2024-09-11T01:22:55.999502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20105,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001118+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:56.001118+0000","last_peered":"2024-09-11T01:22:56.001118+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:56.001118+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25947,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21692,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195088384,"data_stored":195036199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182542,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518600,"kb_used_data":63544,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853240,"statfs":{"total":96636764160,"available":96105717760,"internally_reserved":0,"allocated":65069056,"data_stored":64767762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986315,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488444,"kb_used_data":127788,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883396,"statfs":{"total":96636764160,"available":96136597504,"internally_reserved":0,"allocated":130854912,"data_stored":130557695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659259,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469432,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93902408,"statfs":{"total":96636764160,"available":96156065792,"internally_reserved":0,"allocated":499712,"data_stored":156670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":480177839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":8021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:27.901 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:28.214 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:28.215 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:23:28.233 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":418,"stamp":"2024-09-11T01:23:26.823515+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25947,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21693,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1476476,"kb_used_data":191820,"kb_used_omap":12,"kb_used_meta":1284595,"kb_avail":281639044,"statfs":{"total":289910292480,"available":288398381056,"internally_reserved":0,"allocated":196423680,"data_stored":195482127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1315425284},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-695,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":695,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004286"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":24262,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:23.241372+0000","last_change":"2024-09-11T01:22:54.587427+0000","last_active":"2024-09-11T01:23:23.241372+0000","last_peered":"2024-09-11T01:23:23.241372+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:23:23.241372+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2123,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1949,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2123}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20473,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996631+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:55.996631+0000","last_peered":"2024-09-11T01:22:55.996631+0000","last_clean":"2024-09-11T01:22:55.996631+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:55.996631+0000","last_undegraded":"2024-09-11T01:22:55.996631+0000","last_fullsized":"2024-09-11T01:22:55.996631+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20296,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.149418+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:22:56.149418+0000","last_peered":"2024-09-11T01:22:56.149418+0000","last_clean":"2024-09-11T01:22:56.149418+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:22:56.149418+0000","last_undegraded":"2024-09-11T01:22:56.149418+0000","last_fullsized":"2024-09-11T01:22:56.149418+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20223,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996887+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:55.996887+0000","last_peered":"2024-09-11T01:22:55.996887+0000","last_clean":"2024-09-11T01:22:55.996887+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:55.996887+0000","last_undegraded":"2024-09-11T01:22:55.996887+0000","last_fullsized":"2024-09-11T01:22:55.996887+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20156,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001082+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:56.001082+0000","last_peered":"2024-09-11T01:22:56.001082+0000","last_clean":"2024-09-11T01:22:56.001082+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:56.001082+0000","last_undegraded":"2024-09-11T01:22:56.001082+0000","last_fullsized":"2024-09-11T01:22:56.001082+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20416,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000893+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:56.000893+0000","last_peered":"2024-09-11T01:22:56.000893+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:56.000893+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20079,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000461+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:56.000461+0000","last_peered":"2024-09-11T01:22:56.000461+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:56.000461+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20403,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000263+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:56.000263+0000","last_peered":"2024-09-11T01:22:56.000263+0000","last_clean":"2024-09-11T01:22:56.000263+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:56.000263+0000","last_undegraded":"2024-09-11T01:22:56.000263+0000","last_fullsized":"2024-09-11T01:22:56.000263+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999267+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:55.999267+0000","last_peered":"2024-09-11T01:22:55.999267+0000","last_clean":"2024-09-11T01:22:55.999267+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:55.999267+0000","last_undegraded":"2024-09-11T01:22:55.999267+0000","last_fullsized":"2024-09-11T01:22:55.999267+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28000,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999439+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:55.999439+0000","last_peered":"2024-09-11T01:22:55.999439+0000","last_clean":"2024-09-11T01:22:55.999439+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:55.999439+0000","last_undegraded":"2024-09-11T01:22:55.999439+0000","last_fullsized":"2024-09-11T01:22:55.999439+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:55.994373+0000","last_peered":"2024-09-11T01:22:55.994373+0000","last_clean":"2024-09-11T01:22:55.994373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:55.994373+0000","last_undegraded":"2024-09-11T01:22:55.994373+0000","last_fullsized":"2024-09-11T01:22:55.994373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20015,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001237+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:56.001237+0000","last_peered":"2024-09-11T01:22:56.001237+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:56.001237+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000522+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:56.000522+0000","last_peered":"2024-09-11T01:22:56.000522+0000","last_clean":"2024-09-11T01:22:56.000522+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:56.000522+0000","last_undegraded":"2024-09-11T01:22:56.000522+0000","last_fullsized":"2024-09-11T01:22:56.000522+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20345,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:55.994507+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:55.994507+0000","last_peered":"2024-09-11T01:22:55.994507+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:55.994507+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999168+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:55.999168+0000","last_peered":"2024-09-11T01:22:55.999168+0000","last_clean":"2024-09-11T01:22:55.999168+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:55.999168+0000","last_undegraded":"2024-09-11T01:22:55.999168+0000","last_fullsized":"2024-09-11T01:22:55.999168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27796,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994249+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:55.994249+0000","last_peered":"2024-09-11T01:22:55.994249+0000","last_clean":"2024-09-11T01:22:55.994249+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:55.994249+0000","last_undegraded":"2024-09-11T01:22:55.994249+0000","last_fullsized":"2024-09-11T01:22:55.994249+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20415,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.006380+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:56.006380+0000","last_peered":"2024-09-11T01:22:56.006380+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:56.006380+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003870+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:56.003870+0000","last_peered":"2024-09-11T01:22:56.003870+0000","last_clean":"2024-09-11T01:22:56.003870+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:56.003870+0000","last_undegraded":"2024-09-11T01:22:56.003870+0000","last_fullsized":"2024-09-11T01:22:56.003870+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27934,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003846+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:56.003846+0000","last_peered":"2024-09-11T01:22:56.003846+0000","last_clean":"2024-09-11T01:22:56.003846+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:56.003846+0000","last_undegraded":"2024-09-11T01:22:56.003846+0000","last_fullsized":"2024-09-11T01:22:56.003846+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996759+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:55.996759+0000","last_peered":"2024-09-11T01:22:55.996759+0000","last_clean":"2024-09-11T01:22:55.996759+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:55.996759+0000","last_undegraded":"2024-09-11T01:22:55.996759+0000","last_fullsized":"2024-09-11T01:22:55.996759+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24613,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001573+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:56.001573+0000","last_peered":"2024-09-11T01:22:56.001573+0000","last_clean":"2024-09-11T01:22:56.001573+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:56.001573+0000","last_undegraded":"2024-09-11T01:22:56.001573+0000","last_fullsized":"2024-09-11T01:22:56.001573+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999413+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:55.999413+0000","last_peered":"2024-09-11T01:22:55.999413+0000","last_clean":"2024-09-11T01:22:55.999413+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:55.999413+0000","last_undegraded":"2024-09-11T01:22:55.999413+0000","last_fullsized":"2024-09-11T01:22:55.999413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999502+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:55.999502+0000","last_peered":"2024-09-11T01:22:55.999502+0000","last_clean":"2024-09-11T01:22:55.999502+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:55.999502+0000","last_undegraded":"2024-09-11T01:22:55.999502+0000","last_fullsized":"2024-09-11T01:22:55.999502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20105,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001118+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:56.001118+0000","last_peered":"2024-09-11T01:22:56.001118+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:56.001118+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25947,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21692,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195088384,"data_stored":195036199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182542,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518600,"kb_used_data":63544,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853240,"statfs":{"total":96636764160,"available":96105717760,"internally_reserved":0,"allocated":65069056,"data_stored":64767762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986315,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488444,"kb_used_data":127788,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883396,"statfs":{"total":96636764160,"available":96136597504,"internally_reserved":0,"allocated":130854912,"data_stored":130557695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659259,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469432,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93902408,"statfs":{"total":96636764160,"available":96156065792,"internally_reserved":0,"allocated":499712,"data_stored":156670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":480177839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":8021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:28.235 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:23:28.235 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20266", 'reported_seq': 24262, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:23.241372+0000', 'last_change': '2024-09-11T01:22:54.587427+0000', 'last_active': '2024-09-11T01:23:23.241372+0000', 'last_peered': '2024-09-11T01:23:23.241372+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:23:23.241372+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2123, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20266, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1949, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2123}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:28.235 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:23:28.235 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'20227", 'reported_seq': 20416, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.000893+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:22:56.000893+0000', 'last_peered': '2024-09-11T01:22:56.000893+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:22:56.000893+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16568, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:28.235 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:23:28.235 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20079, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.000461+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:22:56.000461+0000', 'last_peered': '2024-09-11T01:22:56.000461+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:22:56.000461+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:28.235 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:23:28.235 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20015, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.001237+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:22:56.001237+0000', 'last_peered': '2024-09-11T01:22:56.001237+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:22:56.001237+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:28.236 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:23:28.236 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'20207", 'reported_seq': 20345, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:55.994507+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:22:55.994507+0000', 'last_peered': '2024-09-11T01:22:55.994507+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:22:55.994507+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18024, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4024}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:28.236 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:23:28.236 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20264", 'reported_seq': 20415, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.006380+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:22:56.006380+0000', 'last_peered': '2024-09-11T01:22:56.006380+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:22:56.006380+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17001, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:28.236 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:23:28.236 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 20105, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.001118+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:22:56.001118+0000', 'last_peered': '2024-09-11T01:22:56.001118+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:22:56.001118+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:28.237 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:23:28.619 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:28.619 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 25947/6 objects degraded (432450.000%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":745,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1511911424,"bytes_avail":288398381056,"bytes_total":289910292480,"degraded_objects":25947,"degraded_total":6,"degraded_ratio":4324.5,"recovering_objects_per_sec":57,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":695,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:23:28.638 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:23:28.638 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:28.972 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:28.973 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:23:28.993 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":419,"stamp":"2024-09-11T01:23:28.824255+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22105,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1480636,"kb_used_data":191820,"kb_used_omap":12,"kb_used_meta":1288755,"kb_avail":281634884,"statfs":{"total":289910292480,"available":288394121216,"internally_reserved":0,"allocated":196423680,"data_stored":195482127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1319685124},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1107,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1107,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004524"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":25086,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:28.247950+0000","last_change":"2024-09-11T01:22:54.587427+0000","last_active":"2024-09-11T01:23:28.247950+0000","last_peered":"2024-09-11T01:23:28.247950+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:23:28.247950+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1711,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1711}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20473,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996631+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:55.996631+0000","last_peered":"2024-09-11T01:22:55.996631+0000","last_clean":"2024-09-11T01:22:55.996631+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:55.996631+0000","last_undegraded":"2024-09-11T01:22:55.996631+0000","last_fullsized":"2024-09-11T01:22:55.996631+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20296,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.149418+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:22:56.149418+0000","last_peered":"2024-09-11T01:22:56.149418+0000","last_clean":"2024-09-11T01:22:56.149418+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:22:56.149418+0000","last_undegraded":"2024-09-11T01:22:56.149418+0000","last_fullsized":"2024-09-11T01:22:56.149418+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20223,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996887+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:55.996887+0000","last_peered":"2024-09-11T01:22:55.996887+0000","last_clean":"2024-09-11T01:22:55.996887+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:55.996887+0000","last_undegraded":"2024-09-11T01:22:55.996887+0000","last_fullsized":"2024-09-11T01:22:55.996887+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20156,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001082+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:56.001082+0000","last_peered":"2024-09-11T01:22:56.001082+0000","last_clean":"2024-09-11T01:22:56.001082+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:56.001082+0000","last_undegraded":"2024-09-11T01:22:56.001082+0000","last_fullsized":"2024-09-11T01:22:56.001082+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20416,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000893+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:56.000893+0000","last_peered":"2024-09-11T01:22:56.000893+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:56.000893+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20079,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000461+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:56.000461+0000","last_peered":"2024-09-11T01:22:56.000461+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:56.000461+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20403,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000263+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:56.000263+0000","last_peered":"2024-09-11T01:22:56.000263+0000","last_clean":"2024-09-11T01:22:56.000263+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:56.000263+0000","last_undegraded":"2024-09-11T01:22:56.000263+0000","last_fullsized":"2024-09-11T01:22:56.000263+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999267+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:55.999267+0000","last_peered":"2024-09-11T01:22:55.999267+0000","last_clean":"2024-09-11T01:22:55.999267+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:55.999267+0000","last_undegraded":"2024-09-11T01:22:55.999267+0000","last_fullsized":"2024-09-11T01:22:55.999267+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28000,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999439+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:55.999439+0000","last_peered":"2024-09-11T01:22:55.999439+0000","last_clean":"2024-09-11T01:22:55.999439+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:55.999439+0000","last_undegraded":"2024-09-11T01:22:55.999439+0000","last_fullsized":"2024-09-11T01:22:55.999439+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:55.994373+0000","last_peered":"2024-09-11T01:22:55.994373+0000","last_clean":"2024-09-11T01:22:55.994373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:55.994373+0000","last_undegraded":"2024-09-11T01:22:55.994373+0000","last_fullsized":"2024-09-11T01:22:55.994373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20016,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.002796+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:56.002796+0000","last_peered":"2024-09-11T01:22:56.002796+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:56.002796+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000522+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:56.000522+0000","last_peered":"2024-09-11T01:22:56.000522+0000","last_clean":"2024-09-11T01:22:56.000522+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:56.000522+0000","last_undegraded":"2024-09-11T01:22:56.000522+0000","last_fullsized":"2024-09-11T01:22:56.000522+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20345,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:55.994507+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:55.994507+0000","last_peered":"2024-09-11T01:22:55.994507+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:55.994507+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999168+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:55.999168+0000","last_peered":"2024-09-11T01:22:55.999168+0000","last_clean":"2024-09-11T01:22:55.999168+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:55.999168+0000","last_undegraded":"2024-09-11T01:22:55.999168+0000","last_fullsized":"2024-09-11T01:22:55.999168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27796,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994249+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:55.994249+0000","last_peered":"2024-09-11T01:22:55.994249+0000","last_clean":"2024-09-11T01:22:55.994249+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:55.994249+0000","last_undegraded":"2024-09-11T01:22:55.994249+0000","last_fullsized":"2024-09-11T01:22:55.994249+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20415,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.006380+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:56.006380+0000","last_peered":"2024-09-11T01:22:56.006380+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:56.006380+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003870+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:56.003870+0000","last_peered":"2024-09-11T01:22:56.003870+0000","last_clean":"2024-09-11T01:22:56.003870+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:56.003870+0000","last_undegraded":"2024-09-11T01:22:56.003870+0000","last_fullsized":"2024-09-11T01:22:56.003870+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27934,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003846+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:56.003846+0000","last_peered":"2024-09-11T01:22:56.003846+0000","last_clean":"2024-09-11T01:22:56.003846+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:56.003846+0000","last_undegraded":"2024-09-11T01:22:56.003846+0000","last_fullsized":"2024-09-11T01:22:56.003846+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996759+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:55.996759+0000","last_peered":"2024-09-11T01:22:55.996759+0000","last_clean":"2024-09-11T01:22:55.996759+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:55.996759+0000","last_undegraded":"2024-09-11T01:22:55.996759+0000","last_fullsized":"2024-09-11T01:22:55.996759+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24613,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001573+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:56.001573+0000","last_peered":"2024-09-11T01:22:56.001573+0000","last_clean":"2024-09-11T01:22:56.001573+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:56.001573+0000","last_undegraded":"2024-09-11T01:22:56.001573+0000","last_fullsized":"2024-09-11T01:22:56.001573+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999413+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:55.999413+0000","last_peered":"2024-09-11T01:22:55.999413+0000","last_clean":"2024-09-11T01:22:55.999413+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:55.999413+0000","last_undegraded":"2024-09-11T01:22:55.999413+0000","last_fullsized":"2024-09-11T01:22:55.999413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999502+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:55.999502+0000","last_peered":"2024-09-11T01:22:55.999502+0000","last_clean":"2024-09-11T01:22:55.999502+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:55.999502+0000","last_undegraded":"2024-09-11T01:22:55.999502+0000","last_fullsized":"2024-09-11T01:22:55.999502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20105,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001118+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:56.001118+0000","last_peered":"2024-09-11T01:22:56.001118+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:56.001118+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22104,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195088384,"data_stored":195036199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182543,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518600,"kb_used_data":63544,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853240,"statfs":{"total":96636764160,"available":96105717760,"internally_reserved":0,"allocated":65069056,"data_stored":64767762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986315,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488444,"kb_used_data":127788,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883396,"statfs":{"total":96636764160,"available":96136597504,"internally_reserved":0,"allocated":130854912,"data_stored":130557695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659260,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473592,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93898248,"statfs":{"total":96636764160,"available":96151805952,"internally_reserved":0,"allocated":499712,"data_stored":156670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":484437679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":8021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:31.996 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:32.335 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:32.335 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:23:32.354 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":420,"stamp":"2024-09-11T01:23:30.824664+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22105,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1480636,"kb_used_data":191820,"kb_used_omap":12,"kb_used_meta":1288755,"kb_avail":281634884,"statfs":{"total":289910292480,"available":288394121216,"internally_reserved":0,"allocated":196423680,"data_stored":195482127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1319685124},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-771,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":771,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004265"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":25086,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:28.247950+0000","last_change":"2024-09-11T01:22:54.587427+0000","last_active":"2024-09-11T01:23:28.247950+0000","last_peered":"2024-09-11T01:23:28.247950+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:23:28.247950+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1711,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1711}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20473,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996631+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:55.996631+0000","last_peered":"2024-09-11T01:22:55.996631+0000","last_clean":"2024-09-11T01:22:55.996631+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:55.996631+0000","last_undegraded":"2024-09-11T01:22:55.996631+0000","last_fullsized":"2024-09-11T01:22:55.996631+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20296,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.149418+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:22:56.149418+0000","last_peered":"2024-09-11T01:22:56.149418+0000","last_clean":"2024-09-11T01:22:56.149418+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:22:56.149418+0000","last_undegraded":"2024-09-11T01:22:56.149418+0000","last_fullsized":"2024-09-11T01:22:56.149418+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20223,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996887+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:55.996887+0000","last_peered":"2024-09-11T01:22:55.996887+0000","last_clean":"2024-09-11T01:22:55.996887+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:55.996887+0000","last_undegraded":"2024-09-11T01:22:55.996887+0000","last_fullsized":"2024-09-11T01:22:55.996887+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20156,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001082+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:56.001082+0000","last_peered":"2024-09-11T01:22:56.001082+0000","last_clean":"2024-09-11T01:22:56.001082+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:56.001082+0000","last_undegraded":"2024-09-11T01:22:56.001082+0000","last_fullsized":"2024-09-11T01:22:56.001082+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20416,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000893+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:56.000893+0000","last_peered":"2024-09-11T01:22:56.000893+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:56.000893+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20079,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000461+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:56.000461+0000","last_peered":"2024-09-11T01:22:56.000461+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:56.000461+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20403,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000263+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:56.000263+0000","last_peered":"2024-09-11T01:22:56.000263+0000","last_clean":"2024-09-11T01:22:56.000263+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:56.000263+0000","last_undegraded":"2024-09-11T01:22:56.000263+0000","last_fullsized":"2024-09-11T01:22:56.000263+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999267+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:55.999267+0000","last_peered":"2024-09-11T01:22:55.999267+0000","last_clean":"2024-09-11T01:22:55.999267+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:55.999267+0000","last_undegraded":"2024-09-11T01:22:55.999267+0000","last_fullsized":"2024-09-11T01:22:55.999267+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28000,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999439+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:55.999439+0000","last_peered":"2024-09-11T01:22:55.999439+0000","last_clean":"2024-09-11T01:22:55.999439+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:55.999439+0000","last_undegraded":"2024-09-11T01:22:55.999439+0000","last_fullsized":"2024-09-11T01:22:55.999439+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:55.994373+0000","last_peered":"2024-09-11T01:22:55.994373+0000","last_clean":"2024-09-11T01:22:55.994373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:55.994373+0000","last_undegraded":"2024-09-11T01:22:55.994373+0000","last_fullsized":"2024-09-11T01:22:55.994373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20016,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.002796+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:56.002796+0000","last_peered":"2024-09-11T01:22:56.002796+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:56.002796+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000522+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:56.000522+0000","last_peered":"2024-09-11T01:22:56.000522+0000","last_clean":"2024-09-11T01:22:56.000522+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:56.000522+0000","last_undegraded":"2024-09-11T01:22:56.000522+0000","last_fullsized":"2024-09-11T01:22:56.000522+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20345,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:55.994507+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:55.994507+0000","last_peered":"2024-09-11T01:22:55.994507+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:55.994507+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999168+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:55.999168+0000","last_peered":"2024-09-11T01:22:55.999168+0000","last_clean":"2024-09-11T01:22:55.999168+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:55.999168+0000","last_undegraded":"2024-09-11T01:22:55.999168+0000","last_fullsized":"2024-09-11T01:22:55.999168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27796,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994249+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:55.994249+0000","last_peered":"2024-09-11T01:22:55.994249+0000","last_clean":"2024-09-11T01:22:55.994249+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:55.994249+0000","last_undegraded":"2024-09-11T01:22:55.994249+0000","last_fullsized":"2024-09-11T01:22:55.994249+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20415,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.006380+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:56.006380+0000","last_peered":"2024-09-11T01:22:56.006380+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:56.006380+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003870+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:56.003870+0000","last_peered":"2024-09-11T01:22:56.003870+0000","last_clean":"2024-09-11T01:22:56.003870+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:56.003870+0000","last_undegraded":"2024-09-11T01:22:56.003870+0000","last_fullsized":"2024-09-11T01:22:56.003870+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27934,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003846+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:56.003846+0000","last_peered":"2024-09-11T01:22:56.003846+0000","last_clean":"2024-09-11T01:22:56.003846+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:56.003846+0000","last_undegraded":"2024-09-11T01:22:56.003846+0000","last_fullsized":"2024-09-11T01:22:56.003846+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996759+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:55.996759+0000","last_peered":"2024-09-11T01:22:55.996759+0000","last_clean":"2024-09-11T01:22:55.996759+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:55.996759+0000","last_undegraded":"2024-09-11T01:22:55.996759+0000","last_fullsized":"2024-09-11T01:22:55.996759+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24613,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001573+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:56.001573+0000","last_peered":"2024-09-11T01:22:56.001573+0000","last_clean":"2024-09-11T01:22:56.001573+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:56.001573+0000","last_undegraded":"2024-09-11T01:22:56.001573+0000","last_fullsized":"2024-09-11T01:22:56.001573+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999413+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:55.999413+0000","last_peered":"2024-09-11T01:22:55.999413+0000","last_clean":"2024-09-11T01:22:55.999413+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:55.999413+0000","last_undegraded":"2024-09-11T01:22:55.999413+0000","last_fullsized":"2024-09-11T01:22:55.999413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999502+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:55.999502+0000","last_peered":"2024-09-11T01:22:55.999502+0000","last_clean":"2024-09-11T01:22:55.999502+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:55.999502+0000","last_undegraded":"2024-09-11T01:22:55.999502+0000","last_fullsized":"2024-09-11T01:22:55.999502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20105,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001118+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:56.001118+0000","last_peered":"2024-09-11T01:22:56.001118+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:56.001118+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22104,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195088384,"data_stored":195036199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182543,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518600,"kb_used_data":63544,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853240,"statfs":{"total":96636764160,"available":96105717760,"internally_reserved":0,"allocated":65069056,"data_stored":64767762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986316,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488444,"kb_used_data":127788,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883396,"statfs":{"total":96636764160,"available":96136597504,"internally_reserved":0,"allocated":130854912,"data_stored":130557695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659260,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473592,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93898248,"statfs":{"total":96636764160,"available":96151805952,"internally_reserved":0,"allocated":499712,"data_stored":156670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":484437679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":8021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:32.356 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:32.863 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:32.863 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:23:32.884 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":421,"stamp":"2024-09-11T01:23:32.825427+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22105,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1480636,"kb_used_data":191820,"kb_used_omap":12,"kb_used_meta":1288755,"kb_avail":281634884,"statfs":{"total":289910292480,"available":288394121216,"internally_reserved":0,"allocated":196423680,"data_stored":195482127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1319685124},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-771,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":771,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004340"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":25086,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:28.247950+0000","last_change":"2024-09-11T01:22:54.587427+0000","last_active":"2024-09-11T01:23:28.247950+0000","last_peered":"2024-09-11T01:23:28.247950+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:23:28.247950+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1711,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1711}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20473,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996631+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:55.996631+0000","last_peered":"2024-09-11T01:22:55.996631+0000","last_clean":"2024-09-11T01:22:55.996631+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:55.996631+0000","last_undegraded":"2024-09-11T01:22:55.996631+0000","last_fullsized":"2024-09-11T01:22:55.996631+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20296,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.149418+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:22:56.149418+0000","last_peered":"2024-09-11T01:22:56.149418+0000","last_clean":"2024-09-11T01:22:56.149418+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:22:56.149418+0000","last_undegraded":"2024-09-11T01:22:56.149418+0000","last_fullsized":"2024-09-11T01:22:56.149418+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20223,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996887+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:55.996887+0000","last_peered":"2024-09-11T01:22:55.996887+0000","last_clean":"2024-09-11T01:22:55.996887+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:55.996887+0000","last_undegraded":"2024-09-11T01:22:55.996887+0000","last_fullsized":"2024-09-11T01:22:55.996887+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20156,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001082+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:56.001082+0000","last_peered":"2024-09-11T01:22:56.001082+0000","last_clean":"2024-09-11T01:22:56.001082+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:56.001082+0000","last_undegraded":"2024-09-11T01:22:56.001082+0000","last_fullsized":"2024-09-11T01:22:56.001082+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20416,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000893+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:56.000893+0000","last_peered":"2024-09-11T01:22:56.000893+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:56.000893+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20079,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000461+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:56.000461+0000","last_peered":"2024-09-11T01:22:56.000461+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:56.000461+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20403,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000263+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:56.000263+0000","last_peered":"2024-09-11T01:22:56.000263+0000","last_clean":"2024-09-11T01:22:56.000263+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:56.000263+0000","last_undegraded":"2024-09-11T01:22:56.000263+0000","last_fullsized":"2024-09-11T01:22:56.000263+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999267+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:55.999267+0000","last_peered":"2024-09-11T01:22:55.999267+0000","last_clean":"2024-09-11T01:22:55.999267+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:55.999267+0000","last_undegraded":"2024-09-11T01:22:55.999267+0000","last_fullsized":"2024-09-11T01:22:55.999267+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28000,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999439+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:55.999439+0000","last_peered":"2024-09-11T01:22:55.999439+0000","last_clean":"2024-09-11T01:22:55.999439+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:55.999439+0000","last_undegraded":"2024-09-11T01:22:55.999439+0000","last_fullsized":"2024-09-11T01:22:55.999439+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:55.994373+0000","last_peered":"2024-09-11T01:22:55.994373+0000","last_clean":"2024-09-11T01:22:55.994373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:55.994373+0000","last_undegraded":"2024-09-11T01:22:55.994373+0000","last_fullsized":"2024-09-11T01:22:55.994373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20016,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.002796+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:56.002796+0000","last_peered":"2024-09-11T01:22:56.002796+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:56.002796+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000522+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:56.000522+0000","last_peered":"2024-09-11T01:22:56.000522+0000","last_clean":"2024-09-11T01:22:56.000522+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:56.000522+0000","last_undegraded":"2024-09-11T01:22:56.000522+0000","last_fullsized":"2024-09-11T01:22:56.000522+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20345,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:55.994507+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:55.994507+0000","last_peered":"2024-09-11T01:22:55.994507+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:55.994507+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999168+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:55.999168+0000","last_peered":"2024-09-11T01:22:55.999168+0000","last_clean":"2024-09-11T01:22:55.999168+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:55.999168+0000","last_undegraded":"2024-09-11T01:22:55.999168+0000","last_fullsized":"2024-09-11T01:22:55.999168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27796,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994249+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:55.994249+0000","last_peered":"2024-09-11T01:22:55.994249+0000","last_clean":"2024-09-11T01:22:55.994249+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:55.994249+0000","last_undegraded":"2024-09-11T01:22:55.994249+0000","last_fullsized":"2024-09-11T01:22:55.994249+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20415,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.006380+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:56.006380+0000","last_peered":"2024-09-11T01:22:56.006380+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:56.006380+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003870+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:56.003870+0000","last_peered":"2024-09-11T01:22:56.003870+0000","last_clean":"2024-09-11T01:22:56.003870+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:56.003870+0000","last_undegraded":"2024-09-11T01:22:56.003870+0000","last_fullsized":"2024-09-11T01:22:56.003870+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27934,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003846+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:56.003846+0000","last_peered":"2024-09-11T01:22:56.003846+0000","last_clean":"2024-09-11T01:22:56.003846+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:56.003846+0000","last_undegraded":"2024-09-11T01:22:56.003846+0000","last_fullsized":"2024-09-11T01:22:56.003846+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996759+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:55.996759+0000","last_peered":"2024-09-11T01:22:55.996759+0000","last_clean":"2024-09-11T01:22:55.996759+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:55.996759+0000","last_undegraded":"2024-09-11T01:22:55.996759+0000","last_fullsized":"2024-09-11T01:22:55.996759+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24613,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001573+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:56.001573+0000","last_peered":"2024-09-11T01:22:56.001573+0000","last_clean":"2024-09-11T01:22:56.001573+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:56.001573+0000","last_undegraded":"2024-09-11T01:22:56.001573+0000","last_fullsized":"2024-09-11T01:22:56.001573+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999413+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:55.999413+0000","last_peered":"2024-09-11T01:22:55.999413+0000","last_clean":"2024-09-11T01:22:55.999413+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:55.999413+0000","last_undegraded":"2024-09-11T01:22:55.999413+0000","last_fullsized":"2024-09-11T01:22:55.999413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999502+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:55.999502+0000","last_peered":"2024-09-11T01:22:55.999502+0000","last_clean":"2024-09-11T01:22:55.999502+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:55.999502+0000","last_undegraded":"2024-09-11T01:22:55.999502+0000","last_fullsized":"2024-09-11T01:22:55.999502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20105,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001118+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:56.001118+0000","last_peered":"2024-09-11T01:22:56.001118+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:56.001118+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22104,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195088384,"data_stored":195036199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182543,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518600,"kb_used_data":63544,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853240,"statfs":{"total":96636764160,"available":96105717760,"internally_reserved":0,"allocated":65069056,"data_stored":64767762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986316,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488444,"kb_used_data":127788,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883396,"statfs":{"total":96636764160,"available":96136597504,"internally_reserved":0,"allocated":130854912,"data_stored":130557695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659260,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473592,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93898248,"statfs":{"total":96636764160,"available":96151805952,"internally_reserved":0,"allocated":499712,"data_stored":156670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":484437679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":8021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:32.885 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:23:32.885 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20266", 'reported_seq': 25086, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:28.247950+0000', 'last_change': '2024-09-11T01:22:54.587427+0000', 'last_active': '2024-09-11T01:23:28.247950+0000', 'last_peered': '2024-09-11T01:23:28.247950+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:23:28.247950+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1711, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20266, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2361, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1711}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:32.885 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:23:32.885 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'20227", 'reported_seq': 20416, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.000893+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:22:56.000893+0000', 'last_peered': '2024-09-11T01:22:56.000893+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:22:56.000893+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16568, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:32.886 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:23:32.886 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20079, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.000461+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:22:56.000461+0000', 'last_peered': '2024-09-11T01:22:56.000461+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:22:56.000461+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:32.886 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:23:32.886 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20016, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.002796+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:22:56.002796+0000', 'last_peered': '2024-09-11T01:22:56.002796+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:22:56.002796+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:32.886 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:23:32.886 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'20207", 'reported_seq': 20345, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:55.994507+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:22:55.994507+0000', 'last_peered': '2024-09-11T01:22:55.994507+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:22:55.994507+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18024, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4024}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:32.886 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:23:32.886 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20264", 'reported_seq': 20415, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.006380+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:22:56.006380+0000', 'last_peered': '2024-09-11T01:22:56.006380+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:22:56.006380+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17001, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:32.886 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:23:32.886 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 20105, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.001118+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:22:56.001118+0000', 'last_peered': '2024-09-11T01:22:56.001118+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:22:56.001118+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:32.887 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:23:33.295 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:33.295 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 25535/6 objects degraded (425583.333%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":749,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1516171264,"bytes_avail":288394121216,"bytes_total":289910292480,"degraded_objects":25535,"degraded_total":6,"degraded_ratio":4255.833333333333,"recovering_objects_per_sec":64,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":771,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:23:33.315 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:23:33.315 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:33.665 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:33.665 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:23:33.684 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":421,"stamp":"2024-09-11T01:23:32.825427+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22105,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1480636,"kb_used_data":191820,"kb_used_omap":12,"kb_used_meta":1288755,"kb_avail":281634884,"statfs":{"total":289910292480,"available":288394121216,"internally_reserved":0,"allocated":196423680,"data_stored":195482127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1319685124},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-771,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":771,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004340"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":25086,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:28.247950+0000","last_change":"2024-09-11T01:22:54.587427+0000","last_active":"2024-09-11T01:23:28.247950+0000","last_peered":"2024-09-11T01:23:28.247950+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:23:28.247950+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1711,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2361,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1711}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20473,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996631+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:55.996631+0000","last_peered":"2024-09-11T01:22:55.996631+0000","last_clean":"2024-09-11T01:22:55.996631+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:55.996631+0000","last_undegraded":"2024-09-11T01:22:55.996631+0000","last_fullsized":"2024-09-11T01:22:55.996631+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20296,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.149418+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:22:56.149418+0000","last_peered":"2024-09-11T01:22:56.149418+0000","last_clean":"2024-09-11T01:22:56.149418+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:22:56.149418+0000","last_undegraded":"2024-09-11T01:22:56.149418+0000","last_fullsized":"2024-09-11T01:22:56.149418+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20223,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996887+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:55.996887+0000","last_peered":"2024-09-11T01:22:55.996887+0000","last_clean":"2024-09-11T01:22:55.996887+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:55.996887+0000","last_undegraded":"2024-09-11T01:22:55.996887+0000","last_fullsized":"2024-09-11T01:22:55.996887+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20156,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001082+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:56.001082+0000","last_peered":"2024-09-11T01:22:56.001082+0000","last_clean":"2024-09-11T01:22:56.001082+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:56.001082+0000","last_undegraded":"2024-09-11T01:22:56.001082+0000","last_fullsized":"2024-09-11T01:22:56.001082+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20416,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000893+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:56.000893+0000","last_peered":"2024-09-11T01:22:56.000893+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:56.000893+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20079,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000461+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:56.000461+0000","last_peered":"2024-09-11T01:22:56.000461+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:56.000461+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20403,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000263+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:56.000263+0000","last_peered":"2024-09-11T01:22:56.000263+0000","last_clean":"2024-09-11T01:22:56.000263+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:56.000263+0000","last_undegraded":"2024-09-11T01:22:56.000263+0000","last_fullsized":"2024-09-11T01:22:56.000263+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999267+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:55.999267+0000","last_peered":"2024-09-11T01:22:55.999267+0000","last_clean":"2024-09-11T01:22:55.999267+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:55.999267+0000","last_undegraded":"2024-09-11T01:22:55.999267+0000","last_fullsized":"2024-09-11T01:22:55.999267+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28000,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999439+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:55.999439+0000","last_peered":"2024-09-11T01:22:55.999439+0000","last_clean":"2024-09-11T01:22:55.999439+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:55.999439+0000","last_undegraded":"2024-09-11T01:22:55.999439+0000","last_fullsized":"2024-09-11T01:22:55.999439+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:55.994373+0000","last_peered":"2024-09-11T01:22:55.994373+0000","last_clean":"2024-09-11T01:22:55.994373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:55.994373+0000","last_undegraded":"2024-09-11T01:22:55.994373+0000","last_fullsized":"2024-09-11T01:22:55.994373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20016,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.002796+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:56.002796+0000","last_peered":"2024-09-11T01:22:56.002796+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:56.002796+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000522+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:56.000522+0000","last_peered":"2024-09-11T01:22:56.000522+0000","last_clean":"2024-09-11T01:22:56.000522+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:56.000522+0000","last_undegraded":"2024-09-11T01:22:56.000522+0000","last_fullsized":"2024-09-11T01:22:56.000522+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20345,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:55.994507+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:55.994507+0000","last_peered":"2024-09-11T01:22:55.994507+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:55.994507+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999168+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:55.999168+0000","last_peered":"2024-09-11T01:22:55.999168+0000","last_clean":"2024-09-11T01:22:55.999168+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:55.999168+0000","last_undegraded":"2024-09-11T01:22:55.999168+0000","last_fullsized":"2024-09-11T01:22:55.999168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27796,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994249+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:55.994249+0000","last_peered":"2024-09-11T01:22:55.994249+0000","last_clean":"2024-09-11T01:22:55.994249+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:55.994249+0000","last_undegraded":"2024-09-11T01:22:55.994249+0000","last_fullsized":"2024-09-11T01:22:55.994249+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20415,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.006380+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:56.006380+0000","last_peered":"2024-09-11T01:22:56.006380+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:56.006380+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003870+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:56.003870+0000","last_peered":"2024-09-11T01:22:56.003870+0000","last_clean":"2024-09-11T01:22:56.003870+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:56.003870+0000","last_undegraded":"2024-09-11T01:22:56.003870+0000","last_fullsized":"2024-09-11T01:22:56.003870+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27934,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003846+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:56.003846+0000","last_peered":"2024-09-11T01:22:56.003846+0000","last_clean":"2024-09-11T01:22:56.003846+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:56.003846+0000","last_undegraded":"2024-09-11T01:22:56.003846+0000","last_fullsized":"2024-09-11T01:22:56.003846+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996759+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:55.996759+0000","last_peered":"2024-09-11T01:22:55.996759+0000","last_clean":"2024-09-11T01:22:55.996759+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:55.996759+0000","last_undegraded":"2024-09-11T01:22:55.996759+0000","last_fullsized":"2024-09-11T01:22:55.996759+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24613,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001573+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:56.001573+0000","last_peered":"2024-09-11T01:22:56.001573+0000","last_clean":"2024-09-11T01:22:56.001573+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:56.001573+0000","last_undegraded":"2024-09-11T01:22:56.001573+0000","last_fullsized":"2024-09-11T01:22:56.001573+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999413+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:55.999413+0000","last_peered":"2024-09-11T01:22:55.999413+0000","last_clean":"2024-09-11T01:22:55.999413+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:55.999413+0000","last_undegraded":"2024-09-11T01:22:55.999413+0000","last_fullsized":"2024-09-11T01:22:55.999413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999502+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:55.999502+0000","last_peered":"2024-09-11T01:22:55.999502+0000","last_clean":"2024-09-11T01:22:55.999502+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:55.999502+0000","last_undegraded":"2024-09-11T01:22:55.999502+0000","last_fullsized":"2024-09-11T01:22:55.999502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20105,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001118+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:56.001118+0000","last_peered":"2024-09-11T01:22:56.001118+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:56.001118+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25535,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22104,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195088384,"data_stored":195036199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182543,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518600,"kb_used_data":63544,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853240,"statfs":{"total":96636764160,"available":96105717760,"internally_reserved":0,"allocated":65069056,"data_stored":64767762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986316,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488444,"kb_used_data":127788,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883396,"statfs":{"total":96636764160,"available":96136597504,"internally_reserved":0,"allocated":130854912,"data_stored":130557695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659260,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473592,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":473083,"kb_avail":93898248,"statfs":{"total":96636764160,"available":96151805952,"internally_reserved":0,"allocated":499712,"data_stored":156670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":484437679},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":8021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:36.687 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:37.016 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:37.016 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:23:37.036 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":423,"stamp":"2024-09-11T01:23:36.826788+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25064,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22576,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1468156,"kb_used_data":191820,"kb_used_omap":12,"kb_used_meta":1276275,"kb_avail":281647364,"statfs":{"total":289910292480,"available":288406900736,"internally_reserved":0,"allocated":196423680,"data_stored":195482127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1306905604},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-883,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":883,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004153"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":26028,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:33.252043+0000","last_change":"2024-09-11T01:22:54.587427+0000","last_active":"2024-09-11T01:23:33.252043+0000","last_peered":"2024-09-11T01:23:33.252043+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:23:33.252043+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1240,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2832,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1240}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20473,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996631+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:55.996631+0000","last_peered":"2024-09-11T01:22:55.996631+0000","last_clean":"2024-09-11T01:22:55.996631+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:55.996631+0000","last_undegraded":"2024-09-11T01:22:55.996631+0000","last_fullsized":"2024-09-11T01:22:55.996631+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20296,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.149418+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:22:56.149418+0000","last_peered":"2024-09-11T01:22:56.149418+0000","last_clean":"2024-09-11T01:22:56.149418+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:22:56.149418+0000","last_undegraded":"2024-09-11T01:22:56.149418+0000","last_fullsized":"2024-09-11T01:22:56.149418+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20223,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996887+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:55.996887+0000","last_peered":"2024-09-11T01:22:55.996887+0000","last_clean":"2024-09-11T01:22:55.996887+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:55.996887+0000","last_undegraded":"2024-09-11T01:22:55.996887+0000","last_fullsized":"2024-09-11T01:22:55.996887+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20156,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001082+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:56.001082+0000","last_peered":"2024-09-11T01:22:56.001082+0000","last_clean":"2024-09-11T01:22:56.001082+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:56.001082+0000","last_undegraded":"2024-09-11T01:22:56.001082+0000","last_fullsized":"2024-09-11T01:22:56.001082+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20416,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000893+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:56.000893+0000","last_peered":"2024-09-11T01:22:56.000893+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:56.000893+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20079,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000461+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:56.000461+0000","last_peered":"2024-09-11T01:22:56.000461+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:56.000461+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20403,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000263+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:56.000263+0000","last_peered":"2024-09-11T01:22:56.000263+0000","last_clean":"2024-09-11T01:22:56.000263+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:56.000263+0000","last_undegraded":"2024-09-11T01:22:56.000263+0000","last_fullsized":"2024-09-11T01:22:56.000263+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999267+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:55.999267+0000","last_peered":"2024-09-11T01:22:55.999267+0000","last_clean":"2024-09-11T01:22:55.999267+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:55.999267+0000","last_undegraded":"2024-09-11T01:22:55.999267+0000","last_fullsized":"2024-09-11T01:22:55.999267+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28000,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999439+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:55.999439+0000","last_peered":"2024-09-11T01:22:55.999439+0000","last_clean":"2024-09-11T01:22:55.999439+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:55.999439+0000","last_undegraded":"2024-09-11T01:22:55.999439+0000","last_fullsized":"2024-09-11T01:22:55.999439+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:55.994373+0000","last_peered":"2024-09-11T01:22:55.994373+0000","last_clean":"2024-09-11T01:22:55.994373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:55.994373+0000","last_undegraded":"2024-09-11T01:22:55.994373+0000","last_fullsized":"2024-09-11T01:22:55.994373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20016,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.002796+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:56.002796+0000","last_peered":"2024-09-11T01:22:56.002796+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:56.002796+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000522+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:56.000522+0000","last_peered":"2024-09-11T01:22:56.000522+0000","last_clean":"2024-09-11T01:22:56.000522+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:56.000522+0000","last_undegraded":"2024-09-11T01:22:56.000522+0000","last_fullsized":"2024-09-11T01:22:56.000522+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20345,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:55.994507+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:55.994507+0000","last_peered":"2024-09-11T01:22:55.994507+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:55.994507+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999168+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:55.999168+0000","last_peered":"2024-09-11T01:22:55.999168+0000","last_clean":"2024-09-11T01:22:55.999168+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:55.999168+0000","last_undegraded":"2024-09-11T01:22:55.999168+0000","last_fullsized":"2024-09-11T01:22:55.999168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27796,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994249+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:55.994249+0000","last_peered":"2024-09-11T01:22:55.994249+0000","last_clean":"2024-09-11T01:22:55.994249+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:55.994249+0000","last_undegraded":"2024-09-11T01:22:55.994249+0000","last_fullsized":"2024-09-11T01:22:55.994249+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20415,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.006380+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:56.006380+0000","last_peered":"2024-09-11T01:22:56.006380+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:56.006380+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003870+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:56.003870+0000","last_peered":"2024-09-11T01:22:56.003870+0000","last_clean":"2024-09-11T01:22:56.003870+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:56.003870+0000","last_undegraded":"2024-09-11T01:22:56.003870+0000","last_fullsized":"2024-09-11T01:22:56.003870+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27934,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003846+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:56.003846+0000","last_peered":"2024-09-11T01:22:56.003846+0000","last_clean":"2024-09-11T01:22:56.003846+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:56.003846+0000","last_undegraded":"2024-09-11T01:22:56.003846+0000","last_fullsized":"2024-09-11T01:22:56.003846+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996759+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:55.996759+0000","last_peered":"2024-09-11T01:22:55.996759+0000","last_clean":"2024-09-11T01:22:55.996759+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:55.996759+0000","last_undegraded":"2024-09-11T01:22:55.996759+0000","last_fullsized":"2024-09-11T01:22:55.996759+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24613,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001573+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:56.001573+0000","last_peered":"2024-09-11T01:22:56.001573+0000","last_clean":"2024-09-11T01:22:56.001573+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:56.001573+0000","last_undegraded":"2024-09-11T01:22:56.001573+0000","last_fullsized":"2024-09-11T01:22:56.001573+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999413+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:55.999413+0000","last_peered":"2024-09-11T01:22:55.999413+0000","last_clean":"2024-09-11T01:22:55.999413+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:55.999413+0000","last_undegraded":"2024-09-11T01:22:55.999413+0000","last_fullsized":"2024-09-11T01:22:55.999413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999502+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:55.999502+0000","last_peered":"2024-09-11T01:22:55.999502+0000","last_clean":"2024-09-11T01:22:55.999502+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:55.999502+0000","last_undegraded":"2024-09-11T01:22:55.999502+0000","last_fullsized":"2024-09-11T01:22:55.999502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20105,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001118+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:56.001118+0000","last_peered":"2024-09-11T01:22:56.001118+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:56.001118+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25064,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22575,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195088384,"data_stored":195036199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182544,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518600,"kb_used_data":63544,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853240,"statfs":{"total":96636764160,"available":96105717760,"internally_reserved":0,"allocated":65069056,"data_stored":64767762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986317,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488444,"kb_used_data":127788,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883396,"statfs":{"total":96636764160,"available":96136597504,"internally_reserved":0,"allocated":130854912,"data_stored":130557695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659261,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461112,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93910728,"statfs":{"total":96636764160,"available":96164585472,"internally_reserved":0,"allocated":499712,"data_stored":156670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":471658159},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":8021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:37.037 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:37.363 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:37.363 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:23:37.381 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":423,"stamp":"2024-09-11T01:23:36.826788+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25064,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22576,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1468156,"kb_used_data":191820,"kb_used_omap":12,"kb_used_meta":1276275,"kb_avail":281647364,"statfs":{"total":289910292480,"available":288406900736,"internally_reserved":0,"allocated":196423680,"data_stored":195482127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1306905604},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-883,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":883,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004153"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":26028,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:33.252043+0000","last_change":"2024-09-11T01:22:54.587427+0000","last_active":"2024-09-11T01:23:33.252043+0000","last_peered":"2024-09-11T01:23:33.252043+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:23:33.252043+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1240,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2832,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1240}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20473,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996631+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:55.996631+0000","last_peered":"2024-09-11T01:22:55.996631+0000","last_clean":"2024-09-11T01:22:55.996631+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:55.996631+0000","last_undegraded":"2024-09-11T01:22:55.996631+0000","last_fullsized":"2024-09-11T01:22:55.996631+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20296,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.149418+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:22:56.149418+0000","last_peered":"2024-09-11T01:22:56.149418+0000","last_clean":"2024-09-11T01:22:56.149418+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:22:56.149418+0000","last_undegraded":"2024-09-11T01:22:56.149418+0000","last_fullsized":"2024-09-11T01:22:56.149418+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20223,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996887+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:55.996887+0000","last_peered":"2024-09-11T01:22:55.996887+0000","last_clean":"2024-09-11T01:22:55.996887+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:55.996887+0000","last_undegraded":"2024-09-11T01:22:55.996887+0000","last_fullsized":"2024-09-11T01:22:55.996887+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20156,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001082+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:56.001082+0000","last_peered":"2024-09-11T01:22:56.001082+0000","last_clean":"2024-09-11T01:22:56.001082+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:56.001082+0000","last_undegraded":"2024-09-11T01:22:56.001082+0000","last_fullsized":"2024-09-11T01:22:56.001082+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20416,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000893+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:56.000893+0000","last_peered":"2024-09-11T01:22:56.000893+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:56.000893+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20079,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000461+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:56.000461+0000","last_peered":"2024-09-11T01:22:56.000461+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:56.000461+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20403,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000263+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:56.000263+0000","last_peered":"2024-09-11T01:22:56.000263+0000","last_clean":"2024-09-11T01:22:56.000263+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:56.000263+0000","last_undegraded":"2024-09-11T01:22:56.000263+0000","last_fullsized":"2024-09-11T01:22:56.000263+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999267+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:55.999267+0000","last_peered":"2024-09-11T01:22:55.999267+0000","last_clean":"2024-09-11T01:22:55.999267+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:55.999267+0000","last_undegraded":"2024-09-11T01:22:55.999267+0000","last_fullsized":"2024-09-11T01:22:55.999267+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28000,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999439+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:55.999439+0000","last_peered":"2024-09-11T01:22:55.999439+0000","last_clean":"2024-09-11T01:22:55.999439+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:55.999439+0000","last_undegraded":"2024-09-11T01:22:55.999439+0000","last_fullsized":"2024-09-11T01:22:55.999439+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:55.994373+0000","last_peered":"2024-09-11T01:22:55.994373+0000","last_clean":"2024-09-11T01:22:55.994373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:55.994373+0000","last_undegraded":"2024-09-11T01:22:55.994373+0000","last_fullsized":"2024-09-11T01:22:55.994373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20016,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.002796+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:56.002796+0000","last_peered":"2024-09-11T01:22:56.002796+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:56.002796+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000522+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:56.000522+0000","last_peered":"2024-09-11T01:22:56.000522+0000","last_clean":"2024-09-11T01:22:56.000522+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:56.000522+0000","last_undegraded":"2024-09-11T01:22:56.000522+0000","last_fullsized":"2024-09-11T01:22:56.000522+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20345,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:55.994507+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:55.994507+0000","last_peered":"2024-09-11T01:22:55.994507+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:55.994507+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999168+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:55.999168+0000","last_peered":"2024-09-11T01:22:55.999168+0000","last_clean":"2024-09-11T01:22:55.999168+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:55.999168+0000","last_undegraded":"2024-09-11T01:22:55.999168+0000","last_fullsized":"2024-09-11T01:22:55.999168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27796,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994249+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:55.994249+0000","last_peered":"2024-09-11T01:22:55.994249+0000","last_clean":"2024-09-11T01:22:55.994249+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:55.994249+0000","last_undegraded":"2024-09-11T01:22:55.994249+0000","last_fullsized":"2024-09-11T01:22:55.994249+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20415,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.006380+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:56.006380+0000","last_peered":"2024-09-11T01:22:56.006380+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:56.006380+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003870+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:56.003870+0000","last_peered":"2024-09-11T01:22:56.003870+0000","last_clean":"2024-09-11T01:22:56.003870+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:56.003870+0000","last_undegraded":"2024-09-11T01:22:56.003870+0000","last_fullsized":"2024-09-11T01:22:56.003870+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27934,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003846+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:56.003846+0000","last_peered":"2024-09-11T01:22:56.003846+0000","last_clean":"2024-09-11T01:22:56.003846+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:56.003846+0000","last_undegraded":"2024-09-11T01:22:56.003846+0000","last_fullsized":"2024-09-11T01:22:56.003846+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996759+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:55.996759+0000","last_peered":"2024-09-11T01:22:55.996759+0000","last_clean":"2024-09-11T01:22:55.996759+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:55.996759+0000","last_undegraded":"2024-09-11T01:22:55.996759+0000","last_fullsized":"2024-09-11T01:22:55.996759+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24613,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001573+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:56.001573+0000","last_peered":"2024-09-11T01:22:56.001573+0000","last_clean":"2024-09-11T01:22:56.001573+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:56.001573+0000","last_undegraded":"2024-09-11T01:22:56.001573+0000","last_fullsized":"2024-09-11T01:22:56.001573+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999413+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:55.999413+0000","last_peered":"2024-09-11T01:22:55.999413+0000","last_clean":"2024-09-11T01:22:55.999413+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:55.999413+0000","last_undegraded":"2024-09-11T01:22:55.999413+0000","last_fullsized":"2024-09-11T01:22:55.999413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999502+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:55.999502+0000","last_peered":"2024-09-11T01:22:55.999502+0000","last_clean":"2024-09-11T01:22:55.999502+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:55.999502+0000","last_undegraded":"2024-09-11T01:22:55.999502+0000","last_fullsized":"2024-09-11T01:22:55.999502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20105,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001118+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:56.001118+0000","last_peered":"2024-09-11T01:22:56.001118+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:56.001118+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25064,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22575,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195088384,"data_stored":195036199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182544,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518600,"kb_used_data":63544,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853240,"statfs":{"total":96636764160,"available":96105717760,"internally_reserved":0,"allocated":65069056,"data_stored":64767762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986317,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488444,"kb_used_data":127788,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883396,"statfs":{"total":96636764160,"available":96136597504,"internally_reserved":0,"allocated":130854912,"data_stored":130557695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659261,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461112,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93910728,"statfs":{"total":96636764160,"available":96164585472,"internally_reserved":0,"allocated":499712,"data_stored":156670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":471658159},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":8021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:37.382 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:23:37.382 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20266", 'reported_seq': 26028, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:33.252043+0000', 'last_change': '2024-09-11T01:22:54.587427+0000', 'last_active': '2024-09-11T01:23:33.252043+0000', 'last_peered': '2024-09-11T01:23:33.252043+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:23:33.252043+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1240, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20266, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2832, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1240}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:37.383 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:23:37.383 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'20227", 'reported_seq': 20416, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.000893+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:22:56.000893+0000', 'last_peered': '2024-09-11T01:22:56.000893+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:22:56.000893+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16568, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:37.383 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:23:37.383 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20079, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.000461+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:22:56.000461+0000', 'last_peered': '2024-09-11T01:22:56.000461+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:22:56.000461+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:37.383 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:23:37.383 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20016, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.002796+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:22:56.002796+0000', 'last_peered': '2024-09-11T01:22:56.002796+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:22:56.002796+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:37.383 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:23:37.383 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'20207", 'reported_seq': 20345, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:55.994507+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:22:55.994507+0000', 'last_peered': '2024-09-11T01:22:55.994507+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:22:55.994507+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18024, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4024}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:37.383 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:23:37.383 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20264", 'reported_seq': 20415, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.006380+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:22:56.006380+0000', 'last_peered': '2024-09-11T01:22:56.006380+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:22:56.006380+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17001, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:37.384 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:23:37.384 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 20105, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.001118+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:22:56.001118+0000', 'last_peered': '2024-09-11T01:22:56.001118+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:22:56.001118+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:37.384 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:23:37.777 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:37.777 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 25064/6 objects degraded (417733.333%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":754,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1503391744,"bytes_avail":288406900736,"bytes_total":289910292480,"degraded_objects":25064,"degraded_total":6,"degraded_ratio":4177.333333333333,"recovering_objects_per_sec":73,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":883,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:23:37.796 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:23:37.797 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:38.140 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:38.141 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:23:38.160 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":423,"stamp":"2024-09-11T01:23:36.826788+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25064,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22576,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1468156,"kb_used_data":191820,"kb_used_omap":12,"kb_used_meta":1276275,"kb_avail":281647364,"statfs":{"total":289910292480,"available":288406900736,"internally_reserved":0,"allocated":196423680,"data_stored":195482127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1306905604},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-883,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":883,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004153"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":26028,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:33.252043+0000","last_change":"2024-09-11T01:22:54.587427+0000","last_active":"2024-09-11T01:23:33.252043+0000","last_peered":"2024-09-11T01:23:33.252043+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:23:33.252043+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1240,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2832,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1240}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20473,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996631+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:55.996631+0000","last_peered":"2024-09-11T01:22:55.996631+0000","last_clean":"2024-09-11T01:22:55.996631+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:55.996631+0000","last_undegraded":"2024-09-11T01:22:55.996631+0000","last_fullsized":"2024-09-11T01:22:55.996631+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20296,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.149418+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:22:56.149418+0000","last_peered":"2024-09-11T01:22:56.149418+0000","last_clean":"2024-09-11T01:22:56.149418+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:22:56.149418+0000","last_undegraded":"2024-09-11T01:22:56.149418+0000","last_fullsized":"2024-09-11T01:22:56.149418+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20223,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996887+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:55.996887+0000","last_peered":"2024-09-11T01:22:55.996887+0000","last_clean":"2024-09-11T01:22:55.996887+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:55.996887+0000","last_undegraded":"2024-09-11T01:22:55.996887+0000","last_fullsized":"2024-09-11T01:22:55.996887+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20156,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001082+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:56.001082+0000","last_peered":"2024-09-11T01:22:56.001082+0000","last_clean":"2024-09-11T01:22:56.001082+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:56.001082+0000","last_undegraded":"2024-09-11T01:22:56.001082+0000","last_fullsized":"2024-09-11T01:22:56.001082+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20416,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000893+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:56.000893+0000","last_peered":"2024-09-11T01:22:56.000893+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:56.000893+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20079,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000461+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:56.000461+0000","last_peered":"2024-09-11T01:22:56.000461+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:56.000461+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20403,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000263+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:56.000263+0000","last_peered":"2024-09-11T01:22:56.000263+0000","last_clean":"2024-09-11T01:22:56.000263+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:56.000263+0000","last_undegraded":"2024-09-11T01:22:56.000263+0000","last_fullsized":"2024-09-11T01:22:56.000263+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999267+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:55.999267+0000","last_peered":"2024-09-11T01:22:55.999267+0000","last_clean":"2024-09-11T01:22:55.999267+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:55.999267+0000","last_undegraded":"2024-09-11T01:22:55.999267+0000","last_fullsized":"2024-09-11T01:22:55.999267+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28000,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999439+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:55.999439+0000","last_peered":"2024-09-11T01:22:55.999439+0000","last_clean":"2024-09-11T01:22:55.999439+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:55.999439+0000","last_undegraded":"2024-09-11T01:22:55.999439+0000","last_fullsized":"2024-09-11T01:22:55.999439+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:55.994373+0000","last_peered":"2024-09-11T01:22:55.994373+0000","last_clean":"2024-09-11T01:22:55.994373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:55.994373+0000","last_undegraded":"2024-09-11T01:22:55.994373+0000","last_fullsized":"2024-09-11T01:22:55.994373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20016,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.002796+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:56.002796+0000","last_peered":"2024-09-11T01:22:56.002796+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:56.002796+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000522+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:56.000522+0000","last_peered":"2024-09-11T01:22:56.000522+0000","last_clean":"2024-09-11T01:22:56.000522+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:56.000522+0000","last_undegraded":"2024-09-11T01:22:56.000522+0000","last_fullsized":"2024-09-11T01:22:56.000522+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20345,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:55.994507+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:55.994507+0000","last_peered":"2024-09-11T01:22:55.994507+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:55.994507+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":98,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999168+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:55.999168+0000","last_peered":"2024-09-11T01:22:55.999168+0000","last_clean":"2024-09-11T01:22:55.999168+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:55.999168+0000","last_undegraded":"2024-09-11T01:22:55.999168+0000","last_fullsized":"2024-09-11T01:22:55.999168+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27796,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994249+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:55.994249+0000","last_peered":"2024-09-11T01:22:55.994249+0000","last_clean":"2024-09-11T01:22:55.994249+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:55.994249+0000","last_undegraded":"2024-09-11T01:22:55.994249+0000","last_fullsized":"2024-09-11T01:22:55.994249+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20415,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.006380+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:56.006380+0000","last_peered":"2024-09-11T01:22:56.006380+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:56.006380+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003870+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:56.003870+0000","last_peered":"2024-09-11T01:22:56.003870+0000","last_clean":"2024-09-11T01:22:56.003870+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:56.003870+0000","last_undegraded":"2024-09-11T01:22:56.003870+0000","last_fullsized":"2024-09-11T01:22:56.003870+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27934,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003846+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:56.003846+0000","last_peered":"2024-09-11T01:22:56.003846+0000","last_clean":"2024-09-11T01:22:56.003846+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:56.003846+0000","last_undegraded":"2024-09-11T01:22:56.003846+0000","last_fullsized":"2024-09-11T01:22:56.003846+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996759+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:55.996759+0000","last_peered":"2024-09-11T01:22:55.996759+0000","last_clean":"2024-09-11T01:22:55.996759+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:55.996759+0000","last_undegraded":"2024-09-11T01:22:55.996759+0000","last_fullsized":"2024-09-11T01:22:55.996759+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24613,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001573+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:56.001573+0000","last_peered":"2024-09-11T01:22:56.001573+0000","last_clean":"2024-09-11T01:22:56.001573+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:56.001573+0000","last_undegraded":"2024-09-11T01:22:56.001573+0000","last_fullsized":"2024-09-11T01:22:56.001573+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999413+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:55.999413+0000","last_peered":"2024-09-11T01:22:55.999413+0000","last_clean":"2024-09-11T01:22:55.999413+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:55.999413+0000","last_undegraded":"2024-09-11T01:22:55.999413+0000","last_fullsized":"2024-09-11T01:22:55.999413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999502+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:55.999502+0000","last_peered":"2024-09-11T01:22:55.999502+0000","last_clean":"2024-09-11T01:22:55.999502+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:55.999502+0000","last_undegraded":"2024-09-11T01:22:55.999502+0000","last_fullsized":"2024-09-11T01:22:55.999502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20105,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001118+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:56.001118+0000","last_peered":"2024-09-11T01:22:56.001118+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:56.001118+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":25064,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22575,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195088384,"data_stored":195036199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182544,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518600,"kb_used_data":63544,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853240,"statfs":{"total":96636764160,"available":96105717760,"internally_reserved":0,"allocated":65069056,"data_stored":64767762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986317,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488444,"kb_used_data":127788,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883396,"statfs":{"total":96636764160,"available":96136597504,"internally_reserved":0,"allocated":130854912,"data_stored":130557695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659261,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":461112,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":460603,"kb_avail":93910728,"statfs":{"total":96636764160,"available":96164585472,"internally_reserved":0,"allocated":499712,"data_stored":156670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":471658159},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":8021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:41.162 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:41.489 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:41.490 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:23:41.508 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":425,"stamp":"2024-09-11T01:23:40.828105+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24510,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23130,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472316,"kb_used_data":191820,"kb_used_omap":12,"kb_used_meta":1280435,"kb_avail":281643204,"statfs":{"total":289910292480,"available":288402640896,"internally_reserved":0,"allocated":196423680,"data_stored":195482127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1311165444},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1025,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1025,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003849"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":27136,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:38.251982+0000","last_change":"2024-09-11T01:22:54.587427+0000","last_active":"2024-09-11T01:23:38.251982+0000","last_peered":"2024-09-11T01:23:38.251982+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:23:38.251982+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":686,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3386,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-686}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20473,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996631+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:55.996631+0000","last_peered":"2024-09-11T01:22:55.996631+0000","last_clean":"2024-09-11T01:22:55.996631+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:55.996631+0000","last_undegraded":"2024-09-11T01:22:55.996631+0000","last_fullsized":"2024-09-11T01:22:55.996631+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20296,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.149418+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:22:56.149418+0000","last_peered":"2024-09-11T01:22:56.149418+0000","last_clean":"2024-09-11T01:22:56.149418+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:22:56.149418+0000","last_undegraded":"2024-09-11T01:22:56.149418+0000","last_fullsized":"2024-09-11T01:22:56.149418+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20223,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996887+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:55.996887+0000","last_peered":"2024-09-11T01:22:55.996887+0000","last_clean":"2024-09-11T01:22:55.996887+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:55.996887+0000","last_undegraded":"2024-09-11T01:22:55.996887+0000","last_fullsized":"2024-09-11T01:22:55.996887+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20156,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001082+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:56.001082+0000","last_peered":"2024-09-11T01:22:56.001082+0000","last_clean":"2024-09-11T01:22:56.001082+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:56.001082+0000","last_undegraded":"2024-09-11T01:22:56.001082+0000","last_fullsized":"2024-09-11T01:22:56.001082+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20416,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000893+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:56.000893+0000","last_peered":"2024-09-11T01:22:56.000893+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:56.000893+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20079,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000461+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:56.000461+0000","last_peered":"2024-09-11T01:22:56.000461+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:56.000461+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20403,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000263+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:56.000263+0000","last_peered":"2024-09-11T01:22:56.000263+0000","last_clean":"2024-09-11T01:22:56.000263+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:56.000263+0000","last_undegraded":"2024-09-11T01:22:56.000263+0000","last_fullsized":"2024-09-11T01:22:56.000263+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999267+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:55.999267+0000","last_peered":"2024-09-11T01:22:55.999267+0000","last_clean":"2024-09-11T01:22:55.999267+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:55.999267+0000","last_undegraded":"2024-09-11T01:22:55.999267+0000","last_fullsized":"2024-09-11T01:22:55.999267+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28000,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999439+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:55.999439+0000","last_peered":"2024-09-11T01:22:55.999439+0000","last_clean":"2024-09-11T01:22:55.999439+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:55.999439+0000","last_undegraded":"2024-09-11T01:22:55.999439+0000","last_fullsized":"2024-09-11T01:22:55.999439+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:55.994373+0000","last_peered":"2024-09-11T01:22:55.994373+0000","last_clean":"2024-09-11T01:22:55.994373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:55.994373+0000","last_undegraded":"2024-09-11T01:22:55.994373+0000","last_fullsized":"2024-09-11T01:22:55.994373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20016,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.002796+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:56.002796+0000","last_peered":"2024-09-11T01:22:56.002796+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:56.002796+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000522+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:56.000522+0000","last_peered":"2024-09-11T01:22:56.000522+0000","last_clean":"2024-09-11T01:22:56.000522+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:56.000522+0000","last_undegraded":"2024-09-11T01:22:56.000522+0000","last_fullsized":"2024-09-11T01:22:56.000522+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20345,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:55.994507+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:55.994507+0000","last_peered":"2024-09-11T01:22:55.994507+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:55.994507+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999238+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:55.999238+0000","last_peered":"2024-09-11T01:22:55.999238+0000","last_clean":"2024-09-11T01:22:55.999238+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:55.999238+0000","last_undegraded":"2024-09-11T01:22:55.999238+0000","last_fullsized":"2024-09-11T01:22:55.999238+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27796,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994249+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:55.994249+0000","last_peered":"2024-09-11T01:22:55.994249+0000","last_clean":"2024-09-11T01:22:55.994249+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:55.994249+0000","last_undegraded":"2024-09-11T01:22:55.994249+0000","last_fullsized":"2024-09-11T01:22:55.994249+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20415,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.006380+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:56.006380+0000","last_peered":"2024-09-11T01:22:56.006380+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:56.006380+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003986+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:56.003986+0000","last_peered":"2024-09-11T01:22:56.003986+0000","last_clean":"2024-09-11T01:22:56.003986+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:56.003986+0000","last_undegraded":"2024-09-11T01:22:56.003986+0000","last_fullsized":"2024-09-11T01:22:56.003986+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27934,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003846+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:56.003846+0000","last_peered":"2024-09-11T01:22:56.003846+0000","last_clean":"2024-09-11T01:22:56.003846+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:56.003846+0000","last_undegraded":"2024-09-11T01:22:56.003846+0000","last_fullsized":"2024-09-11T01:22:56.003846+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996759+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:55.996759+0000","last_peered":"2024-09-11T01:22:55.996759+0000","last_clean":"2024-09-11T01:22:55.996759+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:55.996759+0000","last_undegraded":"2024-09-11T01:22:55.996759+0000","last_fullsized":"2024-09-11T01:22:55.996759+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24613,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001573+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:56.001573+0000","last_peered":"2024-09-11T01:22:56.001573+0000","last_clean":"2024-09-11T01:22:56.001573+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:56.001573+0000","last_undegraded":"2024-09-11T01:22:56.001573+0000","last_fullsized":"2024-09-11T01:22:56.001573+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999413+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:55.999413+0000","last_peered":"2024-09-11T01:22:55.999413+0000","last_clean":"2024-09-11T01:22:55.999413+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:55.999413+0000","last_undegraded":"2024-09-11T01:22:55.999413+0000","last_fullsized":"2024-09-11T01:22:55.999413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999502+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:55.999502+0000","last_peered":"2024-09-11T01:22:55.999502+0000","last_clean":"2024-09-11T01:22:55.999502+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:55.999502+0000","last_undegraded":"2024-09-11T01:22:55.999502+0000","last_fullsized":"2024-09-11T01:22:55.999502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20105,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001118+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:56.001118+0000","last_peered":"2024-09-11T01:22:56.001118+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:56.001118+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24510,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23129,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195088384,"data_stored":195036199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182545,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518600,"kb_used_data":63544,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853240,"statfs":{"total":96636764160,"available":96105717760,"internally_reserved":0,"allocated":65069056,"data_stored":64767762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986318,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488444,"kb_used_data":127788,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883396,"statfs":{"total":96636764160,"available":96136597504,"internally_reserved":0,"allocated":130854912,"data_stored":130557695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659262,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465272,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93906568,"statfs":{"total":96636764160,"available":96160325632,"internally_reserved":0,"allocated":499712,"data_stored":156670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475917999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":8021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:41.510 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:41.821 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:41.821 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:23:41.839 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":425,"stamp":"2024-09-11T01:23:40.828105+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24510,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23130,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472316,"kb_used_data":191820,"kb_used_omap":12,"kb_used_meta":1280435,"kb_avail":281643204,"statfs":{"total":289910292480,"available":288402640896,"internally_reserved":0,"allocated":196423680,"data_stored":195482127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1311165444},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1025,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1025,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003849"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":27136,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:38.251982+0000","last_change":"2024-09-11T01:22:54.587427+0000","last_active":"2024-09-11T01:23:38.251982+0000","last_peered":"2024-09-11T01:23:38.251982+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:23:38.251982+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":686,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3386,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-686}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20473,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996631+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:55.996631+0000","last_peered":"2024-09-11T01:22:55.996631+0000","last_clean":"2024-09-11T01:22:55.996631+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:55.996631+0000","last_undegraded":"2024-09-11T01:22:55.996631+0000","last_fullsized":"2024-09-11T01:22:55.996631+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20296,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.149418+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:22:56.149418+0000","last_peered":"2024-09-11T01:22:56.149418+0000","last_clean":"2024-09-11T01:22:56.149418+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:22:56.149418+0000","last_undegraded":"2024-09-11T01:22:56.149418+0000","last_fullsized":"2024-09-11T01:22:56.149418+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20223,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996887+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:55.996887+0000","last_peered":"2024-09-11T01:22:55.996887+0000","last_clean":"2024-09-11T01:22:55.996887+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:55.996887+0000","last_undegraded":"2024-09-11T01:22:55.996887+0000","last_fullsized":"2024-09-11T01:22:55.996887+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20156,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001082+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:56.001082+0000","last_peered":"2024-09-11T01:22:56.001082+0000","last_clean":"2024-09-11T01:22:56.001082+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:56.001082+0000","last_undegraded":"2024-09-11T01:22:56.001082+0000","last_fullsized":"2024-09-11T01:22:56.001082+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20416,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000893+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:56.000893+0000","last_peered":"2024-09-11T01:22:56.000893+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:56.000893+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20079,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000461+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:56.000461+0000","last_peered":"2024-09-11T01:22:56.000461+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:56.000461+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20403,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000263+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:56.000263+0000","last_peered":"2024-09-11T01:22:56.000263+0000","last_clean":"2024-09-11T01:22:56.000263+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:56.000263+0000","last_undegraded":"2024-09-11T01:22:56.000263+0000","last_fullsized":"2024-09-11T01:22:56.000263+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999267+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:55.999267+0000","last_peered":"2024-09-11T01:22:55.999267+0000","last_clean":"2024-09-11T01:22:55.999267+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:55.999267+0000","last_undegraded":"2024-09-11T01:22:55.999267+0000","last_fullsized":"2024-09-11T01:22:55.999267+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28000,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999439+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:55.999439+0000","last_peered":"2024-09-11T01:22:55.999439+0000","last_clean":"2024-09-11T01:22:55.999439+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:55.999439+0000","last_undegraded":"2024-09-11T01:22:55.999439+0000","last_fullsized":"2024-09-11T01:22:55.999439+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:55.994373+0000","last_peered":"2024-09-11T01:22:55.994373+0000","last_clean":"2024-09-11T01:22:55.994373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:55.994373+0000","last_undegraded":"2024-09-11T01:22:55.994373+0000","last_fullsized":"2024-09-11T01:22:55.994373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20016,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.002796+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:56.002796+0000","last_peered":"2024-09-11T01:22:56.002796+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:56.002796+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000522+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:56.000522+0000","last_peered":"2024-09-11T01:22:56.000522+0000","last_clean":"2024-09-11T01:22:56.000522+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:56.000522+0000","last_undegraded":"2024-09-11T01:22:56.000522+0000","last_fullsized":"2024-09-11T01:22:56.000522+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20345,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:55.994507+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:55.994507+0000","last_peered":"2024-09-11T01:22:55.994507+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:55.994507+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999238+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:55.999238+0000","last_peered":"2024-09-11T01:22:55.999238+0000","last_clean":"2024-09-11T01:22:55.999238+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:55.999238+0000","last_undegraded":"2024-09-11T01:22:55.999238+0000","last_fullsized":"2024-09-11T01:22:55.999238+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27796,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994249+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:55.994249+0000","last_peered":"2024-09-11T01:22:55.994249+0000","last_clean":"2024-09-11T01:22:55.994249+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:55.994249+0000","last_undegraded":"2024-09-11T01:22:55.994249+0000","last_fullsized":"2024-09-11T01:22:55.994249+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20415,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.006380+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:56.006380+0000","last_peered":"2024-09-11T01:22:56.006380+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:56.006380+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003986+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:56.003986+0000","last_peered":"2024-09-11T01:22:56.003986+0000","last_clean":"2024-09-11T01:22:56.003986+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:56.003986+0000","last_undegraded":"2024-09-11T01:22:56.003986+0000","last_fullsized":"2024-09-11T01:22:56.003986+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27934,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003846+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:56.003846+0000","last_peered":"2024-09-11T01:22:56.003846+0000","last_clean":"2024-09-11T01:22:56.003846+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:56.003846+0000","last_undegraded":"2024-09-11T01:22:56.003846+0000","last_fullsized":"2024-09-11T01:22:56.003846+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996759+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:55.996759+0000","last_peered":"2024-09-11T01:22:55.996759+0000","last_clean":"2024-09-11T01:22:55.996759+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:55.996759+0000","last_undegraded":"2024-09-11T01:22:55.996759+0000","last_fullsized":"2024-09-11T01:22:55.996759+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24613,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001573+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:56.001573+0000","last_peered":"2024-09-11T01:22:56.001573+0000","last_clean":"2024-09-11T01:22:56.001573+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:56.001573+0000","last_undegraded":"2024-09-11T01:22:56.001573+0000","last_fullsized":"2024-09-11T01:22:56.001573+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999413+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:55.999413+0000","last_peered":"2024-09-11T01:22:55.999413+0000","last_clean":"2024-09-11T01:22:55.999413+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:55.999413+0000","last_undegraded":"2024-09-11T01:22:55.999413+0000","last_fullsized":"2024-09-11T01:22:55.999413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999502+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:55.999502+0000","last_peered":"2024-09-11T01:22:55.999502+0000","last_clean":"2024-09-11T01:22:55.999502+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:55.999502+0000","last_undegraded":"2024-09-11T01:22:55.999502+0000","last_fullsized":"2024-09-11T01:22:55.999502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20105,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001118+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:56.001118+0000","last_peered":"2024-09-11T01:22:56.001118+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:56.001118+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24510,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23129,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195088384,"data_stored":195036199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182545,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518600,"kb_used_data":63544,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853240,"statfs":{"total":96636764160,"available":96105717760,"internally_reserved":0,"allocated":65069056,"data_stored":64767762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986318,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488444,"kb_used_data":127788,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883396,"statfs":{"total":96636764160,"available":96136597504,"internally_reserved":0,"allocated":130854912,"data_stored":130557695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659262,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465272,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93906568,"statfs":{"total":96636764160,"available":96160325632,"internally_reserved":0,"allocated":499712,"data_stored":156670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475917999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":8021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:41.840 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:23:41.840 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20266", 'reported_seq': 27136, 'reported_epoch': 53, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:38.251982+0000', 'last_change': '2024-09-11T01:22:54.587427+0000', 'last_active': '2024-09-11T01:23:38.251982+0000', 'last_peered': '2024-09-11T01:23:38.251982+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:23:38.251982+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 686, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20266, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3386, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -686}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:41.841 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:23:41.841 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'20227", 'reported_seq': 20416, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.000893+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:22:56.000893+0000', 'last_peered': '2024-09-11T01:22:56.000893+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:22:56.000893+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16568, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:41.841 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:23:41.841 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20079, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.000461+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:22:56.000461+0000', 'last_peered': '2024-09-11T01:22:56.000461+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:22:56.000461+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:41.841 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:23:41.841 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20016, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.002796+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:22:56.002796+0000', 'last_peered': '2024-09-11T01:22:56.002796+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:22:56.002796+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:41.841 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:23:41.841 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'20207", 'reported_seq': 20345, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:55.994507+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:22:55.994507+0000', 'last_peered': '2024-09-11T01:22:55.994507+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:22:55.994507+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18024, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4024}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:41.841 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:23:41.841 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20264", 'reported_seq': 20415, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.006380+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:22:56.006380+0000', 'last_peered': '2024-09-11T01:22:56.006380+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:22:56.006380+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17001, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:41.842 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:23:41.842 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 20105, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.001118+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:22:56.001118+0000', 'last_peered': '2024-09-11T01:22:56.001118+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:22:56.001118+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:41.842 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:23:42.233 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:42.233 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 24510/6 objects degraded (408500.000%), 7 pgs degraded, 7 pgs undersized","count":14},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":758,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":53,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":7},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1507651584,"bytes_avail":288402640896,"bytes_total":289910292480,"degraded_objects":24510,"degraded_total":6,"degraded_ratio":4085,"recovering_objects_per_sec":85,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1025,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:23:42.253 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:23:42.253 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:42.584 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:42.585 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:23:42.602 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":425,"stamp":"2024-09-11T01:23:40.828105+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24510,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23130,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472316,"kb_used_data":191820,"kb_used_omap":12,"kb_used_meta":1280435,"kb_avail":281643204,"statfs":{"total":289910292480,"available":288402640896,"internally_reserved":0,"allocated":196423680,"data_stored":195482127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1311165444},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1025,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1025,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003849"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":27136,"reported_epoch":53,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:38.251982+0000","last_change":"2024-09-11T01:22:54.587427+0000","last_active":"2024-09-11T01:23:38.251982+0000","last_peered":"2024-09-11T01:23:38.251982+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:23:38.251982+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":686,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3386,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-686}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20473,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996631+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:55.996631+0000","last_peered":"2024-09-11T01:22:55.996631+0000","last_clean":"2024-09-11T01:22:55.996631+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:55.996631+0000","last_undegraded":"2024-09-11T01:22:55.996631+0000","last_fullsized":"2024-09-11T01:22:55.996631+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20296,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.149418+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:22:56.149418+0000","last_peered":"2024-09-11T01:22:56.149418+0000","last_clean":"2024-09-11T01:22:56.149418+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:22:56.149418+0000","last_undegraded":"2024-09-11T01:22:56.149418+0000","last_fullsized":"2024-09-11T01:22:56.149418+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20223,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996887+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:55.996887+0000","last_peered":"2024-09-11T01:22:55.996887+0000","last_clean":"2024-09-11T01:22:55.996887+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:55.996887+0000","last_undegraded":"2024-09-11T01:22:55.996887+0000","last_fullsized":"2024-09-11T01:22:55.996887+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20156,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001082+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:56.001082+0000","last_peered":"2024-09-11T01:22:56.001082+0000","last_clean":"2024-09-11T01:22:56.001082+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:56.001082+0000","last_undegraded":"2024-09-11T01:22:56.001082+0000","last_fullsized":"2024-09-11T01:22:56.001082+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20416,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000893+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:56.000893+0000","last_peered":"2024-09-11T01:22:56.000893+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:56.000893+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20079,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000461+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:56.000461+0000","last_peered":"2024-09-11T01:22:56.000461+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:56.000461+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20403,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000263+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:56.000263+0000","last_peered":"2024-09-11T01:22:56.000263+0000","last_clean":"2024-09-11T01:22:56.000263+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:56.000263+0000","last_undegraded":"2024-09-11T01:22:56.000263+0000","last_fullsized":"2024-09-11T01:22:56.000263+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999267+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:55.999267+0000","last_peered":"2024-09-11T01:22:55.999267+0000","last_clean":"2024-09-11T01:22:55.999267+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:55.999267+0000","last_undegraded":"2024-09-11T01:22:55.999267+0000","last_fullsized":"2024-09-11T01:22:55.999267+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28000,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999439+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:55.999439+0000","last_peered":"2024-09-11T01:22:55.999439+0000","last_clean":"2024-09-11T01:22:55.999439+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:55.999439+0000","last_undegraded":"2024-09-11T01:22:55.999439+0000","last_fullsized":"2024-09-11T01:22:55.999439+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:55.994373+0000","last_peered":"2024-09-11T01:22:55.994373+0000","last_clean":"2024-09-11T01:22:55.994373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:55.994373+0000","last_undegraded":"2024-09-11T01:22:55.994373+0000","last_fullsized":"2024-09-11T01:22:55.994373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20016,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.002796+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:56.002796+0000","last_peered":"2024-09-11T01:22:56.002796+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:56.002796+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000522+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:56.000522+0000","last_peered":"2024-09-11T01:22:56.000522+0000","last_clean":"2024-09-11T01:22:56.000522+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:56.000522+0000","last_undegraded":"2024-09-11T01:22:56.000522+0000","last_fullsized":"2024-09-11T01:22:56.000522+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20345,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:55.994507+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:55.994507+0000","last_peered":"2024-09-11T01:22:55.994507+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:55.994507+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999238+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:55.999238+0000","last_peered":"2024-09-11T01:22:55.999238+0000","last_clean":"2024-09-11T01:22:55.999238+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:55.999238+0000","last_undegraded":"2024-09-11T01:22:55.999238+0000","last_fullsized":"2024-09-11T01:22:55.999238+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27796,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994249+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:55.994249+0000","last_peered":"2024-09-11T01:22:55.994249+0000","last_clean":"2024-09-11T01:22:55.994249+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:55.994249+0000","last_undegraded":"2024-09-11T01:22:55.994249+0000","last_fullsized":"2024-09-11T01:22:55.994249+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20415,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.006380+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:56.006380+0000","last_peered":"2024-09-11T01:22:56.006380+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:56.006380+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003986+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:56.003986+0000","last_peered":"2024-09-11T01:22:56.003986+0000","last_clean":"2024-09-11T01:22:56.003986+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:56.003986+0000","last_undegraded":"2024-09-11T01:22:56.003986+0000","last_fullsized":"2024-09-11T01:22:56.003986+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27934,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003846+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:56.003846+0000","last_peered":"2024-09-11T01:22:56.003846+0000","last_clean":"2024-09-11T01:22:56.003846+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:56.003846+0000","last_undegraded":"2024-09-11T01:22:56.003846+0000","last_fullsized":"2024-09-11T01:22:56.003846+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996759+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:55.996759+0000","last_peered":"2024-09-11T01:22:55.996759+0000","last_clean":"2024-09-11T01:22:55.996759+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:55.996759+0000","last_undegraded":"2024-09-11T01:22:55.996759+0000","last_fullsized":"2024-09-11T01:22:55.996759+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24613,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001573+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:56.001573+0000","last_peered":"2024-09-11T01:22:56.001573+0000","last_clean":"2024-09-11T01:22:56.001573+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:56.001573+0000","last_undegraded":"2024-09-11T01:22:56.001573+0000","last_fullsized":"2024-09-11T01:22:56.001573+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999413+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:55.999413+0000","last_peered":"2024-09-11T01:22:55.999413+0000","last_clean":"2024-09-11T01:22:55.999413+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:55.999413+0000","last_undegraded":"2024-09-11T01:22:55.999413+0000","last_fullsized":"2024-09-11T01:22:55.999413+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999502+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:55.999502+0000","last_peered":"2024-09-11T01:22:55.999502+0000","last_clean":"2024-09-11T01:22:55.999502+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:55.999502+0000","last_undegraded":"2024-09-11T01:22:55.999502+0000","last_fullsized":"2024-09-11T01:22:55.999502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20105,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001118+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:56.001118+0000","last_peered":"2024-09-11T01:22:56.001118+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:56.001118+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":24510,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23129,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195088384,"data_stored":195036199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182545,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518600,"kb_used_data":63544,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853240,"statfs":{"total":96636764160,"available":96105717760,"internally_reserved":0,"allocated":65069056,"data_stored":64767762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986318,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488444,"kb_used_data":127788,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883396,"statfs":{"total":96636764160,"available":96136597504,"internally_reserved":0,"allocated":130854912,"data_stored":130557695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659262,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465272,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93906568,"statfs":{"total":96636764160,"available":96160325632,"internally_reserved":0,"allocated":499712,"data_stored":156670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475917999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":8021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:45.604 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:45.960 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:45.960 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:23:45.984 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":429,"stamp":"2024-09-11T01:23:44.965531+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23824,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23816,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1476476,"kb_used_data":191820,"kb_used_omap":12,"kb_used_meta":1284595,"kb_avail":281639044,"statfs":{"total":289910292480,"available":288398381056,"internally_reserved":0,"allocated":196423680,"data_stored":195482127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1315425284},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1240,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1240,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.138742"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28509,"reported_epoch":53,"state":"active+undersized+remapped","last_fresh":"2024-09-11T01:23:43.104782+0000","last_change":"2024-09-11T01:23:43.105339+0000","last_active":"2024-09-11T01:23:43.104782+0000","last_peered":"2024-09-11T01:23:43.104782+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:23:43.104782+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20473,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996631+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:55.996631+0000","last_peered":"2024-09-11T01:22:55.996631+0000","last_clean":"2024-09-11T01:22:55.996631+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:55.996631+0000","last_undegraded":"2024-09-11T01:22:55.996631+0000","last_fullsized":"2024-09-11T01:22:55.996631+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20296,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.149418+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:22:56.149418+0000","last_peered":"2024-09-11T01:22:56.149418+0000","last_clean":"2024-09-11T01:22:56.149418+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:22:56.149418+0000","last_undegraded":"2024-09-11T01:22:56.149418+0000","last_fullsized":"2024-09-11T01:22:56.149418+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20223,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996887+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:55.996887+0000","last_peered":"2024-09-11T01:22:55.996887+0000","last_clean":"2024-09-11T01:22:55.996887+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:55.996887+0000","last_undegraded":"2024-09-11T01:22:55.996887+0000","last_fullsized":"2024-09-11T01:22:55.996887+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20156,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001082+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:56.001082+0000","last_peered":"2024-09-11T01:22:56.001082+0000","last_clean":"2024-09-11T01:22:56.001082+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:56.001082+0000","last_undegraded":"2024-09-11T01:22:56.001082+0000","last_fullsized":"2024-09-11T01:22:56.001082+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20416,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000893+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:56.000893+0000","last_peered":"2024-09-11T01:22:56.000893+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:56.000893+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20079,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000461+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:56.000461+0000","last_peered":"2024-09-11T01:22:56.000461+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:56.000461+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20403,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000263+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:56.000263+0000","last_peered":"2024-09-11T01:22:56.000263+0000","last_clean":"2024-09-11T01:22:56.000263+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:56.000263+0000","last_undegraded":"2024-09-11T01:22:56.000263+0000","last_fullsized":"2024-09-11T01:22:56.000263+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999267+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:55.999267+0000","last_peered":"2024-09-11T01:22:55.999267+0000","last_clean":"2024-09-11T01:22:55.999267+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:55.999267+0000","last_undegraded":"2024-09-11T01:22:55.999267+0000","last_fullsized":"2024-09-11T01:22:55.999267+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28000,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999439+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:55.999439+0000","last_peered":"2024-09-11T01:22:55.999439+0000","last_clean":"2024-09-11T01:22:55.999439+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:55.999439+0000","last_undegraded":"2024-09-11T01:22:55.999439+0000","last_fullsized":"2024-09-11T01:22:55.999439+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:55.994373+0000","last_peered":"2024-09-11T01:22:55.994373+0000","last_clean":"2024-09-11T01:22:55.994373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:55.994373+0000","last_undegraded":"2024-09-11T01:22:55.994373+0000","last_fullsized":"2024-09-11T01:22:55.994373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20016,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.002796+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:56.002796+0000","last_peered":"2024-09-11T01:22:56.002796+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:56.002796+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000522+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:56.000522+0000","last_peered":"2024-09-11T01:22:56.000522+0000","last_clean":"2024-09-11T01:22:56.000522+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:56.000522+0000","last_undegraded":"2024-09-11T01:22:56.000522+0000","last_fullsized":"2024-09-11T01:22:56.000522+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20345,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:55.994507+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:55.994507+0000","last_peered":"2024-09-11T01:22:55.994507+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:55.994507+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999238+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:55.999238+0000","last_peered":"2024-09-11T01:22:55.999238+0000","last_clean":"2024-09-11T01:22:55.999238+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:55.999238+0000","last_undegraded":"2024-09-11T01:22:55.999238+0000","last_fullsized":"2024-09-11T01:22:55.999238+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27796,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994249+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:55.994249+0000","last_peered":"2024-09-11T01:22:55.994249+0000","last_clean":"2024-09-11T01:22:55.994249+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:55.994249+0000","last_undegraded":"2024-09-11T01:22:55.994249+0000","last_fullsized":"2024-09-11T01:22:55.994249+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20415,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.006380+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:56.006380+0000","last_peered":"2024-09-11T01:22:56.006380+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:56.006380+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003986+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:56.003986+0000","last_peered":"2024-09-11T01:22:56.003986+0000","last_clean":"2024-09-11T01:22:56.003986+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:56.003986+0000","last_undegraded":"2024-09-11T01:22:56.003986+0000","last_fullsized":"2024-09-11T01:22:56.003986+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27934,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003846+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:56.003846+0000","last_peered":"2024-09-11T01:22:56.003846+0000","last_clean":"2024-09-11T01:22:56.003846+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:56.003846+0000","last_undegraded":"2024-09-11T01:22:56.003846+0000","last_fullsized":"2024-09-11T01:22:56.003846+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996759+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:55.996759+0000","last_peered":"2024-09-11T01:22:55.996759+0000","last_clean":"2024-09-11T01:22:55.996759+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:55.996759+0000","last_undegraded":"2024-09-11T01:22:55.996759+0000","last_fullsized":"2024-09-11T01:22:55.996759+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24613,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001573+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:56.001573+0000","last_peered":"2024-09-11T01:22:56.001573+0000","last_clean":"2024-09-11T01:22:56.001573+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:56.001573+0000","last_undegraded":"2024-09-11T01:22:56.001573+0000","last_fullsized":"2024-09-11T01:22:56.001573+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999467+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:55.999467+0000","last_peered":"2024-09-11T01:22:55.999467+0000","last_clean":"2024-09-11T01:22:55.999467+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:55.999467+0000","last_undegraded":"2024-09-11T01:22:55.999467+0000","last_fullsized":"2024-09-11T01:22:55.999467+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999502+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:55.999502+0000","last_peered":"2024-09-11T01:22:55.999502+0000","last_clean":"2024-09-11T01:22:55.999502+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:55.999502+0000","last_undegraded":"2024-09-11T01:22:55.999502+0000","last_fullsized":"2024-09-11T01:22:55.999502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20105,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001118+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:56.001118+0000","last_peered":"2024-09-11T01:22:56.001118+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:56.001118+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23824,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23815,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195080192,"data_stored":195034965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182546,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518600,"kb_used_data":63544,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853240,"statfs":{"total":96636764160,"available":96105717760,"internally_reserved":0,"allocated":65069056,"data_stored":64767762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986318,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488444,"kb_used_data":127788,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883396,"statfs":{"total":96636764160,"available":96136597504,"internally_reserved":0,"allocated":130854912,"data_stored":130557695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659263,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469432,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93902408,"statfs":{"total":96636764160,"available":96156065792,"internally_reserved":0,"allocated":499712,"data_stored":156670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":480177839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:45.985 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:46.326 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:46.327 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:23:46.346 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":429,"stamp":"2024-09-11T01:23:44.965531+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23824,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23816,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1476476,"kb_used_data":191820,"kb_used_omap":12,"kb_used_meta":1284595,"kb_avail":281639044,"statfs":{"total":289910292480,"available":288398381056,"internally_reserved":0,"allocated":196423680,"data_stored":195482127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1315425284},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1240,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1240,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.138742"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28509,"reported_epoch":53,"state":"active+undersized+remapped","last_fresh":"2024-09-11T01:23:43.104782+0000","last_change":"2024-09-11T01:23:43.105339+0000","last_active":"2024-09-11T01:23:43.104782+0000","last_peered":"2024-09-11T01:23:43.104782+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:23:43.104782+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20473,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996631+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:22:55.996631+0000","last_peered":"2024-09-11T01:22:55.996631+0000","last_clean":"2024-09-11T01:22:55.996631+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:22:55.996631+0000","last_undegraded":"2024-09-11T01:22:55.996631+0000","last_fullsized":"2024-09-11T01:22:55.996631+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20296,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.149418+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:22:56.149418+0000","last_peered":"2024-09-11T01:22:56.149418+0000","last_clean":"2024-09-11T01:22:56.149418+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:22:56.149418+0000","last_undegraded":"2024-09-11T01:22:56.149418+0000","last_fullsized":"2024-09-11T01:22:56.149418+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20223,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996887+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:22:55.996887+0000","last_peered":"2024-09-11T01:22:55.996887+0000","last_clean":"2024-09-11T01:22:55.996887+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:22:55.996887+0000","last_undegraded":"2024-09-11T01:22:55.996887+0000","last_fullsized":"2024-09-11T01:22:55.996887+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20156,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001082+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:56.001082+0000","last_peered":"2024-09-11T01:22:56.001082+0000","last_clean":"2024-09-11T01:22:56.001082+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:56.001082+0000","last_undegraded":"2024-09-11T01:22:56.001082+0000","last_fullsized":"2024-09-11T01:22:56.001082+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20416,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000893+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:56.000893+0000","last_peered":"2024-09-11T01:22:56.000893+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:56.000893+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20079,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000461+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:56.000461+0000","last_peered":"2024-09-11T01:22:56.000461+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:56.000461+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20403,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000263+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:56.000263+0000","last_peered":"2024-09-11T01:22:56.000263+0000","last_clean":"2024-09-11T01:22:56.000263+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:56.000263+0000","last_undegraded":"2024-09-11T01:22:56.000263+0000","last_fullsized":"2024-09-11T01:22:56.000263+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999267+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:55.999267+0000","last_peered":"2024-09-11T01:22:55.999267+0000","last_clean":"2024-09-11T01:22:55.999267+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:55.999267+0000","last_undegraded":"2024-09-11T01:22:55.999267+0000","last_fullsized":"2024-09-11T01:22:55.999267+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28000,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999439+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:55.999439+0000","last_peered":"2024-09-11T01:22:55.999439+0000","last_clean":"2024-09-11T01:22:55.999439+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:55.999439+0000","last_undegraded":"2024-09-11T01:22:55.999439+0000","last_fullsized":"2024-09-11T01:22:55.999439+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994373+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:22:55.994373+0000","last_peered":"2024-09-11T01:22:55.994373+0000","last_clean":"2024-09-11T01:22:55.994373+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:22:55.994373+0000","last_undegraded":"2024-09-11T01:22:55.994373+0000","last_fullsized":"2024-09-11T01:22:55.994373+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20016,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.002796+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:56.002796+0000","last_peered":"2024-09-11T01:22:56.002796+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:56.002796+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000522+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:56.000522+0000","last_peered":"2024-09-11T01:22:56.000522+0000","last_clean":"2024-09-11T01:22:56.000522+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:56.000522+0000","last_undegraded":"2024-09-11T01:22:56.000522+0000","last_fullsized":"2024-09-11T01:22:56.000522+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20345,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:55.994507+0000","last_change":"2024-09-11T01:13:10.206683+0000","last_active":"2024-09-11T01:22:55.994507+0000","last_peered":"2024-09-11T01:22:55.994507+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:22:55.994507+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4026,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-4024}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999238+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:55.999238+0000","last_peered":"2024-09-11T01:22:55.999238+0000","last_clean":"2024-09-11T01:22:55.999238+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:55.999238+0000","last_undegraded":"2024-09-11T01:22:55.999238+0000","last_fullsized":"2024-09-11T01:22:55.999238+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27796,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.994249+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:22:55.994249+0000","last_peered":"2024-09-11T01:22:55.994249+0000","last_clean":"2024-09-11T01:22:55.994249+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:22:55.994249+0000","last_undegraded":"2024-09-11T01:22:55.994249+0000","last_fullsized":"2024-09-11T01:22:55.994249+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20415,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.006380+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:56.006380+0000","last_peered":"2024-09-11T01:22:56.006380+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:56.006380+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003986+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:56.003986+0000","last_peered":"2024-09-11T01:22:56.003986+0000","last_clean":"2024-09-11T01:22:56.003986+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:56.003986+0000","last_undegraded":"2024-09-11T01:22:56.003986+0000","last_fullsized":"2024-09-11T01:22:56.003986+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27934,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003846+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:56.003846+0000","last_peered":"2024-09-11T01:22:56.003846+0000","last_clean":"2024-09-11T01:22:56.003846+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:56.003846+0000","last_undegraded":"2024-09-11T01:22:56.003846+0000","last_fullsized":"2024-09-11T01:22:56.003846+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.996759+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:22:55.996759+0000","last_peered":"2024-09-11T01:22:55.996759+0000","last_clean":"2024-09-11T01:22:55.996759+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:22:55.996759+0000","last_undegraded":"2024-09-11T01:22:55.996759+0000","last_fullsized":"2024-09-11T01:22:55.996759+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24613,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001573+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:56.001573+0000","last_peered":"2024-09-11T01:22:56.001573+0000","last_clean":"2024-09-11T01:22:56.001573+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:56.001573+0000","last_undegraded":"2024-09-11T01:22:56.001573+0000","last_fullsized":"2024-09-11T01:22:56.001573+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999467+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:55.999467+0000","last_peered":"2024-09-11T01:22:55.999467+0000","last_clean":"2024-09-11T01:22:55.999467+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:55.999467+0000","last_undegraded":"2024-09-11T01:22:55.999467+0000","last_fullsized":"2024-09-11T01:22:55.999467+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999502+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:55.999502+0000","last_peered":"2024-09-11T01:22:55.999502+0000","last_clean":"2024-09-11T01:22:55.999502+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:55.999502+0000","last_undegraded":"2024-09-11T01:22:55.999502+0000","last_fullsized":"2024-09-11T01:22:55.999502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20105,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001118+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:56.001118+0000","last_peered":"2024-09-11T01:22:56.001118+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:56.001118+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23824,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23815,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195080192,"data_stored":195034965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182546,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518600,"kb_used_data":63544,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853240,"statfs":{"total":96636764160,"available":96105717760,"internally_reserved":0,"allocated":65069056,"data_stored":64767762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986318,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488444,"kb_used_data":127788,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883396,"statfs":{"total":96636764160,"available":96136597504,"internally_reserved":0,"allocated":130854912,"data_stored":130557695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659263,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469432,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93902408,"statfs":{"total":96636764160,"available":96156065792,"internally_reserved":0,"allocated":499712,"data_stored":156670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":480177839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:46.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.f is not active+clean 2024-09-11T01:23:46.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.f', 'version': "49'20266", 'reported_seq': 28509, 'reported_epoch': 53, 'state': 'active+undersized+remapped', 'last_fresh': '2024-09-11T01:23:43.104782+0000', 'last_change': '2024-09-11T01:23:43.105339+0000', 'last_active': '2024-09-11T01:23:43.104782+0000', 'last_peered': '2024-09-11T01:23:43.104782+0000', 'last_clean': '2024-09-11T01:13:07.617385+0000', 'last_became_active': '2024-09-11T01:13:10.266564+0000', 'last_became_peered': '2024-09-11T01:13:10.266564+0000', 'last_unstale': '2024-09-11T01:23:43.104782+0000', 'last_undegraded': '2024-09-11T01:13:09.803631+0000', 'last_fullsized': '2024-09-11T01:13:09.657251+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'objects_scrubbed': 0, 'log_size': 10066, 'log_dups_size': 0, 'ondisk_log_size': 10066, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 20266, 'num_write_kb': 40532, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 4072, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '1', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:46.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:23:46.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'20227", 'reported_seq': 20416, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.000893+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:22:56.000893+0000', 'last_peered': '2024-09-11T01:22:56.000893+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:22:56.000893+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16568, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:46.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:23:46.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20079, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.000461+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:22:56.000461+0000', 'last_peered': '2024-09-11T01:22:56.000461+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:22:56.000461+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:46.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:23:46.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20016, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.002796+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:22:56.002796+0000', 'last_peered': '2024-09-11T01:22:56.002796+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:22:56.002796+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:46.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:23:46.348 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'20207", 'reported_seq': 20345, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:55.994507+0000', 'last_change': '2024-09-11T01:13:10.206683+0000', 'last_active': '2024-09-11T01:22:55.994507+0000', 'last_peered': '2024-09-11T01:22:55.994507+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:22:55.994507+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 4026, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18024, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -4024}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:46.349 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:23:46.349 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20264", 'reported_seq': 20415, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.006380+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:22:56.006380+0000', 'last_peered': '2024-09-11T01:22:56.006380+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:22:56.006380+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17001, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:46.349 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:23:46.349 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 20105, 'reported_epoch': 53, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:22:56.001118+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:22:56.001118+0000', 'last_peered': '2024-09-11T01:22:56.001118+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:22:56.001118+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:46.349 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:23:46.786 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:46.786 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 23824/6 objects degraded (397066.667%), 6 pgs degraded, 7 pgs undersized","count":13},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":763,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":17},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":6},{"state_name":"active+undersized+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1511911424,"bytes_avail":288398381056,"bytes_total":289910292480,"degraded_objects":23824,"degraded_total":6,"degraded_ratio":3970.6666666666661,"recovering_objects_per_sec":123,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1240,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:23:46.811 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:23:46.811 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:47.166 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:47.166 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:23:47.184 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":430,"stamp":"2024-09-11T01:23:46.829797+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23628,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24012,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":41,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1476492,"kb_used_data":191836,"kb_used_omap":12,"kb_used_meta":1284595,"kb_avail":281639028,"statfs":{"total":289910292480,"available":288398364672,"internally_reserved":0,"allocated":196440064,"data_stored":195487866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1315425284},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-882,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":882,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002374"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28509,"reported_epoch":53,"state":"active+undersized+remapped","last_fresh":"2024-09-11T01:23:43.104782+0000","last_change":"2024-09-11T01:23:43.105339+0000","last_active":"2024-09-11T01:23:43.104782+0000","last_peered":"2024-09-11T01:23:43.104782+0000","last_clean":"2024-09-11T01:13:07.617385+0000","last_became_active":"2024-09-11T01:13:10.266564+0000","last_became_peered":"2024-09-11T01:13:10.266564+0000","last_unstale":"2024-09-11T01:23:43.104782+0000","last_undegraded":"2024-09-11T01:13:09.803631+0000","last_fullsized":"2024-09-11T01:13:09.657251+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","1","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20477,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975727+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:23:44.975727+0000","last_peered":"2024-09-11T01:23:44.975727+0000","last_clean":"2024-09-11T01:23:44.975727+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:23:44.975727+0000","last_undegraded":"2024-09-11T01:23:44.975727+0000","last_fullsized":"2024-09-11T01:23:44.975727+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20296,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.149418+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:22:56.149418+0000","last_peered":"2024-09-11T01:22:56.149418+0000","last_clean":"2024-09-11T01:22:56.149418+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:22:56.149418+0000","last_undegraded":"2024-09-11T01:22:56.149418+0000","last_fullsized":"2024-09-11T01:22:56.149418+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20227,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975565+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:23:44.975565+0000","last_peered":"2024-09-11T01:23:44.975565+0000","last_clean":"2024-09-11T01:23:44.975565+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:23:44.975565+0000","last_undegraded":"2024-09-11T01:23:44.975565+0000","last_fullsized":"2024-09-11T01:23:44.975565+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20156,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001082+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:22:56.001082+0000","last_peered":"2024-09-11T01:22:56.001082+0000","last_clean":"2024-09-11T01:22:56.001082+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:22:56.001082+0000","last_undegraded":"2024-09-11T01:22:56.001082+0000","last_fullsized":"2024-09-11T01:22:56.001082+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20416,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000893+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:22:56.000893+0000","last_peered":"2024-09-11T01:22:56.000893+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:22:56.000893+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20079,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.000461+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:22:56.000461+0000","last_peered":"2024-09-11T01:22:56.000461+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:22:56.000461+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20403,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000263+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:22:56.000263+0000","last_peered":"2024-09-11T01:22:56.000263+0000","last_clean":"2024-09-11T01:22:56.000263+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:22:56.000263+0000","last_undegraded":"2024-09-11T01:22:56.000263+0000","last_fullsized":"2024-09-11T01:22:56.000263+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999267+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:22:55.999267+0000","last_peered":"2024-09-11T01:22:55.999267+0000","last_clean":"2024-09-11T01:22:55.999267+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:22:55.999267+0000","last_undegraded":"2024-09-11T01:22:55.999267+0000","last_fullsized":"2024-09-11T01:22:55.999267+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28000,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999439+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:22:55.999439+0000","last_peered":"2024-09-11T01:22:55.999439+0000","last_clean":"2024-09-11T01:22:55.999439+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:22:55.999439+0000","last_undegraded":"2024-09-11T01:22:55.999439+0000","last_fullsized":"2024-09-11T01:22:55.999439+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975400+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:23:44.975400+0000","last_peered":"2024-09-11T01:23:44.975400+0000","last_clean":"2024-09-11T01:23:44.975400+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:23:44.975400+0000","last_undegraded":"2024-09-11T01:23:44.975400+0000","last_fullsized":"2024-09-11T01:23:44.975400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20016,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.002796+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:22:56.002796+0000","last_peered":"2024-09-11T01:22:56.002796+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:22:56.002796+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":120,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.000522+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:22:56.000522+0000","last_peered":"2024-09-11T01:22:56.000522+0000","last_clean":"2024-09-11T01:22:56.000522+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:22:56.000522+0000","last_undegraded":"2024-09-11T01:22:56.000522+0000","last_fullsized":"2024-09-11T01:22:56.000522+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20742,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:45.364757+0000","last_change":"2024-09-11T01:23:43.106212+0000","last_active":"2024-09-11T01:23:45.364757+0000","last_peered":"2024-09-11T01:23:45.364757+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:23:45.364757+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3830,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":196,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3830}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":99,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999238+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:22:55.999238+0000","last_peered":"2024-09-11T01:22:55.999238+0000","last_clean":"2024-09-11T01:22:55.999238+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:22:55.999238+0000","last_undegraded":"2024-09-11T01:22:55.999238+0000","last_fullsized":"2024-09-11T01:22:55.999238+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27800,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975312+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:23:44.975312+0000","last_peered":"2024-09-11T01:23:44.975312+0000","last_clean":"2024-09-11T01:23:44.975312+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:23:44.975312+0000","last_undegraded":"2024-09-11T01:23:44.975312+0000","last_fullsized":"2024-09-11T01:23:44.975312+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20415,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.006380+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:22:56.006380+0000","last_peered":"2024-09-11T01:22:56.006380+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:22:56.006380+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003986+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:22:56.003986+0000","last_peered":"2024-09-11T01:22:56.003986+0000","last_clean":"2024-09-11T01:22:56.003986+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:22:56.003986+0000","last_undegraded":"2024-09-11T01:22:56.003986+0000","last_fullsized":"2024-09-11T01:22:56.003986+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27934,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.003846+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:22:56.003846+0000","last_peered":"2024-09-11T01:22:56.003846+0000","last_clean":"2024-09-11T01:22:56.003846+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:22:56.003846+0000","last_undegraded":"2024-09-11T01:22:56.003846+0000","last_fullsized":"2024-09-11T01:22:56.003846+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975642+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:23:44.975642+0000","last_peered":"2024-09-11T01:23:44.975642+0000","last_clean":"2024-09-11T01:23:44.975642+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:23:44.975642+0000","last_undegraded":"2024-09-11T01:23:44.975642+0000","last_fullsized":"2024-09-11T01:23:44.975642+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24613,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:56.001573+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:22:56.001573+0000","last_peered":"2024-09-11T01:22:56.001573+0000","last_clean":"2024-09-11T01:22:56.001573+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:22:56.001573+0000","last_undegraded":"2024-09-11T01:22:56.001573+0000","last_fullsized":"2024-09-11T01:22:56.001573+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":89,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999467+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:22:55.999467+0000","last_peered":"2024-09-11T01:22:55.999467+0000","last_clean":"2024-09-11T01:22:55.999467+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:22:55.999467+0000","last_undegraded":"2024-09-11T01:22:55.999467+0000","last_fullsized":"2024-09-11T01:22:55.999467+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":93,"reported_epoch":53,"state":"active+clean","last_fresh":"2024-09-11T01:22:55.999502+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:22:55.999502+0000","last_peered":"2024-09-11T01:22:55.999502+0000","last_clean":"2024-09-11T01:22:55.999502+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:22:55.999502+0000","last_undegraded":"2024-09-11T01:22:55.999502+0000","last_fullsized":"2024-09-11T01:22:55.999502+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20105,"reported_epoch":53,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:22:56.001118+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:22:56.001118+0000","last_peered":"2024-09-11T01:22:56.001118+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:22:56.001118+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23628,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24011,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195080192,"data_stored":195034965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":25,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182546,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518600,"kb_used_data":63544,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853240,"statfs":{"total":96636764160,"available":96105717760,"internally_reserved":0,"allocated":65069056,"data_stored":64767762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986319,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488460,"kb_used_data":127804,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883380,"statfs":{"total":96636764160,"available":96136581120,"internally_reserved":0,"allocated":130871296,"data_stored":130563434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659263,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":469432,"kb_used_data":488,"kb_used_omap":4,"kb_used_meta":468923,"kb_avail":93902408,"statfs":{"total":96636764160,"available":96156065792,"internally_reserved":0,"allocated":499712,"data_stored":156670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":480177839},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:50.186 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:50.618 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:50.619 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:23:50.635 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":431,"stamp":"2024-09-11T01:23:48.830333+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23628,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24012,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1484836,"kb_used_data":191860,"kb_used_omap":12,"kb_used_meta":1292915,"kb_avail":281630684,"statfs":{"total":289910292480,"available":288389820416,"internally_reserved":0,"allocated":196464640,"data_stored":195498110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1323944964},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-882,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":882,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002228"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28519,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:45.031826+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:23:45.031826+0000","last_peered":"2024-09-11T01:23:45.031826+0000","last_clean":"2024-09-11T01:23:45.031826+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:23:45.031826+0000","last_undegraded":"2024-09-11T01:23:45.031826+0000","last_fullsized":"2024-09-11T01:23:45.031826+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20477,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975727+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:23:44.975727+0000","last_peered":"2024-09-11T01:23:44.975727+0000","last_clean":"2024-09-11T01:23:44.975727+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:23:44.975727+0000","last_undegraded":"2024-09-11T01:23:44.975727+0000","last_fullsized":"2024-09-11T01:23:44.975727+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20300,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20227,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975565+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:23:44.975565+0000","last_peered":"2024-09-11T01:23:44.975565+0000","last_clean":"2024-09-11T01:23:44.975565+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:23:44.975565+0000","last_undegraded":"2024-09-11T01:23:44.975565+0000","last_fullsized":"2024-09-11T01:23:44.975565+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20160,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.972521+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:23:44.972521+0000","last_peered":"2024-09-11T01:23:44.972521+0000","last_clean":"2024-09-11T01:23:44.972521+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:23:44.972521+0000","last_undegraded":"2024-09-11T01:23:44.972521+0000","last_fullsized":"2024-09-11T01:23:44.972521+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20420,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.970396+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:23:44.970396+0000","last_peered":"2024-09-11T01:23:44.970396+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:23:44.970396+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20084,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.969107+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:23:44.969107+0000","last_peered":"2024-09-11T01:23:44.969107+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:23:44.969107+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20407,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.967353+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:23:44.967353+0000","last_peered":"2024-09-11T01:23:44.967353+0000","last_clean":"2024-09-11T01:23:44.967353+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:23:44.967353+0000","last_undegraded":"2024-09-11T01:23:44.967353+0000","last_fullsized":"2024-09-11T01:23:44.967353+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966055+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:23:44.966055+0000","last_peered":"2024-09-11T01:23:44.966055+0000","last_clean":"2024-09-11T01:23:44.966055+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:23:44.966055+0000","last_undegraded":"2024-09-11T01:23:44.966055+0000","last_fullsized":"2024-09-11T01:23:44.966055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28004,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975400+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:23:44.975400+0000","last_peered":"2024-09-11T01:23:44.975400+0000","last_clean":"2024-09-11T01:23:44.975400+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:23:44.975400+0000","last_undegraded":"2024-09-11T01:23:44.975400+0000","last_fullsized":"2024-09-11T01:23:44.975400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20020,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.972355+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:23:44.972355+0000","last_peered":"2024-09-11T01:23:44.972355+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:23:44.972355+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.970279+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:23:44.970279+0000","last_peered":"2024-09-11T01:23:44.970279+0000","last_clean":"2024-09-11T01:23:44.970279+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:23:44.970279+0000","last_undegraded":"2024-09-11T01:23:44.970279+0000","last_fullsized":"2024-09-11T01:23:44.970279+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":20742,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:45.364757+0000","last_change":"2024-09-11T01:23:43.106212+0000","last_active":"2024-09-11T01:23:45.364757+0000","last_peered":"2024-09-11T01:23:45.364757+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:23:45.364757+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3830,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":196,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3830}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966373+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:23:44.966373+0000","last_peered":"2024-09-11T01:23:44.966373+0000","last_clean":"2024-09-11T01:23:44.966373+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:23:44.966373+0000","last_undegraded":"2024-09-11T01:23:44.966373+0000","last_fullsized":"2024-09-11T01:23:44.966373+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27800,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975312+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:23:44.975312+0000","last_peered":"2024-09-11T01:23:44.975312+0000","last_clean":"2024-09-11T01:23:44.975312+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:23:44.975312+0000","last_undegraded":"2024-09-11T01:23:44.975312+0000","last_fullsized":"2024-09-11T01:23:44.975312+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20419,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:45.031607+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:23:45.031607+0000","last_peered":"2024-09-11T01:23:45.031607+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:23:45.031607+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969716+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:23:44.969716+0000","last_peered":"2024-09-11T01:23:44.969716+0000","last_clean":"2024-09-11T01:23:44.969716+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:23:44.969716+0000","last_undegraded":"2024-09-11T01:23:44.969716+0000","last_fullsized":"2024-09-11T01:23:44.969716+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27938,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969619+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:23:44.969619+0000","last_peered":"2024-09-11T01:23:44.969619+0000","last_clean":"2024-09-11T01:23:44.969619+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:23:44.969619+0000","last_undegraded":"2024-09-11T01:23:44.969619+0000","last_fullsized":"2024-09-11T01:23:44.969619+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975642+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:23:44.975642+0000","last_peered":"2024-09-11T01:23:44.975642+0000","last_clean":"2024-09-11T01:23:44.975642+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:23:44.975642+0000","last_undegraded":"2024-09-11T01:23:44.975642+0000","last_fullsized":"2024-09-11T01:23:44.975642+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24617,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969597+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:23:44.969597+0000","last_peered":"2024-09-11T01:23:44.969597+0000","last_clean":"2024-09-11T01:23:44.969597+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:23:44.969597+0000","last_undegraded":"2024-09-11T01:23:44.969597+0000","last_fullsized":"2024-09-11T01:23:44.969597+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969610+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:23:44.969610+0000","last_peered":"2024-09-11T01:23:44.969610+0000","last_clean":"2024-09-11T01:23:44.969610+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:23:44.969610+0000","last_undegraded":"2024-09-11T01:23:44.969610+0000","last_fullsized":"2024-09-11T01:23:44.969610+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966535+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:23:44.966535+0000","last_peered":"2024-09-11T01:23:44.966535+0000","last_clean":"2024-09-11T01:23:44.966535+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:23:44.966535+0000","last_undegraded":"2024-09-11T01:23:44.966535+0000","last_fullsized":"2024-09-11T01:23:44.966535+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20109,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.967341+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:23:44.967341+0000","last_peered":"2024-09-11T01:23:44.967341+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:23:44.967341+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":23628,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24011,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195080192,"data_stored":195034965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182547,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518616,"kb_used_data":63560,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853224,"statfs":{"total":96636764160,"available":96105701376,"internally_reserved":0,"allocated":65085440,"data_stored":64773501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986319,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488460,"kb_used_data":127804,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883380,"statfs":{"total":96636764160,"available":96136581120,"internally_reserved":0,"allocated":130871296,"data_stored":130563434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659264,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":477760,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":477243,"kb_avail":93894080,"statfs":{"total":96636764160,"available":96147537920,"internally_reserved":0,"allocated":507904,"data_stored":161175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":488697519},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:50.636 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:50.998 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:50.998 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:23:51.015 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2024-09-11T01:23:50.831025+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22859,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24781,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1484836,"kb_used_data":191860,"kb_used_omap":12,"kb_used_meta":1292915,"kb_avail":281630684,"statfs":{"total":289910292480,"available":288389820416,"internally_reserved":0,"allocated":196464640,"data_stored":195498110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1323944964},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1651,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1651,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002262"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28519,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:45.031826+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:23:45.031826+0000","last_peered":"2024-09-11T01:23:45.031826+0000","last_clean":"2024-09-11T01:23:45.031826+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:23:45.031826+0000","last_undegraded":"2024-09-11T01:23:45.031826+0000","last_fullsized":"2024-09-11T01:23:45.031826+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20477,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975727+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:23:44.975727+0000","last_peered":"2024-09-11T01:23:44.975727+0000","last_clean":"2024-09-11T01:23:44.975727+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:23:44.975727+0000","last_undegraded":"2024-09-11T01:23:44.975727+0000","last_fullsized":"2024-09-11T01:23:44.975727+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20300,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20227,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975565+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:23:44.975565+0000","last_peered":"2024-09-11T01:23:44.975565+0000","last_clean":"2024-09-11T01:23:44.975565+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:23:44.975565+0000","last_undegraded":"2024-09-11T01:23:44.975565+0000","last_fullsized":"2024-09-11T01:23:44.975565+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20160,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.972521+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:23:44.972521+0000","last_peered":"2024-09-11T01:23:44.972521+0000","last_clean":"2024-09-11T01:23:44.972521+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:23:44.972521+0000","last_undegraded":"2024-09-11T01:23:44.972521+0000","last_fullsized":"2024-09-11T01:23:44.972521+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20420,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.970396+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:23:44.970396+0000","last_peered":"2024-09-11T01:23:44.970396+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:23:44.970396+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20084,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.969107+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:23:44.969107+0000","last_peered":"2024-09-11T01:23:44.969107+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:23:44.969107+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20407,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.967353+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:23:44.967353+0000","last_peered":"2024-09-11T01:23:44.967353+0000","last_clean":"2024-09-11T01:23:44.967353+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:23:44.967353+0000","last_undegraded":"2024-09-11T01:23:44.967353+0000","last_fullsized":"2024-09-11T01:23:44.967353+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966055+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:23:44.966055+0000","last_peered":"2024-09-11T01:23:44.966055+0000","last_clean":"2024-09-11T01:23:44.966055+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:23:44.966055+0000","last_undegraded":"2024-09-11T01:23:44.966055+0000","last_fullsized":"2024-09-11T01:23:44.966055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28004,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975400+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:23:44.975400+0000","last_peered":"2024-09-11T01:23:44.975400+0000","last_clean":"2024-09-11T01:23:44.975400+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:23:44.975400+0000","last_undegraded":"2024-09-11T01:23:44.975400+0000","last_fullsized":"2024-09-11T01:23:44.975400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20020,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.972355+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:23:44.972355+0000","last_peered":"2024-09-11T01:23:44.972355+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:23:44.972355+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.970279+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:23:44.970279+0000","last_peered":"2024-09-11T01:23:44.970279+0000","last_clean":"2024-09-11T01:23:44.970279+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:23:44.970279+0000","last_undegraded":"2024-09-11T01:23:44.970279+0000","last_fullsized":"2024-09-11T01:23:44.970279+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":22280,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:50.372167+0000","last_change":"2024-09-11T01:23:43.106212+0000","last_active":"2024-09-11T01:23:50.372167+0000","last_peered":"2024-09-11T01:23:50.372167+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:23:50.372167+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3061,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":965,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3061}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966373+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:23:44.966373+0000","last_peered":"2024-09-11T01:23:44.966373+0000","last_clean":"2024-09-11T01:23:44.966373+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:23:44.966373+0000","last_undegraded":"2024-09-11T01:23:44.966373+0000","last_fullsized":"2024-09-11T01:23:44.966373+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27800,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975312+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:23:44.975312+0000","last_peered":"2024-09-11T01:23:44.975312+0000","last_clean":"2024-09-11T01:23:44.975312+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:23:44.975312+0000","last_undegraded":"2024-09-11T01:23:44.975312+0000","last_fullsized":"2024-09-11T01:23:44.975312+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20419,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:45.031607+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:23:45.031607+0000","last_peered":"2024-09-11T01:23:45.031607+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:23:45.031607+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969716+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:23:44.969716+0000","last_peered":"2024-09-11T01:23:44.969716+0000","last_clean":"2024-09-11T01:23:44.969716+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:23:44.969716+0000","last_undegraded":"2024-09-11T01:23:44.969716+0000","last_fullsized":"2024-09-11T01:23:44.969716+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27938,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969619+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:23:44.969619+0000","last_peered":"2024-09-11T01:23:44.969619+0000","last_clean":"2024-09-11T01:23:44.969619+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:23:44.969619+0000","last_undegraded":"2024-09-11T01:23:44.969619+0000","last_fullsized":"2024-09-11T01:23:44.969619+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975642+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:23:44.975642+0000","last_peered":"2024-09-11T01:23:44.975642+0000","last_clean":"2024-09-11T01:23:44.975642+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:23:44.975642+0000","last_undegraded":"2024-09-11T01:23:44.975642+0000","last_fullsized":"2024-09-11T01:23:44.975642+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24617,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969597+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:23:44.969597+0000","last_peered":"2024-09-11T01:23:44.969597+0000","last_clean":"2024-09-11T01:23:44.969597+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:23:44.969597+0000","last_undegraded":"2024-09-11T01:23:44.969597+0000","last_fullsized":"2024-09-11T01:23:44.969597+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969610+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:23:44.969610+0000","last_peered":"2024-09-11T01:23:44.969610+0000","last_clean":"2024-09-11T01:23:44.969610+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:23:44.969610+0000","last_undegraded":"2024-09-11T01:23:44.969610+0000","last_fullsized":"2024-09-11T01:23:44.969610+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966535+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:23:44.966535+0000","last_peered":"2024-09-11T01:23:44.966535+0000","last_clean":"2024-09-11T01:23:44.966535+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:23:44.966535+0000","last_undegraded":"2024-09-11T01:23:44.966535+0000","last_fullsized":"2024-09-11T01:23:44.966535+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20109,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.967341+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:23:44.967341+0000","last_peered":"2024-09-11T01:23:44.967341+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:23:44.967341+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22859,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24780,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195080192,"data_stored":195034965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182547,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518616,"kb_used_data":63560,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853224,"statfs":{"total":96636764160,"available":96105701376,"internally_reserved":0,"allocated":65085440,"data_stored":64773501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986320,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488460,"kb_used_data":127804,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883380,"statfs":{"total":96636764160,"available":96136581120,"internally_reserved":0,"allocated":130871296,"data_stored":130563434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659264,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":477760,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":477243,"kb_avail":93894080,"statfs":{"total":96636764160,"available":96147537920,"internally_reserved":0,"allocated":507904,"data_stored":161175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":488697519},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:51.017 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:23:51.017 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'20227", 'reported_seq': 20420, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:44.970396+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:23:44.970396+0000', 'last_peered': '2024-09-11T01:23:44.970396+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:23:44.970396+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16568, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:51.017 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:23:51.017 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20084, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:44.969107+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:23:44.969107+0000', 'last_peered': '2024-09-11T01:23:44.969107+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:23:44.969107+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:51.017 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:23:51.017 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20020, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:44.972355+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:23:44.972355+0000', 'last_peered': '2024-09-11T01:23:44.972355+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:23:44.972355+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:51.017 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:23:51.017 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'20207", 'reported_seq': 22280, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:50.372167+0000', 'last_change': '2024-09-11T01:23:43.106212+0000', 'last_active': '2024-09-11T01:23:50.372167+0000', 'last_peered': '2024-09-11T01:23:50.372167+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:23:50.372167+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3061, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18024, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 965, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3061}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:51.018 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:23:51.018 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20264", 'reported_seq': 20419, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:45.031607+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:23:45.031607+0000', 'last_peered': '2024-09-11T01:23:45.031607+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:23:45.031607+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17001, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:51.018 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:23:51.018 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 20109, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:44.967341+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:23:44.967341+0000', 'last_peered': '2024-09-11T01:23:44.967341+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:23:44.967341+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:51.018 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:23:51.483 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:51.483 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 22859/6 objects degraded (380983.333%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":767,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1520472064,"bytes_avail":288389820416,"bytes_total":289910292480,"degraded_objects":22859,"degraded_total":6,"degraded_ratio":3809.8333333333339,"recovering_objects_per_sec":206,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1651,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:23:51.507 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:23:51.507 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:51.833 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:51.833 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:23:51.851 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":432,"stamp":"2024-09-11T01:23:50.831025+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22859,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24781,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1484836,"kb_used_data":191860,"kb_used_omap":12,"kb_used_meta":1292915,"kb_avail":281630684,"statfs":{"total":289910292480,"available":288389820416,"internally_reserved":0,"allocated":196464640,"data_stored":195498110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1323944964},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1651,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1651,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002262"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28519,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:45.031826+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:23:45.031826+0000","last_peered":"2024-09-11T01:23:45.031826+0000","last_clean":"2024-09-11T01:23:45.031826+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:23:45.031826+0000","last_undegraded":"2024-09-11T01:23:45.031826+0000","last_fullsized":"2024-09-11T01:23:45.031826+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20477,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975727+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:23:44.975727+0000","last_peered":"2024-09-11T01:23:44.975727+0000","last_clean":"2024-09-11T01:23:44.975727+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:23:44.975727+0000","last_undegraded":"2024-09-11T01:23:44.975727+0000","last_fullsized":"2024-09-11T01:23:44.975727+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20300,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20227,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975565+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:23:44.975565+0000","last_peered":"2024-09-11T01:23:44.975565+0000","last_clean":"2024-09-11T01:23:44.975565+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:23:44.975565+0000","last_undegraded":"2024-09-11T01:23:44.975565+0000","last_fullsized":"2024-09-11T01:23:44.975565+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20160,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.972521+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:23:44.972521+0000","last_peered":"2024-09-11T01:23:44.972521+0000","last_clean":"2024-09-11T01:23:44.972521+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:23:44.972521+0000","last_undegraded":"2024-09-11T01:23:44.972521+0000","last_fullsized":"2024-09-11T01:23:44.972521+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20420,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.970396+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:23:44.970396+0000","last_peered":"2024-09-11T01:23:44.970396+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:23:44.970396+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20084,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.969107+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:23:44.969107+0000","last_peered":"2024-09-11T01:23:44.969107+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:23:44.969107+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20407,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.967353+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:23:44.967353+0000","last_peered":"2024-09-11T01:23:44.967353+0000","last_clean":"2024-09-11T01:23:44.967353+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:23:44.967353+0000","last_undegraded":"2024-09-11T01:23:44.967353+0000","last_fullsized":"2024-09-11T01:23:44.967353+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966055+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:23:44.966055+0000","last_peered":"2024-09-11T01:23:44.966055+0000","last_clean":"2024-09-11T01:23:44.966055+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:23:44.966055+0000","last_undegraded":"2024-09-11T01:23:44.966055+0000","last_fullsized":"2024-09-11T01:23:44.966055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28004,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975400+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:23:44.975400+0000","last_peered":"2024-09-11T01:23:44.975400+0000","last_clean":"2024-09-11T01:23:44.975400+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:23:44.975400+0000","last_undegraded":"2024-09-11T01:23:44.975400+0000","last_fullsized":"2024-09-11T01:23:44.975400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20020,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.972355+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:23:44.972355+0000","last_peered":"2024-09-11T01:23:44.972355+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:23:44.972355+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.970279+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:23:44.970279+0000","last_peered":"2024-09-11T01:23:44.970279+0000","last_clean":"2024-09-11T01:23:44.970279+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:23:44.970279+0000","last_undegraded":"2024-09-11T01:23:44.970279+0000","last_fullsized":"2024-09-11T01:23:44.970279+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":22280,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:50.372167+0000","last_change":"2024-09-11T01:23:43.106212+0000","last_active":"2024-09-11T01:23:50.372167+0000","last_peered":"2024-09-11T01:23:50.372167+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:23:50.372167+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3061,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":965,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3061}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966373+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:23:44.966373+0000","last_peered":"2024-09-11T01:23:44.966373+0000","last_clean":"2024-09-11T01:23:44.966373+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:23:44.966373+0000","last_undegraded":"2024-09-11T01:23:44.966373+0000","last_fullsized":"2024-09-11T01:23:44.966373+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27800,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975312+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:23:44.975312+0000","last_peered":"2024-09-11T01:23:44.975312+0000","last_clean":"2024-09-11T01:23:44.975312+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:23:44.975312+0000","last_undegraded":"2024-09-11T01:23:44.975312+0000","last_fullsized":"2024-09-11T01:23:44.975312+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20419,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:45.031607+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:23:45.031607+0000","last_peered":"2024-09-11T01:23:45.031607+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:23:45.031607+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969716+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:23:44.969716+0000","last_peered":"2024-09-11T01:23:44.969716+0000","last_clean":"2024-09-11T01:23:44.969716+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:23:44.969716+0000","last_undegraded":"2024-09-11T01:23:44.969716+0000","last_fullsized":"2024-09-11T01:23:44.969716+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27938,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969619+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:23:44.969619+0000","last_peered":"2024-09-11T01:23:44.969619+0000","last_clean":"2024-09-11T01:23:44.969619+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:23:44.969619+0000","last_undegraded":"2024-09-11T01:23:44.969619+0000","last_fullsized":"2024-09-11T01:23:44.969619+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975642+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:23:44.975642+0000","last_peered":"2024-09-11T01:23:44.975642+0000","last_clean":"2024-09-11T01:23:44.975642+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:23:44.975642+0000","last_undegraded":"2024-09-11T01:23:44.975642+0000","last_fullsized":"2024-09-11T01:23:44.975642+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24617,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969597+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:23:44.969597+0000","last_peered":"2024-09-11T01:23:44.969597+0000","last_clean":"2024-09-11T01:23:44.969597+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:23:44.969597+0000","last_undegraded":"2024-09-11T01:23:44.969597+0000","last_fullsized":"2024-09-11T01:23:44.969597+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969610+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:23:44.969610+0000","last_peered":"2024-09-11T01:23:44.969610+0000","last_clean":"2024-09-11T01:23:44.969610+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:23:44.969610+0000","last_undegraded":"2024-09-11T01:23:44.969610+0000","last_fullsized":"2024-09-11T01:23:44.969610+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966535+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:23:44.966535+0000","last_peered":"2024-09-11T01:23:44.966535+0000","last_clean":"2024-09-11T01:23:44.966535+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:23:44.966535+0000","last_undegraded":"2024-09-11T01:23:44.966535+0000","last_fullsized":"2024-09-11T01:23:44.966535+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20109,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.967341+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:23:44.967341+0000","last_peered":"2024-09-11T01:23:44.967341+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:23:44.967341+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22859,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24780,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195080192,"data_stored":195034965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182547,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518616,"kb_used_data":63560,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853224,"statfs":{"total":96636764160,"available":96105701376,"internally_reserved":0,"allocated":65085440,"data_stored":64773501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986320,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488460,"kb_used_data":127804,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883380,"statfs":{"total":96636764160,"available":96136581120,"internally_reserved":0,"allocated":130871296,"data_stored":130563434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659264,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":477760,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":477243,"kb_avail":93894080,"statfs":{"total":96636764160,"available":96147537920,"internally_reserved":0,"allocated":507904,"data_stored":161175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":488697519},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:54.854 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:55.299 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:55.299 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:23:55.317 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":434,"stamp":"2024-09-11T01:23:54.831982+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22859,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24781,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472356,"kb_used_data":191860,"kb_used_omap":12,"kb_used_meta":1280435,"kb_avail":281643164,"statfs":{"total":289910292480,"available":288402599936,"internally_reserved":0,"allocated":196464640,"data_stored":195498110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1311165444},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-965,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":965,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002632"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28519,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:45.031826+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:23:45.031826+0000","last_peered":"2024-09-11T01:23:45.031826+0000","last_clean":"2024-09-11T01:23:45.031826+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:23:45.031826+0000","last_undegraded":"2024-09-11T01:23:45.031826+0000","last_fullsized":"2024-09-11T01:23:45.031826+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20477,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975727+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:23:44.975727+0000","last_peered":"2024-09-11T01:23:44.975727+0000","last_clean":"2024-09-11T01:23:44.975727+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:23:44.975727+0000","last_undegraded":"2024-09-11T01:23:44.975727+0000","last_fullsized":"2024-09-11T01:23:44.975727+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20300,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20227,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975565+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:23:44.975565+0000","last_peered":"2024-09-11T01:23:44.975565+0000","last_clean":"2024-09-11T01:23:44.975565+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:23:44.975565+0000","last_undegraded":"2024-09-11T01:23:44.975565+0000","last_fullsized":"2024-09-11T01:23:44.975565+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20160,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.972521+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:23:44.972521+0000","last_peered":"2024-09-11T01:23:44.972521+0000","last_clean":"2024-09-11T01:23:44.972521+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:23:44.972521+0000","last_undegraded":"2024-09-11T01:23:44.972521+0000","last_fullsized":"2024-09-11T01:23:44.972521+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20421,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.971608+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:23:44.971608+0000","last_peered":"2024-09-11T01:23:44.971608+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:23:44.971608+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20084,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.969107+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:23:44.969107+0000","last_peered":"2024-09-11T01:23:44.969107+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:23:44.969107+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20407,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.967353+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:23:44.967353+0000","last_peered":"2024-09-11T01:23:44.967353+0000","last_clean":"2024-09-11T01:23:44.967353+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:23:44.967353+0000","last_undegraded":"2024-09-11T01:23:44.967353+0000","last_fullsized":"2024-09-11T01:23:44.967353+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966055+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:23:44.966055+0000","last_peered":"2024-09-11T01:23:44.966055+0000","last_clean":"2024-09-11T01:23:44.966055+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:23:44.966055+0000","last_undegraded":"2024-09-11T01:23:44.966055+0000","last_fullsized":"2024-09-11T01:23:44.966055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28004,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975400+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:23:44.975400+0000","last_peered":"2024-09-11T01:23:44.975400+0000","last_clean":"2024-09-11T01:23:44.975400+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:23:44.975400+0000","last_undegraded":"2024-09-11T01:23:44.975400+0000","last_fullsized":"2024-09-11T01:23:44.975400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20020,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.972355+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:23:44.972355+0000","last_peered":"2024-09-11T01:23:44.972355+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:23:44.972355+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.970279+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:23:44.970279+0000","last_peered":"2024-09-11T01:23:44.970279+0000","last_clean":"2024-09-11T01:23:44.970279+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:23:44.970279+0000","last_undegraded":"2024-09-11T01:23:44.970279+0000","last_fullsized":"2024-09-11T01:23:44.970279+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":22280,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:50.372167+0000","last_change":"2024-09-11T01:23:43.106212+0000","last_active":"2024-09-11T01:23:50.372167+0000","last_peered":"2024-09-11T01:23:50.372167+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:23:50.372167+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3061,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":965,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3061}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966373+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:23:44.966373+0000","last_peered":"2024-09-11T01:23:44.966373+0000","last_clean":"2024-09-11T01:23:44.966373+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:23:44.966373+0000","last_undegraded":"2024-09-11T01:23:44.966373+0000","last_fullsized":"2024-09-11T01:23:44.966373+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27800,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975312+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:23:44.975312+0000","last_peered":"2024-09-11T01:23:44.975312+0000","last_clean":"2024-09-11T01:23:44.975312+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:23:44.975312+0000","last_undegraded":"2024-09-11T01:23:44.975312+0000","last_fullsized":"2024-09-11T01:23:44.975312+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20419,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:45.031607+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:23:45.031607+0000","last_peered":"2024-09-11T01:23:45.031607+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:23:45.031607+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969716+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:23:44.969716+0000","last_peered":"2024-09-11T01:23:44.969716+0000","last_clean":"2024-09-11T01:23:44.969716+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:23:44.969716+0000","last_undegraded":"2024-09-11T01:23:44.969716+0000","last_fullsized":"2024-09-11T01:23:44.969716+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27938,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969619+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:23:44.969619+0000","last_peered":"2024-09-11T01:23:44.969619+0000","last_clean":"2024-09-11T01:23:44.969619+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:23:44.969619+0000","last_undegraded":"2024-09-11T01:23:44.969619+0000","last_fullsized":"2024-09-11T01:23:44.969619+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975642+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:23:44.975642+0000","last_peered":"2024-09-11T01:23:44.975642+0000","last_clean":"2024-09-11T01:23:44.975642+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:23:44.975642+0000","last_undegraded":"2024-09-11T01:23:44.975642+0000","last_fullsized":"2024-09-11T01:23:44.975642+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24617,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969597+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:23:44.969597+0000","last_peered":"2024-09-11T01:23:44.969597+0000","last_clean":"2024-09-11T01:23:44.969597+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:23:44.969597+0000","last_undegraded":"2024-09-11T01:23:44.969597+0000","last_fullsized":"2024-09-11T01:23:44.969597+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969610+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:23:44.969610+0000","last_peered":"2024-09-11T01:23:44.969610+0000","last_clean":"2024-09-11T01:23:44.969610+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:23:44.969610+0000","last_undegraded":"2024-09-11T01:23:44.969610+0000","last_fullsized":"2024-09-11T01:23:44.969610+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966535+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:23:44.966535+0000","last_peered":"2024-09-11T01:23:44.966535+0000","last_clean":"2024-09-11T01:23:44.966535+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:23:44.966535+0000","last_undegraded":"2024-09-11T01:23:44.966535+0000","last_fullsized":"2024-09-11T01:23:44.966535+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20109,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.967341+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:23:44.967341+0000","last_peered":"2024-09-11T01:23:44.967341+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:23:44.967341+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22859,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24780,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195080192,"data_stored":195034965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182548,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518616,"kb_used_data":63560,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853224,"statfs":{"total":96636764160,"available":96105701376,"internally_reserved":0,"allocated":65085440,"data_stored":64773501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986320,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488460,"kb_used_data":127804,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883380,"statfs":{"total":96636764160,"available":96136581120,"internally_reserved":0,"allocated":130871296,"data_stored":130563434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659265,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465280,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93906560,"statfs":{"total":96636764160,"available":96160317440,"internally_reserved":0,"allocated":507904,"data_stored":161175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475917999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:55.319 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:55.680 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:55.681 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:23:55.699 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":434,"stamp":"2024-09-11T01:23:54.831982+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22859,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24781,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472356,"kb_used_data":191860,"kb_used_omap":12,"kb_used_meta":1280435,"kb_avail":281643164,"statfs":{"total":289910292480,"available":288402599936,"internally_reserved":0,"allocated":196464640,"data_stored":195498110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1311165444},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-965,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":965,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002632"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28519,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:45.031826+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:23:45.031826+0000","last_peered":"2024-09-11T01:23:45.031826+0000","last_clean":"2024-09-11T01:23:45.031826+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:23:45.031826+0000","last_undegraded":"2024-09-11T01:23:45.031826+0000","last_fullsized":"2024-09-11T01:23:45.031826+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20477,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975727+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:23:44.975727+0000","last_peered":"2024-09-11T01:23:44.975727+0000","last_clean":"2024-09-11T01:23:44.975727+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:23:44.975727+0000","last_undegraded":"2024-09-11T01:23:44.975727+0000","last_fullsized":"2024-09-11T01:23:44.975727+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20300,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20227,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975565+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:23:44.975565+0000","last_peered":"2024-09-11T01:23:44.975565+0000","last_clean":"2024-09-11T01:23:44.975565+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:23:44.975565+0000","last_undegraded":"2024-09-11T01:23:44.975565+0000","last_fullsized":"2024-09-11T01:23:44.975565+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20160,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.972521+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:23:44.972521+0000","last_peered":"2024-09-11T01:23:44.972521+0000","last_clean":"2024-09-11T01:23:44.972521+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:23:44.972521+0000","last_undegraded":"2024-09-11T01:23:44.972521+0000","last_fullsized":"2024-09-11T01:23:44.972521+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20421,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.971608+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:23:44.971608+0000","last_peered":"2024-09-11T01:23:44.971608+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:23:44.971608+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20084,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.969107+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:23:44.969107+0000","last_peered":"2024-09-11T01:23:44.969107+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:23:44.969107+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20407,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.967353+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:23:44.967353+0000","last_peered":"2024-09-11T01:23:44.967353+0000","last_clean":"2024-09-11T01:23:44.967353+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:23:44.967353+0000","last_undegraded":"2024-09-11T01:23:44.967353+0000","last_fullsized":"2024-09-11T01:23:44.967353+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966055+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:23:44.966055+0000","last_peered":"2024-09-11T01:23:44.966055+0000","last_clean":"2024-09-11T01:23:44.966055+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:23:44.966055+0000","last_undegraded":"2024-09-11T01:23:44.966055+0000","last_fullsized":"2024-09-11T01:23:44.966055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28004,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975400+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:23:44.975400+0000","last_peered":"2024-09-11T01:23:44.975400+0000","last_clean":"2024-09-11T01:23:44.975400+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:23:44.975400+0000","last_undegraded":"2024-09-11T01:23:44.975400+0000","last_fullsized":"2024-09-11T01:23:44.975400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20020,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.972355+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:23:44.972355+0000","last_peered":"2024-09-11T01:23:44.972355+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:23:44.972355+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.970279+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:23:44.970279+0000","last_peered":"2024-09-11T01:23:44.970279+0000","last_clean":"2024-09-11T01:23:44.970279+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:23:44.970279+0000","last_undegraded":"2024-09-11T01:23:44.970279+0000","last_fullsized":"2024-09-11T01:23:44.970279+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":22280,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:50.372167+0000","last_change":"2024-09-11T01:23:43.106212+0000","last_active":"2024-09-11T01:23:50.372167+0000","last_peered":"2024-09-11T01:23:50.372167+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:23:50.372167+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3061,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":965,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3061}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966373+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:23:44.966373+0000","last_peered":"2024-09-11T01:23:44.966373+0000","last_clean":"2024-09-11T01:23:44.966373+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:23:44.966373+0000","last_undegraded":"2024-09-11T01:23:44.966373+0000","last_fullsized":"2024-09-11T01:23:44.966373+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27800,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975312+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:23:44.975312+0000","last_peered":"2024-09-11T01:23:44.975312+0000","last_clean":"2024-09-11T01:23:44.975312+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:23:44.975312+0000","last_undegraded":"2024-09-11T01:23:44.975312+0000","last_fullsized":"2024-09-11T01:23:44.975312+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20419,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:45.031607+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:23:45.031607+0000","last_peered":"2024-09-11T01:23:45.031607+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:23:45.031607+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969716+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:23:44.969716+0000","last_peered":"2024-09-11T01:23:44.969716+0000","last_clean":"2024-09-11T01:23:44.969716+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:23:44.969716+0000","last_undegraded":"2024-09-11T01:23:44.969716+0000","last_fullsized":"2024-09-11T01:23:44.969716+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27938,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969619+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:23:44.969619+0000","last_peered":"2024-09-11T01:23:44.969619+0000","last_clean":"2024-09-11T01:23:44.969619+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:23:44.969619+0000","last_undegraded":"2024-09-11T01:23:44.969619+0000","last_fullsized":"2024-09-11T01:23:44.969619+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975642+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:23:44.975642+0000","last_peered":"2024-09-11T01:23:44.975642+0000","last_clean":"2024-09-11T01:23:44.975642+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:23:44.975642+0000","last_undegraded":"2024-09-11T01:23:44.975642+0000","last_fullsized":"2024-09-11T01:23:44.975642+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24617,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969597+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:23:44.969597+0000","last_peered":"2024-09-11T01:23:44.969597+0000","last_clean":"2024-09-11T01:23:44.969597+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:23:44.969597+0000","last_undegraded":"2024-09-11T01:23:44.969597+0000","last_fullsized":"2024-09-11T01:23:44.969597+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969610+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:23:44.969610+0000","last_peered":"2024-09-11T01:23:44.969610+0000","last_clean":"2024-09-11T01:23:44.969610+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:23:44.969610+0000","last_undegraded":"2024-09-11T01:23:44.969610+0000","last_fullsized":"2024-09-11T01:23:44.969610+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966535+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:23:44.966535+0000","last_peered":"2024-09-11T01:23:44.966535+0000","last_clean":"2024-09-11T01:23:44.966535+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:23:44.966535+0000","last_undegraded":"2024-09-11T01:23:44.966535+0000","last_fullsized":"2024-09-11T01:23:44.966535+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20109,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.967341+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:23:44.967341+0000","last_peered":"2024-09-11T01:23:44.967341+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:23:44.967341+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22859,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24780,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195080192,"data_stored":195034965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182548,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518616,"kb_used_data":63560,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853224,"statfs":{"total":96636764160,"available":96105701376,"internally_reserved":0,"allocated":65085440,"data_stored":64773501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986320,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488460,"kb_used_data":127804,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883380,"statfs":{"total":96636764160,"available":96136581120,"internally_reserved":0,"allocated":130871296,"data_stored":130563434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659265,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465280,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93906560,"statfs":{"total":96636764160,"available":96160317440,"internally_reserved":0,"allocated":507904,"data_stored":161175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475917999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:55.700 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:23:55.700 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'20227", 'reported_seq': 20421, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:44.971608+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:23:44.971608+0000', 'last_peered': '2024-09-11T01:23:44.971608+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:23:44.971608+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16568, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:55.701 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:23:55.701 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20084, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:44.969107+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:23:44.969107+0000', 'last_peered': '2024-09-11T01:23:44.969107+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:23:44.969107+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:55.701 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:23:55.701 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20020, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:44.972355+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:23:44.972355+0000', 'last_peered': '2024-09-11T01:23:44.972355+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:23:44.972355+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:55.701 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:23:55.701 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'20207", 'reported_seq': 22280, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:50.372167+0000', 'last_change': '2024-09-11T01:23:43.106212+0000', 'last_active': '2024-09-11T01:23:50.372167+0000', 'last_peered': '2024-09-11T01:23:50.372167+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:23:50.372167+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3061, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18024, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 965, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -3061}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:23:55.701 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:23:55.701 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20264", 'reported_seq': 20419, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:45.031607+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:23:45.031607+0000', 'last_peered': '2024-09-11T01:23:45.031607+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:23:45.031607+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17001, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:55.701 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:23:55.701 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 20109, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:44.967341+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:23:44.967341+0000', 'last_peered': '2024-09-11T01:23:44.967341+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:23:44.967341+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:23:55.702 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:23:56.099 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:56.099 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 22859/6 objects degraded (380983.333%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":772,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1507692544,"bytes_avail":288402599936,"bytes_total":289910292480,"degraded_objects":22859,"degraded_total":6,"degraded_ratio":3809.8333333333339,"recovering_objects_per_sec":96,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":965,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:23:56.119 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:23:56.119 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:56.435 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:56.436 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:23:56.454 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":434,"stamp":"2024-09-11T01:23:54.831982+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22859,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24781,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472356,"kb_used_data":191860,"kb_used_omap":12,"kb_used_meta":1280435,"kb_avail":281643164,"statfs":{"total":289910292480,"available":288402599936,"internally_reserved":0,"allocated":196464640,"data_stored":195498110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1311165444},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-965,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":965,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002632"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28519,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:45.031826+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:23:45.031826+0000","last_peered":"2024-09-11T01:23:45.031826+0000","last_clean":"2024-09-11T01:23:45.031826+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:23:45.031826+0000","last_undegraded":"2024-09-11T01:23:45.031826+0000","last_fullsized":"2024-09-11T01:23:45.031826+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20477,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975727+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:23:44.975727+0000","last_peered":"2024-09-11T01:23:44.975727+0000","last_clean":"2024-09-11T01:23:44.975727+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:23:44.975727+0000","last_undegraded":"2024-09-11T01:23:44.975727+0000","last_fullsized":"2024-09-11T01:23:44.975727+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20300,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20227,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975565+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:23:44.975565+0000","last_peered":"2024-09-11T01:23:44.975565+0000","last_clean":"2024-09-11T01:23:44.975565+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:23:44.975565+0000","last_undegraded":"2024-09-11T01:23:44.975565+0000","last_fullsized":"2024-09-11T01:23:44.975565+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20160,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.972521+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:23:44.972521+0000","last_peered":"2024-09-11T01:23:44.972521+0000","last_clean":"2024-09-11T01:23:44.972521+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:23:44.972521+0000","last_undegraded":"2024-09-11T01:23:44.972521+0000","last_fullsized":"2024-09-11T01:23:44.972521+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20421,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.971608+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:23:44.971608+0000","last_peered":"2024-09-11T01:23:44.971608+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:23:44.971608+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20084,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.969107+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:23:44.969107+0000","last_peered":"2024-09-11T01:23:44.969107+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:23:44.969107+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20407,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.967353+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:23:44.967353+0000","last_peered":"2024-09-11T01:23:44.967353+0000","last_clean":"2024-09-11T01:23:44.967353+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:23:44.967353+0000","last_undegraded":"2024-09-11T01:23:44.967353+0000","last_fullsized":"2024-09-11T01:23:44.967353+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966055+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:23:44.966055+0000","last_peered":"2024-09-11T01:23:44.966055+0000","last_clean":"2024-09-11T01:23:44.966055+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:23:44.966055+0000","last_undegraded":"2024-09-11T01:23:44.966055+0000","last_fullsized":"2024-09-11T01:23:44.966055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28004,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975400+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:23:44.975400+0000","last_peered":"2024-09-11T01:23:44.975400+0000","last_clean":"2024-09-11T01:23:44.975400+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:23:44.975400+0000","last_undegraded":"2024-09-11T01:23:44.975400+0000","last_fullsized":"2024-09-11T01:23:44.975400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20020,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.972355+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:23:44.972355+0000","last_peered":"2024-09-11T01:23:44.972355+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:23:44.972355+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.970279+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:23:44.970279+0000","last_peered":"2024-09-11T01:23:44.970279+0000","last_clean":"2024-09-11T01:23:44.970279+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:23:44.970279+0000","last_undegraded":"2024-09-11T01:23:44.970279+0000","last_fullsized":"2024-09-11T01:23:44.970279+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":22280,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:50.372167+0000","last_change":"2024-09-11T01:23:43.106212+0000","last_active":"2024-09-11T01:23:50.372167+0000","last_peered":"2024-09-11T01:23:50.372167+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:23:50.372167+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3061,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":965,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-3061}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966373+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:23:44.966373+0000","last_peered":"2024-09-11T01:23:44.966373+0000","last_clean":"2024-09-11T01:23:44.966373+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:23:44.966373+0000","last_undegraded":"2024-09-11T01:23:44.966373+0000","last_fullsized":"2024-09-11T01:23:44.966373+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27800,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975312+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:23:44.975312+0000","last_peered":"2024-09-11T01:23:44.975312+0000","last_clean":"2024-09-11T01:23:44.975312+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:23:44.975312+0000","last_undegraded":"2024-09-11T01:23:44.975312+0000","last_fullsized":"2024-09-11T01:23:44.975312+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20419,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:45.031607+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:23:45.031607+0000","last_peered":"2024-09-11T01:23:45.031607+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:23:45.031607+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969716+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:23:44.969716+0000","last_peered":"2024-09-11T01:23:44.969716+0000","last_clean":"2024-09-11T01:23:44.969716+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:23:44.969716+0000","last_undegraded":"2024-09-11T01:23:44.969716+0000","last_fullsized":"2024-09-11T01:23:44.969716+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27938,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969619+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:23:44.969619+0000","last_peered":"2024-09-11T01:23:44.969619+0000","last_clean":"2024-09-11T01:23:44.969619+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:23:44.969619+0000","last_undegraded":"2024-09-11T01:23:44.969619+0000","last_fullsized":"2024-09-11T01:23:44.969619+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975642+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:23:44.975642+0000","last_peered":"2024-09-11T01:23:44.975642+0000","last_clean":"2024-09-11T01:23:44.975642+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:23:44.975642+0000","last_undegraded":"2024-09-11T01:23:44.975642+0000","last_fullsized":"2024-09-11T01:23:44.975642+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24617,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969597+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:23:44.969597+0000","last_peered":"2024-09-11T01:23:44.969597+0000","last_clean":"2024-09-11T01:23:44.969597+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:23:44.969597+0000","last_undegraded":"2024-09-11T01:23:44.969597+0000","last_fullsized":"2024-09-11T01:23:44.969597+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969610+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:23:44.969610+0000","last_peered":"2024-09-11T01:23:44.969610+0000","last_clean":"2024-09-11T01:23:44.969610+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:23:44.969610+0000","last_undegraded":"2024-09-11T01:23:44.969610+0000","last_fullsized":"2024-09-11T01:23:44.969610+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966535+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:23:44.966535+0000","last_peered":"2024-09-11T01:23:44.966535+0000","last_clean":"2024-09-11T01:23:44.966535+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:23:44.966535+0000","last_undegraded":"2024-09-11T01:23:44.966535+0000","last_fullsized":"2024-09-11T01:23:44.966535+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20109,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.967341+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:23:44.967341+0000","last_peered":"2024-09-11T01:23:44.967341+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:23:44.967341+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22859,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":24780,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195080192,"data_stored":195034965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182548,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518616,"kb_used_data":63560,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853224,"statfs":{"total":96636764160,"available":96105701376,"internally_reserved":0,"allocated":65085440,"data_stored":64773501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986320,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488460,"kb_used_data":127804,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883380,"statfs":{"total":96636764160,"available":96136581120,"internally_reserved":0,"allocated":130871296,"data_stored":130563434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659265,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465280,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93906560,"statfs":{"total":96636764160,"available":96160317440,"internally_reserved":0,"allocated":507904,"data_stored":161175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475917999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:59.456 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:23:59.779 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:23:59.780 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:23:59.797 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":436,"stamp":"2024-09-11T01:23:58.833354+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25116,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472356,"kb_used_data":191860,"kb_used_omap":12,"kb_used_meta":1280435,"kb_avail":281643164,"statfs":{"total":289910292480,"available":288402599936,"internally_reserved":0,"allocated":196464640,"data_stored":195498110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1311165444},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1104,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003557"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28519,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:45.031826+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:23:45.031826+0000","last_peered":"2024-09-11T01:23:45.031826+0000","last_clean":"2024-09-11T01:23:45.031826+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:23:45.031826+0000","last_undegraded":"2024-09-11T01:23:45.031826+0000","last_fullsized":"2024-09-11T01:23:45.031826+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20477,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975727+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:23:44.975727+0000","last_peered":"2024-09-11T01:23:44.975727+0000","last_clean":"2024-09-11T01:23:44.975727+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:23:44.975727+0000","last_undegraded":"2024-09-11T01:23:44.975727+0000","last_fullsized":"2024-09-11T01:23:44.975727+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20300,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20227,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975565+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:23:44.975565+0000","last_peered":"2024-09-11T01:23:44.975565+0000","last_clean":"2024-09-11T01:23:44.975565+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:23:44.975565+0000","last_undegraded":"2024-09-11T01:23:44.975565+0000","last_fullsized":"2024-09-11T01:23:44.975565+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20160,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.972521+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:23:44.972521+0000","last_peered":"2024-09-11T01:23:44.972521+0000","last_clean":"2024-09-11T01:23:44.972521+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:23:44.972521+0000","last_undegraded":"2024-09-11T01:23:44.972521+0000","last_fullsized":"2024-09-11T01:23:44.972521+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20421,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.971608+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:23:44.971608+0000","last_peered":"2024-09-11T01:23:44.971608+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:23:44.971608+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20084,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.969107+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:23:44.969107+0000","last_peered":"2024-09-11T01:23:44.969107+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:23:44.969107+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20407,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.967353+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:23:44.967353+0000","last_peered":"2024-09-11T01:23:44.967353+0000","last_clean":"2024-09-11T01:23:44.967353+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:23:44.967353+0000","last_undegraded":"2024-09-11T01:23:44.967353+0000","last_fullsized":"2024-09-11T01:23:44.967353+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966055+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:23:44.966055+0000","last_peered":"2024-09-11T01:23:44.966055+0000","last_clean":"2024-09-11T01:23:44.966055+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:23:44.966055+0000","last_undegraded":"2024-09-11T01:23:44.966055+0000","last_fullsized":"2024-09-11T01:23:44.966055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28004,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975400+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:23:44.975400+0000","last_peered":"2024-09-11T01:23:44.975400+0000","last_clean":"2024-09-11T01:23:44.975400+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:23:44.975400+0000","last_undegraded":"2024-09-11T01:23:44.975400+0000","last_fullsized":"2024-09-11T01:23:44.975400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20020,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.972355+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:23:44.972355+0000","last_peered":"2024-09-11T01:23:44.972355+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:23:44.972355+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.970279+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:23:44.970279+0000","last_peered":"2024-09-11T01:23:44.970279+0000","last_clean":"2024-09-11T01:23:44.970279+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:23:44.970279+0000","last_undegraded":"2024-09-11T01:23:44.970279+0000","last_fullsized":"2024-09-11T01:23:44.970279+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":22950,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:55.368201+0000","last_change":"2024-09-11T01:23:43.106212+0000","last_active":"2024-09-11T01:23:55.368201+0000","last_peered":"2024-09-11T01:23:55.368201+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:23:55.368201+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2726,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2726}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966373+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:23:44.966373+0000","last_peered":"2024-09-11T01:23:44.966373+0000","last_clean":"2024-09-11T01:23:44.966373+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:23:44.966373+0000","last_undegraded":"2024-09-11T01:23:44.966373+0000","last_fullsized":"2024-09-11T01:23:44.966373+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27800,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975312+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:23:44.975312+0000","last_peered":"2024-09-11T01:23:44.975312+0000","last_clean":"2024-09-11T01:23:44.975312+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:23:44.975312+0000","last_undegraded":"2024-09-11T01:23:44.975312+0000","last_fullsized":"2024-09-11T01:23:44.975312+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20419,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:45.031607+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:23:45.031607+0000","last_peered":"2024-09-11T01:23:45.031607+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:23:45.031607+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969716+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:23:44.969716+0000","last_peered":"2024-09-11T01:23:44.969716+0000","last_clean":"2024-09-11T01:23:44.969716+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:23:44.969716+0000","last_undegraded":"2024-09-11T01:23:44.969716+0000","last_fullsized":"2024-09-11T01:23:44.969716+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27938,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969619+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:23:44.969619+0000","last_peered":"2024-09-11T01:23:44.969619+0000","last_clean":"2024-09-11T01:23:44.969619+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:23:44.969619+0000","last_undegraded":"2024-09-11T01:23:44.969619+0000","last_fullsized":"2024-09-11T01:23:44.969619+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975642+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:23:44.975642+0000","last_peered":"2024-09-11T01:23:44.975642+0000","last_clean":"2024-09-11T01:23:44.975642+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:23:44.975642+0000","last_undegraded":"2024-09-11T01:23:44.975642+0000","last_fullsized":"2024-09-11T01:23:44.975642+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24617,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969597+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:23:44.969597+0000","last_peered":"2024-09-11T01:23:44.969597+0000","last_clean":"2024-09-11T01:23:44.969597+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:23:44.969597+0000","last_undegraded":"2024-09-11T01:23:44.969597+0000","last_fullsized":"2024-09-11T01:23:44.969597+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969610+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:23:44.969610+0000","last_peered":"2024-09-11T01:23:44.969610+0000","last_clean":"2024-09-11T01:23:44.969610+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:23:44.969610+0000","last_undegraded":"2024-09-11T01:23:44.969610+0000","last_fullsized":"2024-09-11T01:23:44.969610+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966535+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:23:44.966535+0000","last_peered":"2024-09-11T01:23:44.966535+0000","last_clean":"2024-09-11T01:23:44.966535+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:23:44.966535+0000","last_undegraded":"2024-09-11T01:23:44.966535+0000","last_fullsized":"2024-09-11T01:23:44.966535+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20109,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.967341+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:23:44.967341+0000","last_peered":"2024-09-11T01:23:44.967341+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:23:44.967341+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25115,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195080192,"data_stored":195034965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182549,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518616,"kb_used_data":63560,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853224,"statfs":{"total":96636764160,"available":96105701376,"internally_reserved":0,"allocated":65085440,"data_stored":64773501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986321,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488460,"kb_used_data":127804,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883380,"statfs":{"total":96636764160,"available":96136581120,"internally_reserved":0,"allocated":130871296,"data_stored":130563434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659266,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465280,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93906560,"statfs":{"total":96636764160,"available":96160317440,"internally_reserved":0,"allocated":507904,"data_stored":161175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475917999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:23:59.798 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:00.122 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:00.123 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:24:00.142 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":436,"stamp":"2024-09-11T01:23:58.833354+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25116,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472356,"kb_used_data":191860,"kb_used_omap":12,"kb_used_meta":1280435,"kb_avail":281643164,"statfs":{"total":289910292480,"available":288402599936,"internally_reserved":0,"allocated":196464640,"data_stored":195498110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1311165444},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1104,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1104,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003557"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28519,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:45.031826+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:23:45.031826+0000","last_peered":"2024-09-11T01:23:45.031826+0000","last_clean":"2024-09-11T01:23:45.031826+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:23:45.031826+0000","last_undegraded":"2024-09-11T01:23:45.031826+0000","last_fullsized":"2024-09-11T01:23:45.031826+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20477,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975727+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:23:44.975727+0000","last_peered":"2024-09-11T01:23:44.975727+0000","last_clean":"2024-09-11T01:23:44.975727+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:23:44.975727+0000","last_undegraded":"2024-09-11T01:23:44.975727+0000","last_fullsized":"2024-09-11T01:23:44.975727+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20300,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20227,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975565+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:23:44.975565+0000","last_peered":"2024-09-11T01:23:44.975565+0000","last_clean":"2024-09-11T01:23:44.975565+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:23:44.975565+0000","last_undegraded":"2024-09-11T01:23:44.975565+0000","last_fullsized":"2024-09-11T01:23:44.975565+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20160,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.972521+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:23:44.972521+0000","last_peered":"2024-09-11T01:23:44.972521+0000","last_clean":"2024-09-11T01:23:44.972521+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:23:44.972521+0000","last_undegraded":"2024-09-11T01:23:44.972521+0000","last_fullsized":"2024-09-11T01:23:44.972521+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20421,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.971608+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:23:44.971608+0000","last_peered":"2024-09-11T01:23:44.971608+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:23:44.971608+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20084,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.969107+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:23:44.969107+0000","last_peered":"2024-09-11T01:23:44.969107+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:23:44.969107+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20407,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.967353+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:23:44.967353+0000","last_peered":"2024-09-11T01:23:44.967353+0000","last_clean":"2024-09-11T01:23:44.967353+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:23:44.967353+0000","last_undegraded":"2024-09-11T01:23:44.967353+0000","last_fullsized":"2024-09-11T01:23:44.967353+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966055+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:23:44.966055+0000","last_peered":"2024-09-11T01:23:44.966055+0000","last_clean":"2024-09-11T01:23:44.966055+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:23:44.966055+0000","last_undegraded":"2024-09-11T01:23:44.966055+0000","last_fullsized":"2024-09-11T01:23:44.966055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28004,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975400+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:23:44.975400+0000","last_peered":"2024-09-11T01:23:44.975400+0000","last_clean":"2024-09-11T01:23:44.975400+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:23:44.975400+0000","last_undegraded":"2024-09-11T01:23:44.975400+0000","last_fullsized":"2024-09-11T01:23:44.975400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20020,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.972355+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:23:44.972355+0000","last_peered":"2024-09-11T01:23:44.972355+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:23:44.972355+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.970279+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:23:44.970279+0000","last_peered":"2024-09-11T01:23:44.970279+0000","last_clean":"2024-09-11T01:23:44.970279+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:23:44.970279+0000","last_undegraded":"2024-09-11T01:23:44.970279+0000","last_fullsized":"2024-09-11T01:23:44.970279+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":22950,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:55.368201+0000","last_change":"2024-09-11T01:23:43.106212+0000","last_active":"2024-09-11T01:23:55.368201+0000","last_peered":"2024-09-11T01:23:55.368201+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:23:55.368201+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2726,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1300,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2726}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966373+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:23:44.966373+0000","last_peered":"2024-09-11T01:23:44.966373+0000","last_clean":"2024-09-11T01:23:44.966373+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:23:44.966373+0000","last_undegraded":"2024-09-11T01:23:44.966373+0000","last_fullsized":"2024-09-11T01:23:44.966373+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27800,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975312+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:23:44.975312+0000","last_peered":"2024-09-11T01:23:44.975312+0000","last_clean":"2024-09-11T01:23:44.975312+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:23:44.975312+0000","last_undegraded":"2024-09-11T01:23:44.975312+0000","last_fullsized":"2024-09-11T01:23:44.975312+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20419,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:45.031607+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:23:45.031607+0000","last_peered":"2024-09-11T01:23:45.031607+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:23:45.031607+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969716+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:23:44.969716+0000","last_peered":"2024-09-11T01:23:44.969716+0000","last_clean":"2024-09-11T01:23:44.969716+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:23:44.969716+0000","last_undegraded":"2024-09-11T01:23:44.969716+0000","last_fullsized":"2024-09-11T01:23:44.969716+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27938,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969619+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:23:44.969619+0000","last_peered":"2024-09-11T01:23:44.969619+0000","last_clean":"2024-09-11T01:23:44.969619+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:23:44.969619+0000","last_undegraded":"2024-09-11T01:23:44.969619+0000","last_fullsized":"2024-09-11T01:23:44.969619+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975642+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:23:44.975642+0000","last_peered":"2024-09-11T01:23:44.975642+0000","last_clean":"2024-09-11T01:23:44.975642+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:23:44.975642+0000","last_undegraded":"2024-09-11T01:23:44.975642+0000","last_fullsized":"2024-09-11T01:23:44.975642+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24617,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969597+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:23:44.969597+0000","last_peered":"2024-09-11T01:23:44.969597+0000","last_clean":"2024-09-11T01:23:44.969597+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:23:44.969597+0000","last_undegraded":"2024-09-11T01:23:44.969597+0000","last_fullsized":"2024-09-11T01:23:44.969597+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969610+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:23:44.969610+0000","last_peered":"2024-09-11T01:23:44.969610+0000","last_clean":"2024-09-11T01:23:44.969610+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:23:44.969610+0000","last_undegraded":"2024-09-11T01:23:44.969610+0000","last_fullsized":"2024-09-11T01:23:44.969610+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966535+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:23:44.966535+0000","last_peered":"2024-09-11T01:23:44.966535+0000","last_clean":"2024-09-11T01:23:44.966535+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:23:44.966535+0000","last_undegraded":"2024-09-11T01:23:44.966535+0000","last_fullsized":"2024-09-11T01:23:44.966535+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20109,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.967341+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:23:44.967341+0000","last_peered":"2024-09-11T01:23:44.967341+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:23:44.967341+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22524,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25115,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195080192,"data_stored":195034965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182549,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518616,"kb_used_data":63560,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853224,"statfs":{"total":96636764160,"available":96105701376,"internally_reserved":0,"allocated":65085440,"data_stored":64773501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986321,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488460,"kb_used_data":127804,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883380,"statfs":{"total":96636764160,"available":96136581120,"internally_reserved":0,"allocated":130871296,"data_stored":130563434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659266,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465280,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93906560,"statfs":{"total":96636764160,"available":96160317440,"internally_reserved":0,"allocated":507904,"data_stored":161175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475917999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:00.143 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:24:00.143 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'20227", 'reported_seq': 20421, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:44.971608+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:23:44.971608+0000', 'last_peered': '2024-09-11T01:23:44.971608+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:23:44.971608+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16568, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:00.143 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:24:00.143 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20084, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:44.969107+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:23:44.969107+0000', 'last_peered': '2024-09-11T01:23:44.969107+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:23:44.969107+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:00.144 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:24:00.144 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20020, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:44.972355+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:23:44.972355+0000', 'last_peered': '2024-09-11T01:23:44.972355+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:23:44.972355+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:00.144 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:24:00.144 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'20207", 'reported_seq': 22950, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:55.368201+0000', 'last_change': '2024-09-11T01:23:43.106212+0000', 'last_active': '2024-09-11T01:23:55.368201+0000', 'last_peered': '2024-09-11T01:23:55.368201+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:23:55.368201+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2726, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18024, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1300, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2726}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:24:00.144 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:24:00.144 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20264", 'reported_seq': 20419, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:45.031607+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:23:45.031607+0000', 'last_peered': '2024-09-11T01:23:45.031607+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:23:45.031607+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17001, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:00.144 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:24:00.144 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 20109, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:44.967341+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:23:44.967341+0000', 'last_peered': '2024-09-11T01:23:44.967341+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:23:44.967341+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:00.145 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:24:00.563 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:00.563 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 22524/6 objects degraded (375400.000%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":777,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1507692544,"bytes_avail":288402599936,"bytes_total":289910292480,"degraded_objects":22524,"degraded_total":6,"degraded_ratio":3754,"recovering_objects_per_sec":91,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1104,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:24:00.582 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:24:00.583 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:00.890 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:00.891 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:24:00.909 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":437,"stamp":"2024-09-11T01:24:00.834249+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25476,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472356,"kb_used_data":191860,"kb_used_omap":12,"kb_used_meta":1280435,"kb_avail":281643164,"statfs":{"total":289910292480,"available":288402599936,"internally_reserved":0,"allocated":196464640,"data_stored":195498110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1311165444},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1464,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1464,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003915"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28519,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:45.031826+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:23:45.031826+0000","last_peered":"2024-09-11T01:23:45.031826+0000","last_clean":"2024-09-11T01:23:45.031826+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:23:45.031826+0000","last_undegraded":"2024-09-11T01:23:45.031826+0000","last_fullsized":"2024-09-11T01:23:45.031826+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20477,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975727+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:23:44.975727+0000","last_peered":"2024-09-11T01:23:44.975727+0000","last_clean":"2024-09-11T01:23:44.975727+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:23:44.975727+0000","last_undegraded":"2024-09-11T01:23:44.975727+0000","last_fullsized":"2024-09-11T01:23:44.975727+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20300,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20227,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975565+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:23:44.975565+0000","last_peered":"2024-09-11T01:23:44.975565+0000","last_clean":"2024-09-11T01:23:44.975565+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:23:44.975565+0000","last_undegraded":"2024-09-11T01:23:44.975565+0000","last_fullsized":"2024-09-11T01:23:44.975565+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20160,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.972521+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:23:44.972521+0000","last_peered":"2024-09-11T01:23:44.972521+0000","last_clean":"2024-09-11T01:23:44.972521+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:23:44.972521+0000","last_undegraded":"2024-09-11T01:23:44.972521+0000","last_fullsized":"2024-09-11T01:23:44.972521+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20421,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.971608+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:23:44.971608+0000","last_peered":"2024-09-11T01:23:44.971608+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:23:44.971608+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20084,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.969107+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:23:44.969107+0000","last_peered":"2024-09-11T01:23:44.969107+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:23:44.969107+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20407,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.967353+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:23:44.967353+0000","last_peered":"2024-09-11T01:23:44.967353+0000","last_clean":"2024-09-11T01:23:44.967353+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:23:44.967353+0000","last_undegraded":"2024-09-11T01:23:44.967353+0000","last_fullsized":"2024-09-11T01:23:44.967353+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966055+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:23:44.966055+0000","last_peered":"2024-09-11T01:23:44.966055+0000","last_clean":"2024-09-11T01:23:44.966055+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:23:44.966055+0000","last_undegraded":"2024-09-11T01:23:44.966055+0000","last_fullsized":"2024-09-11T01:23:44.966055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28004,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975400+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:23:44.975400+0000","last_peered":"2024-09-11T01:23:44.975400+0000","last_clean":"2024-09-11T01:23:44.975400+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:23:44.975400+0000","last_undegraded":"2024-09-11T01:23:44.975400+0000","last_fullsized":"2024-09-11T01:23:44.975400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20020,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.972355+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:23:44.972355+0000","last_peered":"2024-09-11T01:23:44.972355+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:23:44.972355+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.970279+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:23:44.970279+0000","last_peered":"2024-09-11T01:23:44.970279+0000","last_clean":"2024-09-11T01:23:44.970279+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:23:44.970279+0000","last_undegraded":"2024-09-11T01:23:44.970279+0000","last_fullsized":"2024-09-11T01:23:44.970279+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":23670,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:00.368438+0000","last_change":"2024-09-11T01:23:43.106212+0000","last_active":"2024-09-11T01:24:00.368438+0000","last_peered":"2024-09-11T01:24:00.368438+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:24:00.368438+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2366,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1660,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2366}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966373+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:23:44.966373+0000","last_peered":"2024-09-11T01:23:44.966373+0000","last_clean":"2024-09-11T01:23:44.966373+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:23:44.966373+0000","last_undegraded":"2024-09-11T01:23:44.966373+0000","last_fullsized":"2024-09-11T01:23:44.966373+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27800,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975312+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:23:44.975312+0000","last_peered":"2024-09-11T01:23:44.975312+0000","last_clean":"2024-09-11T01:23:44.975312+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:23:44.975312+0000","last_undegraded":"2024-09-11T01:23:44.975312+0000","last_fullsized":"2024-09-11T01:23:44.975312+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20419,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:45.031607+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:23:45.031607+0000","last_peered":"2024-09-11T01:23:45.031607+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:23:45.031607+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969716+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:23:44.969716+0000","last_peered":"2024-09-11T01:23:44.969716+0000","last_clean":"2024-09-11T01:23:44.969716+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:23:44.969716+0000","last_undegraded":"2024-09-11T01:23:44.969716+0000","last_fullsized":"2024-09-11T01:23:44.969716+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27938,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969619+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:23:44.969619+0000","last_peered":"2024-09-11T01:23:44.969619+0000","last_clean":"2024-09-11T01:23:44.969619+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:23:44.969619+0000","last_undegraded":"2024-09-11T01:23:44.969619+0000","last_fullsized":"2024-09-11T01:23:44.969619+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975642+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:23:44.975642+0000","last_peered":"2024-09-11T01:23:44.975642+0000","last_clean":"2024-09-11T01:23:44.975642+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:23:44.975642+0000","last_undegraded":"2024-09-11T01:23:44.975642+0000","last_fullsized":"2024-09-11T01:23:44.975642+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24617,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969597+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:23:44.969597+0000","last_peered":"2024-09-11T01:23:44.969597+0000","last_clean":"2024-09-11T01:23:44.969597+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:23:44.969597+0000","last_undegraded":"2024-09-11T01:23:44.969597+0000","last_fullsized":"2024-09-11T01:23:44.969597+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969610+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:23:44.969610+0000","last_peered":"2024-09-11T01:23:44.969610+0000","last_clean":"2024-09-11T01:23:44.969610+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:23:44.969610+0000","last_undegraded":"2024-09-11T01:23:44.969610+0000","last_fullsized":"2024-09-11T01:23:44.969610+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966535+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:23:44.966535+0000","last_peered":"2024-09-11T01:23:44.966535+0000","last_clean":"2024-09-11T01:23:44.966535+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:23:44.966535+0000","last_undegraded":"2024-09-11T01:23:44.966535+0000","last_fullsized":"2024-09-11T01:23:44.966535+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20109,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.967341+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:23:44.967341+0000","last_peered":"2024-09-11T01:23:44.967341+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:23:44.967341+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25475,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195080192,"data_stored":195034965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182549,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518616,"kb_used_data":63560,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853224,"statfs":{"total":96636764160,"available":96105701376,"internally_reserved":0,"allocated":65085440,"data_stored":64773501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986322,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488460,"kb_used_data":127804,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883380,"statfs":{"total":96636764160,"available":96136581120,"internally_reserved":0,"allocated":130871296,"data_stored":130563434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659266,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465280,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93906560,"statfs":{"total":96636764160,"available":96160317440,"internally_reserved":0,"allocated":507904,"data_stored":161175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475917999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:03.911 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:04.245 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:04.246 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:24:04.263 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":438,"stamp":"2024-09-11T01:24:02.834760+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25476,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472356,"kb_used_data":191860,"kb_used_omap":12,"kb_used_meta":1280435,"kb_avail":281643164,"statfs":{"total":289910292480,"available":288402599936,"internally_reserved":0,"allocated":196464640,"data_stored":195498110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1311165444},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-695,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":695,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003734"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28519,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:45.031826+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:23:45.031826+0000","last_peered":"2024-09-11T01:23:45.031826+0000","last_clean":"2024-09-11T01:23:45.031826+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:23:45.031826+0000","last_undegraded":"2024-09-11T01:23:45.031826+0000","last_fullsized":"2024-09-11T01:23:45.031826+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20477,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975727+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:23:44.975727+0000","last_peered":"2024-09-11T01:23:44.975727+0000","last_clean":"2024-09-11T01:23:44.975727+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:23:44.975727+0000","last_undegraded":"2024-09-11T01:23:44.975727+0000","last_fullsized":"2024-09-11T01:23:44.975727+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20300,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20227,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975565+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:23:44.975565+0000","last_peered":"2024-09-11T01:23:44.975565+0000","last_clean":"2024-09-11T01:23:44.975565+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:23:44.975565+0000","last_undegraded":"2024-09-11T01:23:44.975565+0000","last_fullsized":"2024-09-11T01:23:44.975565+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20160,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.972521+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:23:44.972521+0000","last_peered":"2024-09-11T01:23:44.972521+0000","last_clean":"2024-09-11T01:23:44.972521+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:23:44.972521+0000","last_undegraded":"2024-09-11T01:23:44.972521+0000","last_fullsized":"2024-09-11T01:23:44.972521+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20421,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.971608+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:23:44.971608+0000","last_peered":"2024-09-11T01:23:44.971608+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:23:44.971608+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20084,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.969107+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:23:44.969107+0000","last_peered":"2024-09-11T01:23:44.969107+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:23:44.969107+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20407,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.967353+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:23:44.967353+0000","last_peered":"2024-09-11T01:23:44.967353+0000","last_clean":"2024-09-11T01:23:44.967353+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:23:44.967353+0000","last_undegraded":"2024-09-11T01:23:44.967353+0000","last_fullsized":"2024-09-11T01:23:44.967353+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966055+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:23:44.966055+0000","last_peered":"2024-09-11T01:23:44.966055+0000","last_clean":"2024-09-11T01:23:44.966055+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:23:44.966055+0000","last_undegraded":"2024-09-11T01:23:44.966055+0000","last_fullsized":"2024-09-11T01:23:44.966055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28004,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975400+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:23:44.975400+0000","last_peered":"2024-09-11T01:23:44.975400+0000","last_clean":"2024-09-11T01:23:44.975400+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:23:44.975400+0000","last_undegraded":"2024-09-11T01:23:44.975400+0000","last_fullsized":"2024-09-11T01:23:44.975400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20020,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.972355+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:23:44.972355+0000","last_peered":"2024-09-11T01:23:44.972355+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:23:44.972355+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.970279+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:23:44.970279+0000","last_peered":"2024-09-11T01:23:44.970279+0000","last_clean":"2024-09-11T01:23:44.970279+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:23:44.970279+0000","last_undegraded":"2024-09-11T01:23:44.970279+0000","last_fullsized":"2024-09-11T01:23:44.970279+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":23670,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:00.368438+0000","last_change":"2024-09-11T01:23:43.106212+0000","last_active":"2024-09-11T01:24:00.368438+0000","last_peered":"2024-09-11T01:24:00.368438+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:24:00.368438+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2366,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1660,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2366}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966373+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:23:44.966373+0000","last_peered":"2024-09-11T01:23:44.966373+0000","last_clean":"2024-09-11T01:23:44.966373+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:23:44.966373+0000","last_undegraded":"2024-09-11T01:23:44.966373+0000","last_fullsized":"2024-09-11T01:23:44.966373+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27800,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975312+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:23:44.975312+0000","last_peered":"2024-09-11T01:23:44.975312+0000","last_clean":"2024-09-11T01:23:44.975312+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:23:44.975312+0000","last_undegraded":"2024-09-11T01:23:44.975312+0000","last_fullsized":"2024-09-11T01:23:44.975312+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20419,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:45.031607+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:23:45.031607+0000","last_peered":"2024-09-11T01:23:45.031607+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:23:45.031607+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969716+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:23:44.969716+0000","last_peered":"2024-09-11T01:23:44.969716+0000","last_clean":"2024-09-11T01:23:44.969716+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:23:44.969716+0000","last_undegraded":"2024-09-11T01:23:44.969716+0000","last_fullsized":"2024-09-11T01:23:44.969716+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27938,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969619+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:23:44.969619+0000","last_peered":"2024-09-11T01:23:44.969619+0000","last_clean":"2024-09-11T01:23:44.969619+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:23:44.969619+0000","last_undegraded":"2024-09-11T01:23:44.969619+0000","last_fullsized":"2024-09-11T01:23:44.969619+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975642+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:23:44.975642+0000","last_peered":"2024-09-11T01:23:44.975642+0000","last_clean":"2024-09-11T01:23:44.975642+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:23:44.975642+0000","last_undegraded":"2024-09-11T01:23:44.975642+0000","last_fullsized":"2024-09-11T01:23:44.975642+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24617,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969597+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:23:44.969597+0000","last_peered":"2024-09-11T01:23:44.969597+0000","last_clean":"2024-09-11T01:23:44.969597+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:23:44.969597+0000","last_undegraded":"2024-09-11T01:23:44.969597+0000","last_fullsized":"2024-09-11T01:23:44.969597+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969610+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:23:44.969610+0000","last_peered":"2024-09-11T01:23:44.969610+0000","last_clean":"2024-09-11T01:23:44.969610+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:23:44.969610+0000","last_undegraded":"2024-09-11T01:23:44.969610+0000","last_fullsized":"2024-09-11T01:23:44.969610+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966535+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:23:44.966535+0000","last_peered":"2024-09-11T01:23:44.966535+0000","last_clean":"2024-09-11T01:23:44.966535+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:23:44.966535+0000","last_undegraded":"2024-09-11T01:23:44.966535+0000","last_fullsized":"2024-09-11T01:23:44.966535+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20109,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.967341+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:23:44.967341+0000","last_peered":"2024-09-11T01:23:44.967341+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:23:44.967341+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25475,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195080192,"data_stored":195034965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182549,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518616,"kb_used_data":63560,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853224,"statfs":{"total":96636764160,"available":96105701376,"internally_reserved":0,"allocated":65085440,"data_stored":64773501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986322,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488460,"kb_used_data":127804,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883380,"statfs":{"total":96636764160,"available":96136581120,"internally_reserved":0,"allocated":130871296,"data_stored":130563434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659266,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465280,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93906560,"statfs":{"total":96636764160,"available":96160317440,"internally_reserved":0,"allocated":507904,"data_stored":161175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475917999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:04.265 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:04.583 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:04.584 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:24:04.602 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":438,"stamp":"2024-09-11T01:24:02.834760+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25476,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472356,"kb_used_data":191860,"kb_used_omap":12,"kb_used_meta":1280435,"kb_avail":281643164,"statfs":{"total":289910292480,"available":288402599936,"internally_reserved":0,"allocated":196464640,"data_stored":195498110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1311165444},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-695,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":695,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003734"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28519,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:45.031826+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:23:45.031826+0000","last_peered":"2024-09-11T01:23:45.031826+0000","last_clean":"2024-09-11T01:23:45.031826+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:23:45.031826+0000","last_undegraded":"2024-09-11T01:23:45.031826+0000","last_fullsized":"2024-09-11T01:23:45.031826+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20477,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975727+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:23:44.975727+0000","last_peered":"2024-09-11T01:23:44.975727+0000","last_clean":"2024-09-11T01:23:44.975727+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:23:44.975727+0000","last_undegraded":"2024-09-11T01:23:44.975727+0000","last_fullsized":"2024-09-11T01:23:44.975727+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20300,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20227,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975565+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:23:44.975565+0000","last_peered":"2024-09-11T01:23:44.975565+0000","last_clean":"2024-09-11T01:23:44.975565+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:23:44.975565+0000","last_undegraded":"2024-09-11T01:23:44.975565+0000","last_fullsized":"2024-09-11T01:23:44.975565+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20160,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.972521+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:23:44.972521+0000","last_peered":"2024-09-11T01:23:44.972521+0000","last_clean":"2024-09-11T01:23:44.972521+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:23:44.972521+0000","last_undegraded":"2024-09-11T01:23:44.972521+0000","last_fullsized":"2024-09-11T01:23:44.972521+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20421,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.971608+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:23:44.971608+0000","last_peered":"2024-09-11T01:23:44.971608+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:23:44.971608+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20084,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.969107+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:23:44.969107+0000","last_peered":"2024-09-11T01:23:44.969107+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:23:44.969107+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20407,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.967353+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:23:44.967353+0000","last_peered":"2024-09-11T01:23:44.967353+0000","last_clean":"2024-09-11T01:23:44.967353+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:23:44.967353+0000","last_undegraded":"2024-09-11T01:23:44.967353+0000","last_fullsized":"2024-09-11T01:23:44.967353+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966055+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:23:44.966055+0000","last_peered":"2024-09-11T01:23:44.966055+0000","last_clean":"2024-09-11T01:23:44.966055+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:23:44.966055+0000","last_undegraded":"2024-09-11T01:23:44.966055+0000","last_fullsized":"2024-09-11T01:23:44.966055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28004,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975400+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:23:44.975400+0000","last_peered":"2024-09-11T01:23:44.975400+0000","last_clean":"2024-09-11T01:23:44.975400+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:23:44.975400+0000","last_undegraded":"2024-09-11T01:23:44.975400+0000","last_fullsized":"2024-09-11T01:23:44.975400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20020,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.972355+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:23:44.972355+0000","last_peered":"2024-09-11T01:23:44.972355+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:23:44.972355+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.970279+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:23:44.970279+0000","last_peered":"2024-09-11T01:23:44.970279+0000","last_clean":"2024-09-11T01:23:44.970279+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:23:44.970279+0000","last_undegraded":"2024-09-11T01:23:44.970279+0000","last_fullsized":"2024-09-11T01:23:44.970279+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":23670,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:00.368438+0000","last_change":"2024-09-11T01:23:43.106212+0000","last_active":"2024-09-11T01:24:00.368438+0000","last_peered":"2024-09-11T01:24:00.368438+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:24:00.368438+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2366,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1660,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2366}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966373+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:23:44.966373+0000","last_peered":"2024-09-11T01:23:44.966373+0000","last_clean":"2024-09-11T01:23:44.966373+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:23:44.966373+0000","last_undegraded":"2024-09-11T01:23:44.966373+0000","last_fullsized":"2024-09-11T01:23:44.966373+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27800,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975312+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:23:44.975312+0000","last_peered":"2024-09-11T01:23:44.975312+0000","last_clean":"2024-09-11T01:23:44.975312+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:23:44.975312+0000","last_undegraded":"2024-09-11T01:23:44.975312+0000","last_fullsized":"2024-09-11T01:23:44.975312+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20419,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:45.031607+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:23:45.031607+0000","last_peered":"2024-09-11T01:23:45.031607+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:23:45.031607+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969716+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:23:44.969716+0000","last_peered":"2024-09-11T01:23:44.969716+0000","last_clean":"2024-09-11T01:23:44.969716+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:23:44.969716+0000","last_undegraded":"2024-09-11T01:23:44.969716+0000","last_fullsized":"2024-09-11T01:23:44.969716+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27938,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969619+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:23:44.969619+0000","last_peered":"2024-09-11T01:23:44.969619+0000","last_clean":"2024-09-11T01:23:44.969619+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:23:44.969619+0000","last_undegraded":"2024-09-11T01:23:44.969619+0000","last_fullsized":"2024-09-11T01:23:44.969619+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975642+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:23:44.975642+0000","last_peered":"2024-09-11T01:23:44.975642+0000","last_clean":"2024-09-11T01:23:44.975642+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:23:44.975642+0000","last_undegraded":"2024-09-11T01:23:44.975642+0000","last_fullsized":"2024-09-11T01:23:44.975642+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24617,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969597+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:23:44.969597+0000","last_peered":"2024-09-11T01:23:44.969597+0000","last_clean":"2024-09-11T01:23:44.969597+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:23:44.969597+0000","last_undegraded":"2024-09-11T01:23:44.969597+0000","last_fullsized":"2024-09-11T01:23:44.969597+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969610+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:23:44.969610+0000","last_peered":"2024-09-11T01:23:44.969610+0000","last_clean":"2024-09-11T01:23:44.969610+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:23:44.969610+0000","last_undegraded":"2024-09-11T01:23:44.969610+0000","last_fullsized":"2024-09-11T01:23:44.969610+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":97,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966535+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:23:44.966535+0000","last_peered":"2024-09-11T01:23:44.966535+0000","last_clean":"2024-09-11T01:23:44.966535+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:23:44.966535+0000","last_undegraded":"2024-09-11T01:23:44.966535+0000","last_fullsized":"2024-09-11T01:23:44.966535+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20109,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.967341+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:23:44.967341+0000","last_peered":"2024-09-11T01:23:44.967341+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:23:44.967341+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25475,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195080192,"data_stored":195034965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182549,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518616,"kb_used_data":63560,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853224,"statfs":{"total":96636764160,"available":96105701376,"internally_reserved":0,"allocated":65085440,"data_stored":64773501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986322,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488460,"kb_used_data":127804,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883380,"statfs":{"total":96636764160,"available":96136581120,"internally_reserved":0,"allocated":130871296,"data_stored":130563434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659266,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":465280,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":464763,"kb_avail":93906560,"statfs":{"total":96636764160,"available":96160317440,"internally_reserved":0,"allocated":507904,"data_stored":161175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":475917999},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:04.603 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:24:04.603 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'20227", 'reported_seq': 20421, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:44.971608+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:23:44.971608+0000', 'last_peered': '2024-09-11T01:23:44.971608+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:23:44.971608+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16568, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:04.603 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:24:04.603 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20084, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:44.969107+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:23:44.969107+0000', 'last_peered': '2024-09-11T01:23:44.969107+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:23:44.969107+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:04.604 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:24:04.604 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20020, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:44.972355+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:23:44.972355+0000', 'last_peered': '2024-09-11T01:23:44.972355+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:23:44.972355+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:04.604 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:24:04.604 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'20207", 'reported_seq': 23670, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:00.368438+0000', 'last_change': '2024-09-11T01:23:43.106212+0000', 'last_active': '2024-09-11T01:24:00.368438+0000', 'last_peered': '2024-09-11T01:24:00.368438+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:24:00.368438+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2366, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18024, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1660, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -2366}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:24:04.604 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:24:04.604 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20264", 'reported_seq': 20419, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:45.031607+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:23:45.031607+0000', 'last_peered': '2024-09-11T01:23:45.031607+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:23:45.031607+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17001, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:04.604 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:24:04.604 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 20109, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:44.967341+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:23:44.967341+0000', 'last_peered': '2024-09-11T01:23:44.967341+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:23:44.967341+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:04.604 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:24:04.987 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:04.987 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 22164/6 objects degraded (369400.000%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":781,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1507692544,"bytes_avail":288402599936,"bytes_total":289910292480,"degraded_objects":22164,"degraded_total":6,"degraded_ratio":3694,"recovering_objects_per_sec":57,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":695,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:24:05.007 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:24:05.008 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:05.320 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:05.321 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:24:05.338 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":439,"stamp":"2024-09-11T01:24:04.835340+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25476,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1494564,"kb_used_data":191860,"kb_used_omap":12,"kb_used_meta":1302643,"kb_avail":281620956,"statfs":{"total":289910292480,"available":288379858944,"internally_reserved":0,"allocated":196464640,"data_stored":195498110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1333906436},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-695,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":695,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003916"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28519,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:45.031826+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:23:45.031826+0000","last_peered":"2024-09-11T01:23:45.031826+0000","last_clean":"2024-09-11T01:23:45.031826+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:23:45.031826+0000","last_undegraded":"2024-09-11T01:23:45.031826+0000","last_fullsized":"2024-09-11T01:23:45.031826+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20477,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975727+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:23:44.975727+0000","last_peered":"2024-09-11T01:23:44.975727+0000","last_clean":"2024-09-11T01:23:44.975727+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:23:44.975727+0000","last_undegraded":"2024-09-11T01:23:44.975727+0000","last_fullsized":"2024-09-11T01:23:44.975727+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20300,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20227,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975565+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:23:44.975565+0000","last_peered":"2024-09-11T01:23:44.975565+0000","last_clean":"2024-09-11T01:23:44.975565+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:23:44.975565+0000","last_undegraded":"2024-09-11T01:23:44.975565+0000","last_fullsized":"2024-09-11T01:23:44.975565+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20160,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.972521+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:23:44.972521+0000","last_peered":"2024-09-11T01:23:44.972521+0000","last_clean":"2024-09-11T01:23:44.972521+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:23:44.972521+0000","last_undegraded":"2024-09-11T01:23:44.972521+0000","last_fullsized":"2024-09-11T01:23:44.972521+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20421,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.971608+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:23:44.971608+0000","last_peered":"2024-09-11T01:23:44.971608+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:23:44.971608+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20084,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.969107+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:23:44.969107+0000","last_peered":"2024-09-11T01:23:44.969107+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:23:44.969107+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20407,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.967353+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:23:44.967353+0000","last_peered":"2024-09-11T01:23:44.967353+0000","last_clean":"2024-09-11T01:23:44.967353+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:23:44.967353+0000","last_undegraded":"2024-09-11T01:23:44.967353+0000","last_fullsized":"2024-09-11T01:23:44.967353+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966055+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:23:44.966055+0000","last_peered":"2024-09-11T01:23:44.966055+0000","last_clean":"2024-09-11T01:23:44.966055+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:23:44.966055+0000","last_undegraded":"2024-09-11T01:23:44.966055+0000","last_fullsized":"2024-09-11T01:23:44.966055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28004,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975400+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:23:44.975400+0000","last_peered":"2024-09-11T01:23:44.975400+0000","last_clean":"2024-09-11T01:23:44.975400+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:23:44.975400+0000","last_undegraded":"2024-09-11T01:23:44.975400+0000","last_fullsized":"2024-09-11T01:23:44.975400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20020,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.972355+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:23:44.972355+0000","last_peered":"2024-09-11T01:23:44.972355+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:23:44.972355+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.970279+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:23:44.970279+0000","last_peered":"2024-09-11T01:23:44.970279+0000","last_clean":"2024-09-11T01:23:44.970279+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:23:44.970279+0000","last_undegraded":"2024-09-11T01:23:44.970279+0000","last_fullsized":"2024-09-11T01:23:44.970279+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":23670,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:00.368438+0000","last_change":"2024-09-11T01:23:43.106212+0000","last_active":"2024-09-11T01:24:00.368438+0000","last_peered":"2024-09-11T01:24:00.368438+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:24:00.368438+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2366,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1660,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-2366}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966373+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:23:44.966373+0000","last_peered":"2024-09-11T01:23:44.966373+0000","last_clean":"2024-09-11T01:23:44.966373+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:23:44.966373+0000","last_undegraded":"2024-09-11T01:23:44.966373+0000","last_fullsized":"2024-09-11T01:23:44.966373+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27800,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975312+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:23:44.975312+0000","last_peered":"2024-09-11T01:23:44.975312+0000","last_clean":"2024-09-11T01:23:44.975312+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:23:44.975312+0000","last_undegraded":"2024-09-11T01:23:44.975312+0000","last_fullsized":"2024-09-11T01:23:44.975312+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20419,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:45.031607+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:23:45.031607+0000","last_peered":"2024-09-11T01:23:45.031607+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:23:45.031607+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969716+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:23:44.969716+0000","last_peered":"2024-09-11T01:23:44.969716+0000","last_clean":"2024-09-11T01:23:44.969716+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:23:44.969716+0000","last_undegraded":"2024-09-11T01:23:44.969716+0000","last_fullsized":"2024-09-11T01:23:44.969716+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27938,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969619+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:23:44.969619+0000","last_peered":"2024-09-11T01:23:44.969619+0000","last_clean":"2024-09-11T01:23:44.969619+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:23:44.969619+0000","last_undegraded":"2024-09-11T01:23:44.969619+0000","last_fullsized":"2024-09-11T01:23:44.969619+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975642+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:23:44.975642+0000","last_peered":"2024-09-11T01:23:44.975642+0000","last_clean":"2024-09-11T01:23:44.975642+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:23:44.975642+0000","last_undegraded":"2024-09-11T01:23:44.975642+0000","last_fullsized":"2024-09-11T01:23:44.975642+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24617,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969597+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:23:44.969597+0000","last_peered":"2024-09-11T01:23:44.969597+0000","last_clean":"2024-09-11T01:23:44.969597+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:23:44.969597+0000","last_undegraded":"2024-09-11T01:23:44.969597+0000","last_fullsized":"2024-09-11T01:23:44.969597+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969610+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:23:44.969610+0000","last_peered":"2024-09-11T01:23:44.969610+0000","last_clean":"2024-09-11T01:23:44.969610+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:23:44.969610+0000","last_undegraded":"2024-09-11T01:23:44.969610+0000","last_fullsized":"2024-09-11T01:23:44.969610+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966594+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:23:44.966594+0000","last_peered":"2024-09-11T01:23:44.966594+0000","last_clean":"2024-09-11T01:23:44.966594+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:23:44.966594+0000","last_undegraded":"2024-09-11T01:23:44.966594+0000","last_fullsized":"2024-09-11T01:23:44.966594+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20109,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.967341+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:23:44.967341+0000","last_peered":"2024-09-11T01:23:44.967341+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:23:44.967341+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":22164,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25475,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195080192,"data_stored":195034965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182550,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518616,"kb_used_data":63560,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853224,"statfs":{"total":96636764160,"available":96105701376,"internally_reserved":0,"allocated":65085440,"data_stored":64773501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986322,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488460,"kb_used_data":127804,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883380,"statfs":{"total":96636764160,"available":96136581120,"internally_reserved":0,"allocated":130871296,"data_stored":130563434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659267,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487488,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":486971,"kb_avail":93884352,"statfs":{"total":96636764160,"available":96137576448,"internally_reserved":0,"allocated":507904,"data_stored":161175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":498658991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:08.341 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:08.667 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:08.667 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:24:08.685 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":440,"stamp":"2024-09-11T01:24:06.835850+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21780,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25860,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1494564,"kb_used_data":191860,"kb_used_omap":12,"kb_used_meta":1302643,"kb_avail":281620956,"statfs":{"total":289910292480,"available":288379858944,"internally_reserved":0,"allocated":196464640,"data_stored":195498110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1333906436},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1079,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1079,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003868"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28519,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:45.031826+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:23:45.031826+0000","last_peered":"2024-09-11T01:23:45.031826+0000","last_clean":"2024-09-11T01:23:45.031826+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:23:45.031826+0000","last_undegraded":"2024-09-11T01:23:45.031826+0000","last_fullsized":"2024-09-11T01:23:45.031826+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20477,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975727+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:23:44.975727+0000","last_peered":"2024-09-11T01:23:44.975727+0000","last_clean":"2024-09-11T01:23:44.975727+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:23:44.975727+0000","last_undegraded":"2024-09-11T01:23:44.975727+0000","last_fullsized":"2024-09-11T01:23:44.975727+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20300,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20227,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975565+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:23:44.975565+0000","last_peered":"2024-09-11T01:23:44.975565+0000","last_clean":"2024-09-11T01:23:44.975565+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:23:44.975565+0000","last_undegraded":"2024-09-11T01:23:44.975565+0000","last_fullsized":"2024-09-11T01:23:44.975565+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20160,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.972521+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:23:44.972521+0000","last_peered":"2024-09-11T01:23:44.972521+0000","last_clean":"2024-09-11T01:23:44.972521+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:23:44.972521+0000","last_undegraded":"2024-09-11T01:23:44.972521+0000","last_fullsized":"2024-09-11T01:23:44.972521+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20421,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.971608+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:23:44.971608+0000","last_peered":"2024-09-11T01:23:44.971608+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:23:44.971608+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20084,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.969107+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:23:44.969107+0000","last_peered":"2024-09-11T01:23:44.969107+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:23:44.969107+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20407,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.967353+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:23:44.967353+0000","last_peered":"2024-09-11T01:23:44.967353+0000","last_clean":"2024-09-11T01:23:44.967353+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:23:44.967353+0000","last_undegraded":"2024-09-11T01:23:44.967353+0000","last_fullsized":"2024-09-11T01:23:44.967353+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966055+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:23:44.966055+0000","last_peered":"2024-09-11T01:23:44.966055+0000","last_clean":"2024-09-11T01:23:44.966055+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:23:44.966055+0000","last_undegraded":"2024-09-11T01:23:44.966055+0000","last_fullsized":"2024-09-11T01:23:44.966055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28004,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975400+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:23:44.975400+0000","last_peered":"2024-09-11T01:23:44.975400+0000","last_clean":"2024-09-11T01:23:44.975400+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:23:44.975400+0000","last_undegraded":"2024-09-11T01:23:44.975400+0000","last_fullsized":"2024-09-11T01:23:44.975400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20020,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.972355+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:23:44.972355+0000","last_peered":"2024-09-11T01:23:44.972355+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:23:44.972355+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.970279+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:23:44.970279+0000","last_peered":"2024-09-11T01:23:44.970279+0000","last_clean":"2024-09-11T01:23:44.970279+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:23:44.970279+0000","last_undegraded":"2024-09-11T01:23:44.970279+0000","last_fullsized":"2024-09-11T01:23:44.970279+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":24438,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:05.373982+0000","last_change":"2024-09-11T01:23:43.106212+0000","last_active":"2024-09-11T01:24:05.373982+0000","last_peered":"2024-09-11T01:24:05.373982+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:24:05.373982+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2044,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1982}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966373+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:23:44.966373+0000","last_peered":"2024-09-11T01:23:44.966373+0000","last_clean":"2024-09-11T01:23:44.966373+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:23:44.966373+0000","last_undegraded":"2024-09-11T01:23:44.966373+0000","last_fullsized":"2024-09-11T01:23:44.966373+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27800,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975312+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:23:44.975312+0000","last_peered":"2024-09-11T01:23:44.975312+0000","last_clean":"2024-09-11T01:23:44.975312+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:23:44.975312+0000","last_undegraded":"2024-09-11T01:23:44.975312+0000","last_fullsized":"2024-09-11T01:23:44.975312+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20419,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:45.031607+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:23:45.031607+0000","last_peered":"2024-09-11T01:23:45.031607+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:23:45.031607+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969716+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:23:44.969716+0000","last_peered":"2024-09-11T01:23:44.969716+0000","last_clean":"2024-09-11T01:23:44.969716+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:23:44.969716+0000","last_undegraded":"2024-09-11T01:23:44.969716+0000","last_fullsized":"2024-09-11T01:23:44.969716+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27938,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969619+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:23:44.969619+0000","last_peered":"2024-09-11T01:23:44.969619+0000","last_clean":"2024-09-11T01:23:44.969619+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:23:44.969619+0000","last_undegraded":"2024-09-11T01:23:44.969619+0000","last_fullsized":"2024-09-11T01:23:44.969619+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975642+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:23:44.975642+0000","last_peered":"2024-09-11T01:23:44.975642+0000","last_clean":"2024-09-11T01:23:44.975642+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:23:44.975642+0000","last_undegraded":"2024-09-11T01:23:44.975642+0000","last_fullsized":"2024-09-11T01:23:44.975642+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24617,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969597+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:23:44.969597+0000","last_peered":"2024-09-11T01:23:44.969597+0000","last_clean":"2024-09-11T01:23:44.969597+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:23:44.969597+0000","last_undegraded":"2024-09-11T01:23:44.969597+0000","last_fullsized":"2024-09-11T01:23:44.969597+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969610+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:23:44.969610+0000","last_peered":"2024-09-11T01:23:44.969610+0000","last_clean":"2024-09-11T01:23:44.969610+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:23:44.969610+0000","last_undegraded":"2024-09-11T01:23:44.969610+0000","last_fullsized":"2024-09-11T01:23:44.969610+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966594+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:23:44.966594+0000","last_peered":"2024-09-11T01:23:44.966594+0000","last_clean":"2024-09-11T01:23:44.966594+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:23:44.966594+0000","last_undegraded":"2024-09-11T01:23:44.966594+0000","last_fullsized":"2024-09-11T01:23:44.966594+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20109,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.967341+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:23:44.967341+0000","last_peered":"2024-09-11T01:23:44.967341+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:23:44.967341+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21780,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25859,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195080192,"data_stored":195034965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182550,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518616,"kb_used_data":63560,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853224,"statfs":{"total":96636764160,"available":96105701376,"internally_reserved":0,"allocated":65085440,"data_stored":64773501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986323,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488460,"kb_used_data":127804,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883380,"statfs":{"total":96636764160,"available":96136581120,"internally_reserved":0,"allocated":130871296,"data_stored":130563434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659267,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487488,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":486971,"kb_avail":93884352,"statfs":{"total":96636764160,"available":96137576448,"internally_reserved":0,"allocated":507904,"data_stored":161175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":498658991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:08.687 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:08.993 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:08.994 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:24:09.011 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":441,"stamp":"2024-09-11T01:24:08.836493+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21780,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25860,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1498724,"kb_used_data":191860,"kb_used_omap":12,"kb_used_meta":1306803,"kb_avail":281616796,"statfs":{"total":289910292480,"available":288375599104,"internally_reserved":0,"allocated":196464640,"data_stored":195498110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1338166276},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-744,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":744,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003795"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28519,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:45.031826+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:23:45.031826+0000","last_peered":"2024-09-11T01:23:45.031826+0000","last_clean":"2024-09-11T01:23:45.031826+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:23:45.031826+0000","last_undegraded":"2024-09-11T01:23:45.031826+0000","last_fullsized":"2024-09-11T01:23:45.031826+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20477,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975727+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:23:44.975727+0000","last_peered":"2024-09-11T01:23:44.975727+0000","last_clean":"2024-09-11T01:23:44.975727+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:23:44.975727+0000","last_undegraded":"2024-09-11T01:23:44.975727+0000","last_fullsized":"2024-09-11T01:23:44.975727+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20300,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20227,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975565+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:23:44.975565+0000","last_peered":"2024-09-11T01:23:44.975565+0000","last_clean":"2024-09-11T01:23:44.975565+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:23:44.975565+0000","last_undegraded":"2024-09-11T01:23:44.975565+0000","last_fullsized":"2024-09-11T01:23:44.975565+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20160,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.972521+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:23:44.972521+0000","last_peered":"2024-09-11T01:23:44.972521+0000","last_clean":"2024-09-11T01:23:44.972521+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:23:44.972521+0000","last_undegraded":"2024-09-11T01:23:44.972521+0000","last_fullsized":"2024-09-11T01:23:44.972521+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20421,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.971608+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:23:44.971608+0000","last_peered":"2024-09-11T01:23:44.971608+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:23:44.971608+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20084,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.969107+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:23:44.969107+0000","last_peered":"2024-09-11T01:23:44.969107+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:23:44.969107+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20407,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.967353+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:23:44.967353+0000","last_peered":"2024-09-11T01:23:44.967353+0000","last_clean":"2024-09-11T01:23:44.967353+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:23:44.967353+0000","last_undegraded":"2024-09-11T01:23:44.967353+0000","last_fullsized":"2024-09-11T01:23:44.967353+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966055+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:23:44.966055+0000","last_peered":"2024-09-11T01:23:44.966055+0000","last_clean":"2024-09-11T01:23:44.966055+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:23:44.966055+0000","last_undegraded":"2024-09-11T01:23:44.966055+0000","last_fullsized":"2024-09-11T01:23:44.966055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28004,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975400+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:23:44.975400+0000","last_peered":"2024-09-11T01:23:44.975400+0000","last_clean":"2024-09-11T01:23:44.975400+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:23:44.975400+0000","last_undegraded":"2024-09-11T01:23:44.975400+0000","last_fullsized":"2024-09-11T01:23:44.975400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20020,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.972355+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:23:44.972355+0000","last_peered":"2024-09-11T01:23:44.972355+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:23:44.972355+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.970279+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:23:44.970279+0000","last_peered":"2024-09-11T01:23:44.970279+0000","last_clean":"2024-09-11T01:23:44.970279+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:23:44.970279+0000","last_undegraded":"2024-09-11T01:23:44.970279+0000","last_fullsized":"2024-09-11T01:23:44.970279+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":24438,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:05.373982+0000","last_change":"2024-09-11T01:23:43.106212+0000","last_active":"2024-09-11T01:24:05.373982+0000","last_peered":"2024-09-11T01:24:05.373982+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:24:05.373982+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2044,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1982}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966373+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:23:44.966373+0000","last_peered":"2024-09-11T01:23:44.966373+0000","last_clean":"2024-09-11T01:23:44.966373+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:23:44.966373+0000","last_undegraded":"2024-09-11T01:23:44.966373+0000","last_fullsized":"2024-09-11T01:23:44.966373+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27800,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975312+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:23:44.975312+0000","last_peered":"2024-09-11T01:23:44.975312+0000","last_clean":"2024-09-11T01:23:44.975312+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:23:44.975312+0000","last_undegraded":"2024-09-11T01:23:44.975312+0000","last_fullsized":"2024-09-11T01:23:44.975312+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20419,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:45.031607+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:23:45.031607+0000","last_peered":"2024-09-11T01:23:45.031607+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:23:45.031607+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969716+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:23:44.969716+0000","last_peered":"2024-09-11T01:23:44.969716+0000","last_clean":"2024-09-11T01:23:44.969716+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:23:44.969716+0000","last_undegraded":"2024-09-11T01:23:44.969716+0000","last_fullsized":"2024-09-11T01:23:44.969716+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27938,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969619+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:23:44.969619+0000","last_peered":"2024-09-11T01:23:44.969619+0000","last_clean":"2024-09-11T01:23:44.969619+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:23:44.969619+0000","last_undegraded":"2024-09-11T01:23:44.969619+0000","last_fullsized":"2024-09-11T01:23:44.969619+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975642+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:23:44.975642+0000","last_peered":"2024-09-11T01:23:44.975642+0000","last_clean":"2024-09-11T01:23:44.975642+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:23:44.975642+0000","last_undegraded":"2024-09-11T01:23:44.975642+0000","last_fullsized":"2024-09-11T01:23:44.975642+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24617,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969597+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:23:44.969597+0000","last_peered":"2024-09-11T01:23:44.969597+0000","last_clean":"2024-09-11T01:23:44.969597+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:23:44.969597+0000","last_undegraded":"2024-09-11T01:23:44.969597+0000","last_fullsized":"2024-09-11T01:23:44.969597+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969610+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:23:44.969610+0000","last_peered":"2024-09-11T01:23:44.969610+0000","last_clean":"2024-09-11T01:23:44.969610+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:23:44.969610+0000","last_undegraded":"2024-09-11T01:23:44.969610+0000","last_fullsized":"2024-09-11T01:23:44.969610+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966594+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:23:44.966594+0000","last_peered":"2024-09-11T01:23:44.966594+0000","last_clean":"2024-09-11T01:23:44.966594+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:23:44.966594+0000","last_undegraded":"2024-09-11T01:23:44.966594+0000","last_fullsized":"2024-09-11T01:23:44.966594+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20109,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.967341+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:23:44.967341+0000","last_peered":"2024-09-11T01:23:44.967341+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:23:44.967341+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21780,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25859,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195080192,"data_stored":195034965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182551,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518616,"kb_used_data":63560,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853224,"statfs":{"total":96636764160,"available":96105701376,"internally_reserved":0,"allocated":65085440,"data_stored":64773501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986323,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488460,"kb_used_data":127804,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883380,"statfs":{"total":96636764160,"available":96136581120,"internally_reserved":0,"allocated":130871296,"data_stored":130563434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659268,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491648,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93880192,"statfs":{"total":96636764160,"available":96133316608,"internally_reserved":0,"allocated":507904,"data_stored":161175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502918831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:09.013 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:24:09.013 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'20227", 'reported_seq': 20421, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:44.971608+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:23:44.971608+0000', 'last_peered': '2024-09-11T01:23:44.971608+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:23:44.971608+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16568, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:09.013 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:24:09.013 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20084, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:44.969107+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:23:44.969107+0000', 'last_peered': '2024-09-11T01:23:44.969107+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:23:44.969107+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:09.013 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:24:09.013 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20020, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:44.972355+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:23:44.972355+0000', 'last_peered': '2024-09-11T01:23:44.972355+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:23:44.972355+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:09.013 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:24:09.013 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'20207", 'reported_seq': 24438, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:05.373982+0000', 'last_change': '2024-09-11T01:23:43.106212+0000', 'last_active': '2024-09-11T01:24:05.373982+0000', 'last_peered': '2024-09-11T01:24:05.373982+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:24:05.373982+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1982, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18024, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2044, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1982}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:24:09.014 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:24:09.014 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20264", 'reported_seq': 20419, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:45.031607+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:23:45.031607+0000', 'last_peered': '2024-09-11T01:23:45.031607+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:23:45.031607+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17001, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:09.014 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:24:09.014 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 20109, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:44.967341+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:23:44.967341+0000', 'last_peered': '2024-09-11T01:23:44.967341+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:23:44.967341+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:09.014 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:24:09.394 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:09.394 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 21780/6 objects degraded (363000.000%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":785,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1534693376,"bytes_avail":288375599104,"bytes_total":289910292480,"degraded_objects":21780,"degraded_total":6,"degraded_ratio":3630,"recovering_objects_per_sec":61,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":744,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:24:09.413 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:24:09.413 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:09.723 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:09.723 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:24:09.742 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":441,"stamp":"2024-09-11T01:24:08.836493+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21780,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25860,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1498724,"kb_used_data":191860,"kb_used_omap":12,"kb_used_meta":1306803,"kb_avail":281616796,"statfs":{"total":289910292480,"available":288375599104,"internally_reserved":0,"allocated":196464640,"data_stored":195498110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1338166276},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-744,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":744,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003795"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28519,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:45.031826+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:23:45.031826+0000","last_peered":"2024-09-11T01:23:45.031826+0000","last_clean":"2024-09-11T01:23:45.031826+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:23:45.031826+0000","last_undegraded":"2024-09-11T01:23:45.031826+0000","last_fullsized":"2024-09-11T01:23:45.031826+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20477,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975727+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:23:44.975727+0000","last_peered":"2024-09-11T01:23:44.975727+0000","last_clean":"2024-09-11T01:23:44.975727+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:23:44.975727+0000","last_undegraded":"2024-09-11T01:23:44.975727+0000","last_fullsized":"2024-09-11T01:23:44.975727+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20300,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20227,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975565+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:23:44.975565+0000","last_peered":"2024-09-11T01:23:44.975565+0000","last_clean":"2024-09-11T01:23:44.975565+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:23:44.975565+0000","last_undegraded":"2024-09-11T01:23:44.975565+0000","last_fullsized":"2024-09-11T01:23:44.975565+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20160,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.972521+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:23:44.972521+0000","last_peered":"2024-09-11T01:23:44.972521+0000","last_clean":"2024-09-11T01:23:44.972521+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:23:44.972521+0000","last_undegraded":"2024-09-11T01:23:44.972521+0000","last_fullsized":"2024-09-11T01:23:44.972521+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20421,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.971608+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:23:44.971608+0000","last_peered":"2024-09-11T01:23:44.971608+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:23:44.971608+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20084,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.969107+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:23:44.969107+0000","last_peered":"2024-09-11T01:23:44.969107+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:23:44.969107+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20407,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.967353+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:23:44.967353+0000","last_peered":"2024-09-11T01:23:44.967353+0000","last_clean":"2024-09-11T01:23:44.967353+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:23:44.967353+0000","last_undegraded":"2024-09-11T01:23:44.967353+0000","last_fullsized":"2024-09-11T01:23:44.967353+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966055+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:23:44.966055+0000","last_peered":"2024-09-11T01:23:44.966055+0000","last_clean":"2024-09-11T01:23:44.966055+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:23:44.966055+0000","last_undegraded":"2024-09-11T01:23:44.966055+0000","last_fullsized":"2024-09-11T01:23:44.966055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28004,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975400+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:23:44.975400+0000","last_peered":"2024-09-11T01:23:44.975400+0000","last_clean":"2024-09-11T01:23:44.975400+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:23:44.975400+0000","last_undegraded":"2024-09-11T01:23:44.975400+0000","last_fullsized":"2024-09-11T01:23:44.975400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20020,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.972355+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:23:44.972355+0000","last_peered":"2024-09-11T01:23:44.972355+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:23:44.972355+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.970279+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:23:44.970279+0000","last_peered":"2024-09-11T01:23:44.970279+0000","last_clean":"2024-09-11T01:23:44.970279+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:23:44.970279+0000","last_undegraded":"2024-09-11T01:23:44.970279+0000","last_fullsized":"2024-09-11T01:23:44.970279+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":24438,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:05.373982+0000","last_change":"2024-09-11T01:23:43.106212+0000","last_active":"2024-09-11T01:24:05.373982+0000","last_peered":"2024-09-11T01:24:05.373982+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:24:05.373982+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1982,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2044,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1982}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966373+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:23:44.966373+0000","last_peered":"2024-09-11T01:23:44.966373+0000","last_clean":"2024-09-11T01:23:44.966373+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:23:44.966373+0000","last_undegraded":"2024-09-11T01:23:44.966373+0000","last_fullsized":"2024-09-11T01:23:44.966373+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27800,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975312+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:23:44.975312+0000","last_peered":"2024-09-11T01:23:44.975312+0000","last_clean":"2024-09-11T01:23:44.975312+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:23:44.975312+0000","last_undegraded":"2024-09-11T01:23:44.975312+0000","last_fullsized":"2024-09-11T01:23:44.975312+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20419,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:45.031607+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:23:45.031607+0000","last_peered":"2024-09-11T01:23:45.031607+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:23:45.031607+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969716+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:23:44.969716+0000","last_peered":"2024-09-11T01:23:44.969716+0000","last_clean":"2024-09-11T01:23:44.969716+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:23:44.969716+0000","last_undegraded":"2024-09-11T01:23:44.969716+0000","last_fullsized":"2024-09-11T01:23:44.969716+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27938,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969619+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:23:44.969619+0000","last_peered":"2024-09-11T01:23:44.969619+0000","last_clean":"2024-09-11T01:23:44.969619+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:23:44.969619+0000","last_undegraded":"2024-09-11T01:23:44.969619+0000","last_fullsized":"2024-09-11T01:23:44.969619+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975642+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:23:44.975642+0000","last_peered":"2024-09-11T01:23:44.975642+0000","last_clean":"2024-09-11T01:23:44.975642+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:23:44.975642+0000","last_undegraded":"2024-09-11T01:23:44.975642+0000","last_fullsized":"2024-09-11T01:23:44.975642+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24617,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969597+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:23:44.969597+0000","last_peered":"2024-09-11T01:23:44.969597+0000","last_clean":"2024-09-11T01:23:44.969597+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:23:44.969597+0000","last_undegraded":"2024-09-11T01:23:44.969597+0000","last_fullsized":"2024-09-11T01:23:44.969597+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969610+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:23:44.969610+0000","last_peered":"2024-09-11T01:23:44.969610+0000","last_clean":"2024-09-11T01:23:44.969610+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:23:44.969610+0000","last_undegraded":"2024-09-11T01:23:44.969610+0000","last_fullsized":"2024-09-11T01:23:44.969610+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966594+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:23:44.966594+0000","last_peered":"2024-09-11T01:23:44.966594+0000","last_clean":"2024-09-11T01:23:44.966594+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:23:44.966594+0000","last_undegraded":"2024-09-11T01:23:44.966594+0000","last_fullsized":"2024-09-11T01:23:44.966594+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20109,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.967341+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:23:44.967341+0000","last_peered":"2024-09-11T01:23:44.967341+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:23:44.967341+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21780,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25859,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195080192,"data_stored":195034965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182551,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518616,"kb_used_data":63560,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853224,"statfs":{"total":96636764160,"available":96105701376,"internally_reserved":0,"allocated":65085440,"data_stored":64773501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986323,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488460,"kb_used_data":127804,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883380,"statfs":{"total":96636764160,"available":96136581120,"internally_reserved":0,"allocated":130871296,"data_stored":130563434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659268,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491648,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93880192,"statfs":{"total":96636764160,"available":96133316608,"internally_reserved":0,"allocated":507904,"data_stored":161175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502918831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:12.744 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:13.071 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:13.071 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:24:13.090 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":443,"stamp":"2024-09-11T01:24:12.837627+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21355,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26285,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1498724,"kb_used_data":191860,"kb_used_omap":12,"kb_used_meta":1306803,"kb_avail":281616796,"statfs":{"total":289910292480,"available":288375599104,"internally_reserved":0,"allocated":196464640,"data_stored":195498110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1338166276},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-809,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":809,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003378"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28519,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:45.031826+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:23:45.031826+0000","last_peered":"2024-09-11T01:23:45.031826+0000","last_clean":"2024-09-11T01:23:45.031826+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:23:45.031826+0000","last_undegraded":"2024-09-11T01:23:45.031826+0000","last_fullsized":"2024-09-11T01:23:45.031826+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20477,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975727+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:23:44.975727+0000","last_peered":"2024-09-11T01:23:44.975727+0000","last_clean":"2024-09-11T01:23:44.975727+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:23:44.975727+0000","last_undegraded":"2024-09-11T01:23:44.975727+0000","last_fullsized":"2024-09-11T01:23:44.975727+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20300,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20227,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975565+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:23:44.975565+0000","last_peered":"2024-09-11T01:23:44.975565+0000","last_clean":"2024-09-11T01:23:44.975565+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:23:44.975565+0000","last_undegraded":"2024-09-11T01:23:44.975565+0000","last_fullsized":"2024-09-11T01:23:44.975565+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20160,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.972521+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:23:44.972521+0000","last_peered":"2024-09-11T01:23:44.972521+0000","last_clean":"2024-09-11T01:23:44.972521+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:23:44.972521+0000","last_undegraded":"2024-09-11T01:23:44.972521+0000","last_fullsized":"2024-09-11T01:23:44.972521+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20421,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.971608+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:23:44.971608+0000","last_peered":"2024-09-11T01:23:44.971608+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:23:44.971608+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20084,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.969107+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:23:44.969107+0000","last_peered":"2024-09-11T01:23:44.969107+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:23:44.969107+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20407,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.967353+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:23:44.967353+0000","last_peered":"2024-09-11T01:23:44.967353+0000","last_clean":"2024-09-11T01:23:44.967353+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:23:44.967353+0000","last_undegraded":"2024-09-11T01:23:44.967353+0000","last_fullsized":"2024-09-11T01:23:44.967353+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966055+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:23:44.966055+0000","last_peered":"2024-09-11T01:23:44.966055+0000","last_clean":"2024-09-11T01:23:44.966055+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:23:44.966055+0000","last_undegraded":"2024-09-11T01:23:44.966055+0000","last_fullsized":"2024-09-11T01:23:44.966055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28004,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975400+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:23:44.975400+0000","last_peered":"2024-09-11T01:23:44.975400+0000","last_clean":"2024-09-11T01:23:44.975400+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:23:44.975400+0000","last_undegraded":"2024-09-11T01:23:44.975400+0000","last_fullsized":"2024-09-11T01:23:44.975400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20020,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.972355+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:23:44.972355+0000","last_peered":"2024-09-11T01:23:44.972355+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:23:44.972355+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.970279+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:23:44.970279+0000","last_peered":"2024-09-11T01:23:44.970279+0000","last_clean":"2024-09-11T01:23:44.970279+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:23:44.970279+0000","last_undegraded":"2024-09-11T01:23:44.970279+0000","last_fullsized":"2024-09-11T01:23:44.970279+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":25288,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:10.375733+0000","last_change":"2024-09-11T01:23:43.106212+0000","last_active":"2024-09-11T01:24:10.375733+0000","last_peered":"2024-09-11T01:24:10.375733+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:24:10.375733+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2469,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1557}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966373+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:23:44.966373+0000","last_peered":"2024-09-11T01:23:44.966373+0000","last_clean":"2024-09-11T01:23:44.966373+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:23:44.966373+0000","last_undegraded":"2024-09-11T01:23:44.966373+0000","last_fullsized":"2024-09-11T01:23:44.966373+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27801,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975364+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:23:44.975364+0000","last_peered":"2024-09-11T01:23:44.975364+0000","last_clean":"2024-09-11T01:23:44.975364+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:23:44.975364+0000","last_undegraded":"2024-09-11T01:23:44.975364+0000","last_fullsized":"2024-09-11T01:23:44.975364+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20419,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:45.031607+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:23:45.031607+0000","last_peered":"2024-09-11T01:23:45.031607+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:23:45.031607+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969716+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:23:44.969716+0000","last_peered":"2024-09-11T01:23:44.969716+0000","last_clean":"2024-09-11T01:23:44.969716+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:23:44.969716+0000","last_undegraded":"2024-09-11T01:23:44.969716+0000","last_fullsized":"2024-09-11T01:23:44.969716+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27938,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969619+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:23:44.969619+0000","last_peered":"2024-09-11T01:23:44.969619+0000","last_clean":"2024-09-11T01:23:44.969619+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:23:44.969619+0000","last_undegraded":"2024-09-11T01:23:44.969619+0000","last_fullsized":"2024-09-11T01:23:44.969619+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975642+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:23:44.975642+0000","last_peered":"2024-09-11T01:23:44.975642+0000","last_clean":"2024-09-11T01:23:44.975642+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:23:44.975642+0000","last_undegraded":"2024-09-11T01:23:44.975642+0000","last_fullsized":"2024-09-11T01:23:44.975642+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24617,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969597+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:23:44.969597+0000","last_peered":"2024-09-11T01:23:44.969597+0000","last_clean":"2024-09-11T01:23:44.969597+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:23:44.969597+0000","last_undegraded":"2024-09-11T01:23:44.969597+0000","last_fullsized":"2024-09-11T01:23:44.969597+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969610+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:23:44.969610+0000","last_peered":"2024-09-11T01:23:44.969610+0000","last_clean":"2024-09-11T01:23:44.969610+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:23:44.969610+0000","last_undegraded":"2024-09-11T01:23:44.969610+0000","last_fullsized":"2024-09-11T01:23:44.969610+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966594+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:23:44.966594+0000","last_peered":"2024-09-11T01:23:44.966594+0000","last_clean":"2024-09-11T01:23:44.966594+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:23:44.966594+0000","last_undegraded":"2024-09-11T01:23:44.966594+0000","last_fullsized":"2024-09-11T01:23:44.966594+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20109,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.967341+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:23:44.967341+0000","last_peered":"2024-09-11T01:23:44.967341+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:23:44.967341+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21355,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26284,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195080192,"data_stored":195034965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182551,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518616,"kb_used_data":63560,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853224,"statfs":{"total":96636764160,"available":96105701376,"internally_reserved":0,"allocated":65085440,"data_stored":64773501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986324,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488460,"kb_used_data":127804,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883380,"statfs":{"total":96636764160,"available":96136581120,"internally_reserved":0,"allocated":130871296,"data_stored":130563434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659268,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491648,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93880192,"statfs":{"total":96636764160,"available":96133316608,"internally_reserved":0,"allocated":507904,"data_stored":161175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502918831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:13.091 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:13.402 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:13.402 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:24:13.420 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":443,"stamp":"2024-09-11T01:24:12.837627+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21355,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26285,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1498724,"kb_used_data":191860,"kb_used_omap":12,"kb_used_meta":1306803,"kb_avail":281616796,"statfs":{"total":289910292480,"available":288375599104,"internally_reserved":0,"allocated":196464640,"data_stored":195498110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1338166276},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-809,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":809,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003378"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28519,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:45.031826+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:23:45.031826+0000","last_peered":"2024-09-11T01:23:45.031826+0000","last_clean":"2024-09-11T01:23:45.031826+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:23:45.031826+0000","last_undegraded":"2024-09-11T01:23:45.031826+0000","last_fullsized":"2024-09-11T01:23:45.031826+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20477,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975727+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:23:44.975727+0000","last_peered":"2024-09-11T01:23:44.975727+0000","last_clean":"2024-09-11T01:23:44.975727+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:23:44.975727+0000","last_undegraded":"2024-09-11T01:23:44.975727+0000","last_fullsized":"2024-09-11T01:23:44.975727+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20300,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20227,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975565+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:23:44.975565+0000","last_peered":"2024-09-11T01:23:44.975565+0000","last_clean":"2024-09-11T01:23:44.975565+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:23:44.975565+0000","last_undegraded":"2024-09-11T01:23:44.975565+0000","last_fullsized":"2024-09-11T01:23:44.975565+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20160,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.972521+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:23:44.972521+0000","last_peered":"2024-09-11T01:23:44.972521+0000","last_clean":"2024-09-11T01:23:44.972521+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:23:44.972521+0000","last_undegraded":"2024-09-11T01:23:44.972521+0000","last_fullsized":"2024-09-11T01:23:44.972521+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20421,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.971608+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:23:44.971608+0000","last_peered":"2024-09-11T01:23:44.971608+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:23:44.971608+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20084,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.969107+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:23:44.969107+0000","last_peered":"2024-09-11T01:23:44.969107+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:23:44.969107+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20407,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.967353+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:23:44.967353+0000","last_peered":"2024-09-11T01:23:44.967353+0000","last_clean":"2024-09-11T01:23:44.967353+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:23:44.967353+0000","last_undegraded":"2024-09-11T01:23:44.967353+0000","last_fullsized":"2024-09-11T01:23:44.967353+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966055+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:23:44.966055+0000","last_peered":"2024-09-11T01:23:44.966055+0000","last_clean":"2024-09-11T01:23:44.966055+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:23:44.966055+0000","last_undegraded":"2024-09-11T01:23:44.966055+0000","last_fullsized":"2024-09-11T01:23:44.966055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28004,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975400+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:23:44.975400+0000","last_peered":"2024-09-11T01:23:44.975400+0000","last_clean":"2024-09-11T01:23:44.975400+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:23:44.975400+0000","last_undegraded":"2024-09-11T01:23:44.975400+0000","last_fullsized":"2024-09-11T01:23:44.975400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20020,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.972355+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:23:44.972355+0000","last_peered":"2024-09-11T01:23:44.972355+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:23:44.972355+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.970279+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:23:44.970279+0000","last_peered":"2024-09-11T01:23:44.970279+0000","last_clean":"2024-09-11T01:23:44.970279+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:23:44.970279+0000","last_undegraded":"2024-09-11T01:23:44.970279+0000","last_fullsized":"2024-09-11T01:23:44.970279+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":25288,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:10.375733+0000","last_change":"2024-09-11T01:23:43.106212+0000","last_active":"2024-09-11T01:24:10.375733+0000","last_peered":"2024-09-11T01:24:10.375733+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:24:10.375733+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2469,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1557}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966373+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:23:44.966373+0000","last_peered":"2024-09-11T01:23:44.966373+0000","last_clean":"2024-09-11T01:23:44.966373+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:23:44.966373+0000","last_undegraded":"2024-09-11T01:23:44.966373+0000","last_fullsized":"2024-09-11T01:23:44.966373+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27801,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975364+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:23:44.975364+0000","last_peered":"2024-09-11T01:23:44.975364+0000","last_clean":"2024-09-11T01:23:44.975364+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:23:44.975364+0000","last_undegraded":"2024-09-11T01:23:44.975364+0000","last_fullsized":"2024-09-11T01:23:44.975364+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20419,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:45.031607+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:23:45.031607+0000","last_peered":"2024-09-11T01:23:45.031607+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:23:45.031607+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969716+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:23:44.969716+0000","last_peered":"2024-09-11T01:23:44.969716+0000","last_clean":"2024-09-11T01:23:44.969716+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:23:44.969716+0000","last_undegraded":"2024-09-11T01:23:44.969716+0000","last_fullsized":"2024-09-11T01:23:44.969716+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27938,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969619+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:23:44.969619+0000","last_peered":"2024-09-11T01:23:44.969619+0000","last_clean":"2024-09-11T01:23:44.969619+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:23:44.969619+0000","last_undegraded":"2024-09-11T01:23:44.969619+0000","last_fullsized":"2024-09-11T01:23:44.969619+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975642+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:23:44.975642+0000","last_peered":"2024-09-11T01:23:44.975642+0000","last_clean":"2024-09-11T01:23:44.975642+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:23:44.975642+0000","last_undegraded":"2024-09-11T01:23:44.975642+0000","last_fullsized":"2024-09-11T01:23:44.975642+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24617,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969597+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:23:44.969597+0000","last_peered":"2024-09-11T01:23:44.969597+0000","last_clean":"2024-09-11T01:23:44.969597+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:23:44.969597+0000","last_undegraded":"2024-09-11T01:23:44.969597+0000","last_fullsized":"2024-09-11T01:23:44.969597+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969610+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:23:44.969610+0000","last_peered":"2024-09-11T01:23:44.969610+0000","last_clean":"2024-09-11T01:23:44.969610+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:23:44.969610+0000","last_undegraded":"2024-09-11T01:23:44.969610+0000","last_fullsized":"2024-09-11T01:23:44.969610+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966594+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:23:44.966594+0000","last_peered":"2024-09-11T01:23:44.966594+0000","last_clean":"2024-09-11T01:23:44.966594+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:23:44.966594+0000","last_undegraded":"2024-09-11T01:23:44.966594+0000","last_fullsized":"2024-09-11T01:23:44.966594+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20109,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.967341+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:23:44.967341+0000","last_peered":"2024-09-11T01:23:44.967341+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:23:44.967341+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21355,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26284,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195080192,"data_stored":195034965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182551,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518616,"kb_used_data":63560,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853224,"statfs":{"total":96636764160,"available":96105701376,"internally_reserved":0,"allocated":65085440,"data_stored":64773501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986324,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488460,"kb_used_data":127804,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883380,"statfs":{"total":96636764160,"available":96136581120,"internally_reserved":0,"allocated":130871296,"data_stored":130563434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659268,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491648,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93880192,"statfs":{"total":96636764160,"available":96133316608,"internally_reserved":0,"allocated":507904,"data_stored":161175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502918831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:13.422 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:24:13.422 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'20227", 'reported_seq': 20421, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:44.971608+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:23:44.971608+0000', 'last_peered': '2024-09-11T01:23:44.971608+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:23:44.971608+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16568, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:13.422 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:24:13.422 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20084, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:44.969107+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:23:44.969107+0000', 'last_peered': '2024-09-11T01:23:44.969107+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:23:44.969107+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:13.422 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:24:13.423 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20020, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:44.972355+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:23:44.972355+0000', 'last_peered': '2024-09-11T01:23:44.972355+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:23:44.972355+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:13.423 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:24:13.423 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'20207", 'reported_seq': 25288, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:10.375733+0000', 'last_change': '2024-09-11T01:23:43.106212+0000', 'last_active': '2024-09-11T01:24:10.375733+0000', 'last_peered': '2024-09-11T01:24:10.375733+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:24:10.375733+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1557, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18024, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2469, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1557}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:24:13.423 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:24:13.423 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20264", 'reported_seq': 20419, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:45.031607+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:23:45.031607+0000', 'last_peered': '2024-09-11T01:23:45.031607+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:23:45.031607+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17001, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:13.423 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:24:13.423 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 20109, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:44.967341+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:23:44.967341+0000', 'last_peered': '2024-09-11T01:23:44.967341+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:23:44.967341+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:13.423 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:24:13.803 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:13.803 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 21355/6 objects degraded (355916.667%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":790,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1534693376,"bytes_avail":288375599104,"bytes_total":289910292480,"degraded_objects":21355,"degraded_total":6,"degraded_ratio":3559.1666666666661,"recovering_objects_per_sec":67,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":809,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:24:13.822 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:24:13.822 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:14.163 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:14.163 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:24:14.182 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":443,"stamp":"2024-09-11T01:24:12.837627+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21355,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26285,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1498724,"kb_used_data":191860,"kb_used_omap":12,"kb_used_meta":1306803,"kb_avail":281616796,"statfs":{"total":289910292480,"available":288375599104,"internally_reserved":0,"allocated":196464640,"data_stored":195498110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1338166276},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-809,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":809,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003378"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28519,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:45.031826+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:23:45.031826+0000","last_peered":"2024-09-11T01:23:45.031826+0000","last_clean":"2024-09-11T01:23:45.031826+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:23:45.031826+0000","last_undegraded":"2024-09-11T01:23:45.031826+0000","last_fullsized":"2024-09-11T01:23:45.031826+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20477,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975727+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:23:44.975727+0000","last_peered":"2024-09-11T01:23:44.975727+0000","last_clean":"2024-09-11T01:23:44.975727+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:23:44.975727+0000","last_undegraded":"2024-09-11T01:23:44.975727+0000","last_fullsized":"2024-09-11T01:23:44.975727+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20300,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20227,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975565+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:23:44.975565+0000","last_peered":"2024-09-11T01:23:44.975565+0000","last_clean":"2024-09-11T01:23:44.975565+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:23:44.975565+0000","last_undegraded":"2024-09-11T01:23:44.975565+0000","last_fullsized":"2024-09-11T01:23:44.975565+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20160,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.972521+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:23:44.972521+0000","last_peered":"2024-09-11T01:23:44.972521+0000","last_clean":"2024-09-11T01:23:44.972521+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:23:44.972521+0000","last_undegraded":"2024-09-11T01:23:44.972521+0000","last_fullsized":"2024-09-11T01:23:44.972521+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20421,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.971608+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:23:44.971608+0000","last_peered":"2024-09-11T01:23:44.971608+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:23:44.971608+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20084,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.969107+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:23:44.969107+0000","last_peered":"2024-09-11T01:23:44.969107+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:23:44.969107+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20407,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.967353+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:23:44.967353+0000","last_peered":"2024-09-11T01:23:44.967353+0000","last_clean":"2024-09-11T01:23:44.967353+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:23:44.967353+0000","last_undegraded":"2024-09-11T01:23:44.967353+0000","last_fullsized":"2024-09-11T01:23:44.967353+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966055+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:23:44.966055+0000","last_peered":"2024-09-11T01:23:44.966055+0000","last_clean":"2024-09-11T01:23:44.966055+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:23:44.966055+0000","last_undegraded":"2024-09-11T01:23:44.966055+0000","last_fullsized":"2024-09-11T01:23:44.966055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28004,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975400+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:23:44.975400+0000","last_peered":"2024-09-11T01:23:44.975400+0000","last_clean":"2024-09-11T01:23:44.975400+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:23:44.975400+0000","last_undegraded":"2024-09-11T01:23:44.975400+0000","last_fullsized":"2024-09-11T01:23:44.975400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20020,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.972355+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:23:44.972355+0000","last_peered":"2024-09-11T01:23:44.972355+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:23:44.972355+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.970279+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:23:44.970279+0000","last_peered":"2024-09-11T01:23:44.970279+0000","last_clean":"2024-09-11T01:23:44.970279+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:23:44.970279+0000","last_undegraded":"2024-09-11T01:23:44.970279+0000","last_fullsized":"2024-09-11T01:23:44.970279+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":25288,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:10.375733+0000","last_change":"2024-09-11T01:23:43.106212+0000","last_active":"2024-09-11T01:24:10.375733+0000","last_peered":"2024-09-11T01:24:10.375733+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:24:10.375733+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1557,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2469,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1557}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966373+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:23:44.966373+0000","last_peered":"2024-09-11T01:23:44.966373+0000","last_clean":"2024-09-11T01:23:44.966373+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:23:44.966373+0000","last_undegraded":"2024-09-11T01:23:44.966373+0000","last_fullsized":"2024-09-11T01:23:44.966373+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27801,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975364+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:23:44.975364+0000","last_peered":"2024-09-11T01:23:44.975364+0000","last_clean":"2024-09-11T01:23:44.975364+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:23:44.975364+0000","last_undegraded":"2024-09-11T01:23:44.975364+0000","last_fullsized":"2024-09-11T01:23:44.975364+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20419,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:45.031607+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:23:45.031607+0000","last_peered":"2024-09-11T01:23:45.031607+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:23:45.031607+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969716+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:23:44.969716+0000","last_peered":"2024-09-11T01:23:44.969716+0000","last_clean":"2024-09-11T01:23:44.969716+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:23:44.969716+0000","last_undegraded":"2024-09-11T01:23:44.969716+0000","last_fullsized":"2024-09-11T01:23:44.969716+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27938,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969619+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:23:44.969619+0000","last_peered":"2024-09-11T01:23:44.969619+0000","last_clean":"2024-09-11T01:23:44.969619+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:23:44.969619+0000","last_undegraded":"2024-09-11T01:23:44.969619+0000","last_fullsized":"2024-09-11T01:23:44.969619+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975642+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:23:44.975642+0000","last_peered":"2024-09-11T01:23:44.975642+0000","last_clean":"2024-09-11T01:23:44.975642+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:23:44.975642+0000","last_undegraded":"2024-09-11T01:23:44.975642+0000","last_fullsized":"2024-09-11T01:23:44.975642+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24617,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969597+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:23:44.969597+0000","last_peered":"2024-09-11T01:23:44.969597+0000","last_clean":"2024-09-11T01:23:44.969597+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:23:44.969597+0000","last_undegraded":"2024-09-11T01:23:44.969597+0000","last_fullsized":"2024-09-11T01:23:44.969597+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969610+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:23:44.969610+0000","last_peered":"2024-09-11T01:23:44.969610+0000","last_clean":"2024-09-11T01:23:44.969610+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:23:44.969610+0000","last_undegraded":"2024-09-11T01:23:44.969610+0000","last_fullsized":"2024-09-11T01:23:44.969610+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966594+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:23:44.966594+0000","last_peered":"2024-09-11T01:23:44.966594+0000","last_clean":"2024-09-11T01:23:44.966594+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:23:44.966594+0000","last_undegraded":"2024-09-11T01:23:44.966594+0000","last_fullsized":"2024-09-11T01:23:44.966594+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20109,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.967341+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:23:44.967341+0000","last_peered":"2024-09-11T01:23:44.967341+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:23:44.967341+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":21355,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26284,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195080192,"data_stored":195034965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182551,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518616,"kb_used_data":63560,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853224,"statfs":{"total":96636764160,"available":96105701376,"internally_reserved":0,"allocated":65085440,"data_stored":64773501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986324,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488460,"kb_used_data":127804,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883380,"statfs":{"total":96636764160,"available":96136581120,"internally_reserved":0,"allocated":130871296,"data_stored":130563434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659268,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491648,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93880192,"statfs":{"total":96636764160,"available":96133316608,"internally_reserved":0,"allocated":507904,"data_stored":161175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502918831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:17.184 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:17.507 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:17.508 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:24:17.529 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":445,"stamp":"2024-09-11T01:24:16.838858+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20885,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26755,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1502884,"kb_used_data":191860,"kb_used_omap":12,"kb_used_meta":1310963,"kb_avail":281612636,"statfs":{"total":289910292480,"available":288371339264,"internally_reserved":0,"allocated":196464640,"data_stored":195498110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1342426116},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1279,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1279,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003517"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28519,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:45.031826+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:23:45.031826+0000","last_peered":"2024-09-11T01:23:45.031826+0000","last_clean":"2024-09-11T01:23:45.031826+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:23:45.031826+0000","last_undegraded":"2024-09-11T01:23:45.031826+0000","last_fullsized":"2024-09-11T01:23:45.031826+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20477,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975727+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:23:44.975727+0000","last_peered":"2024-09-11T01:23:44.975727+0000","last_clean":"2024-09-11T01:23:44.975727+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:23:44.975727+0000","last_undegraded":"2024-09-11T01:23:44.975727+0000","last_fullsized":"2024-09-11T01:23:44.975727+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20300,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20227,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975565+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:23:44.975565+0000","last_peered":"2024-09-11T01:23:44.975565+0000","last_clean":"2024-09-11T01:23:44.975565+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:23:44.975565+0000","last_undegraded":"2024-09-11T01:23:44.975565+0000","last_fullsized":"2024-09-11T01:23:44.975565+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20160,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.972521+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:23:44.972521+0000","last_peered":"2024-09-11T01:23:44.972521+0000","last_clean":"2024-09-11T01:23:44.972521+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:23:44.972521+0000","last_undegraded":"2024-09-11T01:23:44.972521+0000","last_fullsized":"2024-09-11T01:23:44.972521+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20421,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.971608+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:23:44.971608+0000","last_peered":"2024-09-11T01:23:44.971608+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:23:44.971608+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20084,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.969107+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:23:44.969107+0000","last_peered":"2024-09-11T01:23:44.969107+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:23:44.969107+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20407,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.967353+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:23:44.967353+0000","last_peered":"2024-09-11T01:23:44.967353+0000","last_clean":"2024-09-11T01:23:44.967353+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:23:44.967353+0000","last_undegraded":"2024-09-11T01:23:44.967353+0000","last_fullsized":"2024-09-11T01:23:44.967353+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966055+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:23:44.966055+0000","last_peered":"2024-09-11T01:23:44.966055+0000","last_clean":"2024-09-11T01:23:44.966055+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:23:44.966055+0000","last_undegraded":"2024-09-11T01:23:44.966055+0000","last_fullsized":"2024-09-11T01:23:44.966055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28004,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975400+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:23:44.975400+0000","last_peered":"2024-09-11T01:23:44.975400+0000","last_clean":"2024-09-11T01:23:44.975400+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:23:44.975400+0000","last_undegraded":"2024-09-11T01:23:44.975400+0000","last_fullsized":"2024-09-11T01:23:44.975400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20020,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.972355+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:23:44.972355+0000","last_peered":"2024-09-11T01:23:44.972355+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:23:44.972355+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.970279+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:23:44.970279+0000","last_peered":"2024-09-11T01:23:44.970279+0000","last_clean":"2024-09-11T01:23:44.970279+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:23:44.970279+0000","last_undegraded":"2024-09-11T01:23:44.970279+0000","last_fullsized":"2024-09-11T01:23:44.970279+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":26228,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:15.376762+0000","last_change":"2024-09-11T01:23:43.106212+0000","last_active":"2024-09-11T01:24:15.376762+0000","last_peered":"2024-09-11T01:24:15.376762+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:24:15.376762+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1087,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2939,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1087}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966373+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:23:44.966373+0000","last_peered":"2024-09-11T01:23:44.966373+0000","last_clean":"2024-09-11T01:23:44.966373+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:23:44.966373+0000","last_undegraded":"2024-09-11T01:23:44.966373+0000","last_fullsized":"2024-09-11T01:23:44.966373+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27801,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975364+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:23:44.975364+0000","last_peered":"2024-09-11T01:23:44.975364+0000","last_clean":"2024-09-11T01:23:44.975364+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:23:44.975364+0000","last_undegraded":"2024-09-11T01:23:44.975364+0000","last_fullsized":"2024-09-11T01:23:44.975364+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20419,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:45.031607+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:23:45.031607+0000","last_peered":"2024-09-11T01:23:45.031607+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:23:45.031607+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969716+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:23:44.969716+0000","last_peered":"2024-09-11T01:23:44.969716+0000","last_clean":"2024-09-11T01:23:44.969716+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:23:44.969716+0000","last_undegraded":"2024-09-11T01:23:44.969716+0000","last_fullsized":"2024-09-11T01:23:44.969716+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27938,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969619+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:23:44.969619+0000","last_peered":"2024-09-11T01:23:44.969619+0000","last_clean":"2024-09-11T01:23:44.969619+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:23:44.969619+0000","last_undegraded":"2024-09-11T01:23:44.969619+0000","last_fullsized":"2024-09-11T01:23:44.969619+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975642+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:23:44.975642+0000","last_peered":"2024-09-11T01:23:44.975642+0000","last_clean":"2024-09-11T01:23:44.975642+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:23:44.975642+0000","last_undegraded":"2024-09-11T01:23:44.975642+0000","last_fullsized":"2024-09-11T01:23:44.975642+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24617,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969597+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:23:44.969597+0000","last_peered":"2024-09-11T01:23:44.969597+0000","last_clean":"2024-09-11T01:23:44.969597+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:23:44.969597+0000","last_undegraded":"2024-09-11T01:23:44.969597+0000","last_fullsized":"2024-09-11T01:23:44.969597+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969610+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:23:44.969610+0000","last_peered":"2024-09-11T01:23:44.969610+0000","last_clean":"2024-09-11T01:23:44.969610+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:23:44.969610+0000","last_undegraded":"2024-09-11T01:23:44.969610+0000","last_fullsized":"2024-09-11T01:23:44.969610+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966594+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:23:44.966594+0000","last_peered":"2024-09-11T01:23:44.966594+0000","last_clean":"2024-09-11T01:23:44.966594+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:23:44.966594+0000","last_undegraded":"2024-09-11T01:23:44.966594+0000","last_fullsized":"2024-09-11T01:23:44.966594+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20109,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.967341+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:23:44.967341+0000","last_peered":"2024-09-11T01:23:44.967341+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:23:44.967341+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20885,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26754,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195080192,"data_stored":195034965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182552,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518616,"kb_used_data":63560,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853224,"statfs":{"total":96636764160,"available":96105701376,"internally_reserved":0,"allocated":65085440,"data_stored":64773501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986325,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488460,"kb_used_data":127804,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883380,"statfs":{"total":96636764160,"available":96136581120,"internally_reserved":0,"allocated":130871296,"data_stored":130563434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659269,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":495808,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":495291,"kb_avail":93876032,"statfs":{"total":96636764160,"available":96129056768,"internally_reserved":0,"allocated":507904,"data_stored":161175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":507178671},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:17.531 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:17.842 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:17.842 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:24:17.860 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":445,"stamp":"2024-09-11T01:24:16.838858+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20885,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26755,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1502884,"kb_used_data":191860,"kb_used_omap":12,"kb_used_meta":1310963,"kb_avail":281612636,"statfs":{"total":289910292480,"available":288371339264,"internally_reserved":0,"allocated":196464640,"data_stored":195498110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1342426116},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1279,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1279,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003517"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28519,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:45.031826+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:23:45.031826+0000","last_peered":"2024-09-11T01:23:45.031826+0000","last_clean":"2024-09-11T01:23:45.031826+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:23:45.031826+0000","last_undegraded":"2024-09-11T01:23:45.031826+0000","last_fullsized":"2024-09-11T01:23:45.031826+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20477,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975727+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:23:44.975727+0000","last_peered":"2024-09-11T01:23:44.975727+0000","last_clean":"2024-09-11T01:23:44.975727+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:23:44.975727+0000","last_undegraded":"2024-09-11T01:23:44.975727+0000","last_fullsized":"2024-09-11T01:23:44.975727+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20300,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20227,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975565+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:23:44.975565+0000","last_peered":"2024-09-11T01:23:44.975565+0000","last_clean":"2024-09-11T01:23:44.975565+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:23:44.975565+0000","last_undegraded":"2024-09-11T01:23:44.975565+0000","last_fullsized":"2024-09-11T01:23:44.975565+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20160,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.972521+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:23:44.972521+0000","last_peered":"2024-09-11T01:23:44.972521+0000","last_clean":"2024-09-11T01:23:44.972521+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:23:44.972521+0000","last_undegraded":"2024-09-11T01:23:44.972521+0000","last_fullsized":"2024-09-11T01:23:44.972521+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20421,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.971608+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:23:44.971608+0000","last_peered":"2024-09-11T01:23:44.971608+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:23:44.971608+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20084,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.969107+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:23:44.969107+0000","last_peered":"2024-09-11T01:23:44.969107+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:23:44.969107+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20407,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.967353+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:23:44.967353+0000","last_peered":"2024-09-11T01:23:44.967353+0000","last_clean":"2024-09-11T01:23:44.967353+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:23:44.967353+0000","last_undegraded":"2024-09-11T01:23:44.967353+0000","last_fullsized":"2024-09-11T01:23:44.967353+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966055+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:23:44.966055+0000","last_peered":"2024-09-11T01:23:44.966055+0000","last_clean":"2024-09-11T01:23:44.966055+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:23:44.966055+0000","last_undegraded":"2024-09-11T01:23:44.966055+0000","last_fullsized":"2024-09-11T01:23:44.966055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28004,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975400+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:23:44.975400+0000","last_peered":"2024-09-11T01:23:44.975400+0000","last_clean":"2024-09-11T01:23:44.975400+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:23:44.975400+0000","last_undegraded":"2024-09-11T01:23:44.975400+0000","last_fullsized":"2024-09-11T01:23:44.975400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20020,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.972355+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:23:44.972355+0000","last_peered":"2024-09-11T01:23:44.972355+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:23:44.972355+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.970279+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:23:44.970279+0000","last_peered":"2024-09-11T01:23:44.970279+0000","last_clean":"2024-09-11T01:23:44.970279+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:23:44.970279+0000","last_undegraded":"2024-09-11T01:23:44.970279+0000","last_fullsized":"2024-09-11T01:23:44.970279+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":26228,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:15.376762+0000","last_change":"2024-09-11T01:23:43.106212+0000","last_active":"2024-09-11T01:24:15.376762+0000","last_peered":"2024-09-11T01:24:15.376762+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:24:15.376762+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1087,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2939,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1087}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966373+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:23:44.966373+0000","last_peered":"2024-09-11T01:23:44.966373+0000","last_clean":"2024-09-11T01:23:44.966373+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:23:44.966373+0000","last_undegraded":"2024-09-11T01:23:44.966373+0000","last_fullsized":"2024-09-11T01:23:44.966373+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27801,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975364+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:23:44.975364+0000","last_peered":"2024-09-11T01:23:44.975364+0000","last_clean":"2024-09-11T01:23:44.975364+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:23:44.975364+0000","last_undegraded":"2024-09-11T01:23:44.975364+0000","last_fullsized":"2024-09-11T01:23:44.975364+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20419,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:45.031607+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:23:45.031607+0000","last_peered":"2024-09-11T01:23:45.031607+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:23:45.031607+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969716+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:23:44.969716+0000","last_peered":"2024-09-11T01:23:44.969716+0000","last_clean":"2024-09-11T01:23:44.969716+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:23:44.969716+0000","last_undegraded":"2024-09-11T01:23:44.969716+0000","last_fullsized":"2024-09-11T01:23:44.969716+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27938,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969619+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:23:44.969619+0000","last_peered":"2024-09-11T01:23:44.969619+0000","last_clean":"2024-09-11T01:23:44.969619+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:23:44.969619+0000","last_undegraded":"2024-09-11T01:23:44.969619+0000","last_fullsized":"2024-09-11T01:23:44.969619+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975642+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:23:44.975642+0000","last_peered":"2024-09-11T01:23:44.975642+0000","last_clean":"2024-09-11T01:23:44.975642+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:23:44.975642+0000","last_undegraded":"2024-09-11T01:23:44.975642+0000","last_fullsized":"2024-09-11T01:23:44.975642+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24617,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969597+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:23:44.969597+0000","last_peered":"2024-09-11T01:23:44.969597+0000","last_clean":"2024-09-11T01:23:44.969597+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:23:44.969597+0000","last_undegraded":"2024-09-11T01:23:44.969597+0000","last_fullsized":"2024-09-11T01:23:44.969597+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969610+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:23:44.969610+0000","last_peered":"2024-09-11T01:23:44.969610+0000","last_clean":"2024-09-11T01:23:44.969610+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:23:44.969610+0000","last_undegraded":"2024-09-11T01:23:44.969610+0000","last_fullsized":"2024-09-11T01:23:44.969610+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966594+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:23:44.966594+0000","last_peered":"2024-09-11T01:23:44.966594+0000","last_clean":"2024-09-11T01:23:44.966594+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:23:44.966594+0000","last_undegraded":"2024-09-11T01:23:44.966594+0000","last_fullsized":"2024-09-11T01:23:44.966594+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20109,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.967341+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:23:44.967341+0000","last_peered":"2024-09-11T01:23:44.967341+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:23:44.967341+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20885,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26754,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195080192,"data_stored":195034965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182552,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518616,"kb_used_data":63560,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853224,"statfs":{"total":96636764160,"available":96105701376,"internally_reserved":0,"allocated":65085440,"data_stored":64773501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986325,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488460,"kb_used_data":127804,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883380,"statfs":{"total":96636764160,"available":96136581120,"internally_reserved":0,"allocated":130871296,"data_stored":130563434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659269,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":495808,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":495291,"kb_avail":93876032,"statfs":{"total":96636764160,"available":96129056768,"internally_reserved":0,"allocated":507904,"data_stored":161175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":507178671},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:17.862 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:24:17.862 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'20227", 'reported_seq': 20421, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:44.971608+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:23:44.971608+0000', 'last_peered': '2024-09-11T01:23:44.971608+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:23:44.971608+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16568, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:17.862 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:24:17.862 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20084, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:44.969107+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:23:44.969107+0000', 'last_peered': '2024-09-11T01:23:44.969107+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:23:44.969107+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:17.862 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:24:17.862 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20020, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:44.972355+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:23:44.972355+0000', 'last_peered': '2024-09-11T01:23:44.972355+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:23:44.972355+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:17.862 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:24:17.863 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'20207", 'reported_seq': 26228, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:15.376762+0000', 'last_change': '2024-09-11T01:23:43.106212+0000', 'last_active': '2024-09-11T01:24:15.376762+0000', 'last_peered': '2024-09-11T01:24:15.376762+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:24:15.376762+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1087, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18024, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2939, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -1087}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:24:17.863 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:24:17.863 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20264", 'reported_seq': 20419, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:45.031607+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:23:45.031607+0000', 'last_peered': '2024-09-11T01:23:45.031607+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:23:45.031607+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17001, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:17.863 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:24:17.863 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 20109, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:44.967341+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:23:44.967341+0000', 'last_peered': '2024-09-11T01:23:44.967341+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:23:44.967341+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:17.863 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:24:18.240 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:18.240 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 20885/6 objects degraded (348083.333%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":794,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1538953216,"bytes_avail":288371339264,"bytes_total":289910292480,"degraded_objects":20885,"degraded_total":6,"degraded_ratio":3480.8333333333339,"recovering_objects_per_sec":106,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1279,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:24:18.259 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:24:18.259 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:18.560 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:18.560 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:24:18.578 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":445,"stamp":"2024-09-11T01:24:16.838858+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20885,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26755,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1502884,"kb_used_data":191860,"kb_used_omap":12,"kb_used_meta":1310963,"kb_avail":281612636,"statfs":{"total":289910292480,"available":288371339264,"internally_reserved":0,"allocated":196464640,"data_stored":195498110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1342426116},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1279,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1279,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003517"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28519,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:45.031826+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:23:45.031826+0000","last_peered":"2024-09-11T01:23:45.031826+0000","last_clean":"2024-09-11T01:23:45.031826+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:23:45.031826+0000","last_undegraded":"2024-09-11T01:23:45.031826+0000","last_fullsized":"2024-09-11T01:23:45.031826+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20477,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975727+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:23:44.975727+0000","last_peered":"2024-09-11T01:23:44.975727+0000","last_clean":"2024-09-11T01:23:44.975727+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:23:44.975727+0000","last_undegraded":"2024-09-11T01:23:44.975727+0000","last_fullsized":"2024-09-11T01:23:44.975727+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20300,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20227,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975565+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:23:44.975565+0000","last_peered":"2024-09-11T01:23:44.975565+0000","last_clean":"2024-09-11T01:23:44.975565+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:23:44.975565+0000","last_undegraded":"2024-09-11T01:23:44.975565+0000","last_fullsized":"2024-09-11T01:23:44.975565+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20160,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.972521+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:23:44.972521+0000","last_peered":"2024-09-11T01:23:44.972521+0000","last_clean":"2024-09-11T01:23:44.972521+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:23:44.972521+0000","last_undegraded":"2024-09-11T01:23:44.972521+0000","last_fullsized":"2024-09-11T01:23:44.972521+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20421,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.971608+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:23:44.971608+0000","last_peered":"2024-09-11T01:23:44.971608+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:23:44.971608+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20084,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.969107+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:23:44.969107+0000","last_peered":"2024-09-11T01:23:44.969107+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:23:44.969107+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20407,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.967353+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:23:44.967353+0000","last_peered":"2024-09-11T01:23:44.967353+0000","last_clean":"2024-09-11T01:23:44.967353+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:23:44.967353+0000","last_undegraded":"2024-09-11T01:23:44.967353+0000","last_fullsized":"2024-09-11T01:23:44.967353+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966055+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:23:44.966055+0000","last_peered":"2024-09-11T01:23:44.966055+0000","last_clean":"2024-09-11T01:23:44.966055+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:23:44.966055+0000","last_undegraded":"2024-09-11T01:23:44.966055+0000","last_fullsized":"2024-09-11T01:23:44.966055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28004,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975400+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:23:44.975400+0000","last_peered":"2024-09-11T01:23:44.975400+0000","last_clean":"2024-09-11T01:23:44.975400+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:23:44.975400+0000","last_undegraded":"2024-09-11T01:23:44.975400+0000","last_fullsized":"2024-09-11T01:23:44.975400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20020,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.972355+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:23:44.972355+0000","last_peered":"2024-09-11T01:23:44.972355+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:23:44.972355+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.970279+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:23:44.970279+0000","last_peered":"2024-09-11T01:23:44.970279+0000","last_clean":"2024-09-11T01:23:44.970279+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:23:44.970279+0000","last_undegraded":"2024-09-11T01:23:44.970279+0000","last_fullsized":"2024-09-11T01:23:44.970279+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":26228,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:15.376762+0000","last_change":"2024-09-11T01:23:43.106212+0000","last_active":"2024-09-11T01:24:15.376762+0000","last_peered":"2024-09-11T01:24:15.376762+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:24:15.376762+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1087,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2939,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-1087}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966373+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:23:44.966373+0000","last_peered":"2024-09-11T01:23:44.966373+0000","last_clean":"2024-09-11T01:23:44.966373+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:23:44.966373+0000","last_undegraded":"2024-09-11T01:23:44.966373+0000","last_fullsized":"2024-09-11T01:23:44.966373+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27801,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975364+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:23:44.975364+0000","last_peered":"2024-09-11T01:23:44.975364+0000","last_clean":"2024-09-11T01:23:44.975364+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:23:44.975364+0000","last_undegraded":"2024-09-11T01:23:44.975364+0000","last_fullsized":"2024-09-11T01:23:44.975364+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20419,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:45.031607+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:23:45.031607+0000","last_peered":"2024-09-11T01:23:45.031607+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:23:45.031607+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969716+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:23:44.969716+0000","last_peered":"2024-09-11T01:23:44.969716+0000","last_clean":"2024-09-11T01:23:44.969716+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:23:44.969716+0000","last_undegraded":"2024-09-11T01:23:44.969716+0000","last_fullsized":"2024-09-11T01:23:44.969716+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27938,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969619+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:23:44.969619+0000","last_peered":"2024-09-11T01:23:44.969619+0000","last_clean":"2024-09-11T01:23:44.969619+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:23:44.969619+0000","last_undegraded":"2024-09-11T01:23:44.969619+0000","last_fullsized":"2024-09-11T01:23:44.969619+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975642+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:23:44.975642+0000","last_peered":"2024-09-11T01:23:44.975642+0000","last_clean":"2024-09-11T01:23:44.975642+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:23:44.975642+0000","last_undegraded":"2024-09-11T01:23:44.975642+0000","last_fullsized":"2024-09-11T01:23:44.975642+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24617,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969597+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:23:44.969597+0000","last_peered":"2024-09-11T01:23:44.969597+0000","last_clean":"2024-09-11T01:23:44.969597+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:23:44.969597+0000","last_undegraded":"2024-09-11T01:23:44.969597+0000","last_fullsized":"2024-09-11T01:23:44.969597+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969610+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:23:44.969610+0000","last_peered":"2024-09-11T01:23:44.969610+0000","last_clean":"2024-09-11T01:23:44.969610+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:23:44.969610+0000","last_undegraded":"2024-09-11T01:23:44.969610+0000","last_fullsized":"2024-09-11T01:23:44.969610+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966594+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:23:44.966594+0000","last_peered":"2024-09-11T01:23:44.966594+0000","last_clean":"2024-09-11T01:23:44.966594+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:23:44.966594+0000","last_undegraded":"2024-09-11T01:23:44.966594+0000","last_fullsized":"2024-09-11T01:23:44.966594+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20109,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.967341+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:23:44.967341+0000","last_peered":"2024-09-11T01:23:44.967341+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:23:44.967341+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20885,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":26754,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195080192,"data_stored":195034965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182552,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518616,"kb_used_data":63560,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853224,"statfs":{"total":96636764160,"available":96105701376,"internally_reserved":0,"allocated":65085440,"data_stored":64773501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986325,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488460,"kb_used_data":127804,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883380,"statfs":{"total":96636764160,"available":96136581120,"internally_reserved":0,"allocated":130871296,"data_stored":130563434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659269,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":495808,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":495291,"kb_avail":93876032,"statfs":{"total":96636764160,"available":96129056768,"internally_reserved":0,"allocated":507904,"data_stored":161175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":507178671},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:21.581 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:21.925 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:21.926 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:24:21.944 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":447,"stamp":"2024-09-11T01:24:20.840241+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20335,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27305,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1490404,"kb_used_data":191860,"kb_used_omap":12,"kb_used_meta":1298483,"kb_avail":281625116,"statfs":{"total":289910292480,"available":288384118784,"internally_reserved":0,"allocated":196464640,"data_stored":195498110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1329646596},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1445,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1445,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003747"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28519,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:45.031826+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:23:45.031826+0000","last_peered":"2024-09-11T01:23:45.031826+0000","last_clean":"2024-09-11T01:23:45.031826+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:23:45.031826+0000","last_undegraded":"2024-09-11T01:23:45.031826+0000","last_fullsized":"2024-09-11T01:23:45.031826+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20477,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975727+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:23:44.975727+0000","last_peered":"2024-09-11T01:23:44.975727+0000","last_clean":"2024-09-11T01:23:44.975727+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:23:44.975727+0000","last_undegraded":"2024-09-11T01:23:44.975727+0000","last_fullsized":"2024-09-11T01:23:44.975727+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20300,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20227,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975565+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:23:44.975565+0000","last_peered":"2024-09-11T01:23:44.975565+0000","last_clean":"2024-09-11T01:23:44.975565+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:23:44.975565+0000","last_undegraded":"2024-09-11T01:23:44.975565+0000","last_fullsized":"2024-09-11T01:23:44.975565+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20160,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.972521+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:23:44.972521+0000","last_peered":"2024-09-11T01:23:44.972521+0000","last_clean":"2024-09-11T01:23:44.972521+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:23:44.972521+0000","last_undegraded":"2024-09-11T01:23:44.972521+0000","last_fullsized":"2024-09-11T01:23:44.972521+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20421,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.971608+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:23:44.971608+0000","last_peered":"2024-09-11T01:23:44.971608+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:23:44.971608+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20084,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.969107+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:23:44.969107+0000","last_peered":"2024-09-11T01:23:44.969107+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:23:44.969107+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20407,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.967353+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:23:44.967353+0000","last_peered":"2024-09-11T01:23:44.967353+0000","last_clean":"2024-09-11T01:23:44.967353+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:23:44.967353+0000","last_undegraded":"2024-09-11T01:23:44.967353+0000","last_fullsized":"2024-09-11T01:23:44.967353+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966055+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:23:44.966055+0000","last_peered":"2024-09-11T01:23:44.966055+0000","last_clean":"2024-09-11T01:23:44.966055+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:23:44.966055+0000","last_undegraded":"2024-09-11T01:23:44.966055+0000","last_fullsized":"2024-09-11T01:23:44.966055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28004,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975400+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:23:44.975400+0000","last_peered":"2024-09-11T01:23:44.975400+0000","last_clean":"2024-09-11T01:23:44.975400+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:23:44.975400+0000","last_undegraded":"2024-09-11T01:23:44.975400+0000","last_fullsized":"2024-09-11T01:23:44.975400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20020,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.972355+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:23:44.972355+0000","last_peered":"2024-09-11T01:23:44.972355+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:23:44.972355+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.970279+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:23:44.970279+0000","last_peered":"2024-09-11T01:23:44.970279+0000","last_clean":"2024-09-11T01:23:44.970279+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:23:44.970279+0000","last_undegraded":"2024-09-11T01:23:44.970279+0000","last_fullsized":"2024-09-11T01:23:44.970279+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":27328,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:20.380683+0000","last_change":"2024-09-11T01:23:43.106212+0000","last_active":"2024-09-11T01:24:20.380683+0000","last_peered":"2024-09-11T01:24:20.380683+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:24:20.380683+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":537,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3489,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-537}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966373+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:23:44.966373+0000","last_peered":"2024-09-11T01:23:44.966373+0000","last_clean":"2024-09-11T01:23:44.966373+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:23:44.966373+0000","last_undegraded":"2024-09-11T01:23:44.966373+0000","last_fullsized":"2024-09-11T01:23:44.966373+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27801,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975364+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:23:44.975364+0000","last_peered":"2024-09-11T01:23:44.975364+0000","last_clean":"2024-09-11T01:23:44.975364+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:23:44.975364+0000","last_undegraded":"2024-09-11T01:23:44.975364+0000","last_fullsized":"2024-09-11T01:23:44.975364+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20419,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:45.031607+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:23:45.031607+0000","last_peered":"2024-09-11T01:23:45.031607+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:23:45.031607+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969716+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:23:44.969716+0000","last_peered":"2024-09-11T01:23:44.969716+0000","last_clean":"2024-09-11T01:23:44.969716+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:23:44.969716+0000","last_undegraded":"2024-09-11T01:23:44.969716+0000","last_fullsized":"2024-09-11T01:23:44.969716+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27938,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969619+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:23:44.969619+0000","last_peered":"2024-09-11T01:23:44.969619+0000","last_clean":"2024-09-11T01:23:44.969619+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:23:44.969619+0000","last_undegraded":"2024-09-11T01:23:44.969619+0000","last_fullsized":"2024-09-11T01:23:44.969619+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975642+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:23:44.975642+0000","last_peered":"2024-09-11T01:23:44.975642+0000","last_clean":"2024-09-11T01:23:44.975642+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:23:44.975642+0000","last_undegraded":"2024-09-11T01:23:44.975642+0000","last_fullsized":"2024-09-11T01:23:44.975642+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24617,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969597+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:23:44.969597+0000","last_peered":"2024-09-11T01:23:44.969597+0000","last_clean":"2024-09-11T01:23:44.969597+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:23:44.969597+0000","last_undegraded":"2024-09-11T01:23:44.969597+0000","last_fullsized":"2024-09-11T01:23:44.969597+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969610+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:23:44.969610+0000","last_peered":"2024-09-11T01:23:44.969610+0000","last_clean":"2024-09-11T01:23:44.969610+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:23:44.969610+0000","last_undegraded":"2024-09-11T01:23:44.969610+0000","last_fullsized":"2024-09-11T01:23:44.969610+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966594+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:23:44.966594+0000","last_peered":"2024-09-11T01:23:44.966594+0000","last_clean":"2024-09-11T01:23:44.966594+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:23:44.966594+0000","last_undegraded":"2024-09-11T01:23:44.966594+0000","last_fullsized":"2024-09-11T01:23:44.966594+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20109,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.967341+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:23:44.967341+0000","last_peered":"2024-09-11T01:23:44.967341+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:23:44.967341+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20335,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27304,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195080192,"data_stored":195034965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182553,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518616,"kb_used_data":63560,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853224,"statfs":{"total":96636764160,"available":96105701376,"internally_reserved":0,"allocated":65085440,"data_stored":64773501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986326,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488460,"kb_used_data":127804,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883380,"statfs":{"total":96636764160,"available":96136581120,"internally_reserved":0,"allocated":130871296,"data_stored":130563434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659270,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483328,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":482811,"kb_avail":93888512,"statfs":{"total":96636764160,"available":96141836288,"internally_reserved":0,"allocated":507904,"data_stored":161175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494399151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:21.945 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:22.263 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:22.264 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:24:22.281 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":447,"stamp":"2024-09-11T01:24:20.840241+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20335,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27305,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1490404,"kb_used_data":191860,"kb_used_omap":12,"kb_used_meta":1298483,"kb_avail":281625116,"statfs":{"total":289910292480,"available":288384118784,"internally_reserved":0,"allocated":196464640,"data_stored":195498110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1329646596},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1445,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1445,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003747"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28519,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:45.031826+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:23:45.031826+0000","last_peered":"2024-09-11T01:23:45.031826+0000","last_clean":"2024-09-11T01:23:45.031826+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:23:45.031826+0000","last_undegraded":"2024-09-11T01:23:45.031826+0000","last_fullsized":"2024-09-11T01:23:45.031826+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20477,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975727+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:23:44.975727+0000","last_peered":"2024-09-11T01:23:44.975727+0000","last_clean":"2024-09-11T01:23:44.975727+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:23:44.975727+0000","last_undegraded":"2024-09-11T01:23:44.975727+0000","last_fullsized":"2024-09-11T01:23:44.975727+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20300,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20227,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975565+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:23:44.975565+0000","last_peered":"2024-09-11T01:23:44.975565+0000","last_clean":"2024-09-11T01:23:44.975565+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:23:44.975565+0000","last_undegraded":"2024-09-11T01:23:44.975565+0000","last_fullsized":"2024-09-11T01:23:44.975565+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20160,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.972521+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:23:44.972521+0000","last_peered":"2024-09-11T01:23:44.972521+0000","last_clean":"2024-09-11T01:23:44.972521+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:23:44.972521+0000","last_undegraded":"2024-09-11T01:23:44.972521+0000","last_fullsized":"2024-09-11T01:23:44.972521+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20421,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.971608+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:23:44.971608+0000","last_peered":"2024-09-11T01:23:44.971608+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:23:44.971608+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20084,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.969107+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:23:44.969107+0000","last_peered":"2024-09-11T01:23:44.969107+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:23:44.969107+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20407,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.967353+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:23:44.967353+0000","last_peered":"2024-09-11T01:23:44.967353+0000","last_clean":"2024-09-11T01:23:44.967353+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:23:44.967353+0000","last_undegraded":"2024-09-11T01:23:44.967353+0000","last_fullsized":"2024-09-11T01:23:44.967353+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966055+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:23:44.966055+0000","last_peered":"2024-09-11T01:23:44.966055+0000","last_clean":"2024-09-11T01:23:44.966055+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:23:44.966055+0000","last_undegraded":"2024-09-11T01:23:44.966055+0000","last_fullsized":"2024-09-11T01:23:44.966055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28004,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975400+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:23:44.975400+0000","last_peered":"2024-09-11T01:23:44.975400+0000","last_clean":"2024-09-11T01:23:44.975400+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:23:44.975400+0000","last_undegraded":"2024-09-11T01:23:44.975400+0000","last_fullsized":"2024-09-11T01:23:44.975400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20020,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.972355+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:23:44.972355+0000","last_peered":"2024-09-11T01:23:44.972355+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:23:44.972355+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.970279+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:23:44.970279+0000","last_peered":"2024-09-11T01:23:44.970279+0000","last_clean":"2024-09-11T01:23:44.970279+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:23:44.970279+0000","last_undegraded":"2024-09-11T01:23:44.970279+0000","last_fullsized":"2024-09-11T01:23:44.970279+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":27328,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:20.380683+0000","last_change":"2024-09-11T01:23:43.106212+0000","last_active":"2024-09-11T01:24:20.380683+0000","last_peered":"2024-09-11T01:24:20.380683+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:24:20.380683+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":537,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3489,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-537}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966373+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:23:44.966373+0000","last_peered":"2024-09-11T01:23:44.966373+0000","last_clean":"2024-09-11T01:23:44.966373+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:23:44.966373+0000","last_undegraded":"2024-09-11T01:23:44.966373+0000","last_fullsized":"2024-09-11T01:23:44.966373+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27801,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975364+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:23:44.975364+0000","last_peered":"2024-09-11T01:23:44.975364+0000","last_clean":"2024-09-11T01:23:44.975364+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:23:44.975364+0000","last_undegraded":"2024-09-11T01:23:44.975364+0000","last_fullsized":"2024-09-11T01:23:44.975364+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20419,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:45.031607+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:23:45.031607+0000","last_peered":"2024-09-11T01:23:45.031607+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:23:45.031607+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969716+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:23:44.969716+0000","last_peered":"2024-09-11T01:23:44.969716+0000","last_clean":"2024-09-11T01:23:44.969716+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:23:44.969716+0000","last_undegraded":"2024-09-11T01:23:44.969716+0000","last_fullsized":"2024-09-11T01:23:44.969716+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27938,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969619+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:23:44.969619+0000","last_peered":"2024-09-11T01:23:44.969619+0000","last_clean":"2024-09-11T01:23:44.969619+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:23:44.969619+0000","last_undegraded":"2024-09-11T01:23:44.969619+0000","last_fullsized":"2024-09-11T01:23:44.969619+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975642+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:23:44.975642+0000","last_peered":"2024-09-11T01:23:44.975642+0000","last_clean":"2024-09-11T01:23:44.975642+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:23:44.975642+0000","last_undegraded":"2024-09-11T01:23:44.975642+0000","last_fullsized":"2024-09-11T01:23:44.975642+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24617,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969597+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:23:44.969597+0000","last_peered":"2024-09-11T01:23:44.969597+0000","last_clean":"2024-09-11T01:23:44.969597+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:23:44.969597+0000","last_undegraded":"2024-09-11T01:23:44.969597+0000","last_fullsized":"2024-09-11T01:23:44.969597+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969610+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:23:44.969610+0000","last_peered":"2024-09-11T01:23:44.969610+0000","last_clean":"2024-09-11T01:23:44.969610+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:23:44.969610+0000","last_undegraded":"2024-09-11T01:23:44.969610+0000","last_fullsized":"2024-09-11T01:23:44.969610+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966594+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:23:44.966594+0000","last_peered":"2024-09-11T01:23:44.966594+0000","last_clean":"2024-09-11T01:23:44.966594+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:23:44.966594+0000","last_undegraded":"2024-09-11T01:23:44.966594+0000","last_fullsized":"2024-09-11T01:23:44.966594+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20109,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.967341+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:23:44.967341+0000","last_peered":"2024-09-11T01:23:44.967341+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:23:44.967341+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20335,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27304,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195080192,"data_stored":195034965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182553,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518616,"kb_used_data":63560,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853224,"statfs":{"total":96636764160,"available":96105701376,"internally_reserved":0,"allocated":65085440,"data_stored":64773501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986326,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488460,"kb_used_data":127804,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883380,"statfs":{"total":96636764160,"available":96136581120,"internally_reserved":0,"allocated":130871296,"data_stored":130563434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659270,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483328,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":482811,"kb_avail":93888512,"statfs":{"total":96636764160,"available":96141836288,"internally_reserved":0,"allocated":507904,"data_stored":161175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494399151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:22.282 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:24:22.283 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'20227", 'reported_seq': 20421, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:44.971608+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:23:44.971608+0000', 'last_peered': '2024-09-11T01:23:44.971608+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:23:44.971608+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16568, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:22.283 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:24:22.283 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20084, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:44.969107+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:23:44.969107+0000', 'last_peered': '2024-09-11T01:23:44.969107+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:23:44.969107+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:22.283 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:24:22.283 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20020, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:44.972355+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:23:44.972355+0000', 'last_peered': '2024-09-11T01:23:44.972355+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:23:44.972355+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:22.283 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:24:22.283 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'20207", 'reported_seq': 27328, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:20.380683+0000', 'last_change': '2024-09-11T01:23:43.106212+0000', 'last_active': '2024-09-11T01:24:20.380683+0000', 'last_peered': '2024-09-11T01:24:20.380683+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:24:20.380683+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 537, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18024, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3489, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -537}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:24:22.283 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:24:22.283 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20264", 'reported_seq': 20419, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:45.031607+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:23:45.031607+0000', 'last_peered': '2024-09-11T01:23:45.031607+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:23:45.031607+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17001, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:22.283 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:24:22.284 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 20109, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:44.967341+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:23:44.967341+0000', 'last_peered': '2024-09-11T01:23:44.967341+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:23:44.967341+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:22.284 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:24:22.680 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:22.680 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 20335/6 objects degraded (338916.667%), 6 pgs degraded, 6 pgs undersized","count":12},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":799,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":55,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":6},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":18},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1526173696,"bytes_avail":288384118784,"bytes_total":289910292480,"degraded_objects":20335,"degraded_total":6,"degraded_ratio":3389.1666666666661,"recovering_objects_per_sec":120,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1445,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:24:22.698 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:24:22.699 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:23.014 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:23.015 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:24:23.032 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":448,"stamp":"2024-09-11T01:24:22.840912+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20335,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27305,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1490404,"kb_used_data":191860,"kb_used_omap":12,"kb_used_meta":1298483,"kb_avail":281625116,"statfs":{"total":289910292480,"available":288384118784,"internally_reserved":0,"allocated":196464640,"data_stored":195498110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1329646596},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1020,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1020,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003898"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28519,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:45.031826+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:23:45.031826+0000","last_peered":"2024-09-11T01:23:45.031826+0000","last_clean":"2024-09-11T01:23:45.031826+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:23:45.031826+0000","last_undegraded":"2024-09-11T01:23:45.031826+0000","last_fullsized":"2024-09-11T01:23:45.031826+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20477,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975727+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:23:44.975727+0000","last_peered":"2024-09-11T01:23:44.975727+0000","last_clean":"2024-09-11T01:23:44.975727+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:23:44.975727+0000","last_undegraded":"2024-09-11T01:23:44.975727+0000","last_fullsized":"2024-09-11T01:23:44.975727+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20300,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20227,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975565+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:23:44.975565+0000","last_peered":"2024-09-11T01:23:44.975565+0000","last_clean":"2024-09-11T01:23:44.975565+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:23:44.975565+0000","last_undegraded":"2024-09-11T01:23:44.975565+0000","last_fullsized":"2024-09-11T01:23:44.975565+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20160,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.972521+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:23:44.972521+0000","last_peered":"2024-09-11T01:23:44.972521+0000","last_clean":"2024-09-11T01:23:44.972521+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:23:44.972521+0000","last_undegraded":"2024-09-11T01:23:44.972521+0000","last_fullsized":"2024-09-11T01:23:44.972521+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20421,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.971608+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:23:44.971608+0000","last_peered":"2024-09-11T01:23:44.971608+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:23:44.971608+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20084,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.969107+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:23:44.969107+0000","last_peered":"2024-09-11T01:23:44.969107+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:23:44.969107+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20407,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.967353+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:23:44.967353+0000","last_peered":"2024-09-11T01:23:44.967353+0000","last_clean":"2024-09-11T01:23:44.967353+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:23:44.967353+0000","last_undegraded":"2024-09-11T01:23:44.967353+0000","last_fullsized":"2024-09-11T01:23:44.967353+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966055+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:23:44.966055+0000","last_peered":"2024-09-11T01:23:44.966055+0000","last_clean":"2024-09-11T01:23:44.966055+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:23:44.966055+0000","last_undegraded":"2024-09-11T01:23:44.966055+0000","last_fullsized":"2024-09-11T01:23:44.966055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28004,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975400+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:23:44.975400+0000","last_peered":"2024-09-11T01:23:44.975400+0000","last_clean":"2024-09-11T01:23:44.975400+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:23:44.975400+0000","last_undegraded":"2024-09-11T01:23:44.975400+0000","last_fullsized":"2024-09-11T01:23:44.975400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20020,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.972355+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:23:44.972355+0000","last_peered":"2024-09-11T01:23:44.972355+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:23:44.972355+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.970279+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:23:44.970279+0000","last_peered":"2024-09-11T01:23:44.970279+0000","last_clean":"2024-09-11T01:23:44.970279+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:23:44.970279+0000","last_undegraded":"2024-09-11T01:23:44.970279+0000","last_fullsized":"2024-09-11T01:23:44.970279+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":27328,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:20.380683+0000","last_change":"2024-09-11T01:23:43.106212+0000","last_active":"2024-09-11T01:24:20.380683+0000","last_peered":"2024-09-11T01:24:20.380683+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:24:20.380683+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":537,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3489,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-537}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966373+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:23:44.966373+0000","last_peered":"2024-09-11T01:23:44.966373+0000","last_clean":"2024-09-11T01:23:44.966373+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:23:44.966373+0000","last_undegraded":"2024-09-11T01:23:44.966373+0000","last_fullsized":"2024-09-11T01:23:44.966373+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27801,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975364+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:23:44.975364+0000","last_peered":"2024-09-11T01:23:44.975364+0000","last_clean":"2024-09-11T01:23:44.975364+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:23:44.975364+0000","last_undegraded":"2024-09-11T01:23:44.975364+0000","last_fullsized":"2024-09-11T01:23:44.975364+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20419,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:45.031607+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:23:45.031607+0000","last_peered":"2024-09-11T01:23:45.031607+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:23:45.031607+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969716+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:23:44.969716+0000","last_peered":"2024-09-11T01:23:44.969716+0000","last_clean":"2024-09-11T01:23:44.969716+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:23:44.969716+0000","last_undegraded":"2024-09-11T01:23:44.969716+0000","last_fullsized":"2024-09-11T01:23:44.969716+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27938,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969619+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:23:44.969619+0000","last_peered":"2024-09-11T01:23:44.969619+0000","last_clean":"2024-09-11T01:23:44.969619+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:23:44.969619+0000","last_undegraded":"2024-09-11T01:23:44.969619+0000","last_fullsized":"2024-09-11T01:23:44.969619+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975642+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:23:44.975642+0000","last_peered":"2024-09-11T01:23:44.975642+0000","last_clean":"2024-09-11T01:23:44.975642+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:23:44.975642+0000","last_undegraded":"2024-09-11T01:23:44.975642+0000","last_fullsized":"2024-09-11T01:23:44.975642+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24617,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969597+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:23:44.969597+0000","last_peered":"2024-09-11T01:23:44.969597+0000","last_clean":"2024-09-11T01:23:44.969597+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:23:44.969597+0000","last_undegraded":"2024-09-11T01:23:44.969597+0000","last_fullsized":"2024-09-11T01:23:44.969597+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969610+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:23:44.969610+0000","last_peered":"2024-09-11T01:23:44.969610+0000","last_clean":"2024-09-11T01:23:44.969610+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:23:44.969610+0000","last_undegraded":"2024-09-11T01:23:44.969610+0000","last_fullsized":"2024-09-11T01:23:44.969610+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966594+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:23:44.966594+0000","last_peered":"2024-09-11T01:23:44.966594+0000","last_clean":"2024-09-11T01:23:44.966594+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:23:44.966594+0000","last_undegraded":"2024-09-11T01:23:44.966594+0000","last_fullsized":"2024-09-11T01:23:44.966594+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20109,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.967341+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:23:44.967341+0000","last_peered":"2024-09-11T01:23:44.967341+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:23:44.967341+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20335,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27304,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195080192,"data_stored":195034965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182553,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518616,"kb_used_data":63560,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853224,"statfs":{"total":96636764160,"available":96105701376,"internally_reserved":0,"allocated":65085440,"data_stored":64773501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986326,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488460,"kb_used_data":127804,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883380,"statfs":{"total":96636764160,"available":96136581120,"internally_reserved":0,"allocated":130871296,"data_stored":130563434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659270,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483328,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":482811,"kb_avail":93888512,"statfs":{"total":96636764160,"available":96141836288,"internally_reserved":0,"allocated":507904,"data_stored":161175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494399151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:26.034 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:26.364 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:26.365 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:24:26.384 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":451,"stamp":"2024-09-11T01:24:25.132651+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20335,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27305,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1494564,"kb_used_data":191860,"kb_used_omap":12,"kb_used_meta":1302643,"kb_avail":281620956,"statfs":{"total":289910292480,"available":288379858944,"internally_reserved":0,"allocated":196464640,"data_stored":195498110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1333906436},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-550,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":550,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.293793"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28519,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:45.031826+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:23:45.031826+0000","last_peered":"2024-09-11T01:23:45.031826+0000","last_clean":"2024-09-11T01:23:45.031826+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:23:45.031826+0000","last_undegraded":"2024-09-11T01:23:45.031826+0000","last_fullsized":"2024-09-11T01:23:45.031826+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20477,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975727+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:23:44.975727+0000","last_peered":"2024-09-11T01:23:44.975727+0000","last_clean":"2024-09-11T01:23:44.975727+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:23:44.975727+0000","last_undegraded":"2024-09-11T01:23:44.975727+0000","last_fullsized":"2024-09-11T01:23:44.975727+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20300,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20227,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975565+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:23:44.975565+0000","last_peered":"2024-09-11T01:23:44.975565+0000","last_clean":"2024-09-11T01:23:44.975565+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:23:44.975565+0000","last_undegraded":"2024-09-11T01:23:44.975565+0000","last_fullsized":"2024-09-11T01:23:44.975565+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20160,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.972521+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:23:44.972521+0000","last_peered":"2024-09-11T01:23:44.972521+0000","last_clean":"2024-09-11T01:23:44.972521+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:23:44.972521+0000","last_undegraded":"2024-09-11T01:23:44.972521+0000","last_fullsized":"2024-09-11T01:23:44.972521+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20421,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.971608+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:23:44.971608+0000","last_peered":"2024-09-11T01:23:44.971608+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:23:44.971608+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20084,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.969107+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:23:44.969107+0000","last_peered":"2024-09-11T01:23:44.969107+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:23:44.969107+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20407,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.967353+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:23:44.967353+0000","last_peered":"2024-09-11T01:23:44.967353+0000","last_clean":"2024-09-11T01:23:44.967353+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:23:44.967353+0000","last_undegraded":"2024-09-11T01:23:44.967353+0000","last_fullsized":"2024-09-11T01:23:44.967353+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966055+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:23:44.966055+0000","last_peered":"2024-09-11T01:23:44.966055+0000","last_clean":"2024-09-11T01:23:44.966055+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:23:44.966055+0000","last_undegraded":"2024-09-11T01:23:44.966055+0000","last_fullsized":"2024-09-11T01:23:44.966055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28004,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975400+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:23:44.975400+0000","last_peered":"2024-09-11T01:23:44.975400+0000","last_clean":"2024-09-11T01:23:44.975400+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:23:44.975400+0000","last_undegraded":"2024-09-11T01:23:44.975400+0000","last_fullsized":"2024-09-11T01:23:44.975400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20020,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.972355+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:23:44.972355+0000","last_peered":"2024-09-11T01:23:44.972355+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:23:44.972355+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.970279+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:23:44.970279+0000","last_peered":"2024-09-11T01:23:44.970279+0000","last_clean":"2024-09-11T01:23:44.970279+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:23:44.970279+0000","last_undegraded":"2024-09-11T01:23:44.970279+0000","last_fullsized":"2024-09-11T01:23:44.970279+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":27328,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:20.380683+0000","last_change":"2024-09-11T01:23:43.106212+0000","last_active":"2024-09-11T01:24:20.380683+0000","last_peered":"2024-09-11T01:24:20.380683+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:24:20.380683+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":537,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3489,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-537}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966373+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:23:44.966373+0000","last_peered":"2024-09-11T01:23:44.966373+0000","last_clean":"2024-09-11T01:23:44.966373+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:23:44.966373+0000","last_undegraded":"2024-09-11T01:23:44.966373+0000","last_fullsized":"2024-09-11T01:23:44.966373+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27801,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975364+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:23:44.975364+0000","last_peered":"2024-09-11T01:23:44.975364+0000","last_clean":"2024-09-11T01:23:44.975364+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:23:44.975364+0000","last_undegraded":"2024-09-11T01:23:44.975364+0000","last_fullsized":"2024-09-11T01:23:44.975364+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20419,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:45.031607+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:23:45.031607+0000","last_peered":"2024-09-11T01:23:45.031607+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:23:45.031607+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969716+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:23:44.969716+0000","last_peered":"2024-09-11T01:23:44.969716+0000","last_clean":"2024-09-11T01:23:44.969716+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:23:44.969716+0000","last_undegraded":"2024-09-11T01:23:44.969716+0000","last_fullsized":"2024-09-11T01:23:44.969716+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27939,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969676+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:23:44.969676+0000","last_peered":"2024-09-11T01:23:44.969676+0000","last_clean":"2024-09-11T01:23:44.969676+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:23:44.969676+0000","last_undegraded":"2024-09-11T01:23:44.969676+0000","last_fullsized":"2024-09-11T01:23:44.969676+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975642+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:23:44.975642+0000","last_peered":"2024-09-11T01:23:44.975642+0000","last_clean":"2024-09-11T01:23:44.975642+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:23:44.975642+0000","last_undegraded":"2024-09-11T01:23:44.975642+0000","last_fullsized":"2024-09-11T01:23:44.975642+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24617,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969597+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:23:44.969597+0000","last_peered":"2024-09-11T01:23:44.969597+0000","last_clean":"2024-09-11T01:23:44.969597+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:23:44.969597+0000","last_undegraded":"2024-09-11T01:23:44.969597+0000","last_fullsized":"2024-09-11T01:23:44.969597+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969610+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:23:44.969610+0000","last_peered":"2024-09-11T01:23:44.969610+0000","last_clean":"2024-09-11T01:23:44.969610+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:23:44.969610+0000","last_undegraded":"2024-09-11T01:23:44.969610+0000","last_fullsized":"2024-09-11T01:23:44.969610+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966594+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:23:44.966594+0000","last_peered":"2024-09-11T01:23:44.966594+0000","last_clean":"2024-09-11T01:23:44.966594+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:23:44.966594+0000","last_undegraded":"2024-09-11T01:23:44.966594+0000","last_fullsized":"2024-09-11T01:23:44.966594+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20109,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.967341+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:23:44.967341+0000","last_peered":"2024-09-11T01:23:44.967341+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:23:44.967341+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20335,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27304,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195080192,"data_stored":195034965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182554,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518616,"kb_used_data":63560,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853224,"statfs":{"total":96636764160,"available":96105701376,"internally_reserved":0,"allocated":65085440,"data_stored":64773501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986326,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488460,"kb_used_data":127804,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883380,"statfs":{"total":96636764160,"available":96136581120,"internally_reserved":0,"allocated":130871296,"data_stored":130563434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659271,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487488,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":486971,"kb_avail":93884352,"statfs":{"total":96636764160,"available":96137576448,"internally_reserved":0,"allocated":507904,"data_stored":161175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":498658991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:26.386 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:26.735 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:26.736 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:24:26.753 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":451,"stamp":"2024-09-11T01:24:25.132651+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20335,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27305,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":42,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":61,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1494564,"kb_used_data":191860,"kb_used_omap":12,"kb_used_meta":1302643,"kb_avail":281620956,"statfs":{"total":289910292480,"available":288379858944,"internally_reserved":0,"allocated":196464640,"data_stored":195498110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1333906436},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-550,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":550,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.293793"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28519,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:45.031826+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:23:45.031826+0000","last_peered":"2024-09-11T01:23:45.031826+0000","last_clean":"2024-09-11T01:23:45.031826+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:23:45.031826+0000","last_undegraded":"2024-09-11T01:23:45.031826+0000","last_fullsized":"2024-09-11T01:23:45.031826+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20477,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975727+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:23:44.975727+0000","last_peered":"2024-09-11T01:23:44.975727+0000","last_clean":"2024-09-11T01:23:44.975727+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:23:44.975727+0000","last_undegraded":"2024-09-11T01:23:44.975727+0000","last_fullsized":"2024-09-11T01:23:44.975727+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20300,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20227,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975565+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:23:44.975565+0000","last_peered":"2024-09-11T01:23:44.975565+0000","last_clean":"2024-09-11T01:23:44.975565+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:23:44.975565+0000","last_undegraded":"2024-09-11T01:23:44.975565+0000","last_fullsized":"2024-09-11T01:23:44.975565+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20160,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.972521+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:23:44.972521+0000","last_peered":"2024-09-11T01:23:44.972521+0000","last_clean":"2024-09-11T01:23:44.972521+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:23:44.972521+0000","last_undegraded":"2024-09-11T01:23:44.972521+0000","last_fullsized":"2024-09-11T01:23:44.972521+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20421,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.971608+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:23:44.971608+0000","last_peered":"2024-09-11T01:23:44.971608+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:23:44.971608+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20084,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.969107+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:23:44.969107+0000","last_peered":"2024-09-11T01:23:44.969107+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:23:44.969107+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20407,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.967353+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:23:44.967353+0000","last_peered":"2024-09-11T01:23:44.967353+0000","last_clean":"2024-09-11T01:23:44.967353+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:23:44.967353+0000","last_undegraded":"2024-09-11T01:23:44.967353+0000","last_fullsized":"2024-09-11T01:23:44.967353+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966055+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:23:44.966055+0000","last_peered":"2024-09-11T01:23:44.966055+0000","last_clean":"2024-09-11T01:23:44.966055+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:23:44.966055+0000","last_undegraded":"2024-09-11T01:23:44.966055+0000","last_fullsized":"2024-09-11T01:23:44.966055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28004,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975400+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:23:44.975400+0000","last_peered":"2024-09-11T01:23:44.975400+0000","last_clean":"2024-09-11T01:23:44.975400+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:23:44.975400+0000","last_undegraded":"2024-09-11T01:23:44.975400+0000","last_fullsized":"2024-09-11T01:23:44.975400+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20020,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.972355+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:23:44.972355+0000","last_peered":"2024-09-11T01:23:44.972355+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:23:44.972355+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.970279+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:23:44.970279+0000","last_peered":"2024-09-11T01:23:44.970279+0000","last_clean":"2024-09-11T01:23:44.970279+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:23:44.970279+0000","last_undegraded":"2024-09-11T01:23:44.970279+0000","last_fullsized":"2024-09-11T01:23:44.970279+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":27328,"reported_epoch":55,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:20.380683+0000","last_change":"2024-09-11T01:23:43.106212+0000","last_active":"2024-09-11T01:24:20.380683+0000","last_peered":"2024-09-11T01:24:20.380683+0000","last_clean":"2024-09-11T01:13:07.645090+0000","last_became_active":"2024-09-11T01:13:10.198575+0000","last_became_peered":"2024-09-11T01:13:10.198575+0000","last_unstale":"2024-09-11T01:24:20.380683+0000","last_undegraded":"2024-09-11T01:13:09.737420+0000","last_fullsized":"2024-09-11T01:13:09.661982+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"no scrub is scheduled","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":537,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3489,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2],"avail_no_missing":["2","0"],"object_location_counts":[{"shards":"2","objects":-537}],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966373+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:23:44.966373+0000","last_peered":"2024-09-11T01:23:44.966373+0000","last_clean":"2024-09-11T01:23:44.966373+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:23:44.966373+0000","last_undegraded":"2024-09-11T01:23:44.966373+0000","last_fullsized":"2024-09-11T01:23:44.966373+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27801,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975364+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:23:44.975364+0000","last_peered":"2024-09-11T01:23:44.975364+0000","last_clean":"2024-09-11T01:23:44.975364+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:23:44.975364+0000","last_undegraded":"2024-09-11T01:23:44.975364+0000","last_fullsized":"2024-09-11T01:23:44.975364+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20419,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:45.031607+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:23:45.031607+0000","last_peered":"2024-09-11T01:23:45.031607+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:23:45.031607+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969716+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:23:44.969716+0000","last_peered":"2024-09-11T01:23:44.969716+0000","last_clean":"2024-09-11T01:23:44.969716+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:23:44.969716+0000","last_undegraded":"2024-09-11T01:23:44.969716+0000","last_fullsized":"2024-09-11T01:23:44.969716+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27939,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969676+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:23:44.969676+0000","last_peered":"2024-09-11T01:23:44.969676+0000","last_clean":"2024-09-11T01:23:44.969676+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:23:44.969676+0000","last_undegraded":"2024-09-11T01:23:44.969676+0000","last_fullsized":"2024-09-11T01:23:44.969676+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.975642+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:23:44.975642+0000","last_peered":"2024-09-11T01:23:44.975642+0000","last_clean":"2024-09-11T01:23:44.975642+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:23:44.975642+0000","last_undegraded":"2024-09-11T01:23:44.975642+0000","last_fullsized":"2024-09-11T01:23:44.975642+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24617,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969597+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:23:44.969597+0000","last_peered":"2024-09-11T01:23:44.969597+0000","last_clean":"2024-09-11T01:23:44.969597+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:23:44.969597+0000","last_undegraded":"2024-09-11T01:23:44.969597+0000","last_fullsized":"2024-09-11T01:23:44.969597+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969610+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:23:44.969610+0000","last_peered":"2024-09-11T01:23:44.969610+0000","last_clean":"2024-09-11T01:23:44.969610+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:23:44.969610+0000","last_undegraded":"2024-09-11T01:23:44.969610+0000","last_fullsized":"2024-09-11T01:23:44.969610+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966594+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:23:44.966594+0000","last_peered":"2024-09-11T01:23:44.966594+0000","last_clean":"2024-09-11T01:23:44.966594+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:23:44.966594+0000","last_undegraded":"2024-09-11T01:23:44.966594+0000","last_fullsized":"2024-09-11T01:23:44.966594+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20109,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.967341+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:23:44.967341+0000","last_peered":"2024-09-11T01:23:44.967341+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:23:44.967341+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":20335,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27304,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":195080192,"data_stored":195034965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":26,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182554,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518616,"kb_used_data":63560,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853224,"statfs":{"total":96636764160,"available":96105701376,"internally_reserved":0,"allocated":65085440,"data_stored":64773501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986326,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":488460,"kb_used_data":127804,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93883380,"statfs":{"total":96636764160,"available":96136581120,"internally_reserved":0,"allocated":130871296,"data_stored":130563434,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659271,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487488,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":486971,"kb_avail":93884352,"statfs":{"total":96636764160,"available":96137576448,"internally_reserved":0,"allocated":507904,"data_stored":161175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":498658991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":130412544,"data_stored":130409065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:26.755 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:24:26.755 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'20227", 'reported_seq': 20421, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:44.971608+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:23:44.971608+0000', 'last_peered': '2024-09-11T01:23:44.971608+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:23:44.971608+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16568, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:26.755 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:24:26.755 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20084, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:44.969107+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:23:44.969107+0000', 'last_peered': '2024-09-11T01:23:44.969107+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:23:44.969107+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:26.755 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:24:26.755 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20020, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:44.972355+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:23:44.972355+0000', 'last_peered': '2024-09-11T01:23:44.972355+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:23:44.972355+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:26.755 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.1 is not active+clean 2024-09-11T01:24:26.756 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.1', 'version': "49'20207", 'reported_seq': 27328, 'reported_epoch': 55, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:20.380683+0000', 'last_change': '2024-09-11T01:23:43.106212+0000', 'last_active': '2024-09-11T01:24:20.380683+0000', 'last_peered': '2024-09-11T01:24:20.380683+0000', 'last_clean': '2024-09-11T01:13:07.645090+0000', 'last_became_active': '2024-09-11T01:13:10.198575+0000', 'last_became_peered': '2024-09-11T01:13:10.198575+0000', 'last_unstale': '2024-09-11T01:24:20.380683+0000', 'last_undegraded': '2024-09-11T01:13:09.737420+0000', 'last_fullsized': '2024-09-11T01:13:09.661982+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2223", 'last_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:41.868472+0000', 'objects_scrubbed': 2178, 'log_size': 10007, 'log_dups_size': 0, 'ondisk_log_size': 10007, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 1, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 963, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 537, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 18024, 'num_write_kb': 31692, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3489, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2], 'avail_no_missing': ['2', '0'], 'object_location_counts': [{'shards': '2', 'objects': -537}], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-11T01:24:26.756 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:24:26.756 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20264", 'reported_seq': 20419, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:45.031607+0000', 'last_change': '2024-09-11T01:13:10.278546+0000', 'last_active': '2024-09-11T01:23:45.031607+0000', 'last_peered': '2024-09-11T01:23:45.031607+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:23:45.031607+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3951, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17001, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3950}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:26.756 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:24:26.756 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 20109, 'reported_epoch': 55, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:23:44.967341+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:23:44.967341+0000', 'last_peered': '2024-09-11T01:23:44.967341+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:23:44.967341+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:26.756 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:24:27.163 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:27.163 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 19798/6 objects degraded (329966.667%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":803,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":5}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1529724928,"bytes_avail":288380567552,"bytes_total":289910292480,"degraded_objects":19798,"degraded_total":6,"degraded_ratio":3299.6666666666661,"recovering_objects_per_sec":135,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1087,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:24:27.185 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:24:27.185 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:27.983 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:27.983 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:24:28.003 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":452,"stamp":"2024-09-11T01:24:26.842148+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19798,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27842,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1493872,"kb_used_data":191168,"kb_used_omap":12,"kb_used_meta":1302643,"kb_avail":281621648,"statfs":{"total":289910292480,"available":288380567552,"internally_reserved":0,"allocated":195756032,"data_stored":194778807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1333906436},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1087,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1087,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002646"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28519,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:45.031826+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:23:45.031826+0000","last_peered":"2024-09-11T01:23:45.031826+0000","last_clean":"2024-09-11T01:23:45.031826+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:23:45.031826+0000","last_undegraded":"2024-09-11T01:23:45.031826+0000","last_fullsized":"2024-09-11T01:23:45.031826+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20481,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138008+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:24:25.138008+0000","last_peered":"2024-09-11T01:24:25.138008+0000","last_clean":"2024-09-11T01:24:25.138008+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:24:25.138008+0000","last_undegraded":"2024-09-11T01:24:25.138008+0000","last_fullsized":"2024-09-11T01:24:25.138008+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20300,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20231,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.133993+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:24:25.133993+0000","last_peered":"2024-09-11T01:24:25.133993+0000","last_clean":"2024-09-11T01:24:25.133993+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:24:25.133993+0000","last_undegraded":"2024-09-11T01:24:25.133993+0000","last_fullsized":"2024-09-11T01:24:25.133993+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20160,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.972521+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:23:44.972521+0000","last_peered":"2024-09-11T01:23:44.972521+0000","last_clean":"2024-09-11T01:23:44.972521+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:23:44.972521+0000","last_undegraded":"2024-09-11T01:23:44.972521+0000","last_fullsized":"2024-09-11T01:23:44.972521+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20421,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.971608+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:23:44.971608+0000","last_peered":"2024-09-11T01:23:44.971608+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:23:44.971608+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20084,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.969107+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:23:44.969107+0000","last_peered":"2024-09-11T01:23:44.969107+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:23:44.969107+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20407,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.967353+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:23:44.967353+0000","last_peered":"2024-09-11T01:23:44.967353+0000","last_clean":"2024-09-11T01:23:44.967353+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:23:44.967353+0000","last_undegraded":"2024-09-11T01:23:44.967353+0000","last_fullsized":"2024-09-11T01:23:44.967353+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966055+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:23:44.966055+0000","last_peered":"2024-09-11T01:23:44.966055+0000","last_clean":"2024-09-11T01:23:44.966055+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:23:44.966055+0000","last_undegraded":"2024-09-11T01:23:44.966055+0000","last_fullsized":"2024-09-11T01:23:44.966055+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28004,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969165+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:23:44.969165+0000","last_peered":"2024-09-11T01:23:44.969165+0000","last_clean":"2024-09-11T01:23:44.969165+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:23:44.969165+0000","last_undegraded":"2024-09-11T01:23:44.969165+0000","last_fullsized":"2024-09-11T01:23:44.969165+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.134096+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:24:25.134096+0000","last_peered":"2024-09-11T01:24:25.134096+0000","last_clean":"2024-09-11T01:24:25.134096+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:24:25.134096+0000","last_undegraded":"2024-09-11T01:24:25.134096+0000","last_fullsized":"2024-09-11T01:24:25.134096+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20020,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.972355+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:23:44.972355+0000","last_peered":"2024-09-11T01:23:44.972355+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:23:44.972355+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":124,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.970279+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:23:44.970279+0000","last_peered":"2024-09-11T01:23:44.970279+0000","last_clean":"2024-09-11T01:23:44.970279+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:23:44.970279+0000","last_undegraded":"2024-09-11T01:23:44.970279+0000","last_fullsized":"2024-09-11T01:23:44.970279+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28413,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.265719+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:24:25.265719+0000","last_peered":"2024-09-11T01:24:25.265719+0000","last_clean":"2024-09-11T01:24:25.265719+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:24:25.265719+0000","last_undegraded":"2024-09-11T01:24:25.265719+0000","last_fullsized":"2024-09-11T01:24:25.265719+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":103,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966373+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:23:44.966373+0000","last_peered":"2024-09-11T01:23:44.966373+0000","last_clean":"2024-09-11T01:23:44.966373+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:23:44.966373+0000","last_undegraded":"2024-09-11T01:23:44.966373+0000","last_fullsized":"2024-09-11T01:23:44.966373+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27805,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.132991+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:24:25.132991+0000","last_peered":"2024-09-11T01:24:25.132991+0000","last_clean":"2024-09-11T01:24:25.132991+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:24:25.132991+0000","last_undegraded":"2024-09-11T01:24:25.132991+0000","last_fullsized":"2024-09-11T01:24:25.132991+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20419,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:45.031607+0000","last_change":"2024-09-11T01:13:10.278546+0000","last_active":"2024-09-11T01:23:45.031607+0000","last_peered":"2024-09-11T01:23:45.031607+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:23:45.031607+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3951,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3950}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969716+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:23:44.969716+0000","last_peered":"2024-09-11T01:23:44.969716+0000","last_clean":"2024-09-11T01:23:44.969716+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:23:44.969716+0000","last_undegraded":"2024-09-11T01:23:44.969716+0000","last_fullsized":"2024-09-11T01:23:44.969716+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27939,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969676+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:23:44.969676+0000","last_peered":"2024-09-11T01:23:44.969676+0000","last_clean":"2024-09-11T01:23:44.969676+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:23:44.969676+0000","last_undegraded":"2024-09-11T01:23:44.969676+0000","last_fullsized":"2024-09-11T01:23:44.969676+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.137971+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:24:25.137971+0000","last_peered":"2024-09-11T01:24:25.137971+0000","last_clean":"2024-09-11T01:24:25.137971+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:24:25.137971+0000","last_undegraded":"2024-09-11T01:24:25.137971+0000","last_fullsized":"2024-09-11T01:24:25.137971+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24617,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969597+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:23:44.969597+0000","last_peered":"2024-09-11T01:23:44.969597+0000","last_clean":"2024-09-11T01:23:44.969597+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:23:44.969597+0000","last_undegraded":"2024-09-11T01:23:44.969597+0000","last_fullsized":"2024-09-11T01:23:44.969597+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":93,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.969610+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:23:44.969610+0000","last_peered":"2024-09-11T01:23:44.969610+0000","last_clean":"2024-09-11T01:23:44.969610+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:23:44.969610+0000","last_undegraded":"2024-09-11T01:23:44.969610+0000","last_fullsized":"2024-09-11T01:23:44.969610+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":98,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-11T01:23:44.966594+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:23:44.966594+0000","last_peered":"2024-09-11T01:23:44.966594+0000","last_clean":"2024-09-11T01:23:44.966594+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:23:44.966594+0000","last_undegraded":"2024-09-11T01:23:44.966594+0000","last_fullsized":"2024-09-11T01:23:44.966594+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20109,"reported_epoch":55,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:23:44.967341+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:23:44.967341+0000","last_peered":"2024-09-11T01:23:44.967341+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:23:44.967341+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19798,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27841,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":194355200,"data_stored":194309973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182554,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518616,"kb_used_data":63560,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853224,"statfs":{"total":96636764160,"available":96105701376,"internally_reserved":0,"allocated":65085440,"data_stored":64773501,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986327,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487768,"kb_used_data":127112,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93884072,"statfs":{"total":96636764160,"available":96137289728,"internally_reserved":0,"allocated":130162688,"data_stored":129844131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659271,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487488,"kb_used_data":496,"kb_used_omap":4,"kb_used_meta":486971,"kb_avail":93884352,"statfs":{"total":96636764160,"available":96137576448,"internally_reserved":0,"allocated":507904,"data_stored":161175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":498658991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":6787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":129687552,"data_stored":129684073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:31.005 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:31.352 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:31.352 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:24:31.370 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":454,"stamp":"2024-09-11T01:24:30.843179+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19196,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28444,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1496260,"kb_used_data":189396,"kb_used_omap":12,"kb_used_meta":1306803,"kb_avail":281619260,"statfs":{"total":289910292480,"available":288378122240,"internally_reserved":0,"allocated":193941504,"data_stored":192946368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1338166276},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002266"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28523,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139109+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:24:25.139109+0000","last_peered":"2024-09-11T01:24:25.139109+0000","last_clean":"2024-09-11T01:24:25.139109+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:24:25.139109+0000","last_undegraded":"2024-09-11T01:24:25.139109+0000","last_fullsized":"2024-09-11T01:24:25.139109+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20481,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138008+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:24:25.138008+0000","last_peered":"2024-09-11T01:24:25.138008+0000","last_clean":"2024-09-11T01:24:25.138008+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:24:25.138008+0000","last_undegraded":"2024-09-11T01:24:25.138008+0000","last_fullsized":"2024-09-11T01:24:25.138008+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20304,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142521+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:24:25.142521+0000","last_peered":"2024-09-11T01:24:25.142521+0000","last_clean":"2024-09-11T01:24:25.142521+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:24:25.142521+0000","last_undegraded":"2024-09-11T01:24:25.142521+0000","last_fullsized":"2024-09-11T01:24:25.142521+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20231,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.133993+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:24:25.133993+0000","last_peered":"2024-09-11T01:24:25.133993+0000","last_clean":"2024-09-11T01:24:25.133993+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:24:25.133993+0000","last_undegraded":"2024-09-11T01:24:25.133993+0000","last_fullsized":"2024-09-11T01:24:25.133993+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20164,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135797+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:24:25.135797+0000","last_peered":"2024-09-11T01:24:25.135797+0000","last_clean":"2024-09-11T01:24:25.135797+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:24:25.135797+0000","last_undegraded":"2024-09-11T01:24:25.135797+0000","last_fullsized":"2024-09-11T01:24:25.135797+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20425,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135738+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:24:25.135738+0000","last_peered":"2024-09-11T01:24:25.135738+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:24:25.135738+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20088,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135679+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:24:25.135679+0000","last_peered":"2024-09-11T01:24:25.135679+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:24:25.135679+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20411,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138636+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:24:25.138636+0000","last_peered":"2024-09-11T01:24:25.138636+0000","last_clean":"2024-09-11T01:24:25.138636+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:24:25.138636+0000","last_undegraded":"2024-09-11T01:24:25.138636+0000","last_fullsized":"2024-09-11T01:24:25.138636+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139906+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:24:25.139906+0000","last_peered":"2024-09-11T01:24:25.139906+0000","last_clean":"2024-09-11T01:24:25.139906+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:24:25.139906+0000","last_undegraded":"2024-09-11T01:24:25.139906+0000","last_fullsized":"2024-09-11T01:24:25.139906+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28008,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142257+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:24:25.142257+0000","last_peered":"2024-09-11T01:24:25.142257+0000","last_clean":"2024-09-11T01:24:25.142257+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:24:25.142257+0000","last_undegraded":"2024-09-11T01:24:25.142257+0000","last_fullsized":"2024-09-11T01:24:25.142257+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.134096+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:24:25.134096+0000","last_peered":"2024-09-11T01:24:25.134096+0000","last_clean":"2024-09-11T01:24:25.134096+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:24:25.134096+0000","last_undegraded":"2024-09-11T01:24:25.134096+0000","last_fullsized":"2024-09-11T01:24:25.134096+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20024,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.137211+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:24:25.137211+0000","last_peered":"2024-09-11T01:24:25.137211+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:24:25.137211+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":129,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135666+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:24:25.135666+0000","last_peered":"2024-09-11T01:24:25.135666+0000","last_clean":"2024-09-11T01:24:25.135666+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:24:25.135666+0000","last_undegraded":"2024-09-11T01:24:25.135666+0000","last_fullsized":"2024-09-11T01:24:25.135666+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28413,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.265719+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:24:25.265719+0000","last_peered":"2024-09-11T01:24:25.265719+0000","last_clean":"2024-09-11T01:24:25.265719+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:24:25.265719+0000","last_undegraded":"2024-09-11T01:24:25.265719+0000","last_fullsized":"2024-09-11T01:24:25.265719+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142149+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:24:25.142149+0000","last_peered":"2024-09-11T01:24:25.142149+0000","last_clean":"2024-09-11T01:24:25.142149+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:24:25.142149+0000","last_undegraded":"2024-09-11T01:24:25.142149+0000","last_fullsized":"2024-09-11T01:24:25.142149+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27805,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.132991+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:24:25.132991+0000","last_peered":"2024-09-11T01:24:25.132991+0000","last_clean":"2024-09-11T01:24:25.132991+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:24:25.132991+0000","last_undegraded":"2024-09-11T01:24:25.132991+0000","last_fullsized":"2024-09-11T01:24:25.132991+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":21628,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:28.263145+0000","last_change":"2024-09-11T01:24:23.706628+0000","last_active":"2024-09-11T01:24:28.263145+0000","last_peered":"2024-09-11T01:24:28.263145+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:24:28.263145+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3349,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":602,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3349}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142487+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:24:25.142487+0000","last_peered":"2024-09-11T01:24:25.142487+0000","last_clean":"2024-09-11T01:24:25.142487+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:24:25.142487+0000","last_undegraded":"2024-09-11T01:24:25.142487+0000","last_fullsized":"2024-09-11T01:24:25.142487+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27943,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142784+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:24:25.142784+0000","last_peered":"2024-09-11T01:24:25.142784+0000","last_clean":"2024-09-11T01:24:25.142784+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:24:25.142784+0000","last_undegraded":"2024-09-11T01:24:25.142784+0000","last_fullsized":"2024-09-11T01:24:25.142784+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.137971+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:24:25.137971+0000","last_peered":"2024-09-11T01:24:25.137971+0000","last_clean":"2024-09-11T01:24:25.137971+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:24:25.137971+0000","last_undegraded":"2024-09-11T01:24:25.137971+0000","last_fullsized":"2024-09-11T01:24:25.137971+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24621,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142615+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:24:25.142615+0000","last_peered":"2024-09-11T01:24:25.142615+0000","last_clean":"2024-09-11T01:24:25.142615+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:24:25.142615+0000","last_undegraded":"2024-09-11T01:24:25.142615+0000","last_fullsized":"2024-09-11T01:24:25.142615+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142712+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:24:25.142712+0000","last_peered":"2024-09-11T01:24:25.142712+0000","last_clean":"2024-09-11T01:24:25.142712+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:24:25.142712+0000","last_undegraded":"2024-09-11T01:24:25.142712+0000","last_fullsized":"2024-09-11T01:24:25.142712+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142785+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:24:25.142785+0000","last_peered":"2024-09-11T01:24:25.142785+0000","last_clean":"2024-09-11T01:24:25.142785+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:24:25.142785+0000","last_undegraded":"2024-09-11T01:24:25.142785+0000","last_fullsized":"2024-09-11T01:24:25.142785+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20113,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135732+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:24:25.135732+0000","last_peered":"2024-09-11T01:24:25.135732+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:24:25.135732+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19196,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28443,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":192507904,"data_stored":192466156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182555,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518632,"kb_used_data":63576,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853208,"statfs":{"total":96636764160,"available":96105684992,"internally_reserved":0,"allocated":65101824,"data_stored":64779190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986328,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485968,"kb_used_data":125312,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93885872,"statfs":{"total":96636764160,"available":96139132928,"internally_reserved":0,"allocated":128319488,"data_stored":128000931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659272,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491660,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93880180,"statfs":{"total":96636764160,"available":96133304320,"internally_reserved":0,"allocated":520192,"data_stored":166247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502918831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127844352,"data_stored":127840873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:31.372 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:31.718 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:31.719 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:24:31.740 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":454,"stamp":"2024-09-11T01:24:30.843179+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19196,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28444,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1496260,"kb_used_data":189396,"kb_used_omap":12,"kb_used_meta":1306803,"kb_avail":281619260,"statfs":{"total":289910292480,"available":288378122240,"internally_reserved":0,"allocated":193941504,"data_stored":192946368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1338166276},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002266"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28523,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139109+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:24:25.139109+0000","last_peered":"2024-09-11T01:24:25.139109+0000","last_clean":"2024-09-11T01:24:25.139109+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:24:25.139109+0000","last_undegraded":"2024-09-11T01:24:25.139109+0000","last_fullsized":"2024-09-11T01:24:25.139109+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20481,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138008+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:24:25.138008+0000","last_peered":"2024-09-11T01:24:25.138008+0000","last_clean":"2024-09-11T01:24:25.138008+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:24:25.138008+0000","last_undegraded":"2024-09-11T01:24:25.138008+0000","last_fullsized":"2024-09-11T01:24:25.138008+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20304,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142521+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:24:25.142521+0000","last_peered":"2024-09-11T01:24:25.142521+0000","last_clean":"2024-09-11T01:24:25.142521+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:24:25.142521+0000","last_undegraded":"2024-09-11T01:24:25.142521+0000","last_fullsized":"2024-09-11T01:24:25.142521+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20231,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.133993+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:24:25.133993+0000","last_peered":"2024-09-11T01:24:25.133993+0000","last_clean":"2024-09-11T01:24:25.133993+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:24:25.133993+0000","last_undegraded":"2024-09-11T01:24:25.133993+0000","last_fullsized":"2024-09-11T01:24:25.133993+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20164,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135797+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:24:25.135797+0000","last_peered":"2024-09-11T01:24:25.135797+0000","last_clean":"2024-09-11T01:24:25.135797+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:24:25.135797+0000","last_undegraded":"2024-09-11T01:24:25.135797+0000","last_fullsized":"2024-09-11T01:24:25.135797+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20425,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135738+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:24:25.135738+0000","last_peered":"2024-09-11T01:24:25.135738+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:24:25.135738+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20088,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135679+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:24:25.135679+0000","last_peered":"2024-09-11T01:24:25.135679+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:24:25.135679+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20411,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138636+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:24:25.138636+0000","last_peered":"2024-09-11T01:24:25.138636+0000","last_clean":"2024-09-11T01:24:25.138636+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:24:25.138636+0000","last_undegraded":"2024-09-11T01:24:25.138636+0000","last_fullsized":"2024-09-11T01:24:25.138636+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139906+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:24:25.139906+0000","last_peered":"2024-09-11T01:24:25.139906+0000","last_clean":"2024-09-11T01:24:25.139906+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:24:25.139906+0000","last_undegraded":"2024-09-11T01:24:25.139906+0000","last_fullsized":"2024-09-11T01:24:25.139906+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28008,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142257+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:24:25.142257+0000","last_peered":"2024-09-11T01:24:25.142257+0000","last_clean":"2024-09-11T01:24:25.142257+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:24:25.142257+0000","last_undegraded":"2024-09-11T01:24:25.142257+0000","last_fullsized":"2024-09-11T01:24:25.142257+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.134096+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:24:25.134096+0000","last_peered":"2024-09-11T01:24:25.134096+0000","last_clean":"2024-09-11T01:24:25.134096+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:24:25.134096+0000","last_undegraded":"2024-09-11T01:24:25.134096+0000","last_fullsized":"2024-09-11T01:24:25.134096+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20024,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.137211+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:24:25.137211+0000","last_peered":"2024-09-11T01:24:25.137211+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:24:25.137211+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":129,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135666+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:24:25.135666+0000","last_peered":"2024-09-11T01:24:25.135666+0000","last_clean":"2024-09-11T01:24:25.135666+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:24:25.135666+0000","last_undegraded":"2024-09-11T01:24:25.135666+0000","last_fullsized":"2024-09-11T01:24:25.135666+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28413,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.265719+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:24:25.265719+0000","last_peered":"2024-09-11T01:24:25.265719+0000","last_clean":"2024-09-11T01:24:25.265719+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:24:25.265719+0000","last_undegraded":"2024-09-11T01:24:25.265719+0000","last_fullsized":"2024-09-11T01:24:25.265719+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142149+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:24:25.142149+0000","last_peered":"2024-09-11T01:24:25.142149+0000","last_clean":"2024-09-11T01:24:25.142149+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:24:25.142149+0000","last_undegraded":"2024-09-11T01:24:25.142149+0000","last_fullsized":"2024-09-11T01:24:25.142149+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27805,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.132991+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:24:25.132991+0000","last_peered":"2024-09-11T01:24:25.132991+0000","last_clean":"2024-09-11T01:24:25.132991+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:24:25.132991+0000","last_undegraded":"2024-09-11T01:24:25.132991+0000","last_fullsized":"2024-09-11T01:24:25.132991+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":21628,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:28.263145+0000","last_change":"2024-09-11T01:24:23.706628+0000","last_active":"2024-09-11T01:24:28.263145+0000","last_peered":"2024-09-11T01:24:28.263145+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:24:28.263145+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3349,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":602,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3349}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142487+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:24:25.142487+0000","last_peered":"2024-09-11T01:24:25.142487+0000","last_clean":"2024-09-11T01:24:25.142487+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:24:25.142487+0000","last_undegraded":"2024-09-11T01:24:25.142487+0000","last_fullsized":"2024-09-11T01:24:25.142487+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27943,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142784+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:24:25.142784+0000","last_peered":"2024-09-11T01:24:25.142784+0000","last_clean":"2024-09-11T01:24:25.142784+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:24:25.142784+0000","last_undegraded":"2024-09-11T01:24:25.142784+0000","last_fullsized":"2024-09-11T01:24:25.142784+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.137971+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:24:25.137971+0000","last_peered":"2024-09-11T01:24:25.137971+0000","last_clean":"2024-09-11T01:24:25.137971+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:24:25.137971+0000","last_undegraded":"2024-09-11T01:24:25.137971+0000","last_fullsized":"2024-09-11T01:24:25.137971+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24621,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142615+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:24:25.142615+0000","last_peered":"2024-09-11T01:24:25.142615+0000","last_clean":"2024-09-11T01:24:25.142615+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:24:25.142615+0000","last_undegraded":"2024-09-11T01:24:25.142615+0000","last_fullsized":"2024-09-11T01:24:25.142615+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142712+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:24:25.142712+0000","last_peered":"2024-09-11T01:24:25.142712+0000","last_clean":"2024-09-11T01:24:25.142712+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:24:25.142712+0000","last_undegraded":"2024-09-11T01:24:25.142712+0000","last_fullsized":"2024-09-11T01:24:25.142712+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142785+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:24:25.142785+0000","last_peered":"2024-09-11T01:24:25.142785+0000","last_clean":"2024-09-11T01:24:25.142785+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:24:25.142785+0000","last_undegraded":"2024-09-11T01:24:25.142785+0000","last_fullsized":"2024-09-11T01:24:25.142785+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20113,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135732+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:24:25.135732+0000","last_peered":"2024-09-11T01:24:25.135732+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:24:25.135732+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19196,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28443,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":192507904,"data_stored":192466156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182555,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518632,"kb_used_data":63576,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853208,"statfs":{"total":96636764160,"available":96105684992,"internally_reserved":0,"allocated":65101824,"data_stored":64779190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986328,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485968,"kb_used_data":125312,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93885872,"statfs":{"total":96636764160,"available":96139132928,"internally_reserved":0,"allocated":128319488,"data_stored":128000931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659272,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491660,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93880180,"statfs":{"total":96636764160,"available":96133304320,"internally_reserved":0,"allocated":520192,"data_stored":166247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502918831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127844352,"data_stored":127840873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:31.742 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:24:31.742 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'20227", 'reported_seq': 20425, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:25.135738+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:24:25.135738+0000', 'last_peered': '2024-09-11T01:24:25.135738+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:24:25.135738+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16568, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:31.742 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:24:31.742 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20088, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:25.135679+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:24:25.135679+0000', 'last_peered': '2024-09-11T01:24:25.135679+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:24:25.135679+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:31.742 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:24:31.742 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20024, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:25.137211+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:24:25.137211+0000', 'last_peered': '2024-09-11T01:24:25.137211+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:24:25.137211+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:31.743 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:24:31.743 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20264", 'reported_seq': 21628, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:28.263145+0000', 'last_change': '2024-09-11T01:24:23.706628+0000', 'last_active': '2024-09-11T01:24:28.263145+0000', 'last_peered': '2024-09-11T01:24:28.263145+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:24:28.263145+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3349, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17001, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 602, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3349}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:31.743 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:24:31.743 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 20113, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:25.135732+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:24:25.135732+0000', 'last_peered': '2024-09-11T01:24:25.135732+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:24:25.135732+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:31.743 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:24:32.174 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:32.174 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 19196/6 objects degraded (319933.333%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":808,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1532170240,"bytes_avail":288378122240,"bytes_total":289910292480,"degraded_objects":19196,"degraded_total":6,"degraded_ratio":3199.3333333333339,"recovering_objects_per_sec":142,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1139,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:24:32.193 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:24:32.194 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:32.504 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:32.505 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:24:32.523 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":454,"stamp":"2024-09-11T01:24:30.843179+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19196,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28444,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1496260,"kb_used_data":189396,"kb_used_omap":12,"kb_used_meta":1306803,"kb_avail":281619260,"statfs":{"total":289910292480,"available":288378122240,"internally_reserved":0,"allocated":193941504,"data_stored":192946368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1338166276},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1139,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1139,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002266"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28523,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139109+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:24:25.139109+0000","last_peered":"2024-09-11T01:24:25.139109+0000","last_clean":"2024-09-11T01:24:25.139109+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:24:25.139109+0000","last_undegraded":"2024-09-11T01:24:25.139109+0000","last_fullsized":"2024-09-11T01:24:25.139109+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20481,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138008+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:24:25.138008+0000","last_peered":"2024-09-11T01:24:25.138008+0000","last_clean":"2024-09-11T01:24:25.138008+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:24:25.138008+0000","last_undegraded":"2024-09-11T01:24:25.138008+0000","last_fullsized":"2024-09-11T01:24:25.138008+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20304,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142521+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:24:25.142521+0000","last_peered":"2024-09-11T01:24:25.142521+0000","last_clean":"2024-09-11T01:24:25.142521+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:24:25.142521+0000","last_undegraded":"2024-09-11T01:24:25.142521+0000","last_fullsized":"2024-09-11T01:24:25.142521+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20231,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.133993+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:24:25.133993+0000","last_peered":"2024-09-11T01:24:25.133993+0000","last_clean":"2024-09-11T01:24:25.133993+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:24:25.133993+0000","last_undegraded":"2024-09-11T01:24:25.133993+0000","last_fullsized":"2024-09-11T01:24:25.133993+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20164,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135797+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:24:25.135797+0000","last_peered":"2024-09-11T01:24:25.135797+0000","last_clean":"2024-09-11T01:24:25.135797+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:24:25.135797+0000","last_undegraded":"2024-09-11T01:24:25.135797+0000","last_fullsized":"2024-09-11T01:24:25.135797+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20425,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135738+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:24:25.135738+0000","last_peered":"2024-09-11T01:24:25.135738+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:24:25.135738+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20088,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135679+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:24:25.135679+0000","last_peered":"2024-09-11T01:24:25.135679+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:24:25.135679+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20411,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138636+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:24:25.138636+0000","last_peered":"2024-09-11T01:24:25.138636+0000","last_clean":"2024-09-11T01:24:25.138636+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:24:25.138636+0000","last_undegraded":"2024-09-11T01:24:25.138636+0000","last_fullsized":"2024-09-11T01:24:25.138636+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139906+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:24:25.139906+0000","last_peered":"2024-09-11T01:24:25.139906+0000","last_clean":"2024-09-11T01:24:25.139906+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:24:25.139906+0000","last_undegraded":"2024-09-11T01:24:25.139906+0000","last_fullsized":"2024-09-11T01:24:25.139906+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28008,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142257+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:24:25.142257+0000","last_peered":"2024-09-11T01:24:25.142257+0000","last_clean":"2024-09-11T01:24:25.142257+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:24:25.142257+0000","last_undegraded":"2024-09-11T01:24:25.142257+0000","last_fullsized":"2024-09-11T01:24:25.142257+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.134096+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:24:25.134096+0000","last_peered":"2024-09-11T01:24:25.134096+0000","last_clean":"2024-09-11T01:24:25.134096+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:24:25.134096+0000","last_undegraded":"2024-09-11T01:24:25.134096+0000","last_fullsized":"2024-09-11T01:24:25.134096+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20024,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.137211+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:24:25.137211+0000","last_peered":"2024-09-11T01:24:25.137211+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:24:25.137211+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":129,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135666+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:24:25.135666+0000","last_peered":"2024-09-11T01:24:25.135666+0000","last_clean":"2024-09-11T01:24:25.135666+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:24:25.135666+0000","last_undegraded":"2024-09-11T01:24:25.135666+0000","last_fullsized":"2024-09-11T01:24:25.135666+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28413,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.265719+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:24:25.265719+0000","last_peered":"2024-09-11T01:24:25.265719+0000","last_clean":"2024-09-11T01:24:25.265719+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:24:25.265719+0000","last_undegraded":"2024-09-11T01:24:25.265719+0000","last_fullsized":"2024-09-11T01:24:25.265719+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142149+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:24:25.142149+0000","last_peered":"2024-09-11T01:24:25.142149+0000","last_clean":"2024-09-11T01:24:25.142149+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:24:25.142149+0000","last_undegraded":"2024-09-11T01:24:25.142149+0000","last_fullsized":"2024-09-11T01:24:25.142149+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27805,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.132991+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:24:25.132991+0000","last_peered":"2024-09-11T01:24:25.132991+0000","last_clean":"2024-09-11T01:24:25.132991+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:24:25.132991+0000","last_undegraded":"2024-09-11T01:24:25.132991+0000","last_fullsized":"2024-09-11T01:24:25.132991+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":21628,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:28.263145+0000","last_change":"2024-09-11T01:24:23.706628+0000","last_active":"2024-09-11T01:24:28.263145+0000","last_peered":"2024-09-11T01:24:28.263145+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:24:28.263145+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3349,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":602,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3349}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142487+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:24:25.142487+0000","last_peered":"2024-09-11T01:24:25.142487+0000","last_clean":"2024-09-11T01:24:25.142487+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:24:25.142487+0000","last_undegraded":"2024-09-11T01:24:25.142487+0000","last_fullsized":"2024-09-11T01:24:25.142487+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27943,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142784+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:24:25.142784+0000","last_peered":"2024-09-11T01:24:25.142784+0000","last_clean":"2024-09-11T01:24:25.142784+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:24:25.142784+0000","last_undegraded":"2024-09-11T01:24:25.142784+0000","last_fullsized":"2024-09-11T01:24:25.142784+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.137971+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:24:25.137971+0000","last_peered":"2024-09-11T01:24:25.137971+0000","last_clean":"2024-09-11T01:24:25.137971+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:24:25.137971+0000","last_undegraded":"2024-09-11T01:24:25.137971+0000","last_fullsized":"2024-09-11T01:24:25.137971+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24621,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142615+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:24:25.142615+0000","last_peered":"2024-09-11T01:24:25.142615+0000","last_clean":"2024-09-11T01:24:25.142615+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:24:25.142615+0000","last_undegraded":"2024-09-11T01:24:25.142615+0000","last_fullsized":"2024-09-11T01:24:25.142615+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142712+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:24:25.142712+0000","last_peered":"2024-09-11T01:24:25.142712+0000","last_clean":"2024-09-11T01:24:25.142712+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:24:25.142712+0000","last_undegraded":"2024-09-11T01:24:25.142712+0000","last_fullsized":"2024-09-11T01:24:25.142712+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142785+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:24:25.142785+0000","last_peered":"2024-09-11T01:24:25.142785+0000","last_clean":"2024-09-11T01:24:25.142785+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:24:25.142785+0000","last_undegraded":"2024-09-11T01:24:25.142785+0000","last_fullsized":"2024-09-11T01:24:25.142785+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20113,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135732+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:24:25.135732+0000","last_peered":"2024-09-11T01:24:25.135732+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:24:25.135732+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":19196,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28443,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":192507904,"data_stored":192466156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182555,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518632,"kb_used_data":63576,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853208,"statfs":{"total":96636764160,"available":96105684992,"internally_reserved":0,"allocated":65101824,"data_stored":64779190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986328,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485968,"kb_used_data":125312,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93885872,"statfs":{"total":96636764160,"available":96139132928,"internally_reserved":0,"allocated":128319488,"data_stored":128000931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659272,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491660,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93880180,"statfs":{"total":96636764160,"available":96133304320,"internally_reserved":0,"allocated":520192,"data_stored":166247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502918831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127844352,"data_stored":127840873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:35.525 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:35.849 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:35.850 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:24:35.868 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":456,"stamp":"2024-09-11T01:24:34.844283+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18530,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29110,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1487940,"kb_used_data":189396,"kb_used_omap":12,"kb_used_meta":1298483,"kb_avail":281627580,"statfs":{"total":289910292480,"available":288386641920,"internally_reserved":0,"allocated":193941504,"data_stored":192946368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1329646596},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1805,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1805,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002996"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28523,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139109+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:24:25.139109+0000","last_peered":"2024-09-11T01:24:25.139109+0000","last_clean":"2024-09-11T01:24:25.139109+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:24:25.139109+0000","last_undegraded":"2024-09-11T01:24:25.139109+0000","last_fullsized":"2024-09-11T01:24:25.139109+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20481,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138008+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:24:25.138008+0000","last_peered":"2024-09-11T01:24:25.138008+0000","last_clean":"2024-09-11T01:24:25.138008+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:24:25.138008+0000","last_undegraded":"2024-09-11T01:24:25.138008+0000","last_fullsized":"2024-09-11T01:24:25.138008+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20304,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142521+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:24:25.142521+0000","last_peered":"2024-09-11T01:24:25.142521+0000","last_clean":"2024-09-11T01:24:25.142521+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:24:25.142521+0000","last_undegraded":"2024-09-11T01:24:25.142521+0000","last_fullsized":"2024-09-11T01:24:25.142521+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20231,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.133993+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:24:25.133993+0000","last_peered":"2024-09-11T01:24:25.133993+0000","last_clean":"2024-09-11T01:24:25.133993+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:24:25.133993+0000","last_undegraded":"2024-09-11T01:24:25.133993+0000","last_fullsized":"2024-09-11T01:24:25.133993+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20164,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135797+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:24:25.135797+0000","last_peered":"2024-09-11T01:24:25.135797+0000","last_clean":"2024-09-11T01:24:25.135797+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:24:25.135797+0000","last_undegraded":"2024-09-11T01:24:25.135797+0000","last_fullsized":"2024-09-11T01:24:25.135797+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20425,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135738+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:24:25.135738+0000","last_peered":"2024-09-11T01:24:25.135738+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:24:25.135738+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20088,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135679+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:24:25.135679+0000","last_peered":"2024-09-11T01:24:25.135679+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:24:25.135679+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20411,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138636+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:24:25.138636+0000","last_peered":"2024-09-11T01:24:25.138636+0000","last_clean":"2024-09-11T01:24:25.138636+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:24:25.138636+0000","last_undegraded":"2024-09-11T01:24:25.138636+0000","last_fullsized":"2024-09-11T01:24:25.138636+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139906+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:24:25.139906+0000","last_peered":"2024-09-11T01:24:25.139906+0000","last_clean":"2024-09-11T01:24:25.139906+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:24:25.139906+0000","last_undegraded":"2024-09-11T01:24:25.139906+0000","last_fullsized":"2024-09-11T01:24:25.139906+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28009,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142299+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:24:25.142299+0000","last_peered":"2024-09-11T01:24:25.142299+0000","last_clean":"2024-09-11T01:24:25.142299+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:24:25.142299+0000","last_undegraded":"2024-09-11T01:24:25.142299+0000","last_fullsized":"2024-09-11T01:24:25.142299+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.134096+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:24:25.134096+0000","last_peered":"2024-09-11T01:24:25.134096+0000","last_clean":"2024-09-11T01:24:25.134096+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:24:25.134096+0000","last_undegraded":"2024-09-11T01:24:25.134096+0000","last_fullsized":"2024-09-11T01:24:25.134096+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20024,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.137211+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:24:25.137211+0000","last_peered":"2024-09-11T01:24:25.137211+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:24:25.137211+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":129,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135666+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:24:25.135666+0000","last_peered":"2024-09-11T01:24:25.135666+0000","last_clean":"2024-09-11T01:24:25.135666+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:24:25.135666+0000","last_undegraded":"2024-09-11T01:24:25.135666+0000","last_fullsized":"2024-09-11T01:24:25.135666+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28413,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.265719+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:24:25.265719+0000","last_peered":"2024-09-11T01:24:25.265719+0000","last_clean":"2024-09-11T01:24:25.265719+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:24:25.265719+0000","last_undegraded":"2024-09-11T01:24:25.265719+0000","last_fullsized":"2024-09-11T01:24:25.265719+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142149+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:24:25.142149+0000","last_peered":"2024-09-11T01:24:25.142149+0000","last_clean":"2024-09-11T01:24:25.142149+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:24:25.142149+0000","last_undegraded":"2024-09-11T01:24:25.142149+0000","last_fullsized":"2024-09-11T01:24:25.142149+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27805,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.132991+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:24:25.132991+0000","last_peered":"2024-09-11T01:24:25.132991+0000","last_clean":"2024-09-11T01:24:25.132991+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:24:25.132991+0000","last_undegraded":"2024-09-11T01:24:25.132991+0000","last_fullsized":"2024-09-11T01:24:25.132991+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":22960,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:33.258570+0000","last_change":"2024-09-11T01:24:23.706628+0000","last_active":"2024-09-11T01:24:33.258570+0000","last_peered":"2024-09-11T01:24:33.258570+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:24:33.258570+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2683,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2683}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142487+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:24:25.142487+0000","last_peered":"2024-09-11T01:24:25.142487+0000","last_clean":"2024-09-11T01:24:25.142487+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:24:25.142487+0000","last_undegraded":"2024-09-11T01:24:25.142487+0000","last_fullsized":"2024-09-11T01:24:25.142487+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27943,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142784+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:24:25.142784+0000","last_peered":"2024-09-11T01:24:25.142784+0000","last_clean":"2024-09-11T01:24:25.142784+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:24:25.142784+0000","last_undegraded":"2024-09-11T01:24:25.142784+0000","last_fullsized":"2024-09-11T01:24:25.142784+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.137971+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:24:25.137971+0000","last_peered":"2024-09-11T01:24:25.137971+0000","last_clean":"2024-09-11T01:24:25.137971+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:24:25.137971+0000","last_undegraded":"2024-09-11T01:24:25.137971+0000","last_fullsized":"2024-09-11T01:24:25.137971+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24621,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142615+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:24:25.142615+0000","last_peered":"2024-09-11T01:24:25.142615+0000","last_clean":"2024-09-11T01:24:25.142615+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:24:25.142615+0000","last_undegraded":"2024-09-11T01:24:25.142615+0000","last_fullsized":"2024-09-11T01:24:25.142615+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142712+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:24:25.142712+0000","last_peered":"2024-09-11T01:24:25.142712+0000","last_clean":"2024-09-11T01:24:25.142712+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:24:25.142712+0000","last_undegraded":"2024-09-11T01:24:25.142712+0000","last_fullsized":"2024-09-11T01:24:25.142712+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142785+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:24:25.142785+0000","last_peered":"2024-09-11T01:24:25.142785+0000","last_clean":"2024-09-11T01:24:25.142785+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:24:25.142785+0000","last_undegraded":"2024-09-11T01:24:25.142785+0000","last_fullsized":"2024-09-11T01:24:25.142785+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20113,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135732+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:24:25.135732+0000","last_peered":"2024-09-11T01:24:25.135732+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:24:25.135732+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18530,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29109,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":192507904,"data_stored":192466156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182556,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518632,"kb_used_data":63576,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853208,"statfs":{"total":96636764160,"available":96105684992,"internally_reserved":0,"allocated":65101824,"data_stored":64779190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986328,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485968,"kb_used_data":125312,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93885872,"statfs":{"total":96636764160,"available":96139132928,"internally_reserved":0,"allocated":128319488,"data_stored":128000931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659273,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483340,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":482811,"kb_avail":93888500,"statfs":{"total":96636764160,"available":96141824000,"internally_reserved":0,"allocated":520192,"data_stored":166247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494399151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127844352,"data_stored":127840873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:35.869 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:36.200 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:36.201 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:24:36.221 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":456,"stamp":"2024-09-11T01:24:34.844283+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18530,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29110,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1487940,"kb_used_data":189396,"kb_used_omap":12,"kb_used_meta":1298483,"kb_avail":281627580,"statfs":{"total":289910292480,"available":288386641920,"internally_reserved":0,"allocated":193941504,"data_stored":192946368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1329646596},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1805,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1805,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002996"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28523,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139109+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:24:25.139109+0000","last_peered":"2024-09-11T01:24:25.139109+0000","last_clean":"2024-09-11T01:24:25.139109+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:24:25.139109+0000","last_undegraded":"2024-09-11T01:24:25.139109+0000","last_fullsized":"2024-09-11T01:24:25.139109+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20481,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138008+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:24:25.138008+0000","last_peered":"2024-09-11T01:24:25.138008+0000","last_clean":"2024-09-11T01:24:25.138008+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:24:25.138008+0000","last_undegraded":"2024-09-11T01:24:25.138008+0000","last_fullsized":"2024-09-11T01:24:25.138008+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20304,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142521+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:24:25.142521+0000","last_peered":"2024-09-11T01:24:25.142521+0000","last_clean":"2024-09-11T01:24:25.142521+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:24:25.142521+0000","last_undegraded":"2024-09-11T01:24:25.142521+0000","last_fullsized":"2024-09-11T01:24:25.142521+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20231,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.133993+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:24:25.133993+0000","last_peered":"2024-09-11T01:24:25.133993+0000","last_clean":"2024-09-11T01:24:25.133993+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:24:25.133993+0000","last_undegraded":"2024-09-11T01:24:25.133993+0000","last_fullsized":"2024-09-11T01:24:25.133993+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20164,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135797+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:24:25.135797+0000","last_peered":"2024-09-11T01:24:25.135797+0000","last_clean":"2024-09-11T01:24:25.135797+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:24:25.135797+0000","last_undegraded":"2024-09-11T01:24:25.135797+0000","last_fullsized":"2024-09-11T01:24:25.135797+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20425,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135738+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:24:25.135738+0000","last_peered":"2024-09-11T01:24:25.135738+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:24:25.135738+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20088,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135679+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:24:25.135679+0000","last_peered":"2024-09-11T01:24:25.135679+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:24:25.135679+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20411,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138636+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:24:25.138636+0000","last_peered":"2024-09-11T01:24:25.138636+0000","last_clean":"2024-09-11T01:24:25.138636+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:24:25.138636+0000","last_undegraded":"2024-09-11T01:24:25.138636+0000","last_fullsized":"2024-09-11T01:24:25.138636+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139906+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:24:25.139906+0000","last_peered":"2024-09-11T01:24:25.139906+0000","last_clean":"2024-09-11T01:24:25.139906+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:24:25.139906+0000","last_undegraded":"2024-09-11T01:24:25.139906+0000","last_fullsized":"2024-09-11T01:24:25.139906+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28009,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142299+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:24:25.142299+0000","last_peered":"2024-09-11T01:24:25.142299+0000","last_clean":"2024-09-11T01:24:25.142299+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:24:25.142299+0000","last_undegraded":"2024-09-11T01:24:25.142299+0000","last_fullsized":"2024-09-11T01:24:25.142299+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.134096+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:24:25.134096+0000","last_peered":"2024-09-11T01:24:25.134096+0000","last_clean":"2024-09-11T01:24:25.134096+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:24:25.134096+0000","last_undegraded":"2024-09-11T01:24:25.134096+0000","last_fullsized":"2024-09-11T01:24:25.134096+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20024,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.137211+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:24:25.137211+0000","last_peered":"2024-09-11T01:24:25.137211+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:24:25.137211+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":129,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135666+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:24:25.135666+0000","last_peered":"2024-09-11T01:24:25.135666+0000","last_clean":"2024-09-11T01:24:25.135666+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:24:25.135666+0000","last_undegraded":"2024-09-11T01:24:25.135666+0000","last_fullsized":"2024-09-11T01:24:25.135666+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28413,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.265719+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:24:25.265719+0000","last_peered":"2024-09-11T01:24:25.265719+0000","last_clean":"2024-09-11T01:24:25.265719+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:24:25.265719+0000","last_undegraded":"2024-09-11T01:24:25.265719+0000","last_fullsized":"2024-09-11T01:24:25.265719+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142149+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:24:25.142149+0000","last_peered":"2024-09-11T01:24:25.142149+0000","last_clean":"2024-09-11T01:24:25.142149+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:24:25.142149+0000","last_undegraded":"2024-09-11T01:24:25.142149+0000","last_fullsized":"2024-09-11T01:24:25.142149+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27805,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.132991+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:24:25.132991+0000","last_peered":"2024-09-11T01:24:25.132991+0000","last_clean":"2024-09-11T01:24:25.132991+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:24:25.132991+0000","last_undegraded":"2024-09-11T01:24:25.132991+0000","last_fullsized":"2024-09-11T01:24:25.132991+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":22960,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:33.258570+0000","last_change":"2024-09-11T01:24:23.706628+0000","last_active":"2024-09-11T01:24:33.258570+0000","last_peered":"2024-09-11T01:24:33.258570+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:24:33.258570+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2683,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2683}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142487+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:24:25.142487+0000","last_peered":"2024-09-11T01:24:25.142487+0000","last_clean":"2024-09-11T01:24:25.142487+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:24:25.142487+0000","last_undegraded":"2024-09-11T01:24:25.142487+0000","last_fullsized":"2024-09-11T01:24:25.142487+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27943,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142784+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:24:25.142784+0000","last_peered":"2024-09-11T01:24:25.142784+0000","last_clean":"2024-09-11T01:24:25.142784+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:24:25.142784+0000","last_undegraded":"2024-09-11T01:24:25.142784+0000","last_fullsized":"2024-09-11T01:24:25.142784+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.137971+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:24:25.137971+0000","last_peered":"2024-09-11T01:24:25.137971+0000","last_clean":"2024-09-11T01:24:25.137971+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:24:25.137971+0000","last_undegraded":"2024-09-11T01:24:25.137971+0000","last_fullsized":"2024-09-11T01:24:25.137971+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24621,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142615+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:24:25.142615+0000","last_peered":"2024-09-11T01:24:25.142615+0000","last_clean":"2024-09-11T01:24:25.142615+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:24:25.142615+0000","last_undegraded":"2024-09-11T01:24:25.142615+0000","last_fullsized":"2024-09-11T01:24:25.142615+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142712+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:24:25.142712+0000","last_peered":"2024-09-11T01:24:25.142712+0000","last_clean":"2024-09-11T01:24:25.142712+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:24:25.142712+0000","last_undegraded":"2024-09-11T01:24:25.142712+0000","last_fullsized":"2024-09-11T01:24:25.142712+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142785+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:24:25.142785+0000","last_peered":"2024-09-11T01:24:25.142785+0000","last_clean":"2024-09-11T01:24:25.142785+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:24:25.142785+0000","last_undegraded":"2024-09-11T01:24:25.142785+0000","last_fullsized":"2024-09-11T01:24:25.142785+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20113,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135732+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:24:25.135732+0000","last_peered":"2024-09-11T01:24:25.135732+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:24:25.135732+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18530,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29109,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":192507904,"data_stored":192466156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182556,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518632,"kb_used_data":63576,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853208,"statfs":{"total":96636764160,"available":96105684992,"internally_reserved":0,"allocated":65101824,"data_stored":64779190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986328,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":485968,"kb_used_data":125312,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93885872,"statfs":{"total":96636764160,"available":96139132928,"internally_reserved":0,"allocated":128319488,"data_stored":128000931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659273,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483340,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":482811,"kb_avail":93888500,"statfs":{"total":96636764160,"available":96141824000,"internally_reserved":0,"allocated":520192,"data_stored":166247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494399151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":127844352,"data_stored":127840873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:36.222 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:24:36.223 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'20227", 'reported_seq': 20425, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:25.135738+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:24:25.135738+0000', 'last_peered': '2024-09-11T01:24:25.135738+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:24:25.135738+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16568, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:36.223 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:24:36.223 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20088, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:25.135679+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:24:25.135679+0000', 'last_peered': '2024-09-11T01:24:25.135679+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:24:25.135679+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:36.223 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:24:36.223 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20024, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:25.137211+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:24:25.137211+0000', 'last_peered': '2024-09-11T01:24:25.137211+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:24:25.137211+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:36.223 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:24:36.223 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20264", 'reported_seq': 22960, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:33.258570+0000', 'last_change': '2024-09-11T01:24:23.706628+0000', 'last_active': '2024-09-11T01:24:33.258570+0000', 'last_peered': '2024-09-11T01:24:33.258570+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:24:33.258570+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2683, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17001, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1268, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2683}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:36.223 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:24:36.223 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 20113, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:25.135732+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:24:25.135732+0000', 'last_peered': '2024-09-11T01:24:25.135732+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:24:25.135732+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:36.224 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:24:36.602 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:36.602 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 18530/6 objects degraded (308833.333%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":813,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1523650560,"bytes_avail":288386641920,"bytes_total":289910292480,"degraded_objects":18530,"degraded_total":6,"degraded_ratio":3088.3333333333339,"recovering_objects_per_sec":180,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1805,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:24:36.623 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:24:36.623 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:36.962 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:36.963 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:24:36.981 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":457,"stamp":"2024-09-11T01:24:36.844936+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18530,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29110,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1486140,"kb_used_data":187596,"kb_used_omap":12,"kb_used_meta":1298483,"kb_avail":281629380,"statfs":{"total":289910292480,"available":288388485120,"internally_reserved":0,"allocated":192098304,"data_stored":191103168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1329646596},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1805,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1805,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.712285"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28523,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139109+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:24:25.139109+0000","last_peered":"2024-09-11T01:24:25.139109+0000","last_clean":"2024-09-11T01:24:25.139109+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:24:25.139109+0000","last_undegraded":"2024-09-11T01:24:25.139109+0000","last_fullsized":"2024-09-11T01:24:25.139109+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20481,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138008+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:24:25.138008+0000","last_peered":"2024-09-11T01:24:25.138008+0000","last_clean":"2024-09-11T01:24:25.138008+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:24:25.138008+0000","last_undegraded":"2024-09-11T01:24:25.138008+0000","last_fullsized":"2024-09-11T01:24:25.138008+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20304,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142521+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:24:25.142521+0000","last_peered":"2024-09-11T01:24:25.142521+0000","last_clean":"2024-09-11T01:24:25.142521+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:24:25.142521+0000","last_undegraded":"2024-09-11T01:24:25.142521+0000","last_fullsized":"2024-09-11T01:24:25.142521+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20231,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.133993+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:24:25.133993+0000","last_peered":"2024-09-11T01:24:25.133993+0000","last_clean":"2024-09-11T01:24:25.133993+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:24:25.133993+0000","last_undegraded":"2024-09-11T01:24:25.133993+0000","last_fullsized":"2024-09-11T01:24:25.133993+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20164,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135797+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:24:25.135797+0000","last_peered":"2024-09-11T01:24:25.135797+0000","last_clean":"2024-09-11T01:24:25.135797+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:24:25.135797+0000","last_undegraded":"2024-09-11T01:24:25.135797+0000","last_fullsized":"2024-09-11T01:24:25.135797+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20425,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135738+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:24:25.135738+0000","last_peered":"2024-09-11T01:24:25.135738+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:24:25.135738+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20088,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135679+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:24:25.135679+0000","last_peered":"2024-09-11T01:24:25.135679+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:24:25.135679+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20411,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138636+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:24:25.138636+0000","last_peered":"2024-09-11T01:24:25.138636+0000","last_clean":"2024-09-11T01:24:25.138636+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:24:25.138636+0000","last_undegraded":"2024-09-11T01:24:25.138636+0000","last_fullsized":"2024-09-11T01:24:25.138636+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139906+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:24:25.139906+0000","last_peered":"2024-09-11T01:24:25.139906+0000","last_clean":"2024-09-11T01:24:25.139906+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:24:25.139906+0000","last_undegraded":"2024-09-11T01:24:25.139906+0000","last_fullsized":"2024-09-11T01:24:25.139906+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28009,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142299+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:24:25.142299+0000","last_peered":"2024-09-11T01:24:25.142299+0000","last_clean":"2024-09-11T01:24:25.142299+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:24:25.142299+0000","last_undegraded":"2024-09-11T01:24:25.142299+0000","last_fullsized":"2024-09-11T01:24:25.142299+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.134096+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:24:25.134096+0000","last_peered":"2024-09-11T01:24:25.134096+0000","last_clean":"2024-09-11T01:24:25.134096+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:24:25.134096+0000","last_undegraded":"2024-09-11T01:24:25.134096+0000","last_fullsized":"2024-09-11T01:24:25.134096+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20024,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.137211+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:24:25.137211+0000","last_peered":"2024-09-11T01:24:25.137211+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:24:25.137211+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":129,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135666+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:24:25.135666+0000","last_peered":"2024-09-11T01:24:25.135666+0000","last_clean":"2024-09-11T01:24:25.135666+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:24:25.135666+0000","last_undegraded":"2024-09-11T01:24:25.135666+0000","last_fullsized":"2024-09-11T01:24:25.135666+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28413,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.265719+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:24:25.265719+0000","last_peered":"2024-09-11T01:24:25.265719+0000","last_clean":"2024-09-11T01:24:25.265719+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:24:25.265719+0000","last_undegraded":"2024-09-11T01:24:25.265719+0000","last_fullsized":"2024-09-11T01:24:25.265719+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142149+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:24:25.142149+0000","last_peered":"2024-09-11T01:24:25.142149+0000","last_clean":"2024-09-11T01:24:25.142149+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:24:25.142149+0000","last_undegraded":"2024-09-11T01:24:25.142149+0000","last_fullsized":"2024-09-11T01:24:25.142149+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27805,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.132991+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:24:25.132991+0000","last_peered":"2024-09-11T01:24:25.132991+0000","last_clean":"2024-09-11T01:24:25.132991+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:24:25.132991+0000","last_undegraded":"2024-09-11T01:24:25.132991+0000","last_fullsized":"2024-09-11T01:24:25.132991+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":22960,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:33.258570+0000","last_change":"2024-09-11T01:24:23.706628+0000","last_active":"2024-09-11T01:24:33.258570+0000","last_peered":"2024-09-11T01:24:33.258570+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:24:33.258570+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2683,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1268,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2683}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142487+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:24:25.142487+0000","last_peered":"2024-09-11T01:24:25.142487+0000","last_clean":"2024-09-11T01:24:25.142487+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:24:25.142487+0000","last_undegraded":"2024-09-11T01:24:25.142487+0000","last_fullsized":"2024-09-11T01:24:25.142487+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27943,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142784+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:24:25.142784+0000","last_peered":"2024-09-11T01:24:25.142784+0000","last_clean":"2024-09-11T01:24:25.142784+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:24:25.142784+0000","last_undegraded":"2024-09-11T01:24:25.142784+0000","last_fullsized":"2024-09-11T01:24:25.142784+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.137971+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:24:25.137971+0000","last_peered":"2024-09-11T01:24:25.137971+0000","last_clean":"2024-09-11T01:24:25.137971+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:24:25.137971+0000","last_undegraded":"2024-09-11T01:24:25.137971+0000","last_fullsized":"2024-09-11T01:24:25.137971+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24621,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142615+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:24:25.142615+0000","last_peered":"2024-09-11T01:24:25.142615+0000","last_clean":"2024-09-11T01:24:25.142615+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:24:25.142615+0000","last_undegraded":"2024-09-11T01:24:25.142615+0000","last_fullsized":"2024-09-11T01:24:25.142615+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142712+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:24:25.142712+0000","last_peered":"2024-09-11T01:24:25.142712+0000","last_clean":"2024-09-11T01:24:25.142712+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:24:25.142712+0000","last_undegraded":"2024-09-11T01:24:25.142712+0000","last_fullsized":"2024-09-11T01:24:25.142712+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142785+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:24:25.142785+0000","last_peered":"2024-09-11T01:24:25.142785+0000","last_clean":"2024-09-11T01:24:25.142785+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:24:25.142785+0000","last_undegraded":"2024-09-11T01:24:25.142785+0000","last_fullsized":"2024-09-11T01:24:25.142785+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20113,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135732+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:24:25.135732+0000","last_peered":"2024-09-11T01:24:25.135732+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:24:25.135732+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18530,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29109,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":190664704,"data_stored":190622956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182556,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518632,"kb_used_data":63576,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853208,"statfs":{"total":96636764160,"available":96105684992,"internally_reserved":0,"allocated":65101824,"data_stored":64779190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986329,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":484168,"kb_used_data":123512,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93887672,"statfs":{"total":96636764160,"available":96140976128,"internally_reserved":0,"allocated":126476288,"data_stored":126157731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659273,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483340,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":482811,"kb_avail":93888500,"statfs":{"total":96636764160,"available":96141824000,"internally_reserved":0,"allocated":520192,"data_stored":166247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494399151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":126001152,"data_stored":125997673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:39.983 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:40.354 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:40.355 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:24:40.375 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":458,"stamp":"2024-09-11T01:24:38.845470+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18194,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29446,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1490300,"kb_used_data":187596,"kb_used_omap":12,"kb_used_meta":1302643,"kb_avail":281625220,"statfs":{"total":289910292480,"available":288384225280,"internally_reserved":0,"allocated":192098304,"data_stored":191103168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1333906436},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1604,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1604,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003321"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28523,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139109+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:24:25.139109+0000","last_peered":"2024-09-11T01:24:25.139109+0000","last_clean":"2024-09-11T01:24:25.139109+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:24:25.139109+0000","last_undegraded":"2024-09-11T01:24:25.139109+0000","last_fullsized":"2024-09-11T01:24:25.139109+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20481,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138008+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:24:25.138008+0000","last_peered":"2024-09-11T01:24:25.138008+0000","last_clean":"2024-09-11T01:24:25.138008+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:24:25.138008+0000","last_undegraded":"2024-09-11T01:24:25.138008+0000","last_fullsized":"2024-09-11T01:24:25.138008+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20304,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142521+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:24:25.142521+0000","last_peered":"2024-09-11T01:24:25.142521+0000","last_clean":"2024-09-11T01:24:25.142521+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:24:25.142521+0000","last_undegraded":"2024-09-11T01:24:25.142521+0000","last_fullsized":"2024-09-11T01:24:25.142521+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20231,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.133993+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:24:25.133993+0000","last_peered":"2024-09-11T01:24:25.133993+0000","last_clean":"2024-09-11T01:24:25.133993+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:24:25.133993+0000","last_undegraded":"2024-09-11T01:24:25.133993+0000","last_fullsized":"2024-09-11T01:24:25.133993+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20164,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135797+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:24:25.135797+0000","last_peered":"2024-09-11T01:24:25.135797+0000","last_clean":"2024-09-11T01:24:25.135797+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:24:25.135797+0000","last_undegraded":"2024-09-11T01:24:25.135797+0000","last_fullsized":"2024-09-11T01:24:25.135797+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20425,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135738+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:24:25.135738+0000","last_peered":"2024-09-11T01:24:25.135738+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:24:25.135738+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20088,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135679+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:24:25.135679+0000","last_peered":"2024-09-11T01:24:25.135679+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:24:25.135679+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20411,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138636+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:24:25.138636+0000","last_peered":"2024-09-11T01:24:25.138636+0000","last_clean":"2024-09-11T01:24:25.138636+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:24:25.138636+0000","last_undegraded":"2024-09-11T01:24:25.138636+0000","last_fullsized":"2024-09-11T01:24:25.138636+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139906+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:24:25.139906+0000","last_peered":"2024-09-11T01:24:25.139906+0000","last_clean":"2024-09-11T01:24:25.139906+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:24:25.139906+0000","last_undegraded":"2024-09-11T01:24:25.139906+0000","last_fullsized":"2024-09-11T01:24:25.139906+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28009,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142299+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:24:25.142299+0000","last_peered":"2024-09-11T01:24:25.142299+0000","last_clean":"2024-09-11T01:24:25.142299+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:24:25.142299+0000","last_undegraded":"2024-09-11T01:24:25.142299+0000","last_fullsized":"2024-09-11T01:24:25.142299+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.134096+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:24:25.134096+0000","last_peered":"2024-09-11T01:24:25.134096+0000","last_clean":"2024-09-11T01:24:25.134096+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:24:25.134096+0000","last_undegraded":"2024-09-11T01:24:25.134096+0000","last_fullsized":"2024-09-11T01:24:25.134096+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20024,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.137211+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:24:25.137211+0000","last_peered":"2024-09-11T01:24:25.137211+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:24:25.137211+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":129,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135666+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:24:25.135666+0000","last_peered":"2024-09-11T01:24:25.135666+0000","last_clean":"2024-09-11T01:24:25.135666+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:24:25.135666+0000","last_undegraded":"2024-09-11T01:24:25.135666+0000","last_fullsized":"2024-09-11T01:24:25.135666+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28413,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.265719+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:24:25.265719+0000","last_peered":"2024-09-11T01:24:25.265719+0000","last_clean":"2024-09-11T01:24:25.265719+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:24:25.265719+0000","last_undegraded":"2024-09-11T01:24:25.265719+0000","last_fullsized":"2024-09-11T01:24:25.265719+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142149+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:24:25.142149+0000","last_peered":"2024-09-11T01:24:25.142149+0000","last_clean":"2024-09-11T01:24:25.142149+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:24:25.142149+0000","last_undegraded":"2024-09-11T01:24:25.142149+0000","last_fullsized":"2024-09-11T01:24:25.142149+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27805,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.132991+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:24:25.132991+0000","last_peered":"2024-09-11T01:24:25.132991+0000","last_clean":"2024-09-11T01:24:25.132991+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:24:25.132991+0000","last_undegraded":"2024-09-11T01:24:25.132991+0000","last_fullsized":"2024-09-11T01:24:25.132991+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":23632,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:38.259606+0000","last_change":"2024-09-11T01:24:23.706628+0000","last_active":"2024-09-11T01:24:38.259606+0000","last_peered":"2024-09-11T01:24:38.259606+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:24:38.259606+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1604,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2347}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142487+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:24:25.142487+0000","last_peered":"2024-09-11T01:24:25.142487+0000","last_clean":"2024-09-11T01:24:25.142487+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:24:25.142487+0000","last_undegraded":"2024-09-11T01:24:25.142487+0000","last_fullsized":"2024-09-11T01:24:25.142487+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27943,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142784+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:24:25.142784+0000","last_peered":"2024-09-11T01:24:25.142784+0000","last_clean":"2024-09-11T01:24:25.142784+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:24:25.142784+0000","last_undegraded":"2024-09-11T01:24:25.142784+0000","last_fullsized":"2024-09-11T01:24:25.142784+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.137971+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:24:25.137971+0000","last_peered":"2024-09-11T01:24:25.137971+0000","last_clean":"2024-09-11T01:24:25.137971+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:24:25.137971+0000","last_undegraded":"2024-09-11T01:24:25.137971+0000","last_fullsized":"2024-09-11T01:24:25.137971+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24621,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142615+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:24:25.142615+0000","last_peered":"2024-09-11T01:24:25.142615+0000","last_clean":"2024-09-11T01:24:25.142615+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:24:25.142615+0000","last_undegraded":"2024-09-11T01:24:25.142615+0000","last_fullsized":"2024-09-11T01:24:25.142615+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142712+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:24:25.142712+0000","last_peered":"2024-09-11T01:24:25.142712+0000","last_clean":"2024-09-11T01:24:25.142712+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:24:25.142712+0000","last_undegraded":"2024-09-11T01:24:25.142712+0000","last_fullsized":"2024-09-11T01:24:25.142712+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142785+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:24:25.142785+0000","last_peered":"2024-09-11T01:24:25.142785+0000","last_clean":"2024-09-11T01:24:25.142785+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:24:25.142785+0000","last_undegraded":"2024-09-11T01:24:25.142785+0000","last_fullsized":"2024-09-11T01:24:25.142785+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20114,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.137564+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:24:25.137564+0000","last_peered":"2024-09-11T01:24:25.137564+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:24:25.137564+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18194,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29445,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":190664704,"data_stored":190622956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182557,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518632,"kb_used_data":63576,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853208,"statfs":{"total":96636764160,"available":96105684992,"internally_reserved":0,"allocated":65101824,"data_stored":64779190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986329,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":484168,"kb_used_data":123512,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93887672,"statfs":{"total":96636764160,"available":96140976128,"internally_reserved":0,"allocated":126476288,"data_stored":126157731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659274,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487500,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":486971,"kb_avail":93884340,"statfs":{"total":96636764160,"available":96137564160,"internally_reserved":0,"allocated":520192,"data_stored":166247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":498658991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":126001152,"data_stored":125997673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:40.377 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:40.718 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:40.719 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:24:40.736 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":458,"stamp":"2024-09-11T01:24:38.845470+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18194,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29446,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1490300,"kb_used_data":187596,"kb_used_omap":12,"kb_used_meta":1302643,"kb_avail":281625220,"statfs":{"total":289910292480,"available":288384225280,"internally_reserved":0,"allocated":192098304,"data_stored":191103168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1333906436},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1604,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1604,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003321"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28523,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139109+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:24:25.139109+0000","last_peered":"2024-09-11T01:24:25.139109+0000","last_clean":"2024-09-11T01:24:25.139109+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:24:25.139109+0000","last_undegraded":"2024-09-11T01:24:25.139109+0000","last_fullsized":"2024-09-11T01:24:25.139109+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20481,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138008+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:24:25.138008+0000","last_peered":"2024-09-11T01:24:25.138008+0000","last_clean":"2024-09-11T01:24:25.138008+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:24:25.138008+0000","last_undegraded":"2024-09-11T01:24:25.138008+0000","last_fullsized":"2024-09-11T01:24:25.138008+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20304,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142521+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:24:25.142521+0000","last_peered":"2024-09-11T01:24:25.142521+0000","last_clean":"2024-09-11T01:24:25.142521+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:24:25.142521+0000","last_undegraded":"2024-09-11T01:24:25.142521+0000","last_fullsized":"2024-09-11T01:24:25.142521+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20231,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.133993+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:24:25.133993+0000","last_peered":"2024-09-11T01:24:25.133993+0000","last_clean":"2024-09-11T01:24:25.133993+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:24:25.133993+0000","last_undegraded":"2024-09-11T01:24:25.133993+0000","last_fullsized":"2024-09-11T01:24:25.133993+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20164,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135797+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:24:25.135797+0000","last_peered":"2024-09-11T01:24:25.135797+0000","last_clean":"2024-09-11T01:24:25.135797+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:24:25.135797+0000","last_undegraded":"2024-09-11T01:24:25.135797+0000","last_fullsized":"2024-09-11T01:24:25.135797+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20425,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135738+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:24:25.135738+0000","last_peered":"2024-09-11T01:24:25.135738+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:24:25.135738+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20088,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135679+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:24:25.135679+0000","last_peered":"2024-09-11T01:24:25.135679+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:24:25.135679+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20411,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138636+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:24:25.138636+0000","last_peered":"2024-09-11T01:24:25.138636+0000","last_clean":"2024-09-11T01:24:25.138636+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:24:25.138636+0000","last_undegraded":"2024-09-11T01:24:25.138636+0000","last_fullsized":"2024-09-11T01:24:25.138636+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139906+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:24:25.139906+0000","last_peered":"2024-09-11T01:24:25.139906+0000","last_clean":"2024-09-11T01:24:25.139906+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:24:25.139906+0000","last_undegraded":"2024-09-11T01:24:25.139906+0000","last_fullsized":"2024-09-11T01:24:25.139906+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28009,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142299+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:24:25.142299+0000","last_peered":"2024-09-11T01:24:25.142299+0000","last_clean":"2024-09-11T01:24:25.142299+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:24:25.142299+0000","last_undegraded":"2024-09-11T01:24:25.142299+0000","last_fullsized":"2024-09-11T01:24:25.142299+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.134096+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:24:25.134096+0000","last_peered":"2024-09-11T01:24:25.134096+0000","last_clean":"2024-09-11T01:24:25.134096+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:24:25.134096+0000","last_undegraded":"2024-09-11T01:24:25.134096+0000","last_fullsized":"2024-09-11T01:24:25.134096+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20024,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.137211+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:24:25.137211+0000","last_peered":"2024-09-11T01:24:25.137211+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:24:25.137211+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":129,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135666+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:24:25.135666+0000","last_peered":"2024-09-11T01:24:25.135666+0000","last_clean":"2024-09-11T01:24:25.135666+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:24:25.135666+0000","last_undegraded":"2024-09-11T01:24:25.135666+0000","last_fullsized":"2024-09-11T01:24:25.135666+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28413,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.265719+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:24:25.265719+0000","last_peered":"2024-09-11T01:24:25.265719+0000","last_clean":"2024-09-11T01:24:25.265719+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:24:25.265719+0000","last_undegraded":"2024-09-11T01:24:25.265719+0000","last_fullsized":"2024-09-11T01:24:25.265719+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142149+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:24:25.142149+0000","last_peered":"2024-09-11T01:24:25.142149+0000","last_clean":"2024-09-11T01:24:25.142149+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:24:25.142149+0000","last_undegraded":"2024-09-11T01:24:25.142149+0000","last_fullsized":"2024-09-11T01:24:25.142149+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27805,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.132991+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:24:25.132991+0000","last_peered":"2024-09-11T01:24:25.132991+0000","last_clean":"2024-09-11T01:24:25.132991+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:24:25.132991+0000","last_undegraded":"2024-09-11T01:24:25.132991+0000","last_fullsized":"2024-09-11T01:24:25.132991+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":23632,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:38.259606+0000","last_change":"2024-09-11T01:24:23.706628+0000","last_active":"2024-09-11T01:24:38.259606+0000","last_peered":"2024-09-11T01:24:38.259606+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:24:38.259606+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1604,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2347}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142487+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:24:25.142487+0000","last_peered":"2024-09-11T01:24:25.142487+0000","last_clean":"2024-09-11T01:24:25.142487+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:24:25.142487+0000","last_undegraded":"2024-09-11T01:24:25.142487+0000","last_fullsized":"2024-09-11T01:24:25.142487+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27943,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142784+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:24:25.142784+0000","last_peered":"2024-09-11T01:24:25.142784+0000","last_clean":"2024-09-11T01:24:25.142784+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:24:25.142784+0000","last_undegraded":"2024-09-11T01:24:25.142784+0000","last_fullsized":"2024-09-11T01:24:25.142784+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.137971+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:24:25.137971+0000","last_peered":"2024-09-11T01:24:25.137971+0000","last_clean":"2024-09-11T01:24:25.137971+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:24:25.137971+0000","last_undegraded":"2024-09-11T01:24:25.137971+0000","last_fullsized":"2024-09-11T01:24:25.137971+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24621,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142615+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:24:25.142615+0000","last_peered":"2024-09-11T01:24:25.142615+0000","last_clean":"2024-09-11T01:24:25.142615+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:24:25.142615+0000","last_undegraded":"2024-09-11T01:24:25.142615+0000","last_fullsized":"2024-09-11T01:24:25.142615+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142712+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:24:25.142712+0000","last_peered":"2024-09-11T01:24:25.142712+0000","last_clean":"2024-09-11T01:24:25.142712+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:24:25.142712+0000","last_undegraded":"2024-09-11T01:24:25.142712+0000","last_fullsized":"2024-09-11T01:24:25.142712+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142785+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:24:25.142785+0000","last_peered":"2024-09-11T01:24:25.142785+0000","last_clean":"2024-09-11T01:24:25.142785+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:24:25.142785+0000","last_undegraded":"2024-09-11T01:24:25.142785+0000","last_fullsized":"2024-09-11T01:24:25.142785+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20114,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.137564+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:24:25.137564+0000","last_peered":"2024-09-11T01:24:25.137564+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:24:25.137564+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18194,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29445,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":190664704,"data_stored":190622956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182557,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518632,"kb_used_data":63576,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853208,"statfs":{"total":96636764160,"available":96105684992,"internally_reserved":0,"allocated":65101824,"data_stored":64779190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986329,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":484168,"kb_used_data":123512,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93887672,"statfs":{"total":96636764160,"available":96140976128,"internally_reserved":0,"allocated":126476288,"data_stored":126157731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659274,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487500,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":486971,"kb_avail":93884340,"statfs":{"total":96636764160,"available":96137564160,"internally_reserved":0,"allocated":520192,"data_stored":166247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":498658991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":126001152,"data_stored":125997673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:40.738 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:24:40.738 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'20227", 'reported_seq': 20425, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:25.135738+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:24:25.135738+0000', 'last_peered': '2024-09-11T01:24:25.135738+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:24:25.135738+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16568, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:40.738 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:24:40.738 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20088, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:25.135679+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:24:25.135679+0000', 'last_peered': '2024-09-11T01:24:25.135679+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:24:25.135679+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:40.738 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:24:40.738 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20024, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:25.137211+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:24:25.137211+0000', 'last_peered': '2024-09-11T01:24:25.137211+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:24:25.137211+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:40.738 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:24:40.738 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20264", 'reported_seq': 23632, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:38.259606+0000', 'last_change': '2024-09-11T01:24:23.706628+0000', 'last_active': '2024-09-11T01:24:38.259606+0000', 'last_peered': '2024-09-11T01:24:38.259606+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:24:38.259606+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2347, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17001, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1604, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2347}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:40.738 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:24:40.739 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 20114, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:25.137564+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:24:25.137564+0000', 'last_peered': '2024-09-11T01:24:25.137564+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:24:25.137564+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:40.739 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:24:41.116 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:41.117 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 18194/6 objects degraded (303233.333%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":817,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1524224000,"bytes_avail":288386068480,"bytes_total":289910292480,"degraded_objects":18194,"degraded_total":6,"degraded_ratio":3032.3333333333339,"recovering_objects_per_sec":83,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1002,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:24:41.135 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:24:41.136 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:41.457 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:41.458 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:24:41.476 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":459,"stamp":"2024-09-11T01:24:40.846147+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18194,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29446,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1488500,"kb_used_data":185796,"kb_used_omap":12,"kb_used_meta":1302643,"kb_avail":281627020,"statfs":{"total":289910292480,"available":288386068480,"internally_reserved":0,"allocated":190255104,"data_stored":189259968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1333906436},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1002,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1002,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003575"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28523,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139109+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:24:25.139109+0000","last_peered":"2024-09-11T01:24:25.139109+0000","last_clean":"2024-09-11T01:24:25.139109+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:24:25.139109+0000","last_undegraded":"2024-09-11T01:24:25.139109+0000","last_fullsized":"2024-09-11T01:24:25.139109+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20481,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138008+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:24:25.138008+0000","last_peered":"2024-09-11T01:24:25.138008+0000","last_clean":"2024-09-11T01:24:25.138008+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:24:25.138008+0000","last_undegraded":"2024-09-11T01:24:25.138008+0000","last_fullsized":"2024-09-11T01:24:25.138008+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20304,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142521+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:24:25.142521+0000","last_peered":"2024-09-11T01:24:25.142521+0000","last_clean":"2024-09-11T01:24:25.142521+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:24:25.142521+0000","last_undegraded":"2024-09-11T01:24:25.142521+0000","last_fullsized":"2024-09-11T01:24:25.142521+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20231,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.133993+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:24:25.133993+0000","last_peered":"2024-09-11T01:24:25.133993+0000","last_clean":"2024-09-11T01:24:25.133993+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:24:25.133993+0000","last_undegraded":"2024-09-11T01:24:25.133993+0000","last_fullsized":"2024-09-11T01:24:25.133993+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20164,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135797+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:24:25.135797+0000","last_peered":"2024-09-11T01:24:25.135797+0000","last_clean":"2024-09-11T01:24:25.135797+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:24:25.135797+0000","last_undegraded":"2024-09-11T01:24:25.135797+0000","last_fullsized":"2024-09-11T01:24:25.135797+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20425,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135738+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:24:25.135738+0000","last_peered":"2024-09-11T01:24:25.135738+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:24:25.135738+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20088,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135679+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:24:25.135679+0000","last_peered":"2024-09-11T01:24:25.135679+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:24:25.135679+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20411,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138636+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:24:25.138636+0000","last_peered":"2024-09-11T01:24:25.138636+0000","last_clean":"2024-09-11T01:24:25.138636+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:24:25.138636+0000","last_undegraded":"2024-09-11T01:24:25.138636+0000","last_fullsized":"2024-09-11T01:24:25.138636+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139906+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:24:25.139906+0000","last_peered":"2024-09-11T01:24:25.139906+0000","last_clean":"2024-09-11T01:24:25.139906+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:24:25.139906+0000","last_undegraded":"2024-09-11T01:24:25.139906+0000","last_fullsized":"2024-09-11T01:24:25.139906+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28009,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142299+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:24:25.142299+0000","last_peered":"2024-09-11T01:24:25.142299+0000","last_clean":"2024-09-11T01:24:25.142299+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:24:25.142299+0000","last_undegraded":"2024-09-11T01:24:25.142299+0000","last_fullsized":"2024-09-11T01:24:25.142299+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.134130+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:24:25.134130+0000","last_peered":"2024-09-11T01:24:25.134130+0000","last_clean":"2024-09-11T01:24:25.134130+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:24:25.134130+0000","last_undegraded":"2024-09-11T01:24:25.134130+0000","last_fullsized":"2024-09-11T01:24:25.134130+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20024,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.137211+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:24:25.137211+0000","last_peered":"2024-09-11T01:24:25.137211+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:24:25.137211+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":129,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135666+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:24:25.135666+0000","last_peered":"2024-09-11T01:24:25.135666+0000","last_clean":"2024-09-11T01:24:25.135666+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:24:25.135666+0000","last_undegraded":"2024-09-11T01:24:25.135666+0000","last_fullsized":"2024-09-11T01:24:25.135666+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28414,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.266137+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:24:25.266137+0000","last_peered":"2024-09-11T01:24:25.266137+0000","last_clean":"2024-09-11T01:24:25.266137+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:24:25.266137+0000","last_undegraded":"2024-09-11T01:24:25.266137+0000","last_fullsized":"2024-09-11T01:24:25.266137+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142149+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:24:25.142149+0000","last_peered":"2024-09-11T01:24:25.142149+0000","last_clean":"2024-09-11T01:24:25.142149+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:24:25.142149+0000","last_undegraded":"2024-09-11T01:24:25.142149+0000","last_fullsized":"2024-09-11T01:24:25.142149+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27805,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.132991+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:24:25.132991+0000","last_peered":"2024-09-11T01:24:25.132991+0000","last_clean":"2024-09-11T01:24:25.132991+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:24:25.132991+0000","last_undegraded":"2024-09-11T01:24:25.132991+0000","last_fullsized":"2024-09-11T01:24:25.132991+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":23632,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:38.259606+0000","last_change":"2024-09-11T01:24:23.706628+0000","last_active":"2024-09-11T01:24:38.259606+0000","last_peered":"2024-09-11T01:24:38.259606+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:24:38.259606+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1604,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2347}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142487+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:24:25.142487+0000","last_peered":"2024-09-11T01:24:25.142487+0000","last_clean":"2024-09-11T01:24:25.142487+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:24:25.142487+0000","last_undegraded":"2024-09-11T01:24:25.142487+0000","last_fullsized":"2024-09-11T01:24:25.142487+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27943,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142784+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:24:25.142784+0000","last_peered":"2024-09-11T01:24:25.142784+0000","last_clean":"2024-09-11T01:24:25.142784+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:24:25.142784+0000","last_undegraded":"2024-09-11T01:24:25.142784+0000","last_fullsized":"2024-09-11T01:24:25.142784+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138018+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:24:25.138018+0000","last_peered":"2024-09-11T01:24:25.138018+0000","last_clean":"2024-09-11T01:24:25.138018+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:24:25.138018+0000","last_undegraded":"2024-09-11T01:24:25.138018+0000","last_fullsized":"2024-09-11T01:24:25.138018+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24621,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142615+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:24:25.142615+0000","last_peered":"2024-09-11T01:24:25.142615+0000","last_clean":"2024-09-11T01:24:25.142615+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:24:25.142615+0000","last_undegraded":"2024-09-11T01:24:25.142615+0000","last_fullsized":"2024-09-11T01:24:25.142615+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142712+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:24:25.142712+0000","last_peered":"2024-09-11T01:24:25.142712+0000","last_clean":"2024-09-11T01:24:25.142712+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:24:25.142712+0000","last_undegraded":"2024-09-11T01:24:25.142712+0000","last_fullsized":"2024-09-11T01:24:25.142712+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142785+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:24:25.142785+0000","last_peered":"2024-09-11T01:24:25.142785+0000","last_clean":"2024-09-11T01:24:25.142785+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:24:25.142785+0000","last_undegraded":"2024-09-11T01:24:25.142785+0000","last_fullsized":"2024-09-11T01:24:25.142785+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20114,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.137564+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:24:25.137564+0000","last_peered":"2024-09-11T01:24:25.137564+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:24:25.137564+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18194,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29445,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":188821504,"data_stored":188779756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182557,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518632,"kb_used_data":63576,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853208,"statfs":{"total":96636764160,"available":96105684992,"internally_reserved":0,"allocated":65101824,"data_stored":64779190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986330,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482368,"kb_used_data":121712,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93889472,"statfs":{"total":96636764160,"available":96142819328,"internally_reserved":0,"allocated":124633088,"data_stored":124314531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659274,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487500,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":486971,"kb_avail":93884340,"statfs":{"total":96636764160,"available":96137564160,"internally_reserved":0,"allocated":520192,"data_stored":166247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":498658991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":124157952,"data_stored":124154473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:44.478 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:44.807 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:44.807 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:24:44.826 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":460,"stamp":"2024-09-11T01:24:42.846519+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18194,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29446,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1488500,"kb_used_data":185796,"kb_used_omap":12,"kb_used_meta":1302643,"kb_avail":281627020,"statfs":{"total":289910292480,"available":288386068480,"internally_reserved":0,"allocated":190255104,"data_stored":189259968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1333906436},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1002,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1002,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003340"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28523,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139109+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:24:25.139109+0000","last_peered":"2024-09-11T01:24:25.139109+0000","last_clean":"2024-09-11T01:24:25.139109+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:24:25.139109+0000","last_undegraded":"2024-09-11T01:24:25.139109+0000","last_fullsized":"2024-09-11T01:24:25.139109+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20481,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138008+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:24:25.138008+0000","last_peered":"2024-09-11T01:24:25.138008+0000","last_clean":"2024-09-11T01:24:25.138008+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:24:25.138008+0000","last_undegraded":"2024-09-11T01:24:25.138008+0000","last_fullsized":"2024-09-11T01:24:25.138008+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20304,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142521+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:24:25.142521+0000","last_peered":"2024-09-11T01:24:25.142521+0000","last_clean":"2024-09-11T01:24:25.142521+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:24:25.142521+0000","last_undegraded":"2024-09-11T01:24:25.142521+0000","last_fullsized":"2024-09-11T01:24:25.142521+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20231,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.133993+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:24:25.133993+0000","last_peered":"2024-09-11T01:24:25.133993+0000","last_clean":"2024-09-11T01:24:25.133993+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:24:25.133993+0000","last_undegraded":"2024-09-11T01:24:25.133993+0000","last_fullsized":"2024-09-11T01:24:25.133993+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20164,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135797+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:24:25.135797+0000","last_peered":"2024-09-11T01:24:25.135797+0000","last_clean":"2024-09-11T01:24:25.135797+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:24:25.135797+0000","last_undegraded":"2024-09-11T01:24:25.135797+0000","last_fullsized":"2024-09-11T01:24:25.135797+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20425,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135738+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:24:25.135738+0000","last_peered":"2024-09-11T01:24:25.135738+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:24:25.135738+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20088,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135679+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:24:25.135679+0000","last_peered":"2024-09-11T01:24:25.135679+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:24:25.135679+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20411,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138636+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:24:25.138636+0000","last_peered":"2024-09-11T01:24:25.138636+0000","last_clean":"2024-09-11T01:24:25.138636+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:24:25.138636+0000","last_undegraded":"2024-09-11T01:24:25.138636+0000","last_fullsized":"2024-09-11T01:24:25.138636+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139906+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:24:25.139906+0000","last_peered":"2024-09-11T01:24:25.139906+0000","last_clean":"2024-09-11T01:24:25.139906+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:24:25.139906+0000","last_undegraded":"2024-09-11T01:24:25.139906+0000","last_fullsized":"2024-09-11T01:24:25.139906+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28009,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142299+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:24:25.142299+0000","last_peered":"2024-09-11T01:24:25.142299+0000","last_clean":"2024-09-11T01:24:25.142299+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:24:25.142299+0000","last_undegraded":"2024-09-11T01:24:25.142299+0000","last_fullsized":"2024-09-11T01:24:25.142299+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.134130+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:24:25.134130+0000","last_peered":"2024-09-11T01:24:25.134130+0000","last_clean":"2024-09-11T01:24:25.134130+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:24:25.134130+0000","last_undegraded":"2024-09-11T01:24:25.134130+0000","last_fullsized":"2024-09-11T01:24:25.134130+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20024,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.137211+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:24:25.137211+0000","last_peered":"2024-09-11T01:24:25.137211+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:24:25.137211+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":129,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135666+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:24:25.135666+0000","last_peered":"2024-09-11T01:24:25.135666+0000","last_clean":"2024-09-11T01:24:25.135666+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:24:25.135666+0000","last_undegraded":"2024-09-11T01:24:25.135666+0000","last_fullsized":"2024-09-11T01:24:25.135666+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28414,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.266137+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:24:25.266137+0000","last_peered":"2024-09-11T01:24:25.266137+0000","last_clean":"2024-09-11T01:24:25.266137+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:24:25.266137+0000","last_undegraded":"2024-09-11T01:24:25.266137+0000","last_fullsized":"2024-09-11T01:24:25.266137+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142149+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:24:25.142149+0000","last_peered":"2024-09-11T01:24:25.142149+0000","last_clean":"2024-09-11T01:24:25.142149+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:24:25.142149+0000","last_undegraded":"2024-09-11T01:24:25.142149+0000","last_fullsized":"2024-09-11T01:24:25.142149+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27805,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.132991+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:24:25.132991+0000","last_peered":"2024-09-11T01:24:25.132991+0000","last_clean":"2024-09-11T01:24:25.132991+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:24:25.132991+0000","last_undegraded":"2024-09-11T01:24:25.132991+0000","last_fullsized":"2024-09-11T01:24:25.132991+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":23632,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:38.259606+0000","last_change":"2024-09-11T01:24:23.706628+0000","last_active":"2024-09-11T01:24:38.259606+0000","last_peered":"2024-09-11T01:24:38.259606+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:24:38.259606+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2347,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1604,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2347}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142487+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:24:25.142487+0000","last_peered":"2024-09-11T01:24:25.142487+0000","last_clean":"2024-09-11T01:24:25.142487+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:24:25.142487+0000","last_undegraded":"2024-09-11T01:24:25.142487+0000","last_fullsized":"2024-09-11T01:24:25.142487+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27943,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142784+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:24:25.142784+0000","last_peered":"2024-09-11T01:24:25.142784+0000","last_clean":"2024-09-11T01:24:25.142784+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:24:25.142784+0000","last_undegraded":"2024-09-11T01:24:25.142784+0000","last_fullsized":"2024-09-11T01:24:25.142784+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138018+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:24:25.138018+0000","last_peered":"2024-09-11T01:24:25.138018+0000","last_clean":"2024-09-11T01:24:25.138018+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:24:25.138018+0000","last_undegraded":"2024-09-11T01:24:25.138018+0000","last_fullsized":"2024-09-11T01:24:25.138018+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24621,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142615+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:24:25.142615+0000","last_peered":"2024-09-11T01:24:25.142615+0000","last_clean":"2024-09-11T01:24:25.142615+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:24:25.142615+0000","last_undegraded":"2024-09-11T01:24:25.142615+0000","last_fullsized":"2024-09-11T01:24:25.142615+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142712+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:24:25.142712+0000","last_peered":"2024-09-11T01:24:25.142712+0000","last_clean":"2024-09-11T01:24:25.142712+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:24:25.142712+0000","last_undegraded":"2024-09-11T01:24:25.142712+0000","last_fullsized":"2024-09-11T01:24:25.142712+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142785+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:24:25.142785+0000","last_peered":"2024-09-11T01:24:25.142785+0000","last_clean":"2024-09-11T01:24:25.142785+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:24:25.142785+0000","last_undegraded":"2024-09-11T01:24:25.142785+0000","last_fullsized":"2024-09-11T01:24:25.142785+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20114,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.137564+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:24:25.137564+0000","last_peered":"2024-09-11T01:24:25.137564+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:24:25.137564+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":18194,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29445,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":188821504,"data_stored":188779756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182557,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518632,"kb_used_data":63576,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853208,"statfs":{"total":96636764160,"available":96105684992,"internally_reserved":0,"allocated":65101824,"data_stored":64779190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986330,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482368,"kb_used_data":121712,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93889472,"statfs":{"total":96636764160,"available":96142819328,"internally_reserved":0,"allocated":124633088,"data_stored":124314531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659274,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487500,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":486971,"kb_avail":93884340,"statfs":{"total":96636764160,"available":96137564160,"internally_reserved":0,"allocated":520192,"data_stored":166247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":498658991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":124157952,"data_stored":124154473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:44.827 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:45.132 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:45.132 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:24:45.149 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":461,"stamp":"2024-09-11T01:24:44.847217+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17832,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29808,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1488500,"kb_used_data":185796,"kb_used_omap":12,"kb_used_meta":1302643,"kb_avail":281627020,"statfs":{"total":289910292480,"available":288386068480,"internally_reserved":0,"allocated":190255104,"data_stored":189259968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1333906436},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1364,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1364,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003540"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28523,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139109+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:24:25.139109+0000","last_peered":"2024-09-11T01:24:25.139109+0000","last_clean":"2024-09-11T01:24:25.139109+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:24:25.139109+0000","last_undegraded":"2024-09-11T01:24:25.139109+0000","last_fullsized":"2024-09-11T01:24:25.139109+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20481,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138008+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:24:25.138008+0000","last_peered":"2024-09-11T01:24:25.138008+0000","last_clean":"2024-09-11T01:24:25.138008+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:24:25.138008+0000","last_undegraded":"2024-09-11T01:24:25.138008+0000","last_fullsized":"2024-09-11T01:24:25.138008+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20304,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142521+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:24:25.142521+0000","last_peered":"2024-09-11T01:24:25.142521+0000","last_clean":"2024-09-11T01:24:25.142521+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:24:25.142521+0000","last_undegraded":"2024-09-11T01:24:25.142521+0000","last_fullsized":"2024-09-11T01:24:25.142521+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20231,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.133993+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:24:25.133993+0000","last_peered":"2024-09-11T01:24:25.133993+0000","last_clean":"2024-09-11T01:24:25.133993+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:24:25.133993+0000","last_undegraded":"2024-09-11T01:24:25.133993+0000","last_fullsized":"2024-09-11T01:24:25.133993+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20164,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135797+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:24:25.135797+0000","last_peered":"2024-09-11T01:24:25.135797+0000","last_clean":"2024-09-11T01:24:25.135797+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:24:25.135797+0000","last_undegraded":"2024-09-11T01:24:25.135797+0000","last_fullsized":"2024-09-11T01:24:25.135797+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20425,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135738+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:24:25.135738+0000","last_peered":"2024-09-11T01:24:25.135738+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:24:25.135738+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20088,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135679+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:24:25.135679+0000","last_peered":"2024-09-11T01:24:25.135679+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:24:25.135679+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20411,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138636+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:24:25.138636+0000","last_peered":"2024-09-11T01:24:25.138636+0000","last_clean":"2024-09-11T01:24:25.138636+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:24:25.138636+0000","last_undegraded":"2024-09-11T01:24:25.138636+0000","last_fullsized":"2024-09-11T01:24:25.138636+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139906+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:24:25.139906+0000","last_peered":"2024-09-11T01:24:25.139906+0000","last_clean":"2024-09-11T01:24:25.139906+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:24:25.139906+0000","last_undegraded":"2024-09-11T01:24:25.139906+0000","last_fullsized":"2024-09-11T01:24:25.139906+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28009,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142299+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:24:25.142299+0000","last_peered":"2024-09-11T01:24:25.142299+0000","last_clean":"2024-09-11T01:24:25.142299+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:24:25.142299+0000","last_undegraded":"2024-09-11T01:24:25.142299+0000","last_fullsized":"2024-09-11T01:24:25.142299+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.134130+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:24:25.134130+0000","last_peered":"2024-09-11T01:24:25.134130+0000","last_clean":"2024-09-11T01:24:25.134130+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:24:25.134130+0000","last_undegraded":"2024-09-11T01:24:25.134130+0000","last_fullsized":"2024-09-11T01:24:25.134130+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20024,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.137211+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:24:25.137211+0000","last_peered":"2024-09-11T01:24:25.137211+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:24:25.137211+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":129,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135666+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:24:25.135666+0000","last_peered":"2024-09-11T01:24:25.135666+0000","last_clean":"2024-09-11T01:24:25.135666+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:24:25.135666+0000","last_undegraded":"2024-09-11T01:24:25.135666+0000","last_fullsized":"2024-09-11T01:24:25.135666+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28414,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.266137+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:24:25.266137+0000","last_peered":"2024-09-11T01:24:25.266137+0000","last_clean":"2024-09-11T01:24:25.266137+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:24:25.266137+0000","last_undegraded":"2024-09-11T01:24:25.266137+0000","last_fullsized":"2024-09-11T01:24:25.266137+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142149+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:24:25.142149+0000","last_peered":"2024-09-11T01:24:25.142149+0000","last_clean":"2024-09-11T01:24:25.142149+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:24:25.142149+0000","last_undegraded":"2024-09-11T01:24:25.142149+0000","last_fullsized":"2024-09-11T01:24:25.142149+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27805,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.132991+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:24:25.132991+0000","last_peered":"2024-09-11T01:24:25.132991+0000","last_clean":"2024-09-11T01:24:25.132991+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:24:25.132991+0000","last_undegraded":"2024-09-11T01:24:25.132991+0000","last_fullsized":"2024-09-11T01:24:25.132991+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":24356,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:43.257010+0000","last_change":"2024-09-11T01:24:23.706628+0000","last_active":"2024-09-11T01:24:43.257010+0000","last_peered":"2024-09-11T01:24:43.257010+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:24:43.257010+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1985,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1966,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1985}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142487+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:24:25.142487+0000","last_peered":"2024-09-11T01:24:25.142487+0000","last_clean":"2024-09-11T01:24:25.142487+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:24:25.142487+0000","last_undegraded":"2024-09-11T01:24:25.142487+0000","last_fullsized":"2024-09-11T01:24:25.142487+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27943,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142784+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:24:25.142784+0000","last_peered":"2024-09-11T01:24:25.142784+0000","last_clean":"2024-09-11T01:24:25.142784+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:24:25.142784+0000","last_undegraded":"2024-09-11T01:24:25.142784+0000","last_fullsized":"2024-09-11T01:24:25.142784+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138018+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:24:25.138018+0000","last_peered":"2024-09-11T01:24:25.138018+0000","last_clean":"2024-09-11T01:24:25.138018+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:24:25.138018+0000","last_undegraded":"2024-09-11T01:24:25.138018+0000","last_fullsized":"2024-09-11T01:24:25.138018+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24622,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142656+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:24:25.142656+0000","last_peered":"2024-09-11T01:24:25.142656+0000","last_clean":"2024-09-11T01:24:25.142656+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:24:25.142656+0000","last_undegraded":"2024-09-11T01:24:25.142656+0000","last_fullsized":"2024-09-11T01:24:25.142656+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142712+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:24:25.142712+0000","last_peered":"2024-09-11T01:24:25.142712+0000","last_clean":"2024-09-11T01:24:25.142712+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:24:25.142712+0000","last_undegraded":"2024-09-11T01:24:25.142712+0000","last_fullsized":"2024-09-11T01:24:25.142712+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142785+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:24:25.142785+0000","last_peered":"2024-09-11T01:24:25.142785+0000","last_clean":"2024-09-11T01:24:25.142785+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:24:25.142785+0000","last_undegraded":"2024-09-11T01:24:25.142785+0000","last_fullsized":"2024-09-11T01:24:25.142785+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20114,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.137564+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:24:25.137564+0000","last_peered":"2024-09-11T01:24:25.137564+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:24:25.137564+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17832,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29807,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":188821504,"data_stored":188779756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182558,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518632,"kb_used_data":63576,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853208,"statfs":{"total":96636764160,"available":96105684992,"internally_reserved":0,"allocated":65101824,"data_stored":64779190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986330,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482368,"kb_used_data":121712,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93889472,"statfs":{"total":96636764160,"available":96142819328,"internally_reserved":0,"allocated":124633088,"data_stored":124314531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659275,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487500,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":486971,"kb_avail":93884340,"statfs":{"total":96636764160,"available":96137564160,"internally_reserved":0,"allocated":520192,"data_stored":166247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":498658991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":124157952,"data_stored":124154473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:45.151 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:24:45.151 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'20227", 'reported_seq': 20425, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:25.135738+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:24:25.135738+0000', 'last_peered': '2024-09-11T01:24:25.135738+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:24:25.135738+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16568, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:45.151 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:24:45.151 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20088, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:25.135679+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:24:25.135679+0000', 'last_peered': '2024-09-11T01:24:25.135679+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:24:25.135679+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:45.151 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:24:45.151 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20024, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:25.137211+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:24:25.137211+0000', 'last_peered': '2024-09-11T01:24:25.137211+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:24:25.137211+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:45.151 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:24:45.151 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20264", 'reported_seq': 24356, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:43.257010+0000', 'last_change': '2024-09-11T01:24:23.706628+0000', 'last_active': '2024-09-11T01:24:43.257010+0000', 'last_peered': '2024-09-11T01:24:43.257010+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:24:43.257010+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1985, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17001, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1966, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1985}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:45.151 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:24:45.152 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 20114, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:25.137564+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:24:25.137564+0000', 'last_peered': '2024-09-11T01:24:25.137564+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:24:25.137564+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:45.152 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:24:45.564 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:45.564 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 17832/6 objects degraded (297200.000%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":822,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1524224000,"bytes_avail":288386068480,"bytes_total":289910292480,"degraded_objects":17832,"degraded_total":6,"degraded_ratio":2972,"recovering_objects_per_sec":113,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1364,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:24:45.583 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:24:45.583 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:45.891 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:45.892 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:24:45.910 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":461,"stamp":"2024-09-11T01:24:44.847217+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17832,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29808,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1488500,"kb_used_data":185796,"kb_used_omap":12,"kb_used_meta":1302643,"kb_avail":281627020,"statfs":{"total":289910292480,"available":288386068480,"internally_reserved":0,"allocated":190255104,"data_stored":189259968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1333906436},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1364,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1364,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003540"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28523,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139109+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:24:25.139109+0000","last_peered":"2024-09-11T01:24:25.139109+0000","last_clean":"2024-09-11T01:24:25.139109+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:24:25.139109+0000","last_undegraded":"2024-09-11T01:24:25.139109+0000","last_fullsized":"2024-09-11T01:24:25.139109+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20481,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138008+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:24:25.138008+0000","last_peered":"2024-09-11T01:24:25.138008+0000","last_clean":"2024-09-11T01:24:25.138008+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:24:25.138008+0000","last_undegraded":"2024-09-11T01:24:25.138008+0000","last_fullsized":"2024-09-11T01:24:25.138008+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20304,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142521+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:24:25.142521+0000","last_peered":"2024-09-11T01:24:25.142521+0000","last_clean":"2024-09-11T01:24:25.142521+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:24:25.142521+0000","last_undegraded":"2024-09-11T01:24:25.142521+0000","last_fullsized":"2024-09-11T01:24:25.142521+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20231,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.133993+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:24:25.133993+0000","last_peered":"2024-09-11T01:24:25.133993+0000","last_clean":"2024-09-11T01:24:25.133993+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:24:25.133993+0000","last_undegraded":"2024-09-11T01:24:25.133993+0000","last_fullsized":"2024-09-11T01:24:25.133993+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20164,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135797+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:24:25.135797+0000","last_peered":"2024-09-11T01:24:25.135797+0000","last_clean":"2024-09-11T01:24:25.135797+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:24:25.135797+0000","last_undegraded":"2024-09-11T01:24:25.135797+0000","last_fullsized":"2024-09-11T01:24:25.135797+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20425,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135738+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:24:25.135738+0000","last_peered":"2024-09-11T01:24:25.135738+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:24:25.135738+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20088,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135679+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:24:25.135679+0000","last_peered":"2024-09-11T01:24:25.135679+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:24:25.135679+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20411,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138636+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:24:25.138636+0000","last_peered":"2024-09-11T01:24:25.138636+0000","last_clean":"2024-09-11T01:24:25.138636+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:24:25.138636+0000","last_undegraded":"2024-09-11T01:24:25.138636+0000","last_fullsized":"2024-09-11T01:24:25.138636+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139906+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:24:25.139906+0000","last_peered":"2024-09-11T01:24:25.139906+0000","last_clean":"2024-09-11T01:24:25.139906+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:24:25.139906+0000","last_undegraded":"2024-09-11T01:24:25.139906+0000","last_fullsized":"2024-09-11T01:24:25.139906+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28009,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142299+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:24:25.142299+0000","last_peered":"2024-09-11T01:24:25.142299+0000","last_clean":"2024-09-11T01:24:25.142299+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:24:25.142299+0000","last_undegraded":"2024-09-11T01:24:25.142299+0000","last_fullsized":"2024-09-11T01:24:25.142299+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.134130+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:24:25.134130+0000","last_peered":"2024-09-11T01:24:25.134130+0000","last_clean":"2024-09-11T01:24:25.134130+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:24:25.134130+0000","last_undegraded":"2024-09-11T01:24:25.134130+0000","last_fullsized":"2024-09-11T01:24:25.134130+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20024,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.137211+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:24:25.137211+0000","last_peered":"2024-09-11T01:24:25.137211+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:24:25.137211+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":129,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135666+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:24:25.135666+0000","last_peered":"2024-09-11T01:24:25.135666+0000","last_clean":"2024-09-11T01:24:25.135666+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:24:25.135666+0000","last_undegraded":"2024-09-11T01:24:25.135666+0000","last_fullsized":"2024-09-11T01:24:25.135666+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28414,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.266137+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:24:25.266137+0000","last_peered":"2024-09-11T01:24:25.266137+0000","last_clean":"2024-09-11T01:24:25.266137+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:24:25.266137+0000","last_undegraded":"2024-09-11T01:24:25.266137+0000","last_fullsized":"2024-09-11T01:24:25.266137+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142149+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:24:25.142149+0000","last_peered":"2024-09-11T01:24:25.142149+0000","last_clean":"2024-09-11T01:24:25.142149+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:24:25.142149+0000","last_undegraded":"2024-09-11T01:24:25.142149+0000","last_fullsized":"2024-09-11T01:24:25.142149+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27805,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.132991+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:24:25.132991+0000","last_peered":"2024-09-11T01:24:25.132991+0000","last_clean":"2024-09-11T01:24:25.132991+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:24:25.132991+0000","last_undegraded":"2024-09-11T01:24:25.132991+0000","last_fullsized":"2024-09-11T01:24:25.132991+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":24356,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:43.257010+0000","last_change":"2024-09-11T01:24:23.706628+0000","last_active":"2024-09-11T01:24:43.257010+0000","last_peered":"2024-09-11T01:24:43.257010+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:24:43.257010+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1985,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1966,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1985}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142487+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:24:25.142487+0000","last_peered":"2024-09-11T01:24:25.142487+0000","last_clean":"2024-09-11T01:24:25.142487+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:24:25.142487+0000","last_undegraded":"2024-09-11T01:24:25.142487+0000","last_fullsized":"2024-09-11T01:24:25.142487+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27943,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142784+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:24:25.142784+0000","last_peered":"2024-09-11T01:24:25.142784+0000","last_clean":"2024-09-11T01:24:25.142784+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:24:25.142784+0000","last_undegraded":"2024-09-11T01:24:25.142784+0000","last_fullsized":"2024-09-11T01:24:25.142784+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138018+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:24:25.138018+0000","last_peered":"2024-09-11T01:24:25.138018+0000","last_clean":"2024-09-11T01:24:25.138018+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:24:25.138018+0000","last_undegraded":"2024-09-11T01:24:25.138018+0000","last_fullsized":"2024-09-11T01:24:25.138018+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24622,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142656+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:24:25.142656+0000","last_peered":"2024-09-11T01:24:25.142656+0000","last_clean":"2024-09-11T01:24:25.142656+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:24:25.142656+0000","last_undegraded":"2024-09-11T01:24:25.142656+0000","last_fullsized":"2024-09-11T01:24:25.142656+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142712+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:24:25.142712+0000","last_peered":"2024-09-11T01:24:25.142712+0000","last_clean":"2024-09-11T01:24:25.142712+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:24:25.142712+0000","last_undegraded":"2024-09-11T01:24:25.142712+0000","last_fullsized":"2024-09-11T01:24:25.142712+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142785+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:24:25.142785+0000","last_peered":"2024-09-11T01:24:25.142785+0000","last_clean":"2024-09-11T01:24:25.142785+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:24:25.142785+0000","last_undegraded":"2024-09-11T01:24:25.142785+0000","last_fullsized":"2024-09-11T01:24:25.142785+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20114,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.137564+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:24:25.137564+0000","last_peered":"2024-09-11T01:24:25.137564+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:24:25.137564+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17832,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":29807,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":188821504,"data_stored":188779756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182558,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518632,"kb_used_data":63576,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853208,"statfs":{"total":96636764160,"available":96105684992,"internally_reserved":0,"allocated":65101824,"data_stored":64779190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986330,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":482368,"kb_used_data":121712,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93889472,"statfs":{"total":96636764160,"available":96142819328,"internally_reserved":0,"allocated":124633088,"data_stored":124314531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659275,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487500,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":486971,"kb_avail":93884340,"statfs":{"total":96636764160,"available":96137564160,"internally_reserved":0,"allocated":520192,"data_stored":166247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":498658991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":124157952,"data_stored":124154473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:48.912 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:49.252 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:49.253 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:24:49.271 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":463,"stamp":"2024-09-11T01:24:48.848270+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17409,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30231,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1490860,"kb_used_data":183996,"kb_used_omap":12,"kb_used_meta":1306803,"kb_avail":281624660,"statfs":{"total":289910292480,"available":288383651840,"internally_reserved":0,"allocated":188411904,"data_stored":187416768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1338166276},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1121,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1121,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003333"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28523,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139109+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:24:25.139109+0000","last_peered":"2024-09-11T01:24:25.139109+0000","last_clean":"2024-09-11T01:24:25.139109+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:24:25.139109+0000","last_undegraded":"2024-09-11T01:24:25.139109+0000","last_fullsized":"2024-09-11T01:24:25.139109+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20481,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138008+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:24:25.138008+0000","last_peered":"2024-09-11T01:24:25.138008+0000","last_clean":"2024-09-11T01:24:25.138008+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:24:25.138008+0000","last_undegraded":"2024-09-11T01:24:25.138008+0000","last_fullsized":"2024-09-11T01:24:25.138008+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20304,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142521+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:24:25.142521+0000","last_peered":"2024-09-11T01:24:25.142521+0000","last_clean":"2024-09-11T01:24:25.142521+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:24:25.142521+0000","last_undegraded":"2024-09-11T01:24:25.142521+0000","last_fullsized":"2024-09-11T01:24:25.142521+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20231,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.133993+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:24:25.133993+0000","last_peered":"2024-09-11T01:24:25.133993+0000","last_clean":"2024-09-11T01:24:25.133993+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:24:25.133993+0000","last_undegraded":"2024-09-11T01:24:25.133993+0000","last_fullsized":"2024-09-11T01:24:25.133993+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20164,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135797+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:24:25.135797+0000","last_peered":"2024-09-11T01:24:25.135797+0000","last_clean":"2024-09-11T01:24:25.135797+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:24:25.135797+0000","last_undegraded":"2024-09-11T01:24:25.135797+0000","last_fullsized":"2024-09-11T01:24:25.135797+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20425,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135738+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:24:25.135738+0000","last_peered":"2024-09-11T01:24:25.135738+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:24:25.135738+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20088,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135679+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:24:25.135679+0000","last_peered":"2024-09-11T01:24:25.135679+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:24:25.135679+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20411,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138636+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:24:25.138636+0000","last_peered":"2024-09-11T01:24:25.138636+0000","last_clean":"2024-09-11T01:24:25.138636+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:24:25.138636+0000","last_undegraded":"2024-09-11T01:24:25.138636+0000","last_fullsized":"2024-09-11T01:24:25.138636+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139906+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:24:25.139906+0000","last_peered":"2024-09-11T01:24:25.139906+0000","last_clean":"2024-09-11T01:24:25.139906+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:24:25.139906+0000","last_undegraded":"2024-09-11T01:24:25.139906+0000","last_fullsized":"2024-09-11T01:24:25.139906+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28009,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142299+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:24:25.142299+0000","last_peered":"2024-09-11T01:24:25.142299+0000","last_clean":"2024-09-11T01:24:25.142299+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:24:25.142299+0000","last_undegraded":"2024-09-11T01:24:25.142299+0000","last_fullsized":"2024-09-11T01:24:25.142299+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.134130+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:24:25.134130+0000","last_peered":"2024-09-11T01:24:25.134130+0000","last_clean":"2024-09-11T01:24:25.134130+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:24:25.134130+0000","last_undegraded":"2024-09-11T01:24:25.134130+0000","last_fullsized":"2024-09-11T01:24:25.134130+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20024,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.137211+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:24:25.137211+0000","last_peered":"2024-09-11T01:24:25.137211+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:24:25.137211+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":129,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135666+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:24:25.135666+0000","last_peered":"2024-09-11T01:24:25.135666+0000","last_clean":"2024-09-11T01:24:25.135666+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:24:25.135666+0000","last_undegraded":"2024-09-11T01:24:25.135666+0000","last_fullsized":"2024-09-11T01:24:25.135666+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28414,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.266137+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:24:25.266137+0000","last_peered":"2024-09-11T01:24:25.266137+0000","last_clean":"2024-09-11T01:24:25.266137+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:24:25.266137+0000","last_undegraded":"2024-09-11T01:24:25.266137+0000","last_fullsized":"2024-09-11T01:24:25.266137+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142149+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:24:25.142149+0000","last_peered":"2024-09-11T01:24:25.142149+0000","last_clean":"2024-09-11T01:24:25.142149+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:24:25.142149+0000","last_undegraded":"2024-09-11T01:24:25.142149+0000","last_fullsized":"2024-09-11T01:24:25.142149+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27805,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.132991+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:24:25.132991+0000","last_peered":"2024-09-11T01:24:25.132991+0000","last_clean":"2024-09-11T01:24:25.132991+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:24:25.132991+0000","last_undegraded":"2024-09-11T01:24:25.132991+0000","last_fullsized":"2024-09-11T01:24:25.132991+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":25202,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:48.268762+0000","last_change":"2024-09-11T01:24:23.706628+0000","last_active":"2024-09-11T01:24:48.268762+0000","last_peered":"2024-09-11T01:24:48.268762+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:24:48.268762+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1562,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1562}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142487+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:24:25.142487+0000","last_peered":"2024-09-11T01:24:25.142487+0000","last_clean":"2024-09-11T01:24:25.142487+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:24:25.142487+0000","last_undegraded":"2024-09-11T01:24:25.142487+0000","last_fullsized":"2024-09-11T01:24:25.142487+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27943,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142784+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:24:25.142784+0000","last_peered":"2024-09-11T01:24:25.142784+0000","last_clean":"2024-09-11T01:24:25.142784+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:24:25.142784+0000","last_undegraded":"2024-09-11T01:24:25.142784+0000","last_fullsized":"2024-09-11T01:24:25.142784+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138018+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:24:25.138018+0000","last_peered":"2024-09-11T01:24:25.138018+0000","last_clean":"2024-09-11T01:24:25.138018+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:24:25.138018+0000","last_undegraded":"2024-09-11T01:24:25.138018+0000","last_fullsized":"2024-09-11T01:24:25.138018+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24622,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142656+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:24:25.142656+0000","last_peered":"2024-09-11T01:24:25.142656+0000","last_clean":"2024-09-11T01:24:25.142656+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:24:25.142656+0000","last_undegraded":"2024-09-11T01:24:25.142656+0000","last_fullsized":"2024-09-11T01:24:25.142656+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142712+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:24:25.142712+0000","last_peered":"2024-09-11T01:24:25.142712+0000","last_clean":"2024-09-11T01:24:25.142712+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:24:25.142712+0000","last_undegraded":"2024-09-11T01:24:25.142712+0000","last_fullsized":"2024-09-11T01:24:25.142712+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142785+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:24:25.142785+0000","last_peered":"2024-09-11T01:24:25.142785+0000","last_clean":"2024-09-11T01:24:25.142785+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:24:25.142785+0000","last_undegraded":"2024-09-11T01:24:25.142785+0000","last_fullsized":"2024-09-11T01:24:25.142785+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20114,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.137564+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:24:25.137564+0000","last_peered":"2024-09-11T01:24:25.137564+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:24:25.137564+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17409,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30230,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":186978304,"data_stored":186936556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182559,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518632,"kb_used_data":63576,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853208,"statfs":{"total":96636764160,"available":96105684992,"internally_reserved":0,"allocated":65101824,"data_stored":64779190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986331,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480568,"kb_used_data":119912,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93891272,"statfs":{"total":96636764160,"available":96144662528,"internally_reserved":0,"allocated":122789888,"data_stored":122471331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659276,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491660,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93880180,"statfs":{"total":96636764160,"available":96133304320,"internally_reserved":0,"allocated":520192,"data_stored":166247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502918831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":122314752,"data_stored":122311273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:49.273 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:49.581 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:49.582 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:24:49.600 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":463,"stamp":"2024-09-11T01:24:48.848270+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17409,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30231,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1490860,"kb_used_data":183996,"kb_used_omap":12,"kb_used_meta":1306803,"kb_avail":281624660,"statfs":{"total":289910292480,"available":288383651840,"internally_reserved":0,"allocated":188411904,"data_stored":187416768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1338166276},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1121,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1121,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003333"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28523,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139109+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:24:25.139109+0000","last_peered":"2024-09-11T01:24:25.139109+0000","last_clean":"2024-09-11T01:24:25.139109+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:24:25.139109+0000","last_undegraded":"2024-09-11T01:24:25.139109+0000","last_fullsized":"2024-09-11T01:24:25.139109+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20481,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138008+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:24:25.138008+0000","last_peered":"2024-09-11T01:24:25.138008+0000","last_clean":"2024-09-11T01:24:25.138008+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:24:25.138008+0000","last_undegraded":"2024-09-11T01:24:25.138008+0000","last_fullsized":"2024-09-11T01:24:25.138008+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20304,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142521+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:24:25.142521+0000","last_peered":"2024-09-11T01:24:25.142521+0000","last_clean":"2024-09-11T01:24:25.142521+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:24:25.142521+0000","last_undegraded":"2024-09-11T01:24:25.142521+0000","last_fullsized":"2024-09-11T01:24:25.142521+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20231,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.133993+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:24:25.133993+0000","last_peered":"2024-09-11T01:24:25.133993+0000","last_clean":"2024-09-11T01:24:25.133993+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:24:25.133993+0000","last_undegraded":"2024-09-11T01:24:25.133993+0000","last_fullsized":"2024-09-11T01:24:25.133993+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20164,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135797+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:24:25.135797+0000","last_peered":"2024-09-11T01:24:25.135797+0000","last_clean":"2024-09-11T01:24:25.135797+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:24:25.135797+0000","last_undegraded":"2024-09-11T01:24:25.135797+0000","last_fullsized":"2024-09-11T01:24:25.135797+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20425,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135738+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:24:25.135738+0000","last_peered":"2024-09-11T01:24:25.135738+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:24:25.135738+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20088,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135679+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:24:25.135679+0000","last_peered":"2024-09-11T01:24:25.135679+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:24:25.135679+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20411,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138636+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:24:25.138636+0000","last_peered":"2024-09-11T01:24:25.138636+0000","last_clean":"2024-09-11T01:24:25.138636+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:24:25.138636+0000","last_undegraded":"2024-09-11T01:24:25.138636+0000","last_fullsized":"2024-09-11T01:24:25.138636+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139906+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:24:25.139906+0000","last_peered":"2024-09-11T01:24:25.139906+0000","last_clean":"2024-09-11T01:24:25.139906+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:24:25.139906+0000","last_undegraded":"2024-09-11T01:24:25.139906+0000","last_fullsized":"2024-09-11T01:24:25.139906+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28009,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142299+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:24:25.142299+0000","last_peered":"2024-09-11T01:24:25.142299+0000","last_clean":"2024-09-11T01:24:25.142299+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:24:25.142299+0000","last_undegraded":"2024-09-11T01:24:25.142299+0000","last_fullsized":"2024-09-11T01:24:25.142299+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.134130+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:24:25.134130+0000","last_peered":"2024-09-11T01:24:25.134130+0000","last_clean":"2024-09-11T01:24:25.134130+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:24:25.134130+0000","last_undegraded":"2024-09-11T01:24:25.134130+0000","last_fullsized":"2024-09-11T01:24:25.134130+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20024,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.137211+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:24:25.137211+0000","last_peered":"2024-09-11T01:24:25.137211+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:24:25.137211+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":129,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135666+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:24:25.135666+0000","last_peered":"2024-09-11T01:24:25.135666+0000","last_clean":"2024-09-11T01:24:25.135666+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:24:25.135666+0000","last_undegraded":"2024-09-11T01:24:25.135666+0000","last_fullsized":"2024-09-11T01:24:25.135666+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28414,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.266137+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:24:25.266137+0000","last_peered":"2024-09-11T01:24:25.266137+0000","last_clean":"2024-09-11T01:24:25.266137+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:24:25.266137+0000","last_undegraded":"2024-09-11T01:24:25.266137+0000","last_fullsized":"2024-09-11T01:24:25.266137+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142149+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:24:25.142149+0000","last_peered":"2024-09-11T01:24:25.142149+0000","last_clean":"2024-09-11T01:24:25.142149+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:24:25.142149+0000","last_undegraded":"2024-09-11T01:24:25.142149+0000","last_fullsized":"2024-09-11T01:24:25.142149+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27805,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.132991+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:24:25.132991+0000","last_peered":"2024-09-11T01:24:25.132991+0000","last_clean":"2024-09-11T01:24:25.132991+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:24:25.132991+0000","last_undegraded":"2024-09-11T01:24:25.132991+0000","last_fullsized":"2024-09-11T01:24:25.132991+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":25202,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:48.268762+0000","last_change":"2024-09-11T01:24:23.706628+0000","last_active":"2024-09-11T01:24:48.268762+0000","last_peered":"2024-09-11T01:24:48.268762+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:24:48.268762+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1562,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1562}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142487+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:24:25.142487+0000","last_peered":"2024-09-11T01:24:25.142487+0000","last_clean":"2024-09-11T01:24:25.142487+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:24:25.142487+0000","last_undegraded":"2024-09-11T01:24:25.142487+0000","last_fullsized":"2024-09-11T01:24:25.142487+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27943,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142784+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:24:25.142784+0000","last_peered":"2024-09-11T01:24:25.142784+0000","last_clean":"2024-09-11T01:24:25.142784+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:24:25.142784+0000","last_undegraded":"2024-09-11T01:24:25.142784+0000","last_fullsized":"2024-09-11T01:24:25.142784+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138018+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:24:25.138018+0000","last_peered":"2024-09-11T01:24:25.138018+0000","last_clean":"2024-09-11T01:24:25.138018+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:24:25.138018+0000","last_undegraded":"2024-09-11T01:24:25.138018+0000","last_fullsized":"2024-09-11T01:24:25.138018+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24622,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142656+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:24:25.142656+0000","last_peered":"2024-09-11T01:24:25.142656+0000","last_clean":"2024-09-11T01:24:25.142656+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:24:25.142656+0000","last_undegraded":"2024-09-11T01:24:25.142656+0000","last_fullsized":"2024-09-11T01:24:25.142656+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142712+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:24:25.142712+0000","last_peered":"2024-09-11T01:24:25.142712+0000","last_clean":"2024-09-11T01:24:25.142712+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:24:25.142712+0000","last_undegraded":"2024-09-11T01:24:25.142712+0000","last_fullsized":"2024-09-11T01:24:25.142712+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142785+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:24:25.142785+0000","last_peered":"2024-09-11T01:24:25.142785+0000","last_clean":"2024-09-11T01:24:25.142785+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:24:25.142785+0000","last_undegraded":"2024-09-11T01:24:25.142785+0000","last_fullsized":"2024-09-11T01:24:25.142785+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20114,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.137564+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:24:25.137564+0000","last_peered":"2024-09-11T01:24:25.137564+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:24:25.137564+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17409,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30230,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":186978304,"data_stored":186936556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182559,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518632,"kb_used_data":63576,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853208,"statfs":{"total":96636764160,"available":96105684992,"internally_reserved":0,"allocated":65101824,"data_stored":64779190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986331,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480568,"kb_used_data":119912,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93891272,"statfs":{"total":96636764160,"available":96144662528,"internally_reserved":0,"allocated":122789888,"data_stored":122471331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659276,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491660,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93880180,"statfs":{"total":96636764160,"available":96133304320,"internally_reserved":0,"allocated":520192,"data_stored":166247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502918831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":122314752,"data_stored":122311273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:49.601 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:24:49.601 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'20227", 'reported_seq': 20425, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:25.135738+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:24:25.135738+0000', 'last_peered': '2024-09-11T01:24:25.135738+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:24:25.135738+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16568, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:49.601 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:24:49.601 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20088, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:25.135679+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:24:25.135679+0000', 'last_peered': '2024-09-11T01:24:25.135679+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:24:25.135679+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:49.601 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:24:49.601 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20024, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:25.137211+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:24:25.137211+0000', 'last_peered': '2024-09-11T01:24:25.137211+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:24:25.137211+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:49.602 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:24:49.602 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20264", 'reported_seq': 25202, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:48.268762+0000', 'last_change': '2024-09-11T01:24:23.706628+0000', 'last_active': '2024-09-11T01:24:48.268762+0000', 'last_peered': '2024-09-11T01:24:48.268762+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:24:48.268762+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1562, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17001, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2389, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1562}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:49.602 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:24:49.602 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 20114, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:25.137564+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:24:25.137564+0000', 'last_peered': '2024-09-11T01:24:25.137564+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:24:25.137564+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:49.602 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:24:49.999 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:50.000 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 17409/6 objects degraded (290150.000%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":826,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1526640640,"bytes_avail":288383651840,"bytes_total":289910292480,"degraded_objects":17409,"degraded_total":6,"degraded_ratio":2901.5,"recovering_objects_per_sec":93,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1121,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:24:50.019 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:24:50.019 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:50.348 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:50.348 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:24:50.367 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":463,"stamp":"2024-09-11T01:24:48.848270+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17409,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30231,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1490860,"kb_used_data":183996,"kb_used_omap":12,"kb_used_meta":1306803,"kb_avail":281624660,"statfs":{"total":289910292480,"available":288383651840,"internally_reserved":0,"allocated":188411904,"data_stored":187416768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1338166276},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1121,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1121,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003333"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28523,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139109+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:24:25.139109+0000","last_peered":"2024-09-11T01:24:25.139109+0000","last_clean":"2024-09-11T01:24:25.139109+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:24:25.139109+0000","last_undegraded":"2024-09-11T01:24:25.139109+0000","last_fullsized":"2024-09-11T01:24:25.139109+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20481,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138008+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:24:25.138008+0000","last_peered":"2024-09-11T01:24:25.138008+0000","last_clean":"2024-09-11T01:24:25.138008+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:24:25.138008+0000","last_undegraded":"2024-09-11T01:24:25.138008+0000","last_fullsized":"2024-09-11T01:24:25.138008+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20304,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142521+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:24:25.142521+0000","last_peered":"2024-09-11T01:24:25.142521+0000","last_clean":"2024-09-11T01:24:25.142521+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:24:25.142521+0000","last_undegraded":"2024-09-11T01:24:25.142521+0000","last_fullsized":"2024-09-11T01:24:25.142521+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20231,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.133993+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:24:25.133993+0000","last_peered":"2024-09-11T01:24:25.133993+0000","last_clean":"2024-09-11T01:24:25.133993+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:24:25.133993+0000","last_undegraded":"2024-09-11T01:24:25.133993+0000","last_fullsized":"2024-09-11T01:24:25.133993+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20164,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135797+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:24:25.135797+0000","last_peered":"2024-09-11T01:24:25.135797+0000","last_clean":"2024-09-11T01:24:25.135797+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:24:25.135797+0000","last_undegraded":"2024-09-11T01:24:25.135797+0000","last_fullsized":"2024-09-11T01:24:25.135797+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20425,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135738+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:24:25.135738+0000","last_peered":"2024-09-11T01:24:25.135738+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:24:25.135738+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20088,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135679+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:24:25.135679+0000","last_peered":"2024-09-11T01:24:25.135679+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:24:25.135679+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20411,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138636+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:24:25.138636+0000","last_peered":"2024-09-11T01:24:25.138636+0000","last_clean":"2024-09-11T01:24:25.138636+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:24:25.138636+0000","last_undegraded":"2024-09-11T01:24:25.138636+0000","last_fullsized":"2024-09-11T01:24:25.138636+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139906+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:24:25.139906+0000","last_peered":"2024-09-11T01:24:25.139906+0000","last_clean":"2024-09-11T01:24:25.139906+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:24:25.139906+0000","last_undegraded":"2024-09-11T01:24:25.139906+0000","last_fullsized":"2024-09-11T01:24:25.139906+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28009,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142299+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:24:25.142299+0000","last_peered":"2024-09-11T01:24:25.142299+0000","last_clean":"2024-09-11T01:24:25.142299+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:24:25.142299+0000","last_undegraded":"2024-09-11T01:24:25.142299+0000","last_fullsized":"2024-09-11T01:24:25.142299+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.134130+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:24:25.134130+0000","last_peered":"2024-09-11T01:24:25.134130+0000","last_clean":"2024-09-11T01:24:25.134130+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:24:25.134130+0000","last_undegraded":"2024-09-11T01:24:25.134130+0000","last_fullsized":"2024-09-11T01:24:25.134130+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20024,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.137211+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:24:25.137211+0000","last_peered":"2024-09-11T01:24:25.137211+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:24:25.137211+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":129,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135666+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:24:25.135666+0000","last_peered":"2024-09-11T01:24:25.135666+0000","last_clean":"2024-09-11T01:24:25.135666+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:24:25.135666+0000","last_undegraded":"2024-09-11T01:24:25.135666+0000","last_fullsized":"2024-09-11T01:24:25.135666+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28414,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.266137+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:24:25.266137+0000","last_peered":"2024-09-11T01:24:25.266137+0000","last_clean":"2024-09-11T01:24:25.266137+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:24:25.266137+0000","last_undegraded":"2024-09-11T01:24:25.266137+0000","last_fullsized":"2024-09-11T01:24:25.266137+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142149+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:24:25.142149+0000","last_peered":"2024-09-11T01:24:25.142149+0000","last_clean":"2024-09-11T01:24:25.142149+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:24:25.142149+0000","last_undegraded":"2024-09-11T01:24:25.142149+0000","last_fullsized":"2024-09-11T01:24:25.142149+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27805,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.132991+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:24:25.132991+0000","last_peered":"2024-09-11T01:24:25.132991+0000","last_clean":"2024-09-11T01:24:25.132991+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:24:25.132991+0000","last_undegraded":"2024-09-11T01:24:25.132991+0000","last_fullsized":"2024-09-11T01:24:25.132991+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":25202,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:48.268762+0000","last_change":"2024-09-11T01:24:23.706628+0000","last_active":"2024-09-11T01:24:48.268762+0000","last_peered":"2024-09-11T01:24:48.268762+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:24:48.268762+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1562,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1562}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142487+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:24:25.142487+0000","last_peered":"2024-09-11T01:24:25.142487+0000","last_clean":"2024-09-11T01:24:25.142487+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:24:25.142487+0000","last_undegraded":"2024-09-11T01:24:25.142487+0000","last_fullsized":"2024-09-11T01:24:25.142487+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27943,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142784+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:24:25.142784+0000","last_peered":"2024-09-11T01:24:25.142784+0000","last_clean":"2024-09-11T01:24:25.142784+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:24:25.142784+0000","last_undegraded":"2024-09-11T01:24:25.142784+0000","last_fullsized":"2024-09-11T01:24:25.142784+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138018+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:24:25.138018+0000","last_peered":"2024-09-11T01:24:25.138018+0000","last_clean":"2024-09-11T01:24:25.138018+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:24:25.138018+0000","last_undegraded":"2024-09-11T01:24:25.138018+0000","last_fullsized":"2024-09-11T01:24:25.138018+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24622,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142656+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:24:25.142656+0000","last_peered":"2024-09-11T01:24:25.142656+0000","last_clean":"2024-09-11T01:24:25.142656+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:24:25.142656+0000","last_undegraded":"2024-09-11T01:24:25.142656+0000","last_fullsized":"2024-09-11T01:24:25.142656+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142712+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:24:25.142712+0000","last_peered":"2024-09-11T01:24:25.142712+0000","last_clean":"2024-09-11T01:24:25.142712+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:24:25.142712+0000","last_undegraded":"2024-09-11T01:24:25.142712+0000","last_fullsized":"2024-09-11T01:24:25.142712+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142785+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:24:25.142785+0000","last_peered":"2024-09-11T01:24:25.142785+0000","last_clean":"2024-09-11T01:24:25.142785+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:24:25.142785+0000","last_undegraded":"2024-09-11T01:24:25.142785+0000","last_fullsized":"2024-09-11T01:24:25.142785+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20114,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.137564+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:24:25.137564+0000","last_peered":"2024-09-11T01:24:25.137564+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:24:25.137564+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17409,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30230,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":186978304,"data_stored":186936556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182559,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518632,"kb_used_data":63576,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853208,"statfs":{"total":96636764160,"available":96105684992,"internally_reserved":0,"allocated":65101824,"data_stored":64779190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986331,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":480568,"kb_used_data":119912,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93891272,"statfs":{"total":96636764160,"available":96144662528,"internally_reserved":0,"allocated":122789888,"data_stored":122471331,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659276,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491660,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93880180,"statfs":{"total":96636764160,"available":96133304320,"internally_reserved":0,"allocated":520192,"data_stored":166247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502918831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":122314752,"data_stored":122311273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:53.369 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:53.686 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:53.687 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:24:53.704 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":465,"stamp":"2024-09-11T01:24:52.849808+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17409,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30231,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1489060,"kb_used_data":182196,"kb_used_omap":12,"kb_used_meta":1306803,"kb_avail":281626460,"statfs":{"total":289910292480,"available":288385495040,"internally_reserved":0,"allocated":186568704,"data_stored":185573568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1338166276},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-785,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":785,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003660"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28523,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139109+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:24:25.139109+0000","last_peered":"2024-09-11T01:24:25.139109+0000","last_clean":"2024-09-11T01:24:25.139109+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:24:25.139109+0000","last_undegraded":"2024-09-11T01:24:25.139109+0000","last_fullsized":"2024-09-11T01:24:25.139109+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20481,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138008+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:24:25.138008+0000","last_peered":"2024-09-11T01:24:25.138008+0000","last_clean":"2024-09-11T01:24:25.138008+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:24:25.138008+0000","last_undegraded":"2024-09-11T01:24:25.138008+0000","last_fullsized":"2024-09-11T01:24:25.138008+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20304,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142521+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:24:25.142521+0000","last_peered":"2024-09-11T01:24:25.142521+0000","last_clean":"2024-09-11T01:24:25.142521+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:24:25.142521+0000","last_undegraded":"2024-09-11T01:24:25.142521+0000","last_fullsized":"2024-09-11T01:24:25.142521+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20231,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.133993+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:24:25.133993+0000","last_peered":"2024-09-11T01:24:25.133993+0000","last_clean":"2024-09-11T01:24:25.133993+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:24:25.133993+0000","last_undegraded":"2024-09-11T01:24:25.133993+0000","last_fullsized":"2024-09-11T01:24:25.133993+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20164,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135797+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:24:25.135797+0000","last_peered":"2024-09-11T01:24:25.135797+0000","last_clean":"2024-09-11T01:24:25.135797+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:24:25.135797+0000","last_undegraded":"2024-09-11T01:24:25.135797+0000","last_fullsized":"2024-09-11T01:24:25.135797+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20425,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135738+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:24:25.135738+0000","last_peered":"2024-09-11T01:24:25.135738+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:24:25.135738+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20088,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135679+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:24:25.135679+0000","last_peered":"2024-09-11T01:24:25.135679+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:24:25.135679+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20411,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138636+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:24:25.138636+0000","last_peered":"2024-09-11T01:24:25.138636+0000","last_clean":"2024-09-11T01:24:25.138636+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:24:25.138636+0000","last_undegraded":"2024-09-11T01:24:25.138636+0000","last_fullsized":"2024-09-11T01:24:25.138636+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139906+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:24:25.139906+0000","last_peered":"2024-09-11T01:24:25.139906+0000","last_clean":"2024-09-11T01:24:25.139906+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:24:25.139906+0000","last_undegraded":"2024-09-11T01:24:25.139906+0000","last_fullsized":"2024-09-11T01:24:25.139906+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28009,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142299+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:24:25.142299+0000","last_peered":"2024-09-11T01:24:25.142299+0000","last_clean":"2024-09-11T01:24:25.142299+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:24:25.142299+0000","last_undegraded":"2024-09-11T01:24:25.142299+0000","last_fullsized":"2024-09-11T01:24:25.142299+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.134130+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:24:25.134130+0000","last_peered":"2024-09-11T01:24:25.134130+0000","last_clean":"2024-09-11T01:24:25.134130+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:24:25.134130+0000","last_undegraded":"2024-09-11T01:24:25.134130+0000","last_fullsized":"2024-09-11T01:24:25.134130+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20024,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.137211+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:24:25.137211+0000","last_peered":"2024-09-11T01:24:25.137211+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:24:25.137211+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":129,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135666+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:24:25.135666+0000","last_peered":"2024-09-11T01:24:25.135666+0000","last_clean":"2024-09-11T01:24:25.135666+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:24:25.135666+0000","last_undegraded":"2024-09-11T01:24:25.135666+0000","last_fullsized":"2024-09-11T01:24:25.135666+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28414,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.266137+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:24:25.266137+0000","last_peered":"2024-09-11T01:24:25.266137+0000","last_clean":"2024-09-11T01:24:25.266137+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:24:25.266137+0000","last_undegraded":"2024-09-11T01:24:25.266137+0000","last_fullsized":"2024-09-11T01:24:25.266137+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142149+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:24:25.142149+0000","last_peered":"2024-09-11T01:24:25.142149+0000","last_clean":"2024-09-11T01:24:25.142149+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:24:25.142149+0000","last_undegraded":"2024-09-11T01:24:25.142149+0000","last_fullsized":"2024-09-11T01:24:25.142149+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27805,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.132991+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:24:25.132991+0000","last_peered":"2024-09-11T01:24:25.132991+0000","last_clean":"2024-09-11T01:24:25.132991+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:24:25.132991+0000","last_undegraded":"2024-09-11T01:24:25.132991+0000","last_fullsized":"2024-09-11T01:24:25.132991+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":25202,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:48.268762+0000","last_change":"2024-09-11T01:24:23.706628+0000","last_active":"2024-09-11T01:24:48.268762+0000","last_peered":"2024-09-11T01:24:48.268762+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:24:48.268762+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1562,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1562}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142487+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:24:25.142487+0000","last_peered":"2024-09-11T01:24:25.142487+0000","last_clean":"2024-09-11T01:24:25.142487+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:24:25.142487+0000","last_undegraded":"2024-09-11T01:24:25.142487+0000","last_fullsized":"2024-09-11T01:24:25.142487+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27943,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142784+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:24:25.142784+0000","last_peered":"2024-09-11T01:24:25.142784+0000","last_clean":"2024-09-11T01:24:25.142784+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:24:25.142784+0000","last_undegraded":"2024-09-11T01:24:25.142784+0000","last_fullsized":"2024-09-11T01:24:25.142784+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138018+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:24:25.138018+0000","last_peered":"2024-09-11T01:24:25.138018+0000","last_clean":"2024-09-11T01:24:25.138018+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:24:25.138018+0000","last_undegraded":"2024-09-11T01:24:25.138018+0000","last_fullsized":"2024-09-11T01:24:25.138018+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24622,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142656+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:24:25.142656+0000","last_peered":"2024-09-11T01:24:25.142656+0000","last_clean":"2024-09-11T01:24:25.142656+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:24:25.142656+0000","last_undegraded":"2024-09-11T01:24:25.142656+0000","last_fullsized":"2024-09-11T01:24:25.142656+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142712+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:24:25.142712+0000","last_peered":"2024-09-11T01:24:25.142712+0000","last_clean":"2024-09-11T01:24:25.142712+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:24:25.142712+0000","last_undegraded":"2024-09-11T01:24:25.142712+0000","last_fullsized":"2024-09-11T01:24:25.142712+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142785+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:24:25.142785+0000","last_peered":"2024-09-11T01:24:25.142785+0000","last_clean":"2024-09-11T01:24:25.142785+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:24:25.142785+0000","last_undegraded":"2024-09-11T01:24:25.142785+0000","last_fullsized":"2024-09-11T01:24:25.142785+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20114,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.137564+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:24:25.137564+0000","last_peered":"2024-09-11T01:24:25.137564+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:24:25.137564+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17409,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30230,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":185135104,"data_stored":185093356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182559,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518632,"kb_used_data":63576,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853208,"statfs":{"total":96636764160,"available":96105684992,"internally_reserved":0,"allocated":65101824,"data_stored":64779190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986332,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":478768,"kb_used_data":118112,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93893072,"statfs":{"total":96636764160,"available":96146505728,"internally_reserved":0,"allocated":120946688,"data_stored":120628131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659276,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491660,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93880180,"statfs":{"total":96636764160,"available":96133304320,"internally_reserved":0,"allocated":520192,"data_stored":166247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502918831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":120471552,"data_stored":120468073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:53.706 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:54.032 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:54.032 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:24:54.049 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":465,"stamp":"2024-09-11T01:24:52.849808+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17409,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30231,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1489060,"kb_used_data":182196,"kb_used_omap":12,"kb_used_meta":1306803,"kb_avail":281626460,"statfs":{"total":289910292480,"available":288385495040,"internally_reserved":0,"allocated":186568704,"data_stored":185573568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1338166276},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-785,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":785,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003660"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28523,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139109+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:24:25.139109+0000","last_peered":"2024-09-11T01:24:25.139109+0000","last_clean":"2024-09-11T01:24:25.139109+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:24:25.139109+0000","last_undegraded":"2024-09-11T01:24:25.139109+0000","last_fullsized":"2024-09-11T01:24:25.139109+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20481,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138008+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:24:25.138008+0000","last_peered":"2024-09-11T01:24:25.138008+0000","last_clean":"2024-09-11T01:24:25.138008+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:24:25.138008+0000","last_undegraded":"2024-09-11T01:24:25.138008+0000","last_fullsized":"2024-09-11T01:24:25.138008+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20304,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142521+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:24:25.142521+0000","last_peered":"2024-09-11T01:24:25.142521+0000","last_clean":"2024-09-11T01:24:25.142521+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:24:25.142521+0000","last_undegraded":"2024-09-11T01:24:25.142521+0000","last_fullsized":"2024-09-11T01:24:25.142521+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20231,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.133993+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:24:25.133993+0000","last_peered":"2024-09-11T01:24:25.133993+0000","last_clean":"2024-09-11T01:24:25.133993+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:24:25.133993+0000","last_undegraded":"2024-09-11T01:24:25.133993+0000","last_fullsized":"2024-09-11T01:24:25.133993+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20164,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135797+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:24:25.135797+0000","last_peered":"2024-09-11T01:24:25.135797+0000","last_clean":"2024-09-11T01:24:25.135797+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:24:25.135797+0000","last_undegraded":"2024-09-11T01:24:25.135797+0000","last_fullsized":"2024-09-11T01:24:25.135797+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20425,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135738+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:24:25.135738+0000","last_peered":"2024-09-11T01:24:25.135738+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:24:25.135738+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20088,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135679+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:24:25.135679+0000","last_peered":"2024-09-11T01:24:25.135679+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:24:25.135679+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20411,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138636+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:24:25.138636+0000","last_peered":"2024-09-11T01:24:25.138636+0000","last_clean":"2024-09-11T01:24:25.138636+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:24:25.138636+0000","last_undegraded":"2024-09-11T01:24:25.138636+0000","last_fullsized":"2024-09-11T01:24:25.138636+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139906+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:24:25.139906+0000","last_peered":"2024-09-11T01:24:25.139906+0000","last_clean":"2024-09-11T01:24:25.139906+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:24:25.139906+0000","last_undegraded":"2024-09-11T01:24:25.139906+0000","last_fullsized":"2024-09-11T01:24:25.139906+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28009,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142299+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:24:25.142299+0000","last_peered":"2024-09-11T01:24:25.142299+0000","last_clean":"2024-09-11T01:24:25.142299+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:24:25.142299+0000","last_undegraded":"2024-09-11T01:24:25.142299+0000","last_fullsized":"2024-09-11T01:24:25.142299+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.134130+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:24:25.134130+0000","last_peered":"2024-09-11T01:24:25.134130+0000","last_clean":"2024-09-11T01:24:25.134130+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:24:25.134130+0000","last_undegraded":"2024-09-11T01:24:25.134130+0000","last_fullsized":"2024-09-11T01:24:25.134130+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20024,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.137211+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:24:25.137211+0000","last_peered":"2024-09-11T01:24:25.137211+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:24:25.137211+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":129,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135666+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:24:25.135666+0000","last_peered":"2024-09-11T01:24:25.135666+0000","last_clean":"2024-09-11T01:24:25.135666+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:24:25.135666+0000","last_undegraded":"2024-09-11T01:24:25.135666+0000","last_fullsized":"2024-09-11T01:24:25.135666+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28414,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.266137+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:24:25.266137+0000","last_peered":"2024-09-11T01:24:25.266137+0000","last_clean":"2024-09-11T01:24:25.266137+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:24:25.266137+0000","last_undegraded":"2024-09-11T01:24:25.266137+0000","last_fullsized":"2024-09-11T01:24:25.266137+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142149+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:24:25.142149+0000","last_peered":"2024-09-11T01:24:25.142149+0000","last_clean":"2024-09-11T01:24:25.142149+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:24:25.142149+0000","last_undegraded":"2024-09-11T01:24:25.142149+0000","last_fullsized":"2024-09-11T01:24:25.142149+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27805,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.132991+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:24:25.132991+0000","last_peered":"2024-09-11T01:24:25.132991+0000","last_clean":"2024-09-11T01:24:25.132991+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:24:25.132991+0000","last_undegraded":"2024-09-11T01:24:25.132991+0000","last_fullsized":"2024-09-11T01:24:25.132991+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":25202,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:48.268762+0000","last_change":"2024-09-11T01:24:23.706628+0000","last_active":"2024-09-11T01:24:48.268762+0000","last_peered":"2024-09-11T01:24:48.268762+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:24:48.268762+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1562,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1562}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142487+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:24:25.142487+0000","last_peered":"2024-09-11T01:24:25.142487+0000","last_clean":"2024-09-11T01:24:25.142487+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:24:25.142487+0000","last_undegraded":"2024-09-11T01:24:25.142487+0000","last_fullsized":"2024-09-11T01:24:25.142487+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27943,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142784+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:24:25.142784+0000","last_peered":"2024-09-11T01:24:25.142784+0000","last_clean":"2024-09-11T01:24:25.142784+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:24:25.142784+0000","last_undegraded":"2024-09-11T01:24:25.142784+0000","last_fullsized":"2024-09-11T01:24:25.142784+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138018+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:24:25.138018+0000","last_peered":"2024-09-11T01:24:25.138018+0000","last_clean":"2024-09-11T01:24:25.138018+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:24:25.138018+0000","last_undegraded":"2024-09-11T01:24:25.138018+0000","last_fullsized":"2024-09-11T01:24:25.138018+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24622,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142656+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:24:25.142656+0000","last_peered":"2024-09-11T01:24:25.142656+0000","last_clean":"2024-09-11T01:24:25.142656+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:24:25.142656+0000","last_undegraded":"2024-09-11T01:24:25.142656+0000","last_fullsized":"2024-09-11T01:24:25.142656+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142712+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:24:25.142712+0000","last_peered":"2024-09-11T01:24:25.142712+0000","last_clean":"2024-09-11T01:24:25.142712+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:24:25.142712+0000","last_undegraded":"2024-09-11T01:24:25.142712+0000","last_fullsized":"2024-09-11T01:24:25.142712+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142785+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:24:25.142785+0000","last_peered":"2024-09-11T01:24:25.142785+0000","last_clean":"2024-09-11T01:24:25.142785+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:24:25.142785+0000","last_undegraded":"2024-09-11T01:24:25.142785+0000","last_fullsized":"2024-09-11T01:24:25.142785+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20114,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.137564+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:24:25.137564+0000","last_peered":"2024-09-11T01:24:25.137564+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:24:25.137564+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17409,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30230,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":185135104,"data_stored":185093356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182559,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518632,"kb_used_data":63576,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853208,"statfs":{"total":96636764160,"available":96105684992,"internally_reserved":0,"allocated":65101824,"data_stored":64779190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986332,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":478768,"kb_used_data":118112,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93893072,"statfs":{"total":96636764160,"available":96146505728,"internally_reserved":0,"allocated":120946688,"data_stored":120628131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659276,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491660,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93880180,"statfs":{"total":96636764160,"available":96133304320,"internally_reserved":0,"allocated":520192,"data_stored":166247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502918831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":120471552,"data_stored":120468073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:54.050 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:24:54.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'20227", 'reported_seq': 20425, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:25.135738+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:24:25.135738+0000', 'last_peered': '2024-09-11T01:24:25.135738+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:24:25.135738+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16568, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:54.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:24:54.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20088, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:25.135679+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:24:25.135679+0000', 'last_peered': '2024-09-11T01:24:25.135679+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:24:25.135679+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:54.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:24:54.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20024, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:25.137211+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:24:25.137211+0000', 'last_peered': '2024-09-11T01:24:25.137211+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:24:25.137211+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:54.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:24:54.051 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20264", 'reported_seq': 25202, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:48.268762+0000', 'last_change': '2024-09-11T01:24:23.706628+0000', 'last_active': '2024-09-11T01:24:48.268762+0000', 'last_peered': '2024-09-11T01:24:48.268762+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:24:48.268762+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1562, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17001, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2389, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1562}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:54.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:24:54.052 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 20114, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:25.137564+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:24:25.137564+0000', 'last_peered': '2024-09-11T01:24:25.137564+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:24:25.137564+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:54.052 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:24:54.407 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:54.407 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 17409/6 objects degraded (290150.000%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":830,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1524797440,"bytes_avail":288385495040,"bytes_total":289910292480,"degraded_objects":17409,"degraded_total":6,"degraded_ratio":2901.5,"recovering_objects_per_sec":65,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":785,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:24:54.426 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:24:54.426 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:54.733 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:54.734 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:24:54.750 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":465,"stamp":"2024-09-11T01:24:52.849808+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17409,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30231,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1489060,"kb_used_data":182196,"kb_used_omap":12,"kb_used_meta":1306803,"kb_avail":281626460,"statfs":{"total":289910292480,"available":288385495040,"internally_reserved":0,"allocated":186568704,"data_stored":185573568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1338166276},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-785,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":785,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003660"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28523,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139109+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:24:25.139109+0000","last_peered":"2024-09-11T01:24:25.139109+0000","last_clean":"2024-09-11T01:24:25.139109+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:24:25.139109+0000","last_undegraded":"2024-09-11T01:24:25.139109+0000","last_fullsized":"2024-09-11T01:24:25.139109+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20481,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138008+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:24:25.138008+0000","last_peered":"2024-09-11T01:24:25.138008+0000","last_clean":"2024-09-11T01:24:25.138008+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:24:25.138008+0000","last_undegraded":"2024-09-11T01:24:25.138008+0000","last_fullsized":"2024-09-11T01:24:25.138008+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20304,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142521+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:24:25.142521+0000","last_peered":"2024-09-11T01:24:25.142521+0000","last_clean":"2024-09-11T01:24:25.142521+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:24:25.142521+0000","last_undegraded":"2024-09-11T01:24:25.142521+0000","last_fullsized":"2024-09-11T01:24:25.142521+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20231,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.133993+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:24:25.133993+0000","last_peered":"2024-09-11T01:24:25.133993+0000","last_clean":"2024-09-11T01:24:25.133993+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:24:25.133993+0000","last_undegraded":"2024-09-11T01:24:25.133993+0000","last_fullsized":"2024-09-11T01:24:25.133993+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20164,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135797+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:24:25.135797+0000","last_peered":"2024-09-11T01:24:25.135797+0000","last_clean":"2024-09-11T01:24:25.135797+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:24:25.135797+0000","last_undegraded":"2024-09-11T01:24:25.135797+0000","last_fullsized":"2024-09-11T01:24:25.135797+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20425,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135738+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:24:25.135738+0000","last_peered":"2024-09-11T01:24:25.135738+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:24:25.135738+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20088,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135679+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:24:25.135679+0000","last_peered":"2024-09-11T01:24:25.135679+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:24:25.135679+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20411,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138636+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:24:25.138636+0000","last_peered":"2024-09-11T01:24:25.138636+0000","last_clean":"2024-09-11T01:24:25.138636+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:24:25.138636+0000","last_undegraded":"2024-09-11T01:24:25.138636+0000","last_fullsized":"2024-09-11T01:24:25.138636+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139906+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:24:25.139906+0000","last_peered":"2024-09-11T01:24:25.139906+0000","last_clean":"2024-09-11T01:24:25.139906+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:24:25.139906+0000","last_undegraded":"2024-09-11T01:24:25.139906+0000","last_fullsized":"2024-09-11T01:24:25.139906+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28009,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142299+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:24:25.142299+0000","last_peered":"2024-09-11T01:24:25.142299+0000","last_clean":"2024-09-11T01:24:25.142299+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:24:25.142299+0000","last_undegraded":"2024-09-11T01:24:25.142299+0000","last_fullsized":"2024-09-11T01:24:25.142299+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.134130+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:24:25.134130+0000","last_peered":"2024-09-11T01:24:25.134130+0000","last_clean":"2024-09-11T01:24:25.134130+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:24:25.134130+0000","last_undegraded":"2024-09-11T01:24:25.134130+0000","last_fullsized":"2024-09-11T01:24:25.134130+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20024,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.137211+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:24:25.137211+0000","last_peered":"2024-09-11T01:24:25.137211+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:24:25.137211+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":129,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135666+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:24:25.135666+0000","last_peered":"2024-09-11T01:24:25.135666+0000","last_clean":"2024-09-11T01:24:25.135666+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:24:25.135666+0000","last_undegraded":"2024-09-11T01:24:25.135666+0000","last_fullsized":"2024-09-11T01:24:25.135666+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28414,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.266137+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:24:25.266137+0000","last_peered":"2024-09-11T01:24:25.266137+0000","last_clean":"2024-09-11T01:24:25.266137+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:24:25.266137+0000","last_undegraded":"2024-09-11T01:24:25.266137+0000","last_fullsized":"2024-09-11T01:24:25.266137+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142149+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:24:25.142149+0000","last_peered":"2024-09-11T01:24:25.142149+0000","last_clean":"2024-09-11T01:24:25.142149+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:24:25.142149+0000","last_undegraded":"2024-09-11T01:24:25.142149+0000","last_fullsized":"2024-09-11T01:24:25.142149+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27805,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.132991+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:24:25.132991+0000","last_peered":"2024-09-11T01:24:25.132991+0000","last_clean":"2024-09-11T01:24:25.132991+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:24:25.132991+0000","last_undegraded":"2024-09-11T01:24:25.132991+0000","last_fullsized":"2024-09-11T01:24:25.132991+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":25202,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:48.268762+0000","last_change":"2024-09-11T01:24:23.706628+0000","last_active":"2024-09-11T01:24:48.268762+0000","last_peered":"2024-09-11T01:24:48.268762+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:24:48.268762+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1562,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2389,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1562}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142487+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:24:25.142487+0000","last_peered":"2024-09-11T01:24:25.142487+0000","last_clean":"2024-09-11T01:24:25.142487+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:24:25.142487+0000","last_undegraded":"2024-09-11T01:24:25.142487+0000","last_fullsized":"2024-09-11T01:24:25.142487+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27943,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142784+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:24:25.142784+0000","last_peered":"2024-09-11T01:24:25.142784+0000","last_clean":"2024-09-11T01:24:25.142784+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:24:25.142784+0000","last_undegraded":"2024-09-11T01:24:25.142784+0000","last_fullsized":"2024-09-11T01:24:25.142784+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138018+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:24:25.138018+0000","last_peered":"2024-09-11T01:24:25.138018+0000","last_clean":"2024-09-11T01:24:25.138018+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:24:25.138018+0000","last_undegraded":"2024-09-11T01:24:25.138018+0000","last_fullsized":"2024-09-11T01:24:25.138018+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24622,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142656+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:24:25.142656+0000","last_peered":"2024-09-11T01:24:25.142656+0000","last_clean":"2024-09-11T01:24:25.142656+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:24:25.142656+0000","last_undegraded":"2024-09-11T01:24:25.142656+0000","last_fullsized":"2024-09-11T01:24:25.142656+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142712+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:24:25.142712+0000","last_peered":"2024-09-11T01:24:25.142712+0000","last_clean":"2024-09-11T01:24:25.142712+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:24:25.142712+0000","last_undegraded":"2024-09-11T01:24:25.142712+0000","last_fullsized":"2024-09-11T01:24:25.142712+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142785+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:24:25.142785+0000","last_peered":"2024-09-11T01:24:25.142785+0000","last_clean":"2024-09-11T01:24:25.142785+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:24:25.142785+0000","last_undegraded":"2024-09-11T01:24:25.142785+0000","last_fullsized":"2024-09-11T01:24:25.142785+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20114,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.137564+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:24:25.137564+0000","last_peered":"2024-09-11T01:24:25.137564+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:24:25.137564+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":17409,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30230,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":185135104,"data_stored":185093356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182559,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518632,"kb_used_data":63576,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853208,"statfs":{"total":96636764160,"available":96105684992,"internally_reserved":0,"allocated":65101824,"data_stored":64779190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986332,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":478768,"kb_used_data":118112,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93893072,"statfs":{"total":96636764160,"available":96146505728,"internally_reserved":0,"allocated":120946688,"data_stored":120628131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659276,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491660,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93880180,"statfs":{"total":96636764160,"available":96133304320,"internally_reserved":0,"allocated":520192,"data_stored":166247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502918831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":120471552,"data_stored":120468073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:57.753 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:58.108 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:58.109 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:24:58.130 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":467,"stamp":"2024-09-11T01:24:56.851337+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16949,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30691,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1474780,"kb_used_data":180396,"kb_used_omap":12,"kb_used_meta":1294323,"kb_avail":281640740,"statfs":{"total":289910292480,"available":288400117760,"internally_reserved":0,"allocated":184725504,"data_stored":183730368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1325386756},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-883,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":883,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004119"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28523,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139109+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:24:25.139109+0000","last_peered":"2024-09-11T01:24:25.139109+0000","last_clean":"2024-09-11T01:24:25.139109+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:24:25.139109+0000","last_undegraded":"2024-09-11T01:24:25.139109+0000","last_fullsized":"2024-09-11T01:24:25.139109+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20481,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138008+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:24:25.138008+0000","last_peered":"2024-09-11T01:24:25.138008+0000","last_clean":"2024-09-11T01:24:25.138008+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:24:25.138008+0000","last_undegraded":"2024-09-11T01:24:25.138008+0000","last_fullsized":"2024-09-11T01:24:25.138008+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20304,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142521+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:24:25.142521+0000","last_peered":"2024-09-11T01:24:25.142521+0000","last_clean":"2024-09-11T01:24:25.142521+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:24:25.142521+0000","last_undegraded":"2024-09-11T01:24:25.142521+0000","last_fullsized":"2024-09-11T01:24:25.142521+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20231,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.133993+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:24:25.133993+0000","last_peered":"2024-09-11T01:24:25.133993+0000","last_clean":"2024-09-11T01:24:25.133993+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:24:25.133993+0000","last_undegraded":"2024-09-11T01:24:25.133993+0000","last_fullsized":"2024-09-11T01:24:25.133993+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20164,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135797+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:24:25.135797+0000","last_peered":"2024-09-11T01:24:25.135797+0000","last_clean":"2024-09-11T01:24:25.135797+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:24:25.135797+0000","last_undegraded":"2024-09-11T01:24:25.135797+0000","last_fullsized":"2024-09-11T01:24:25.135797+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20425,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135738+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:24:25.135738+0000","last_peered":"2024-09-11T01:24:25.135738+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:24:25.135738+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20088,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135679+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:24:25.135679+0000","last_peered":"2024-09-11T01:24:25.135679+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:24:25.135679+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20411,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138636+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:24:25.138636+0000","last_peered":"2024-09-11T01:24:25.138636+0000","last_clean":"2024-09-11T01:24:25.138636+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:24:25.138636+0000","last_undegraded":"2024-09-11T01:24:25.138636+0000","last_fullsized":"2024-09-11T01:24:25.138636+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139906+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:24:25.139906+0000","last_peered":"2024-09-11T01:24:25.139906+0000","last_clean":"2024-09-11T01:24:25.139906+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:24:25.139906+0000","last_undegraded":"2024-09-11T01:24:25.139906+0000","last_fullsized":"2024-09-11T01:24:25.139906+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28009,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142299+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:24:25.142299+0000","last_peered":"2024-09-11T01:24:25.142299+0000","last_clean":"2024-09-11T01:24:25.142299+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:24:25.142299+0000","last_undegraded":"2024-09-11T01:24:25.142299+0000","last_fullsized":"2024-09-11T01:24:25.142299+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.134130+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:24:25.134130+0000","last_peered":"2024-09-11T01:24:25.134130+0000","last_clean":"2024-09-11T01:24:25.134130+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:24:25.134130+0000","last_undegraded":"2024-09-11T01:24:25.134130+0000","last_fullsized":"2024-09-11T01:24:25.134130+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20024,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.137211+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:24:25.137211+0000","last_peered":"2024-09-11T01:24:25.137211+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:24:25.137211+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":129,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135666+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:24:25.135666+0000","last_peered":"2024-09-11T01:24:25.135666+0000","last_clean":"2024-09-11T01:24:25.135666+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:24:25.135666+0000","last_undegraded":"2024-09-11T01:24:25.135666+0000","last_fullsized":"2024-09-11T01:24:25.135666+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28414,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.266137+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:24:25.266137+0000","last_peered":"2024-09-11T01:24:25.266137+0000","last_clean":"2024-09-11T01:24:25.266137+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:24:25.266137+0000","last_undegraded":"2024-09-11T01:24:25.266137+0000","last_fullsized":"2024-09-11T01:24:25.266137+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142149+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:24:25.142149+0000","last_peered":"2024-09-11T01:24:25.142149+0000","last_clean":"2024-09-11T01:24:25.142149+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:24:25.142149+0000","last_undegraded":"2024-09-11T01:24:25.142149+0000","last_fullsized":"2024-09-11T01:24:25.142149+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27805,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.132991+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:24:25.132991+0000","last_peered":"2024-09-11T01:24:25.132991+0000","last_clean":"2024-09-11T01:24:25.132991+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:24:25.132991+0000","last_undegraded":"2024-09-11T01:24:25.132991+0000","last_fullsized":"2024-09-11T01:24:25.132991+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":26122,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:53.271728+0000","last_change":"2024-09-11T01:24:23.706628+0000","last_active":"2024-09-11T01:24:53.271728+0000","last_peered":"2024-09-11T01:24:53.271728+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:24:53.271728+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1102,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2849,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142487+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:24:25.142487+0000","last_peered":"2024-09-11T01:24:25.142487+0000","last_clean":"2024-09-11T01:24:25.142487+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:24:25.142487+0000","last_undegraded":"2024-09-11T01:24:25.142487+0000","last_fullsized":"2024-09-11T01:24:25.142487+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27943,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142784+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:24:25.142784+0000","last_peered":"2024-09-11T01:24:25.142784+0000","last_clean":"2024-09-11T01:24:25.142784+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:24:25.142784+0000","last_undegraded":"2024-09-11T01:24:25.142784+0000","last_fullsized":"2024-09-11T01:24:25.142784+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138018+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:24:25.138018+0000","last_peered":"2024-09-11T01:24:25.138018+0000","last_clean":"2024-09-11T01:24:25.138018+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:24:25.138018+0000","last_undegraded":"2024-09-11T01:24:25.138018+0000","last_fullsized":"2024-09-11T01:24:25.138018+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24622,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142656+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:24:25.142656+0000","last_peered":"2024-09-11T01:24:25.142656+0000","last_clean":"2024-09-11T01:24:25.142656+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:24:25.142656+0000","last_undegraded":"2024-09-11T01:24:25.142656+0000","last_fullsized":"2024-09-11T01:24:25.142656+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142712+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:24:25.142712+0000","last_peered":"2024-09-11T01:24:25.142712+0000","last_clean":"2024-09-11T01:24:25.142712+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:24:25.142712+0000","last_undegraded":"2024-09-11T01:24:25.142712+0000","last_fullsized":"2024-09-11T01:24:25.142712+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142785+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:24:25.142785+0000","last_peered":"2024-09-11T01:24:25.142785+0000","last_clean":"2024-09-11T01:24:25.142785+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:24:25.142785+0000","last_undegraded":"2024-09-11T01:24:25.142785+0000","last_fullsized":"2024-09-11T01:24:25.142785+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20114,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.137564+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:24:25.137564+0000","last_peered":"2024-09-11T01:24:25.137564+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:24:25.137564+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16949,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30690,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":183291904,"data_stored":183250156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182560,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518632,"kb_used_data":63576,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853208,"statfs":{"total":96636764160,"available":96105684992,"internally_reserved":0,"allocated":65101824,"data_stored":64779190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986333,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476968,"kb_used_data":116312,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93894872,"statfs":{"total":96636764160,"available":96148348928,"internally_reserved":0,"allocated":119103488,"data_stored":118784931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659277,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479180,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93892660,"statfs":{"total":96636764160,"available":96146083840,"internally_reserved":0,"allocated":520192,"data_stored":166247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":490139311},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":118628352,"data_stored":118624873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:58.132 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:58.466 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:58.467 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:24:58.484 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":467,"stamp":"2024-09-11T01:24:56.851337+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16949,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30691,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1474780,"kb_used_data":180396,"kb_used_omap":12,"kb_used_meta":1294323,"kb_avail":281640740,"statfs":{"total":289910292480,"available":288400117760,"internally_reserved":0,"allocated":184725504,"data_stored":183730368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1325386756},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-883,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":883,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004119"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28523,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139109+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:24:25.139109+0000","last_peered":"2024-09-11T01:24:25.139109+0000","last_clean":"2024-09-11T01:24:25.139109+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:24:25.139109+0000","last_undegraded":"2024-09-11T01:24:25.139109+0000","last_fullsized":"2024-09-11T01:24:25.139109+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20481,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138008+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:24:25.138008+0000","last_peered":"2024-09-11T01:24:25.138008+0000","last_clean":"2024-09-11T01:24:25.138008+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:24:25.138008+0000","last_undegraded":"2024-09-11T01:24:25.138008+0000","last_fullsized":"2024-09-11T01:24:25.138008+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20304,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142521+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:24:25.142521+0000","last_peered":"2024-09-11T01:24:25.142521+0000","last_clean":"2024-09-11T01:24:25.142521+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:24:25.142521+0000","last_undegraded":"2024-09-11T01:24:25.142521+0000","last_fullsized":"2024-09-11T01:24:25.142521+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20231,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.133993+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:24:25.133993+0000","last_peered":"2024-09-11T01:24:25.133993+0000","last_clean":"2024-09-11T01:24:25.133993+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:24:25.133993+0000","last_undegraded":"2024-09-11T01:24:25.133993+0000","last_fullsized":"2024-09-11T01:24:25.133993+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20164,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135797+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:24:25.135797+0000","last_peered":"2024-09-11T01:24:25.135797+0000","last_clean":"2024-09-11T01:24:25.135797+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:24:25.135797+0000","last_undegraded":"2024-09-11T01:24:25.135797+0000","last_fullsized":"2024-09-11T01:24:25.135797+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20425,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135738+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:24:25.135738+0000","last_peered":"2024-09-11T01:24:25.135738+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:24:25.135738+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20088,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135679+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:24:25.135679+0000","last_peered":"2024-09-11T01:24:25.135679+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:24:25.135679+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20411,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138636+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:24:25.138636+0000","last_peered":"2024-09-11T01:24:25.138636+0000","last_clean":"2024-09-11T01:24:25.138636+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:24:25.138636+0000","last_undegraded":"2024-09-11T01:24:25.138636+0000","last_fullsized":"2024-09-11T01:24:25.138636+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139906+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:24:25.139906+0000","last_peered":"2024-09-11T01:24:25.139906+0000","last_clean":"2024-09-11T01:24:25.139906+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:24:25.139906+0000","last_undegraded":"2024-09-11T01:24:25.139906+0000","last_fullsized":"2024-09-11T01:24:25.139906+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28009,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142299+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:24:25.142299+0000","last_peered":"2024-09-11T01:24:25.142299+0000","last_clean":"2024-09-11T01:24:25.142299+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:24:25.142299+0000","last_undegraded":"2024-09-11T01:24:25.142299+0000","last_fullsized":"2024-09-11T01:24:25.142299+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.134130+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:24:25.134130+0000","last_peered":"2024-09-11T01:24:25.134130+0000","last_clean":"2024-09-11T01:24:25.134130+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:24:25.134130+0000","last_undegraded":"2024-09-11T01:24:25.134130+0000","last_fullsized":"2024-09-11T01:24:25.134130+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20024,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.137211+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:24:25.137211+0000","last_peered":"2024-09-11T01:24:25.137211+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:24:25.137211+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":129,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135666+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:24:25.135666+0000","last_peered":"2024-09-11T01:24:25.135666+0000","last_clean":"2024-09-11T01:24:25.135666+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:24:25.135666+0000","last_undegraded":"2024-09-11T01:24:25.135666+0000","last_fullsized":"2024-09-11T01:24:25.135666+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28414,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.266137+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:24:25.266137+0000","last_peered":"2024-09-11T01:24:25.266137+0000","last_clean":"2024-09-11T01:24:25.266137+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:24:25.266137+0000","last_undegraded":"2024-09-11T01:24:25.266137+0000","last_fullsized":"2024-09-11T01:24:25.266137+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142149+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:24:25.142149+0000","last_peered":"2024-09-11T01:24:25.142149+0000","last_clean":"2024-09-11T01:24:25.142149+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:24:25.142149+0000","last_undegraded":"2024-09-11T01:24:25.142149+0000","last_fullsized":"2024-09-11T01:24:25.142149+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27805,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.132991+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:24:25.132991+0000","last_peered":"2024-09-11T01:24:25.132991+0000","last_clean":"2024-09-11T01:24:25.132991+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:24:25.132991+0000","last_undegraded":"2024-09-11T01:24:25.132991+0000","last_fullsized":"2024-09-11T01:24:25.132991+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":26122,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:53.271728+0000","last_change":"2024-09-11T01:24:23.706628+0000","last_active":"2024-09-11T01:24:53.271728+0000","last_peered":"2024-09-11T01:24:53.271728+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:24:53.271728+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1102,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2849,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1102}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142487+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:24:25.142487+0000","last_peered":"2024-09-11T01:24:25.142487+0000","last_clean":"2024-09-11T01:24:25.142487+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:24:25.142487+0000","last_undegraded":"2024-09-11T01:24:25.142487+0000","last_fullsized":"2024-09-11T01:24:25.142487+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27943,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142784+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:24:25.142784+0000","last_peered":"2024-09-11T01:24:25.142784+0000","last_clean":"2024-09-11T01:24:25.142784+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:24:25.142784+0000","last_undegraded":"2024-09-11T01:24:25.142784+0000","last_fullsized":"2024-09-11T01:24:25.142784+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138018+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:24:25.138018+0000","last_peered":"2024-09-11T01:24:25.138018+0000","last_clean":"2024-09-11T01:24:25.138018+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:24:25.138018+0000","last_undegraded":"2024-09-11T01:24:25.138018+0000","last_fullsized":"2024-09-11T01:24:25.138018+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24622,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142656+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:24:25.142656+0000","last_peered":"2024-09-11T01:24:25.142656+0000","last_clean":"2024-09-11T01:24:25.142656+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:24:25.142656+0000","last_undegraded":"2024-09-11T01:24:25.142656+0000","last_fullsized":"2024-09-11T01:24:25.142656+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142712+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:24:25.142712+0000","last_peered":"2024-09-11T01:24:25.142712+0000","last_clean":"2024-09-11T01:24:25.142712+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:24:25.142712+0000","last_undegraded":"2024-09-11T01:24:25.142712+0000","last_fullsized":"2024-09-11T01:24:25.142712+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142785+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:24:25.142785+0000","last_peered":"2024-09-11T01:24:25.142785+0000","last_clean":"2024-09-11T01:24:25.142785+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:24:25.142785+0000","last_undegraded":"2024-09-11T01:24:25.142785+0000","last_fullsized":"2024-09-11T01:24:25.142785+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20114,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.137564+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:24:25.137564+0000","last_peered":"2024-09-11T01:24:25.137564+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:24:25.137564+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16949,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":30690,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":183291904,"data_stored":183250156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182560,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518632,"kb_used_data":63576,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853208,"statfs":{"total":96636764160,"available":96105684992,"internally_reserved":0,"allocated":65101824,"data_stored":64779190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986333,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476968,"kb_used_data":116312,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93894872,"statfs":{"total":96636764160,"available":96148348928,"internally_reserved":0,"allocated":119103488,"data_stored":118784931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659277,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479180,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93892660,"statfs":{"total":96636764160,"available":96146083840,"internally_reserved":0,"allocated":520192,"data_stored":166247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":490139311},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":118628352,"data_stored":118624873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:24:58.485 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:24:58.486 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'20227", 'reported_seq': 20425, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:25.135738+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:24:25.135738+0000', 'last_peered': '2024-09-11T01:24:25.135738+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:24:25.135738+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16568, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:58.486 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:24:58.486 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20088, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:25.135679+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:24:25.135679+0000', 'last_peered': '2024-09-11T01:24:25.135679+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:24:25.135679+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:58.486 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:24:58.486 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20024, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:25.137211+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:24:25.137211+0000', 'last_peered': '2024-09-11T01:24:25.137211+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:24:25.137211+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:58.486 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:24:58.486 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20264", 'reported_seq': 26122, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:53.271728+0000', 'last_change': '2024-09-11T01:24:23.706628+0000', 'last_active': '2024-09-11T01:24:53.271728+0000', 'last_peered': '2024-09-11T01:24:53.271728+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:24:53.271728+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1102, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17001, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2849, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1102}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:58.486 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:24:58.486 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 20114, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:25.137564+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:24:25.137564+0000', 'last_peered': '2024-09-11T01:24:25.137564+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:24:25.137564+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:24:58.487 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:24:58.851 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:58.851 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 16949/6 objects degraded (282483.333%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":835,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":57,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":5},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1510174720,"bytes_avail":288400117760,"bytes_total":289910292480,"degraded_objects":16949,"degraded_total":6,"degraded_ratio":2824.8333333333339,"recovering_objects_per_sec":73,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":883,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:24:58.871 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:24:58.872 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:24:59.188 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:24:59.189 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:24:59.205 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":468,"stamp":"2024-09-11T01:24:58.851950+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16377,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31263,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1478940,"kb_used_data":180396,"kb_used_omap":12,"kb_used_meta":1298483,"kb_avail":281636580,"statfs":{"total":289910292480,"available":288395857920,"internally_reserved":0,"allocated":184725504,"data_stored":183730368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1329646596},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1455,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1455,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004300"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28523,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139109+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:24:25.139109+0000","last_peered":"2024-09-11T01:24:25.139109+0000","last_clean":"2024-09-11T01:24:25.139109+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:24:25.139109+0000","last_undegraded":"2024-09-11T01:24:25.139109+0000","last_fullsized":"2024-09-11T01:24:25.139109+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20481,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138008+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:24:25.138008+0000","last_peered":"2024-09-11T01:24:25.138008+0000","last_clean":"2024-09-11T01:24:25.138008+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:24:25.138008+0000","last_undegraded":"2024-09-11T01:24:25.138008+0000","last_fullsized":"2024-09-11T01:24:25.138008+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20304,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142521+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:24:25.142521+0000","last_peered":"2024-09-11T01:24:25.142521+0000","last_clean":"2024-09-11T01:24:25.142521+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:24:25.142521+0000","last_undegraded":"2024-09-11T01:24:25.142521+0000","last_fullsized":"2024-09-11T01:24:25.142521+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20231,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.133993+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:24:25.133993+0000","last_peered":"2024-09-11T01:24:25.133993+0000","last_clean":"2024-09-11T01:24:25.133993+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:24:25.133993+0000","last_undegraded":"2024-09-11T01:24:25.133993+0000","last_fullsized":"2024-09-11T01:24:25.133993+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20164,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135797+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:24:25.135797+0000","last_peered":"2024-09-11T01:24:25.135797+0000","last_clean":"2024-09-11T01:24:25.135797+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:24:25.135797+0000","last_undegraded":"2024-09-11T01:24:25.135797+0000","last_fullsized":"2024-09-11T01:24:25.135797+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20425,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135738+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:24:25.135738+0000","last_peered":"2024-09-11T01:24:25.135738+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:24:25.135738+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20088,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135679+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:24:25.135679+0000","last_peered":"2024-09-11T01:24:25.135679+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:24:25.135679+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20412,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138779+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:24:25.138779+0000","last_peered":"2024-09-11T01:24:25.138779+0000","last_clean":"2024-09-11T01:24:25.138779+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:24:25.138779+0000","last_undegraded":"2024-09-11T01:24:25.138779+0000","last_fullsized":"2024-09-11T01:24:25.138779+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139906+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:24:25.139906+0000","last_peered":"2024-09-11T01:24:25.139906+0000","last_clean":"2024-09-11T01:24:25.139906+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:24:25.139906+0000","last_undegraded":"2024-09-11T01:24:25.139906+0000","last_fullsized":"2024-09-11T01:24:25.139906+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28009,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142299+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:24:25.142299+0000","last_peered":"2024-09-11T01:24:25.142299+0000","last_clean":"2024-09-11T01:24:25.142299+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:24:25.142299+0000","last_undegraded":"2024-09-11T01:24:25.142299+0000","last_fullsized":"2024-09-11T01:24:25.142299+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.134130+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:24:25.134130+0000","last_peered":"2024-09-11T01:24:25.134130+0000","last_clean":"2024-09-11T01:24:25.134130+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:24:25.134130+0000","last_undegraded":"2024-09-11T01:24:25.134130+0000","last_fullsized":"2024-09-11T01:24:25.134130+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20024,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.137211+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:24:25.137211+0000","last_peered":"2024-09-11T01:24:25.137211+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:24:25.137211+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":129,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135666+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:24:25.135666+0000","last_peered":"2024-09-11T01:24:25.135666+0000","last_clean":"2024-09-11T01:24:25.135666+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:24:25.135666+0000","last_undegraded":"2024-09-11T01:24:25.135666+0000","last_fullsized":"2024-09-11T01:24:25.135666+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28414,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.266137+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:24:25.266137+0000","last_peered":"2024-09-11T01:24:25.266137+0000","last_clean":"2024-09-11T01:24:25.266137+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:24:25.266137+0000","last_undegraded":"2024-09-11T01:24:25.266137+0000","last_fullsized":"2024-09-11T01:24:25.266137+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142149+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:24:25.142149+0000","last_peered":"2024-09-11T01:24:25.142149+0000","last_clean":"2024-09-11T01:24:25.142149+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:24:25.142149+0000","last_undegraded":"2024-09-11T01:24:25.142149+0000","last_fullsized":"2024-09-11T01:24:25.142149+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27805,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.132991+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:24:25.132991+0000","last_peered":"2024-09-11T01:24:25.132991+0000","last_clean":"2024-09-11T01:24:25.132991+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:24:25.132991+0000","last_undegraded":"2024-09-11T01:24:25.132991+0000","last_fullsized":"2024-09-11T01:24:25.132991+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":27266,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:58.272037+0000","last_change":"2024-09-11T01:24:23.706628+0000","last_active":"2024-09-11T01:24:58.272037+0000","last_peered":"2024-09-11T01:24:58.272037+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:24:58.272037+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":530,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-530}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142487+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:24:25.142487+0000","last_peered":"2024-09-11T01:24:25.142487+0000","last_clean":"2024-09-11T01:24:25.142487+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:24:25.142487+0000","last_undegraded":"2024-09-11T01:24:25.142487+0000","last_fullsized":"2024-09-11T01:24:25.142487+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27943,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142784+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:24:25.142784+0000","last_peered":"2024-09-11T01:24:25.142784+0000","last_clean":"2024-09-11T01:24:25.142784+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:24:25.142784+0000","last_undegraded":"2024-09-11T01:24:25.142784+0000","last_fullsized":"2024-09-11T01:24:25.142784+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138018+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:24:25.138018+0000","last_peered":"2024-09-11T01:24:25.138018+0000","last_clean":"2024-09-11T01:24:25.138018+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:24:25.138018+0000","last_undegraded":"2024-09-11T01:24:25.138018+0000","last_fullsized":"2024-09-11T01:24:25.138018+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24622,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142656+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:24:25.142656+0000","last_peered":"2024-09-11T01:24:25.142656+0000","last_clean":"2024-09-11T01:24:25.142656+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:24:25.142656+0000","last_undegraded":"2024-09-11T01:24:25.142656+0000","last_fullsized":"2024-09-11T01:24:25.142656+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142712+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:24:25.142712+0000","last_peered":"2024-09-11T01:24:25.142712+0000","last_clean":"2024-09-11T01:24:25.142712+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:24:25.142712+0000","last_undegraded":"2024-09-11T01:24:25.142712+0000","last_fullsized":"2024-09-11T01:24:25.142712+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142785+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:24:25.142785+0000","last_peered":"2024-09-11T01:24:25.142785+0000","last_clean":"2024-09-11T01:24:25.142785+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:24:25.142785+0000","last_undegraded":"2024-09-11T01:24:25.142785+0000","last_fullsized":"2024-09-11T01:24:25.142785+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20114,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.137564+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:24:25.137564+0000","last_peered":"2024-09-11T01:24:25.137564+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:24:25.137564+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16377,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31262,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":183291904,"data_stored":183250156,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182561,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518632,"kb_used_data":63576,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853208,"statfs":{"total":96636764160,"available":96105684992,"internally_reserved":0,"allocated":65101824,"data_stored":64779190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986333,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":476968,"kb_used_data":116312,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93894872,"statfs":{"total":96636764160,"available":96148348928,"internally_reserved":0,"allocated":119103488,"data_stored":118784931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659278,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483340,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":482811,"kb_avail":93888500,"statfs":{"total":96636764160,"available":96141824000,"internally_reserved":0,"allocated":520192,"data_stored":166247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494399151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":118628352,"data_stored":118624873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:02.208 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:02.532 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:02.533 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:25:02.554 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":470,"stamp":"2024-09-11T01:25:02.004082+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16377,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31263,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1477140,"kb_used_data":178596,"kb_used_omap":12,"kb_used_meta":1298483,"kb_avail":281638380,"statfs":{"total":289910292480,"available":288397701120,"internally_reserved":0,"allocated":182882304,"data_stored":181887168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1329646596},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1032,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.155072"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28523,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139109+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:24:25.139109+0000","last_peered":"2024-09-11T01:24:25.139109+0000","last_clean":"2024-09-11T01:24:25.139109+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:24:25.139109+0000","last_undegraded":"2024-09-11T01:24:25.139109+0000","last_fullsized":"2024-09-11T01:24:25.139109+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20481,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138008+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:24:25.138008+0000","last_peered":"2024-09-11T01:24:25.138008+0000","last_clean":"2024-09-11T01:24:25.138008+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:24:25.138008+0000","last_undegraded":"2024-09-11T01:24:25.138008+0000","last_fullsized":"2024-09-11T01:24:25.138008+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20304,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142521+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:24:25.142521+0000","last_peered":"2024-09-11T01:24:25.142521+0000","last_clean":"2024-09-11T01:24:25.142521+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:24:25.142521+0000","last_undegraded":"2024-09-11T01:24:25.142521+0000","last_fullsized":"2024-09-11T01:24:25.142521+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20231,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.133993+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:24:25.133993+0000","last_peered":"2024-09-11T01:24:25.133993+0000","last_clean":"2024-09-11T01:24:25.133993+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:24:25.133993+0000","last_undegraded":"2024-09-11T01:24:25.133993+0000","last_fullsized":"2024-09-11T01:24:25.133993+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20164,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135797+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:24:25.135797+0000","last_peered":"2024-09-11T01:24:25.135797+0000","last_clean":"2024-09-11T01:24:25.135797+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:24:25.135797+0000","last_undegraded":"2024-09-11T01:24:25.135797+0000","last_fullsized":"2024-09-11T01:24:25.135797+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20425,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135738+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:24:25.135738+0000","last_peered":"2024-09-11T01:24:25.135738+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:24:25.135738+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20088,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135679+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:24:25.135679+0000","last_peered":"2024-09-11T01:24:25.135679+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:24:25.135679+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20412,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138779+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:24:25.138779+0000","last_peered":"2024-09-11T01:24:25.138779+0000","last_clean":"2024-09-11T01:24:25.138779+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:24:25.138779+0000","last_undegraded":"2024-09-11T01:24:25.138779+0000","last_fullsized":"2024-09-11T01:24:25.138779+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139906+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:24:25.139906+0000","last_peered":"2024-09-11T01:24:25.139906+0000","last_clean":"2024-09-11T01:24:25.139906+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:24:25.139906+0000","last_undegraded":"2024-09-11T01:24:25.139906+0000","last_fullsized":"2024-09-11T01:24:25.139906+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28009,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142299+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:24:25.142299+0000","last_peered":"2024-09-11T01:24:25.142299+0000","last_clean":"2024-09-11T01:24:25.142299+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:24:25.142299+0000","last_undegraded":"2024-09-11T01:24:25.142299+0000","last_fullsized":"2024-09-11T01:24:25.142299+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.134130+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:24:25.134130+0000","last_peered":"2024-09-11T01:24:25.134130+0000","last_clean":"2024-09-11T01:24:25.134130+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:24:25.134130+0000","last_undegraded":"2024-09-11T01:24:25.134130+0000","last_fullsized":"2024-09-11T01:24:25.134130+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20024,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.137211+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:24:25.137211+0000","last_peered":"2024-09-11T01:24:25.137211+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:24:25.137211+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":129,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135666+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:24:25.135666+0000","last_peered":"2024-09-11T01:24:25.135666+0000","last_clean":"2024-09-11T01:24:25.135666+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:24:25.135666+0000","last_undegraded":"2024-09-11T01:24:25.135666+0000","last_fullsized":"2024-09-11T01:24:25.135666+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28414,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.266137+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:24:25.266137+0000","last_peered":"2024-09-11T01:24:25.266137+0000","last_clean":"2024-09-11T01:24:25.266137+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:24:25.266137+0000","last_undegraded":"2024-09-11T01:24:25.266137+0000","last_fullsized":"2024-09-11T01:24:25.266137+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142149+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:24:25.142149+0000","last_peered":"2024-09-11T01:24:25.142149+0000","last_clean":"2024-09-11T01:24:25.142149+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:24:25.142149+0000","last_undegraded":"2024-09-11T01:24:25.142149+0000","last_fullsized":"2024-09-11T01:24:25.142149+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27805,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.132991+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:24:25.132991+0000","last_peered":"2024-09-11T01:24:25.132991+0000","last_clean":"2024-09-11T01:24:25.132991+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:24:25.132991+0000","last_undegraded":"2024-09-11T01:24:25.132991+0000","last_fullsized":"2024-09-11T01:24:25.132991+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":27266,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:58.272037+0000","last_change":"2024-09-11T01:24:23.706628+0000","last_active":"2024-09-11T01:24:58.272037+0000","last_peered":"2024-09-11T01:24:58.272037+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:24:58.272037+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":530,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-530}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142487+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:24:25.142487+0000","last_peered":"2024-09-11T01:24:25.142487+0000","last_clean":"2024-09-11T01:24:25.142487+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:24:25.142487+0000","last_undegraded":"2024-09-11T01:24:25.142487+0000","last_fullsized":"2024-09-11T01:24:25.142487+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27943,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142784+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:24:25.142784+0000","last_peered":"2024-09-11T01:24:25.142784+0000","last_clean":"2024-09-11T01:24:25.142784+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:24:25.142784+0000","last_undegraded":"2024-09-11T01:24:25.142784+0000","last_fullsized":"2024-09-11T01:24:25.142784+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138018+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:24:25.138018+0000","last_peered":"2024-09-11T01:24:25.138018+0000","last_clean":"2024-09-11T01:24:25.138018+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:24:25.138018+0000","last_undegraded":"2024-09-11T01:24:25.138018+0000","last_fullsized":"2024-09-11T01:24:25.138018+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24622,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142656+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:24:25.142656+0000","last_peered":"2024-09-11T01:24:25.142656+0000","last_clean":"2024-09-11T01:24:25.142656+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:24:25.142656+0000","last_undegraded":"2024-09-11T01:24:25.142656+0000","last_fullsized":"2024-09-11T01:24:25.142656+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142712+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:24:25.142712+0000","last_peered":"2024-09-11T01:24:25.142712+0000","last_clean":"2024-09-11T01:24:25.142712+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:24:25.142712+0000","last_undegraded":"2024-09-11T01:24:25.142712+0000","last_fullsized":"2024-09-11T01:24:25.142712+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142785+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:24:25.142785+0000","last_peered":"2024-09-11T01:24:25.142785+0000","last_clean":"2024-09-11T01:24:25.142785+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:24:25.142785+0000","last_undegraded":"2024-09-11T01:24:25.142785+0000","last_fullsized":"2024-09-11T01:24:25.142785+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20114,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.137564+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:24:25.137564+0000","last_peered":"2024-09-11T01:24:25.137564+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:24:25.137564+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16377,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31262,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":181448704,"data_stored":181406956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182561,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518632,"kb_used_data":63576,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853208,"statfs":{"total":96636764160,"available":96105684992,"internally_reserved":0,"allocated":65101824,"data_stored":64779190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986334,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":475168,"kb_used_data":114512,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93896672,"statfs":{"total":96636764160,"available":96150192128,"internally_reserved":0,"allocated":117260288,"data_stored":116941731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659278,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483340,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":482811,"kb_avail":93888500,"statfs":{"total":96636764160,"available":96141824000,"internally_reserved":0,"allocated":520192,"data_stored":166247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494399151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116785152,"data_stored":116781673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:02.555 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:02.931 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:02.932 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:25:02.955 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":471,"stamp":"2024-09-11T01:25:02.853040+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16377,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31263,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1477140,"kb_used_data":178596,"kb_used_omap":12,"kb_used_meta":1298483,"kb_avail":281638380,"statfs":{"total":289910292480,"available":288397701120,"internally_reserved":0,"allocated":182882304,"data_stored":181887168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1329646596},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1032,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1032,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.003232"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28523,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139109+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:24:25.139109+0000","last_peered":"2024-09-11T01:24:25.139109+0000","last_clean":"2024-09-11T01:24:25.139109+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:24:25.139109+0000","last_undegraded":"2024-09-11T01:24:25.139109+0000","last_fullsized":"2024-09-11T01:24:25.139109+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20481,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138008+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:24:25.138008+0000","last_peered":"2024-09-11T01:24:25.138008+0000","last_clean":"2024-09-11T01:24:25.138008+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:24:25.138008+0000","last_undegraded":"2024-09-11T01:24:25.138008+0000","last_fullsized":"2024-09-11T01:24:25.138008+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20304,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142521+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:24:25.142521+0000","last_peered":"2024-09-11T01:24:25.142521+0000","last_clean":"2024-09-11T01:24:25.142521+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:24:25.142521+0000","last_undegraded":"2024-09-11T01:24:25.142521+0000","last_fullsized":"2024-09-11T01:24:25.142521+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20231,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.133993+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:24:25.133993+0000","last_peered":"2024-09-11T01:24:25.133993+0000","last_clean":"2024-09-11T01:24:25.133993+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:24:25.133993+0000","last_undegraded":"2024-09-11T01:24:25.133993+0000","last_fullsized":"2024-09-11T01:24:25.133993+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20164,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135797+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:24:25.135797+0000","last_peered":"2024-09-11T01:24:25.135797+0000","last_clean":"2024-09-11T01:24:25.135797+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:24:25.135797+0000","last_undegraded":"2024-09-11T01:24:25.135797+0000","last_fullsized":"2024-09-11T01:24:25.135797+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20425,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135738+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:24:25.135738+0000","last_peered":"2024-09-11T01:24:25.135738+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:24:25.135738+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20088,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135679+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:24:25.135679+0000","last_peered":"2024-09-11T01:24:25.135679+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:24:25.135679+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20412,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138779+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:24:25.138779+0000","last_peered":"2024-09-11T01:24:25.138779+0000","last_clean":"2024-09-11T01:24:25.138779+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:24:25.138779+0000","last_undegraded":"2024-09-11T01:24:25.138779+0000","last_fullsized":"2024-09-11T01:24:25.138779+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139906+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:24:25.139906+0000","last_peered":"2024-09-11T01:24:25.139906+0000","last_clean":"2024-09-11T01:24:25.139906+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:24:25.139906+0000","last_undegraded":"2024-09-11T01:24:25.139906+0000","last_fullsized":"2024-09-11T01:24:25.139906+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28009,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142299+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:24:25.142299+0000","last_peered":"2024-09-11T01:24:25.142299+0000","last_clean":"2024-09-11T01:24:25.142299+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:24:25.142299+0000","last_undegraded":"2024-09-11T01:24:25.142299+0000","last_fullsized":"2024-09-11T01:24:25.142299+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.134130+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:24:25.134130+0000","last_peered":"2024-09-11T01:24:25.134130+0000","last_clean":"2024-09-11T01:24:25.134130+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:24:25.134130+0000","last_undegraded":"2024-09-11T01:24:25.134130+0000","last_fullsized":"2024-09-11T01:24:25.134130+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20024,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.137211+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:24:25.137211+0000","last_peered":"2024-09-11T01:24:25.137211+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:24:25.137211+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":129,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135666+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:24:25.135666+0000","last_peered":"2024-09-11T01:24:25.135666+0000","last_clean":"2024-09-11T01:24:25.135666+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:24:25.135666+0000","last_undegraded":"2024-09-11T01:24:25.135666+0000","last_fullsized":"2024-09-11T01:24:25.135666+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28414,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.266137+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:24:25.266137+0000","last_peered":"2024-09-11T01:24:25.266137+0000","last_clean":"2024-09-11T01:24:25.266137+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:24:25.266137+0000","last_undegraded":"2024-09-11T01:24:25.266137+0000","last_fullsized":"2024-09-11T01:24:25.266137+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142149+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:24:25.142149+0000","last_peered":"2024-09-11T01:24:25.142149+0000","last_clean":"2024-09-11T01:24:25.142149+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:24:25.142149+0000","last_undegraded":"2024-09-11T01:24:25.142149+0000","last_fullsized":"2024-09-11T01:24:25.142149+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27805,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.132991+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:24:25.132991+0000","last_peered":"2024-09-11T01:24:25.132991+0000","last_clean":"2024-09-11T01:24:25.132991+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:24:25.132991+0000","last_undegraded":"2024-09-11T01:24:25.132991+0000","last_fullsized":"2024-09-11T01:24:25.132991+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":27266,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:58.272037+0000","last_change":"2024-09-11T01:24:23.706628+0000","last_active":"2024-09-11T01:24:58.272037+0000","last_peered":"2024-09-11T01:24:58.272037+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:24:58.272037+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":530,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-530}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142487+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:24:25.142487+0000","last_peered":"2024-09-11T01:24:25.142487+0000","last_clean":"2024-09-11T01:24:25.142487+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:24:25.142487+0000","last_undegraded":"2024-09-11T01:24:25.142487+0000","last_fullsized":"2024-09-11T01:24:25.142487+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27943,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142784+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:24:25.142784+0000","last_peered":"2024-09-11T01:24:25.142784+0000","last_clean":"2024-09-11T01:24:25.142784+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:24:25.142784+0000","last_undegraded":"2024-09-11T01:24:25.142784+0000","last_fullsized":"2024-09-11T01:24:25.142784+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138018+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:24:25.138018+0000","last_peered":"2024-09-11T01:24:25.138018+0000","last_clean":"2024-09-11T01:24:25.138018+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:24:25.138018+0000","last_undegraded":"2024-09-11T01:24:25.138018+0000","last_fullsized":"2024-09-11T01:24:25.138018+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24622,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142656+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:24:25.142656+0000","last_peered":"2024-09-11T01:24:25.142656+0000","last_clean":"2024-09-11T01:24:25.142656+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:24:25.142656+0000","last_undegraded":"2024-09-11T01:24:25.142656+0000","last_fullsized":"2024-09-11T01:24:25.142656+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142712+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:24:25.142712+0000","last_peered":"2024-09-11T01:24:25.142712+0000","last_clean":"2024-09-11T01:24:25.142712+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:24:25.142712+0000","last_undegraded":"2024-09-11T01:24:25.142712+0000","last_fullsized":"2024-09-11T01:24:25.142712+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142785+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:24:25.142785+0000","last_peered":"2024-09-11T01:24:25.142785+0000","last_clean":"2024-09-11T01:24:25.142785+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:24:25.142785+0000","last_undegraded":"2024-09-11T01:24:25.142785+0000","last_fullsized":"2024-09-11T01:24:25.142785+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20114,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.137564+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:24:25.137564+0000","last_peered":"2024-09-11T01:24:25.137564+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:24:25.137564+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16377,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31262,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":181448704,"data_stored":181406956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182561,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518632,"kb_used_data":63576,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853208,"statfs":{"total":96636764160,"available":96105684992,"internally_reserved":0,"allocated":65101824,"data_stored":64779190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986334,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":475168,"kb_used_data":114512,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93896672,"statfs":{"total":96636764160,"available":96150192128,"internally_reserved":0,"allocated":117260288,"data_stored":116941731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659278,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483340,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":482811,"kb_avail":93888500,"statfs":{"total":96636764160,"available":96141824000,"internally_reserved":0,"allocated":520192,"data_stored":166247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494399151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116785152,"data_stored":116781673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:02.956 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:25:02.957 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'20227", 'reported_seq': 20425, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:25.135738+0000', 'last_change': '2024-09-11T01:13:10.279613+0000', 'last_active': '2024-09-11T01:24:25.135738+0000', 'last_peered': '2024-09-11T01:24:25.135738+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:24:25.135738+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3953, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16568, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3951}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:02.957 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:25:02.957 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20088, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:25.135679+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:24:25.135679+0000', 'last_peered': '2024-09-11T01:24:25.135679+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:24:25.135679+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:02.957 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:25:02.957 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20024, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:25.137211+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:24:25.137211+0000', 'last_peered': '2024-09-11T01:24:25.137211+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:24:25.137211+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:02.957 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.7 is not active+clean 2024-09-11T01:25:02.958 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.7', 'version': "49'20264", 'reported_seq': 27266, 'reported_epoch': 57, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:58.272037+0000', 'last_change': '2024-09-11T01:24:23.706628+0000', 'last_active': '2024-09-11T01:24:58.272037+0000', 'last_peered': '2024-09-11T01:24:58.272037+0000', 'last_clean': '2024-09-11T01:13:07.595114+0000', 'last_became_active': '2024-09-11T01:13:10.266657+0000', 'last_became_peered': '2024-09-11T01:13:10.266657+0000', 'last_unstale': '2024-09-11T01:24:58.272037+0000', 'last_undegraded': '2024-09-11T01:13:09.796243+0000', 'last_fullsized': '2024-09-11T01:13:09.656549+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3338", 'last_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:56.185967+0000', 'objects_scrubbed': 3261, 'log_size': 10064, 'log_dups_size': 0, 'ondisk_log_size': 10064, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 3, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 2055, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 530, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17001, 'num_write_kb': 27480, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3421, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -530}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:02.958 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:25:02.958 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 20114, 'reported_epoch': 57, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:24:25.137564+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:24:25.137564+0000', 'last_peered': '2024-09-11T01:24:25.137564+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:24:25.137564+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:02.958 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:25:03.379 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:03.379 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 16377/6 objects degraded (272950.000%), 5 pgs degraded, 5 pgs undersized","count":10},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":839,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":19},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":4},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1512591360,"bytes_avail":288397701120,"bytes_total":289910292480,"degraded_objects":16377,"degraded_total":6,"degraded_ratio":2729.5,"recovering_objects_per_sec":103,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1032,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:25:03.400 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:25:03.400 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:03.747 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:03.747 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:25:03.766 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":472,"stamp":"2024-09-11T01:25:03.012635+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16377,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31263,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":43,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1477140,"kb_used_data":178596,"kb_used_omap":12,"kb_used_meta":1298483,"kb_avail":281638380,"statfs":{"total":289910292480,"available":288397701120,"internally_reserved":0,"allocated":182882304,"data_stored":181887168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1329646596},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-572,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":572,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.162094"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28523,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139109+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:24:25.139109+0000","last_peered":"2024-09-11T01:24:25.139109+0000","last_clean":"2024-09-11T01:24:25.139109+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:24:25.139109+0000","last_undegraded":"2024-09-11T01:24:25.139109+0000","last_fullsized":"2024-09-11T01:24:25.139109+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20481,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138008+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:24:25.138008+0000","last_peered":"2024-09-11T01:24:25.138008+0000","last_clean":"2024-09-11T01:24:25.138008+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:24:25.138008+0000","last_undegraded":"2024-09-11T01:24:25.138008+0000","last_fullsized":"2024-09-11T01:24:25.138008+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20304,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142521+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:24:25.142521+0000","last_peered":"2024-09-11T01:24:25.142521+0000","last_clean":"2024-09-11T01:24:25.142521+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:24:25.142521+0000","last_undegraded":"2024-09-11T01:24:25.142521+0000","last_fullsized":"2024-09-11T01:24:25.142521+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20231,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.133993+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:24:25.133993+0000","last_peered":"2024-09-11T01:24:25.133993+0000","last_clean":"2024-09-11T01:24:25.133993+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:24:25.133993+0000","last_undegraded":"2024-09-11T01:24:25.133993+0000","last_fullsized":"2024-09-11T01:24:25.133993+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20164,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135797+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:24:25.135797+0000","last_peered":"2024-09-11T01:24:25.135797+0000","last_clean":"2024-09-11T01:24:25.135797+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:24:25.135797+0000","last_undegraded":"2024-09-11T01:24:25.135797+0000","last_fullsized":"2024-09-11T01:24:25.135797+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20425,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135738+0000","last_change":"2024-09-11T01:13:10.279613+0000","last_active":"2024-09-11T01:24:25.135738+0000","last_peered":"2024-09-11T01:24:25.135738+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:24:25.135738+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3953,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3951}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20088,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.135679+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:24:25.135679+0000","last_peered":"2024-09-11T01:24:25.135679+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:24:25.135679+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20412,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138779+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:24:25.138779+0000","last_peered":"2024-09-11T01:24:25.138779+0000","last_clean":"2024-09-11T01:24:25.138779+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:24:25.138779+0000","last_undegraded":"2024-09-11T01:24:25.138779+0000","last_fullsized":"2024-09-11T01:24:25.138779+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.139906+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:24:25.139906+0000","last_peered":"2024-09-11T01:24:25.139906+0000","last_clean":"2024-09-11T01:24:25.139906+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:24:25.139906+0000","last_undegraded":"2024-09-11T01:24:25.139906+0000","last_fullsized":"2024-09-11T01:24:25.139906+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28009,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142299+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:24:25.142299+0000","last_peered":"2024-09-11T01:24:25.142299+0000","last_clean":"2024-09-11T01:24:25.142299+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:24:25.142299+0000","last_undegraded":"2024-09-11T01:24:25.142299+0000","last_fullsized":"2024-09-11T01:24:25.142299+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.134130+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:24:25.134130+0000","last_peered":"2024-09-11T01:24:25.134130+0000","last_clean":"2024-09-11T01:24:25.134130+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:24:25.134130+0000","last_undegraded":"2024-09-11T01:24:25.134130+0000","last_fullsized":"2024-09-11T01:24:25.134130+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20024,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.137211+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:24:25.137211+0000","last_peered":"2024-09-11T01:24:25.137211+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:24:25.137211+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":129,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.135666+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:24:25.135666+0000","last_peered":"2024-09-11T01:24:25.135666+0000","last_clean":"2024-09-11T01:24:25.135666+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:24:25.135666+0000","last_undegraded":"2024-09-11T01:24:25.135666+0000","last_fullsized":"2024-09-11T01:24:25.135666+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28414,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.266137+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:24:25.266137+0000","last_peered":"2024-09-11T01:24:25.266137+0000","last_clean":"2024-09-11T01:24:25.266137+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:24:25.266137+0000","last_undegraded":"2024-09-11T01:24:25.266137+0000","last_fullsized":"2024-09-11T01:24:25.266137+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":107,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142149+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:24:25.142149+0000","last_peered":"2024-09-11T01:24:25.142149+0000","last_clean":"2024-09-11T01:24:25.142149+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:24:25.142149+0000","last_undegraded":"2024-09-11T01:24:25.142149+0000","last_fullsized":"2024-09-11T01:24:25.142149+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27805,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.132991+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:24:25.132991+0000","last_peered":"2024-09-11T01:24:25.132991+0000","last_clean":"2024-09-11T01:24:25.132991+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:24:25.132991+0000","last_undegraded":"2024-09-11T01:24:25.132991+0000","last_fullsized":"2024-09-11T01:24:25.132991+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":27266,"reported_epoch":57,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:58.272037+0000","last_change":"2024-09-11T01:24:23.706628+0000","last_active":"2024-09-11T01:24:58.272037+0000","last_peered":"2024-09-11T01:24:58.272037+0000","last_clean":"2024-09-11T01:13:07.595114+0000","last_became_active":"2024-09-11T01:13:10.266657+0000","last_became_peered":"2024-09-11T01:13:10.266657+0000","last_unstale":"2024-09-11T01:24:58.272037+0000","last_undegraded":"2024-09-11T01:13:09.796243+0000","last_fullsized":"2024-09-11T01:13:09.656549+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"no scrub is scheduled","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":530,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3421,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-530}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142487+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:24:25.142487+0000","last_peered":"2024-09-11T01:24:25.142487+0000","last_clean":"2024-09-11T01:24:25.142487+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:24:25.142487+0000","last_undegraded":"2024-09-11T01:24:25.142487+0000","last_fullsized":"2024-09-11T01:24:25.142487+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27943,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142784+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:24:25.142784+0000","last_peered":"2024-09-11T01:24:25.142784+0000","last_clean":"2024-09-11T01:24:25.142784+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:24:25.142784+0000","last_undegraded":"2024-09-11T01:24:25.142784+0000","last_fullsized":"2024-09-11T01:24:25.142784+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":98,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.138018+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:24:25.138018+0000","last_peered":"2024-09-11T01:24:25.138018+0000","last_clean":"2024-09-11T01:24:25.138018+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:24:25.138018+0000","last_undegraded":"2024-09-11T01:24:25.138018+0000","last_fullsized":"2024-09-11T01:24:25.138018+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24622,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142656+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:24:25.142656+0000","last_peered":"2024-09-11T01:24:25.142656+0000","last_clean":"2024-09-11T01:24:25.142656+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:24:25.142656+0000","last_undegraded":"2024-09-11T01:24:25.142656+0000","last_fullsized":"2024-09-11T01:24:25.142656+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":97,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142712+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:24:25.142712+0000","last_peered":"2024-09-11T01:24:25.142712+0000","last_clean":"2024-09-11T01:24:25.142712+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:24:25.142712+0000","last_undegraded":"2024-09-11T01:24:25.142712+0000","last_fullsized":"2024-09-11T01:24:25.142712+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":102,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-11T01:24:25.142785+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:24:25.142785+0000","last_peered":"2024-09-11T01:24:25.142785+0000","last_clean":"2024-09-11T01:24:25.142785+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:24:25.142785+0000","last_undegraded":"2024-09-11T01:24:25.142785+0000","last_fullsized":"2024-09-11T01:24:25.142785+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20114,"reported_epoch":57,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:24:25.137564+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:24:25.137564+0000","last_peered":"2024-09-11T01:24:25.137564+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:24:25.137564+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":16377,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31262,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":181448704,"data_stored":181406956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":27,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182561,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518632,"kb_used_data":63576,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853208,"statfs":{"total":96636764160,"available":96105684992,"internally_reserved":0,"allocated":65101824,"data_stored":64779190,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986334,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":475168,"kb_used_data":114512,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93896672,"statfs":{"total":96636764160,"available":96150192128,"internally_reserved":0,"allocated":117260288,"data_stored":116941731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659278,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483340,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":482811,"kb_avail":93888500,"statfs":{"total":96636764160,"available":96141824000,"internally_reserved":0,"allocated":520192,"data_stored":166247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494399151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":6170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":116785152,"data_stored":116781673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:06.768 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:07.149 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:07.152 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:25:07.171 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":474,"stamp":"2024-09-11T01:25:06.853854+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15647,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31993,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1479656,"kb_used_data":176952,"kb_used_omap":12,"kb_used_meta":1302643,"kb_avail":281635864,"statfs":{"total":289910292480,"available":288395124736,"internally_reserved":0,"allocated":181198848,"data_stored":180183818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1333906436},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-730,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":730,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001904"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28527,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.017457+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:25:03.017457+0000","last_peered":"2024-09-11T01:25:03.017457+0000","last_clean":"2024-09-11T01:25:03.017457+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:25:03.017457+0000","last_undegraded":"2024-09-11T01:25:03.017457+0000","last_fullsized":"2024-09-11T01:25:03.017457+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20485,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220829+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:25:03.220829+0000","last_peered":"2024-09-11T01:25:03.220829+0000","last_clean":"2024-09-11T01:25:03.220829+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:25:03.220829+0000","last_undegraded":"2024-09-11T01:25:03.220829+0000","last_fullsized":"2024-09-11T01:25:03.220829+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20308,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015371+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:25:03.015371+0000","last_peered":"2024-09-11T01:25:03.015371+0000","last_clean":"2024-09-11T01:25:03.015371+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:25:03.015371+0000","last_undegraded":"2024-09-11T01:25:03.015371+0000","last_fullsized":"2024-09-11T01:25:03.015371+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20235,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220654+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:25:03.220654+0000","last_peered":"2024-09-11T01:25:03.220654+0000","last_clean":"2024-09-11T01:25:03.220654+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:25:03.220654+0000","last_undegraded":"2024-09-11T01:25:03.220654+0000","last_fullsized":"2024-09-11T01:25:03.220654+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20168,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.013822+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:25:03.013822+0000","last_peered":"2024-09-11T01:25:03.013822+0000","last_clean":"2024-09-11T01:25:03.013822+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:25:03.013822+0000","last_undegraded":"2024-09-11T01:25:03.013822+0000","last_fullsized":"2024-09-11T01:25:03.013822+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20830,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:03.278291+0000","last_change":"2024-09-11T01:25:01.602361+0000","last_active":"2024-09-11T01:25:03.278291+0000","last_peered":"2024-09-11T01:25:03.278291+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:25:03.278291+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3753,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":200,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3753}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20092,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:03.013655+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:25:03.013655+0000","last_peered":"2024-09-11T01:25:03.013655+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:25:03.013655+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20416,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.013473+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:25:03.013473+0000","last_peered":"2024-09-11T01:25:03.013473+0000","last_clean":"2024-09-11T01:25:03.013473+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:25:03.013473+0000","last_undegraded":"2024-09-11T01:25:03.013473+0000","last_fullsized":"2024-09-11T01:25:03.013473+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.014898+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:25:03.014898+0000","last_peered":"2024-09-11T01:25:03.014898+0000","last_clean":"2024-09-11T01:25:03.014898+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:25:03.014898+0000","last_undegraded":"2024-09-11T01:25:03.014898+0000","last_fullsized":"2024-09-11T01:25:03.014898+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28013,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015230+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:25:03.015230+0000","last_peered":"2024-09-11T01:25:03.015230+0000","last_clean":"2024-09-11T01:25:03.015230+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:25:03.015230+0000","last_undegraded":"2024-09-11T01:25:03.015230+0000","last_fullsized":"2024-09-11T01:25:03.015230+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.217689+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:25:03.217689+0000","last_peered":"2024-09-11T01:25:03.217689+0000","last_clean":"2024-09-11T01:25:03.217689+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:25:03.217689+0000","last_undegraded":"2024-09-11T01:25:03.217689+0000","last_fullsized":"2024-09-11T01:25:03.217689+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20028,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:03.016969+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:25:03.016969+0000","last_peered":"2024-09-11T01:25:03.016969+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:25:03.016969+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":133,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.013350+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:25:03.013350+0000","last_peered":"2024-09-11T01:25:03.013350+0000","last_clean":"2024-09-11T01:25:03.013350+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:25:03.013350+0000","last_undegraded":"2024-09-11T01:25:03.013350+0000","last_fullsized":"2024-09-11T01:25:03.013350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28418,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220421+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:25:03.220421+0000","last_peered":"2024-09-11T01:25:03.220421+0000","last_clean":"2024-09-11T01:25:03.220421+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:25:03.220421+0000","last_undegraded":"2024-09-11T01:25:03.220421+0000","last_fullsized":"2024-09-11T01:25:03.220421+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015139+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:25:03.015139+0000","last_peered":"2024-09-11T01:25:03.015139+0000","last_clean":"2024-09-11T01:25:03.015139+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:25:03.015139+0000","last_undegraded":"2024-09-11T01:25:03.015139+0000","last_fullsized":"2024-09-11T01:25:03.015139+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27809,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.217608+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:25:03.217608+0000","last_peered":"2024-09-11T01:25:03.217608+0000","last_clean":"2024-09-11T01:25:03.217608+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:25:03.217608+0000","last_undegraded":"2024-09-11T01:25:03.217608+0000","last_fullsized":"2024-09-11T01:25:03.217608+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20419,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.116247+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:25:03.116247+0000","last_peered":"2024-09-11T01:25:03.116247+0000","last_clean":"2024-09-11T01:25:03.116247+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:25:03.116247+0000","last_undegraded":"2024-09-11T01:25:03.116247+0000","last_fullsized":"2024-09-11T01:25:03.116247+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015749+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:25:03.015749+0000","last_peered":"2024-09-11T01:25:03.015749+0000","last_clean":"2024-09-11T01:25:03.015749+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:25:03.015749+0000","last_undegraded":"2024-09-11T01:25:03.015749+0000","last_fullsized":"2024-09-11T01:25:03.015749+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27947,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015630+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:25:03.015630+0000","last_peered":"2024-09-11T01:25:03.015630+0000","last_clean":"2024-09-11T01:25:03.015630+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:25:03.015630+0000","last_undegraded":"2024-09-11T01:25:03.015630+0000","last_fullsized":"2024-09-11T01:25:03.015630+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220797+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:25:03.220797+0000","last_peered":"2024-09-11T01:25:03.220797+0000","last_clean":"2024-09-11T01:25:03.220797+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:25:03.220797+0000","last_undegraded":"2024-09-11T01:25:03.220797+0000","last_fullsized":"2024-09-11T01:25:03.220797+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24626,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015423+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:25:03.015423+0000","last_peered":"2024-09-11T01:25:03.015423+0000","last_clean":"2024-09-11T01:25:03.015423+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:25:03.015423+0000","last_undegraded":"2024-09-11T01:25:03.015423+0000","last_fullsized":"2024-09-11T01:25:03.015423+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015540+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:25:03.015540+0000","last_peered":"2024-09-11T01:25:03.015540+0000","last_clean":"2024-09-11T01:25:03.015540+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:25:03.015540+0000","last_undegraded":"2024-09-11T01:25:03.015540+0000","last_fullsized":"2024-09-11T01:25:03.015540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015405+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:25:03.015405+0000","last_peered":"2024-09-11T01:25:03.015405+0000","last_clean":"2024-09-11T01:25:03.015405+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:25:03.015405+0000","last_undegraded":"2024-09-11T01:25:03.015405+0000","last_fullsized":"2024-09-11T01:25:03.015405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20118,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:03.013826+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:25:03.013826+0000","last_peered":"2024-09-11T01:25:03.013826+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:25:03.013826+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15647,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31992,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":179232768,"data_stored":179197978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182562,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518648,"kb_used_data":63592,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853192,"statfs":{"total":96636764160,"available":96105668608,"internally_reserved":0,"allocated":65118208,"data_stored":64784829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986335,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473508,"kb_used_data":112852,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93898332,"statfs":{"total":96636764160,"available":96151891968,"internally_reserved":0,"allocated":115560448,"data_stored":115231146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659279,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487500,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":486971,"kb_avail":93884340,"statfs":{"total":96636764160,"available":96137564160,"internally_reserved":0,"allocated":520192,"data_stored":167843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":498658991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114577408,"data_stored":114573929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:07.173 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:07.568 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:07.569 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:25:07.590 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":474,"stamp":"2024-09-11T01:25:06.853854+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15647,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31993,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1479656,"kb_used_data":176952,"kb_used_omap":12,"kb_used_meta":1302643,"kb_avail":281635864,"statfs":{"total":289910292480,"available":288395124736,"internally_reserved":0,"allocated":181198848,"data_stored":180183818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1333906436},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-730,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":730,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001904"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28527,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.017457+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:25:03.017457+0000","last_peered":"2024-09-11T01:25:03.017457+0000","last_clean":"2024-09-11T01:25:03.017457+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:25:03.017457+0000","last_undegraded":"2024-09-11T01:25:03.017457+0000","last_fullsized":"2024-09-11T01:25:03.017457+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20485,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220829+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:25:03.220829+0000","last_peered":"2024-09-11T01:25:03.220829+0000","last_clean":"2024-09-11T01:25:03.220829+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:25:03.220829+0000","last_undegraded":"2024-09-11T01:25:03.220829+0000","last_fullsized":"2024-09-11T01:25:03.220829+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20308,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015371+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:25:03.015371+0000","last_peered":"2024-09-11T01:25:03.015371+0000","last_clean":"2024-09-11T01:25:03.015371+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:25:03.015371+0000","last_undegraded":"2024-09-11T01:25:03.015371+0000","last_fullsized":"2024-09-11T01:25:03.015371+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20235,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220654+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:25:03.220654+0000","last_peered":"2024-09-11T01:25:03.220654+0000","last_clean":"2024-09-11T01:25:03.220654+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:25:03.220654+0000","last_undegraded":"2024-09-11T01:25:03.220654+0000","last_fullsized":"2024-09-11T01:25:03.220654+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20168,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.013822+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:25:03.013822+0000","last_peered":"2024-09-11T01:25:03.013822+0000","last_clean":"2024-09-11T01:25:03.013822+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:25:03.013822+0000","last_undegraded":"2024-09-11T01:25:03.013822+0000","last_fullsized":"2024-09-11T01:25:03.013822+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20830,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:03.278291+0000","last_change":"2024-09-11T01:25:01.602361+0000","last_active":"2024-09-11T01:25:03.278291+0000","last_peered":"2024-09-11T01:25:03.278291+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:25:03.278291+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3753,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":200,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3753}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20092,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:03.013655+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:25:03.013655+0000","last_peered":"2024-09-11T01:25:03.013655+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:25:03.013655+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20416,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.013473+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:25:03.013473+0000","last_peered":"2024-09-11T01:25:03.013473+0000","last_clean":"2024-09-11T01:25:03.013473+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:25:03.013473+0000","last_undegraded":"2024-09-11T01:25:03.013473+0000","last_fullsized":"2024-09-11T01:25:03.013473+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.014898+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:25:03.014898+0000","last_peered":"2024-09-11T01:25:03.014898+0000","last_clean":"2024-09-11T01:25:03.014898+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:25:03.014898+0000","last_undegraded":"2024-09-11T01:25:03.014898+0000","last_fullsized":"2024-09-11T01:25:03.014898+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28013,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015230+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:25:03.015230+0000","last_peered":"2024-09-11T01:25:03.015230+0000","last_clean":"2024-09-11T01:25:03.015230+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:25:03.015230+0000","last_undegraded":"2024-09-11T01:25:03.015230+0000","last_fullsized":"2024-09-11T01:25:03.015230+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.217689+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:25:03.217689+0000","last_peered":"2024-09-11T01:25:03.217689+0000","last_clean":"2024-09-11T01:25:03.217689+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:25:03.217689+0000","last_undegraded":"2024-09-11T01:25:03.217689+0000","last_fullsized":"2024-09-11T01:25:03.217689+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20028,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:03.016969+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:25:03.016969+0000","last_peered":"2024-09-11T01:25:03.016969+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:25:03.016969+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":133,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.013350+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:25:03.013350+0000","last_peered":"2024-09-11T01:25:03.013350+0000","last_clean":"2024-09-11T01:25:03.013350+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:25:03.013350+0000","last_undegraded":"2024-09-11T01:25:03.013350+0000","last_fullsized":"2024-09-11T01:25:03.013350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28418,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220421+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:25:03.220421+0000","last_peered":"2024-09-11T01:25:03.220421+0000","last_clean":"2024-09-11T01:25:03.220421+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:25:03.220421+0000","last_undegraded":"2024-09-11T01:25:03.220421+0000","last_fullsized":"2024-09-11T01:25:03.220421+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015139+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:25:03.015139+0000","last_peered":"2024-09-11T01:25:03.015139+0000","last_clean":"2024-09-11T01:25:03.015139+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:25:03.015139+0000","last_undegraded":"2024-09-11T01:25:03.015139+0000","last_fullsized":"2024-09-11T01:25:03.015139+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27809,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.217608+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:25:03.217608+0000","last_peered":"2024-09-11T01:25:03.217608+0000","last_clean":"2024-09-11T01:25:03.217608+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:25:03.217608+0000","last_undegraded":"2024-09-11T01:25:03.217608+0000","last_fullsized":"2024-09-11T01:25:03.217608+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20419,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.116247+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:25:03.116247+0000","last_peered":"2024-09-11T01:25:03.116247+0000","last_clean":"2024-09-11T01:25:03.116247+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:25:03.116247+0000","last_undegraded":"2024-09-11T01:25:03.116247+0000","last_fullsized":"2024-09-11T01:25:03.116247+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015749+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:25:03.015749+0000","last_peered":"2024-09-11T01:25:03.015749+0000","last_clean":"2024-09-11T01:25:03.015749+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:25:03.015749+0000","last_undegraded":"2024-09-11T01:25:03.015749+0000","last_fullsized":"2024-09-11T01:25:03.015749+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27947,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015630+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:25:03.015630+0000","last_peered":"2024-09-11T01:25:03.015630+0000","last_clean":"2024-09-11T01:25:03.015630+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:25:03.015630+0000","last_undegraded":"2024-09-11T01:25:03.015630+0000","last_fullsized":"2024-09-11T01:25:03.015630+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220797+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:25:03.220797+0000","last_peered":"2024-09-11T01:25:03.220797+0000","last_clean":"2024-09-11T01:25:03.220797+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:25:03.220797+0000","last_undegraded":"2024-09-11T01:25:03.220797+0000","last_fullsized":"2024-09-11T01:25:03.220797+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24626,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015423+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:25:03.015423+0000","last_peered":"2024-09-11T01:25:03.015423+0000","last_clean":"2024-09-11T01:25:03.015423+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:25:03.015423+0000","last_undegraded":"2024-09-11T01:25:03.015423+0000","last_fullsized":"2024-09-11T01:25:03.015423+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015540+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:25:03.015540+0000","last_peered":"2024-09-11T01:25:03.015540+0000","last_clean":"2024-09-11T01:25:03.015540+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:25:03.015540+0000","last_undegraded":"2024-09-11T01:25:03.015540+0000","last_fullsized":"2024-09-11T01:25:03.015540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015405+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:25:03.015405+0000","last_peered":"2024-09-11T01:25:03.015405+0000","last_clean":"2024-09-11T01:25:03.015405+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:25:03.015405+0000","last_undegraded":"2024-09-11T01:25:03.015405+0000","last_fullsized":"2024-09-11T01:25:03.015405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20118,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:03.013826+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:25:03.013826+0000","last_peered":"2024-09-11T01:25:03.013826+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:25:03.013826+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15647,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31992,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":179232768,"data_stored":179197978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182562,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518648,"kb_used_data":63592,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853192,"statfs":{"total":96636764160,"available":96105668608,"internally_reserved":0,"allocated":65118208,"data_stored":64784829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986335,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473508,"kb_used_data":112852,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93898332,"statfs":{"total":96636764160,"available":96151891968,"internally_reserved":0,"allocated":115560448,"data_stored":115231146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659279,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487500,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":486971,"kb_avail":93884340,"statfs":{"total":96636764160,"available":96137564160,"internally_reserved":0,"allocated":520192,"data_stored":167843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":498658991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114577408,"data_stored":114573929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:07.592 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:25:07.592 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'20227", 'reported_seq': 20830, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:03.278291+0000', 'last_change': '2024-09-11T01:25:01.602361+0000', 'last_active': '2024-09-11T01:25:03.278291+0000', 'last_peered': '2024-09-11T01:25:03.278291+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:25:03.278291+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3753, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16568, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 200, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3753}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:07.592 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:25:07.592 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20092, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:03.013655+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:25:03.013655+0000', 'last_peered': '2024-09-11T01:25:03.013655+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:25:03.013655+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:07.592 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:25:07.592 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20028, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:03.016969+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:25:03.016969+0000', 'last_peered': '2024-09-11T01:25:03.016969+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:25:03.016969+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:07.592 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:25:07.593 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 20118, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:03.013826+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:25:03.013826+0000', 'last_peered': '2024-09-11T01:25:03.013826+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:25:03.013826+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:07.593 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:25:08.010 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:08.010 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 15647/6 objects degraded (260783.333%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":844,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1515167744,"bytes_avail":288395124736,"bytes_total":289910292480,"degraded_objects":15647,"degraded_total":6,"degraded_ratio":2607.8333333333335,"recovering_objects_per_sec":91,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":730,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:25:08.032 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:25:08.033 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:08.353 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:08.354 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:25:08.371 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":474,"stamp":"2024-09-11T01:25:06.853854+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15647,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31993,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1479656,"kb_used_data":176952,"kb_used_omap":12,"kb_used_meta":1302643,"kb_avail":281635864,"statfs":{"total":289910292480,"available":288395124736,"internally_reserved":0,"allocated":181198848,"data_stored":180183818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1333906436},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-730,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":730,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001904"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28527,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.017457+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:25:03.017457+0000","last_peered":"2024-09-11T01:25:03.017457+0000","last_clean":"2024-09-11T01:25:03.017457+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:25:03.017457+0000","last_undegraded":"2024-09-11T01:25:03.017457+0000","last_fullsized":"2024-09-11T01:25:03.017457+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20485,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220829+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:25:03.220829+0000","last_peered":"2024-09-11T01:25:03.220829+0000","last_clean":"2024-09-11T01:25:03.220829+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:25:03.220829+0000","last_undegraded":"2024-09-11T01:25:03.220829+0000","last_fullsized":"2024-09-11T01:25:03.220829+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20308,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015371+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:25:03.015371+0000","last_peered":"2024-09-11T01:25:03.015371+0000","last_clean":"2024-09-11T01:25:03.015371+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:25:03.015371+0000","last_undegraded":"2024-09-11T01:25:03.015371+0000","last_fullsized":"2024-09-11T01:25:03.015371+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20235,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220654+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:25:03.220654+0000","last_peered":"2024-09-11T01:25:03.220654+0000","last_clean":"2024-09-11T01:25:03.220654+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:25:03.220654+0000","last_undegraded":"2024-09-11T01:25:03.220654+0000","last_fullsized":"2024-09-11T01:25:03.220654+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20168,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.013822+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:25:03.013822+0000","last_peered":"2024-09-11T01:25:03.013822+0000","last_clean":"2024-09-11T01:25:03.013822+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:25:03.013822+0000","last_undegraded":"2024-09-11T01:25:03.013822+0000","last_fullsized":"2024-09-11T01:25:03.013822+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20830,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:03.278291+0000","last_change":"2024-09-11T01:25:01.602361+0000","last_active":"2024-09-11T01:25:03.278291+0000","last_peered":"2024-09-11T01:25:03.278291+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:25:03.278291+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3753,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":200,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3753}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20092,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:03.013655+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:25:03.013655+0000","last_peered":"2024-09-11T01:25:03.013655+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:25:03.013655+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20416,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.013473+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:25:03.013473+0000","last_peered":"2024-09-11T01:25:03.013473+0000","last_clean":"2024-09-11T01:25:03.013473+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:25:03.013473+0000","last_undegraded":"2024-09-11T01:25:03.013473+0000","last_fullsized":"2024-09-11T01:25:03.013473+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.014898+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:25:03.014898+0000","last_peered":"2024-09-11T01:25:03.014898+0000","last_clean":"2024-09-11T01:25:03.014898+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:25:03.014898+0000","last_undegraded":"2024-09-11T01:25:03.014898+0000","last_fullsized":"2024-09-11T01:25:03.014898+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28013,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015230+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:25:03.015230+0000","last_peered":"2024-09-11T01:25:03.015230+0000","last_clean":"2024-09-11T01:25:03.015230+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:25:03.015230+0000","last_undegraded":"2024-09-11T01:25:03.015230+0000","last_fullsized":"2024-09-11T01:25:03.015230+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.217689+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:25:03.217689+0000","last_peered":"2024-09-11T01:25:03.217689+0000","last_clean":"2024-09-11T01:25:03.217689+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:25:03.217689+0000","last_undegraded":"2024-09-11T01:25:03.217689+0000","last_fullsized":"2024-09-11T01:25:03.217689+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20028,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:03.016969+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:25:03.016969+0000","last_peered":"2024-09-11T01:25:03.016969+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:25:03.016969+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":133,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.013350+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:25:03.013350+0000","last_peered":"2024-09-11T01:25:03.013350+0000","last_clean":"2024-09-11T01:25:03.013350+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:25:03.013350+0000","last_undegraded":"2024-09-11T01:25:03.013350+0000","last_fullsized":"2024-09-11T01:25:03.013350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28418,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220421+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:25:03.220421+0000","last_peered":"2024-09-11T01:25:03.220421+0000","last_clean":"2024-09-11T01:25:03.220421+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:25:03.220421+0000","last_undegraded":"2024-09-11T01:25:03.220421+0000","last_fullsized":"2024-09-11T01:25:03.220421+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015139+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:25:03.015139+0000","last_peered":"2024-09-11T01:25:03.015139+0000","last_clean":"2024-09-11T01:25:03.015139+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:25:03.015139+0000","last_undegraded":"2024-09-11T01:25:03.015139+0000","last_fullsized":"2024-09-11T01:25:03.015139+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27809,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.217608+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:25:03.217608+0000","last_peered":"2024-09-11T01:25:03.217608+0000","last_clean":"2024-09-11T01:25:03.217608+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:25:03.217608+0000","last_undegraded":"2024-09-11T01:25:03.217608+0000","last_fullsized":"2024-09-11T01:25:03.217608+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20419,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.116247+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:25:03.116247+0000","last_peered":"2024-09-11T01:25:03.116247+0000","last_clean":"2024-09-11T01:25:03.116247+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:25:03.116247+0000","last_undegraded":"2024-09-11T01:25:03.116247+0000","last_fullsized":"2024-09-11T01:25:03.116247+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015749+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:25:03.015749+0000","last_peered":"2024-09-11T01:25:03.015749+0000","last_clean":"2024-09-11T01:25:03.015749+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:25:03.015749+0000","last_undegraded":"2024-09-11T01:25:03.015749+0000","last_fullsized":"2024-09-11T01:25:03.015749+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27947,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015630+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:25:03.015630+0000","last_peered":"2024-09-11T01:25:03.015630+0000","last_clean":"2024-09-11T01:25:03.015630+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:25:03.015630+0000","last_undegraded":"2024-09-11T01:25:03.015630+0000","last_fullsized":"2024-09-11T01:25:03.015630+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220797+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:25:03.220797+0000","last_peered":"2024-09-11T01:25:03.220797+0000","last_clean":"2024-09-11T01:25:03.220797+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:25:03.220797+0000","last_undegraded":"2024-09-11T01:25:03.220797+0000","last_fullsized":"2024-09-11T01:25:03.220797+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24626,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015423+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:25:03.015423+0000","last_peered":"2024-09-11T01:25:03.015423+0000","last_clean":"2024-09-11T01:25:03.015423+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:25:03.015423+0000","last_undegraded":"2024-09-11T01:25:03.015423+0000","last_fullsized":"2024-09-11T01:25:03.015423+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015540+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:25:03.015540+0000","last_peered":"2024-09-11T01:25:03.015540+0000","last_clean":"2024-09-11T01:25:03.015540+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:25:03.015540+0000","last_undegraded":"2024-09-11T01:25:03.015540+0000","last_fullsized":"2024-09-11T01:25:03.015540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015405+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:25:03.015405+0000","last_peered":"2024-09-11T01:25:03.015405+0000","last_clean":"2024-09-11T01:25:03.015405+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:25:03.015405+0000","last_undegraded":"2024-09-11T01:25:03.015405+0000","last_fullsized":"2024-09-11T01:25:03.015405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20118,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:03.013826+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:25:03.013826+0000","last_peered":"2024-09-11T01:25:03.013826+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:25:03.013826+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":15647,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":31992,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":179232768,"data_stored":179197978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182562,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518648,"kb_used_data":63592,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853192,"statfs":{"total":96636764160,"available":96105668608,"internally_reserved":0,"allocated":65118208,"data_stored":64784829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986335,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":473508,"kb_used_data":112852,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93898332,"statfs":{"total":96636764160,"available":96151891968,"internally_reserved":0,"allocated":115560448,"data_stored":115231146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659279,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487500,"kb_used_data":508,"kb_used_omap":4,"kb_used_meta":486971,"kb_avail":93884340,"statfs":{"total":96636764160,"available":96137564160,"internally_reserved":0,"allocated":520192,"data_stored":167843,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":498658991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":114577408,"data_stored":114573929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:11.377 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:11.759 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:11.759 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:25:11.783 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":476,"stamp":"2024-09-11T01:25:10.854981+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14896,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32744,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1468944,"kb_used_data":174560,"kb_used_omap":12,"kb_used_meta":1294323,"kb_avail":281646576,"statfs":{"total":289910292480,"available":288406093824,"internally_reserved":0,"allocated":178749440,"data_stored":177729027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1325386756},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1481,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1481,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.850899"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28527,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.017457+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:25:03.017457+0000","last_peered":"2024-09-11T01:25:03.017457+0000","last_clean":"2024-09-11T01:25:03.017457+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:25:03.017457+0000","last_undegraded":"2024-09-11T01:25:03.017457+0000","last_fullsized":"2024-09-11T01:25:03.017457+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20485,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220829+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:25:03.220829+0000","last_peered":"2024-09-11T01:25:03.220829+0000","last_clean":"2024-09-11T01:25:03.220829+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:25:03.220829+0000","last_undegraded":"2024-09-11T01:25:03.220829+0000","last_fullsized":"2024-09-11T01:25:03.220829+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20308,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015371+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:25:03.015371+0000","last_peered":"2024-09-11T01:25:03.015371+0000","last_clean":"2024-09-11T01:25:03.015371+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:25:03.015371+0000","last_undegraded":"2024-09-11T01:25:03.015371+0000","last_fullsized":"2024-09-11T01:25:03.015371+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20235,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220654+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:25:03.220654+0000","last_peered":"2024-09-11T01:25:03.220654+0000","last_clean":"2024-09-11T01:25:03.220654+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:25:03.220654+0000","last_undegraded":"2024-09-11T01:25:03.220654+0000","last_fullsized":"2024-09-11T01:25:03.220654+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20168,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.013822+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:25:03.013822+0000","last_peered":"2024-09-11T01:25:03.013822+0000","last_clean":"2024-09-11T01:25:03.013822+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:25:03.013822+0000","last_undegraded":"2024-09-11T01:25:03.013822+0000","last_fullsized":"2024-09-11T01:25:03.013822+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":22332,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:08.280099+0000","last_change":"2024-09-11T01:25:01.602361+0000","last_active":"2024-09-11T01:25:08.280099+0000","last_peered":"2024-09-11T01:25:08.280099+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:25:08.280099+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3002,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3002}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20092,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:03.013655+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:25:03.013655+0000","last_peered":"2024-09-11T01:25:03.013655+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:25:03.013655+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20416,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.013473+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:25:03.013473+0000","last_peered":"2024-09-11T01:25:03.013473+0000","last_clean":"2024-09-11T01:25:03.013473+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:25:03.013473+0000","last_undegraded":"2024-09-11T01:25:03.013473+0000","last_fullsized":"2024-09-11T01:25:03.013473+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.014898+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:25:03.014898+0000","last_peered":"2024-09-11T01:25:03.014898+0000","last_clean":"2024-09-11T01:25:03.014898+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:25:03.014898+0000","last_undegraded":"2024-09-11T01:25:03.014898+0000","last_fullsized":"2024-09-11T01:25:03.014898+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28013,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015230+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:25:03.015230+0000","last_peered":"2024-09-11T01:25:03.015230+0000","last_clean":"2024-09-11T01:25:03.015230+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:25:03.015230+0000","last_undegraded":"2024-09-11T01:25:03.015230+0000","last_fullsized":"2024-09-11T01:25:03.015230+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.217689+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:25:03.217689+0000","last_peered":"2024-09-11T01:25:03.217689+0000","last_clean":"2024-09-11T01:25:03.217689+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:25:03.217689+0000","last_undegraded":"2024-09-11T01:25:03.217689+0000","last_fullsized":"2024-09-11T01:25:03.217689+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20028,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:03.016969+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:25:03.016969+0000","last_peered":"2024-09-11T01:25:03.016969+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:25:03.016969+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":133,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.013350+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:25:03.013350+0000","last_peered":"2024-09-11T01:25:03.013350+0000","last_clean":"2024-09-11T01:25:03.013350+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:25:03.013350+0000","last_undegraded":"2024-09-11T01:25:03.013350+0000","last_fullsized":"2024-09-11T01:25:03.013350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28418,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220421+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:25:03.220421+0000","last_peered":"2024-09-11T01:25:03.220421+0000","last_clean":"2024-09-11T01:25:03.220421+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:25:03.220421+0000","last_undegraded":"2024-09-11T01:25:03.220421+0000","last_fullsized":"2024-09-11T01:25:03.220421+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015139+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:25:03.015139+0000","last_peered":"2024-09-11T01:25:03.015139+0000","last_clean":"2024-09-11T01:25:03.015139+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:25:03.015139+0000","last_undegraded":"2024-09-11T01:25:03.015139+0000","last_fullsized":"2024-09-11T01:25:03.015139+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27809,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.217608+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:25:03.217608+0000","last_peered":"2024-09-11T01:25:03.217608+0000","last_clean":"2024-09-11T01:25:03.217608+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:25:03.217608+0000","last_undegraded":"2024-09-11T01:25:03.217608+0000","last_fullsized":"2024-09-11T01:25:03.217608+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20419,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.116247+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:25:03.116247+0000","last_peered":"2024-09-11T01:25:03.116247+0000","last_clean":"2024-09-11T01:25:03.116247+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:25:03.116247+0000","last_undegraded":"2024-09-11T01:25:03.116247+0000","last_fullsized":"2024-09-11T01:25:03.116247+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015749+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:25:03.015749+0000","last_peered":"2024-09-11T01:25:03.015749+0000","last_clean":"2024-09-11T01:25:03.015749+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:25:03.015749+0000","last_undegraded":"2024-09-11T01:25:03.015749+0000","last_fullsized":"2024-09-11T01:25:03.015749+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27947,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015630+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:25:03.015630+0000","last_peered":"2024-09-11T01:25:03.015630+0000","last_clean":"2024-09-11T01:25:03.015630+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:25:03.015630+0000","last_undegraded":"2024-09-11T01:25:03.015630+0000","last_fullsized":"2024-09-11T01:25:03.015630+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220797+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:25:03.220797+0000","last_peered":"2024-09-11T01:25:03.220797+0000","last_clean":"2024-09-11T01:25:03.220797+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:25:03.220797+0000","last_undegraded":"2024-09-11T01:25:03.220797+0000","last_fullsized":"2024-09-11T01:25:03.220797+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24626,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015423+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:25:03.015423+0000","last_peered":"2024-09-11T01:25:03.015423+0000","last_clean":"2024-09-11T01:25:03.015423+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:25:03.015423+0000","last_undegraded":"2024-09-11T01:25:03.015423+0000","last_fullsized":"2024-09-11T01:25:03.015423+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015540+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:25:03.015540+0000","last_peered":"2024-09-11T01:25:03.015540+0000","last_clean":"2024-09-11T01:25:03.015540+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:25:03.015540+0000","last_undegraded":"2024-09-11T01:25:03.015540+0000","last_fullsized":"2024-09-11T01:25:03.015540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015405+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:25:03.015405+0000","last_peered":"2024-09-11T01:25:03.015405+0000","last_clean":"2024-09-11T01:25:03.015405+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:25:03.015405+0000","last_undegraded":"2024-09-11T01:25:03.015405+0000","last_fullsized":"2024-09-11T01:25:03.015405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20118,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:03.013826+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:25:03.013826+0000","last_peered":"2024-09-11T01:25:03.013826+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:25:03.013826+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14896,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32743,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176775168,"data_stored":176740378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182563,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518648,"kb_used_data":63592,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853192,"statfs":{"total":96636764160,"available":96105668608,"internally_reserved":0,"allocated":65118208,"data_stored":64784829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986336,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":471108,"kb_used_data":110452,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93900732,"statfs":{"total":96636764160,"available":96154349568,"internally_reserved":0,"allocated":113102848,"data_stored":112773546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659280,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479188,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93892652,"statfs":{"total":96636764160,"available":96146075648,"internally_reserved":0,"allocated":528384,"data_stored":170652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":490139311},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":112119808,"data_stored":112116329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:11.784 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:12.104 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:12.105 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:25:12.124 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":476,"stamp":"2024-09-11T01:25:10.854981+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14896,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32744,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1468944,"kb_used_data":174560,"kb_used_omap":12,"kb_used_meta":1294323,"kb_avail":281646576,"statfs":{"total":289910292480,"available":288406093824,"internally_reserved":0,"allocated":178749440,"data_stored":177729027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1325386756},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1481,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1481,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.850899"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28527,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.017457+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:25:03.017457+0000","last_peered":"2024-09-11T01:25:03.017457+0000","last_clean":"2024-09-11T01:25:03.017457+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:25:03.017457+0000","last_undegraded":"2024-09-11T01:25:03.017457+0000","last_fullsized":"2024-09-11T01:25:03.017457+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20485,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220829+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:25:03.220829+0000","last_peered":"2024-09-11T01:25:03.220829+0000","last_clean":"2024-09-11T01:25:03.220829+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:25:03.220829+0000","last_undegraded":"2024-09-11T01:25:03.220829+0000","last_fullsized":"2024-09-11T01:25:03.220829+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20308,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015371+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:25:03.015371+0000","last_peered":"2024-09-11T01:25:03.015371+0000","last_clean":"2024-09-11T01:25:03.015371+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:25:03.015371+0000","last_undegraded":"2024-09-11T01:25:03.015371+0000","last_fullsized":"2024-09-11T01:25:03.015371+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20235,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220654+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:25:03.220654+0000","last_peered":"2024-09-11T01:25:03.220654+0000","last_clean":"2024-09-11T01:25:03.220654+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:25:03.220654+0000","last_undegraded":"2024-09-11T01:25:03.220654+0000","last_fullsized":"2024-09-11T01:25:03.220654+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20168,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.013822+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:25:03.013822+0000","last_peered":"2024-09-11T01:25:03.013822+0000","last_clean":"2024-09-11T01:25:03.013822+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:25:03.013822+0000","last_undegraded":"2024-09-11T01:25:03.013822+0000","last_fullsized":"2024-09-11T01:25:03.013822+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":22332,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:08.280099+0000","last_change":"2024-09-11T01:25:01.602361+0000","last_active":"2024-09-11T01:25:08.280099+0000","last_peered":"2024-09-11T01:25:08.280099+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:25:08.280099+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3002,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3002}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20092,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:03.013655+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:25:03.013655+0000","last_peered":"2024-09-11T01:25:03.013655+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:25:03.013655+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20416,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.013473+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:25:03.013473+0000","last_peered":"2024-09-11T01:25:03.013473+0000","last_clean":"2024-09-11T01:25:03.013473+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:25:03.013473+0000","last_undegraded":"2024-09-11T01:25:03.013473+0000","last_fullsized":"2024-09-11T01:25:03.013473+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.014898+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:25:03.014898+0000","last_peered":"2024-09-11T01:25:03.014898+0000","last_clean":"2024-09-11T01:25:03.014898+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:25:03.014898+0000","last_undegraded":"2024-09-11T01:25:03.014898+0000","last_fullsized":"2024-09-11T01:25:03.014898+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28013,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015230+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:25:03.015230+0000","last_peered":"2024-09-11T01:25:03.015230+0000","last_clean":"2024-09-11T01:25:03.015230+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:25:03.015230+0000","last_undegraded":"2024-09-11T01:25:03.015230+0000","last_fullsized":"2024-09-11T01:25:03.015230+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.217689+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:25:03.217689+0000","last_peered":"2024-09-11T01:25:03.217689+0000","last_clean":"2024-09-11T01:25:03.217689+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:25:03.217689+0000","last_undegraded":"2024-09-11T01:25:03.217689+0000","last_fullsized":"2024-09-11T01:25:03.217689+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20028,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:03.016969+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:25:03.016969+0000","last_peered":"2024-09-11T01:25:03.016969+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:25:03.016969+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":133,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.013350+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:25:03.013350+0000","last_peered":"2024-09-11T01:25:03.013350+0000","last_clean":"2024-09-11T01:25:03.013350+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:25:03.013350+0000","last_undegraded":"2024-09-11T01:25:03.013350+0000","last_fullsized":"2024-09-11T01:25:03.013350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28418,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220421+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:25:03.220421+0000","last_peered":"2024-09-11T01:25:03.220421+0000","last_clean":"2024-09-11T01:25:03.220421+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:25:03.220421+0000","last_undegraded":"2024-09-11T01:25:03.220421+0000","last_fullsized":"2024-09-11T01:25:03.220421+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015139+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:25:03.015139+0000","last_peered":"2024-09-11T01:25:03.015139+0000","last_clean":"2024-09-11T01:25:03.015139+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:25:03.015139+0000","last_undegraded":"2024-09-11T01:25:03.015139+0000","last_fullsized":"2024-09-11T01:25:03.015139+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27809,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.217608+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:25:03.217608+0000","last_peered":"2024-09-11T01:25:03.217608+0000","last_clean":"2024-09-11T01:25:03.217608+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:25:03.217608+0000","last_undegraded":"2024-09-11T01:25:03.217608+0000","last_fullsized":"2024-09-11T01:25:03.217608+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20419,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.116247+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:25:03.116247+0000","last_peered":"2024-09-11T01:25:03.116247+0000","last_clean":"2024-09-11T01:25:03.116247+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:25:03.116247+0000","last_undegraded":"2024-09-11T01:25:03.116247+0000","last_fullsized":"2024-09-11T01:25:03.116247+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015749+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:25:03.015749+0000","last_peered":"2024-09-11T01:25:03.015749+0000","last_clean":"2024-09-11T01:25:03.015749+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:25:03.015749+0000","last_undegraded":"2024-09-11T01:25:03.015749+0000","last_fullsized":"2024-09-11T01:25:03.015749+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27947,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015630+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:25:03.015630+0000","last_peered":"2024-09-11T01:25:03.015630+0000","last_clean":"2024-09-11T01:25:03.015630+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:25:03.015630+0000","last_undegraded":"2024-09-11T01:25:03.015630+0000","last_fullsized":"2024-09-11T01:25:03.015630+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220797+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:25:03.220797+0000","last_peered":"2024-09-11T01:25:03.220797+0000","last_clean":"2024-09-11T01:25:03.220797+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:25:03.220797+0000","last_undegraded":"2024-09-11T01:25:03.220797+0000","last_fullsized":"2024-09-11T01:25:03.220797+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24626,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015423+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:25:03.015423+0000","last_peered":"2024-09-11T01:25:03.015423+0000","last_clean":"2024-09-11T01:25:03.015423+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:25:03.015423+0000","last_undegraded":"2024-09-11T01:25:03.015423+0000","last_fullsized":"2024-09-11T01:25:03.015423+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015540+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:25:03.015540+0000","last_peered":"2024-09-11T01:25:03.015540+0000","last_clean":"2024-09-11T01:25:03.015540+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:25:03.015540+0000","last_undegraded":"2024-09-11T01:25:03.015540+0000","last_fullsized":"2024-09-11T01:25:03.015540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015405+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:25:03.015405+0000","last_peered":"2024-09-11T01:25:03.015405+0000","last_clean":"2024-09-11T01:25:03.015405+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:25:03.015405+0000","last_undegraded":"2024-09-11T01:25:03.015405+0000","last_fullsized":"2024-09-11T01:25:03.015405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20118,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:03.013826+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:25:03.013826+0000","last_peered":"2024-09-11T01:25:03.013826+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:25:03.013826+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14896,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32743,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176775168,"data_stored":176740378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182563,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518648,"kb_used_data":63592,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853192,"statfs":{"total":96636764160,"available":96105668608,"internally_reserved":0,"allocated":65118208,"data_stored":64784829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986336,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":471108,"kb_used_data":110452,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93900732,"statfs":{"total":96636764160,"available":96154349568,"internally_reserved":0,"allocated":113102848,"data_stored":112773546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659280,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479188,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93892652,"statfs":{"total":96636764160,"available":96146075648,"internally_reserved":0,"allocated":528384,"data_stored":170652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":490139311},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":112119808,"data_stored":112116329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:12.125 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:25:12.126 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'20227", 'reported_seq': 22332, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:08.280099+0000', 'last_change': '2024-09-11T01:25:01.602361+0000', 'last_active': '2024-09-11T01:25:08.280099+0000', 'last_peered': '2024-09-11T01:25:08.280099+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:25:08.280099+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3002, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16568, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 951, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3002}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:12.126 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:25:12.126 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20092, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:03.013655+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:25:03.013655+0000', 'last_peered': '2024-09-11T01:25:03.013655+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:25:03.013655+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:12.126 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:25:12.126 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20028, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:03.016969+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:25:03.016969+0000', 'last_peered': '2024-09-11T01:25:03.016969+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:25:03.016969+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:12.126 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:25:12.126 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 20118, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:03.013826+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:25:03.013826+0000', 'last_peered': '2024-09-11T01:25:03.013826+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:25:03.013826+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:12.127 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:25:12.529 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:12.529 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 14896/6 objects degraded (248266.667%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":849,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1504198656,"bytes_avail":288406093824,"bytes_total":289910292480,"degraded_objects":14896,"degraded_total":6,"degraded_ratio":2482.6666666666665,"recovering_objects_per_sec":167,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1481,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:25:12.553 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:25:12.553 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:12.882 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:12.882 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:25:12.902 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":477,"stamp":"2024-09-11T01:25:12.855398+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14896,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32744,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1468944,"kb_used_data":174560,"kb_used_omap":12,"kb_used_meta":1294323,"kb_avail":281646576,"statfs":{"total":289910292480,"available":288406093824,"internally_reserved":0,"allocated":178749440,"data_stored":177729027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1325386756},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1481,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1481,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002357"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28527,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.017457+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:25:03.017457+0000","last_peered":"2024-09-11T01:25:03.017457+0000","last_clean":"2024-09-11T01:25:03.017457+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:25:03.017457+0000","last_undegraded":"2024-09-11T01:25:03.017457+0000","last_fullsized":"2024-09-11T01:25:03.017457+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20485,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220829+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:25:03.220829+0000","last_peered":"2024-09-11T01:25:03.220829+0000","last_clean":"2024-09-11T01:25:03.220829+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:25:03.220829+0000","last_undegraded":"2024-09-11T01:25:03.220829+0000","last_fullsized":"2024-09-11T01:25:03.220829+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20308,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015371+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:25:03.015371+0000","last_peered":"2024-09-11T01:25:03.015371+0000","last_clean":"2024-09-11T01:25:03.015371+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:25:03.015371+0000","last_undegraded":"2024-09-11T01:25:03.015371+0000","last_fullsized":"2024-09-11T01:25:03.015371+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20235,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220654+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:25:03.220654+0000","last_peered":"2024-09-11T01:25:03.220654+0000","last_clean":"2024-09-11T01:25:03.220654+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:25:03.220654+0000","last_undegraded":"2024-09-11T01:25:03.220654+0000","last_fullsized":"2024-09-11T01:25:03.220654+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20168,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.013822+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:25:03.013822+0000","last_peered":"2024-09-11T01:25:03.013822+0000","last_clean":"2024-09-11T01:25:03.013822+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:25:03.013822+0000","last_undegraded":"2024-09-11T01:25:03.013822+0000","last_fullsized":"2024-09-11T01:25:03.013822+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":22332,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:08.280099+0000","last_change":"2024-09-11T01:25:01.602361+0000","last_active":"2024-09-11T01:25:08.280099+0000","last_peered":"2024-09-11T01:25:08.280099+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:25:08.280099+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3002,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3002}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20092,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:03.013655+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:25:03.013655+0000","last_peered":"2024-09-11T01:25:03.013655+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:25:03.013655+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20416,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.013473+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:25:03.013473+0000","last_peered":"2024-09-11T01:25:03.013473+0000","last_clean":"2024-09-11T01:25:03.013473+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:25:03.013473+0000","last_undegraded":"2024-09-11T01:25:03.013473+0000","last_fullsized":"2024-09-11T01:25:03.013473+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.014898+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:25:03.014898+0000","last_peered":"2024-09-11T01:25:03.014898+0000","last_clean":"2024-09-11T01:25:03.014898+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:25:03.014898+0000","last_undegraded":"2024-09-11T01:25:03.014898+0000","last_fullsized":"2024-09-11T01:25:03.014898+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28013,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015230+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:25:03.015230+0000","last_peered":"2024-09-11T01:25:03.015230+0000","last_clean":"2024-09-11T01:25:03.015230+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:25:03.015230+0000","last_undegraded":"2024-09-11T01:25:03.015230+0000","last_fullsized":"2024-09-11T01:25:03.015230+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.217689+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:25:03.217689+0000","last_peered":"2024-09-11T01:25:03.217689+0000","last_clean":"2024-09-11T01:25:03.217689+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:25:03.217689+0000","last_undegraded":"2024-09-11T01:25:03.217689+0000","last_fullsized":"2024-09-11T01:25:03.217689+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20028,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:03.016969+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:25:03.016969+0000","last_peered":"2024-09-11T01:25:03.016969+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:25:03.016969+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":133,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.013350+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:25:03.013350+0000","last_peered":"2024-09-11T01:25:03.013350+0000","last_clean":"2024-09-11T01:25:03.013350+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:25:03.013350+0000","last_undegraded":"2024-09-11T01:25:03.013350+0000","last_fullsized":"2024-09-11T01:25:03.013350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28418,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220421+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:25:03.220421+0000","last_peered":"2024-09-11T01:25:03.220421+0000","last_clean":"2024-09-11T01:25:03.220421+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:25:03.220421+0000","last_undegraded":"2024-09-11T01:25:03.220421+0000","last_fullsized":"2024-09-11T01:25:03.220421+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015139+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:25:03.015139+0000","last_peered":"2024-09-11T01:25:03.015139+0000","last_clean":"2024-09-11T01:25:03.015139+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:25:03.015139+0000","last_undegraded":"2024-09-11T01:25:03.015139+0000","last_fullsized":"2024-09-11T01:25:03.015139+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27809,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.217608+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:25:03.217608+0000","last_peered":"2024-09-11T01:25:03.217608+0000","last_clean":"2024-09-11T01:25:03.217608+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:25:03.217608+0000","last_undegraded":"2024-09-11T01:25:03.217608+0000","last_fullsized":"2024-09-11T01:25:03.217608+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20419,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.116247+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:25:03.116247+0000","last_peered":"2024-09-11T01:25:03.116247+0000","last_clean":"2024-09-11T01:25:03.116247+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:25:03.116247+0000","last_undegraded":"2024-09-11T01:25:03.116247+0000","last_fullsized":"2024-09-11T01:25:03.116247+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015749+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:25:03.015749+0000","last_peered":"2024-09-11T01:25:03.015749+0000","last_clean":"2024-09-11T01:25:03.015749+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:25:03.015749+0000","last_undegraded":"2024-09-11T01:25:03.015749+0000","last_fullsized":"2024-09-11T01:25:03.015749+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27947,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015630+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:25:03.015630+0000","last_peered":"2024-09-11T01:25:03.015630+0000","last_clean":"2024-09-11T01:25:03.015630+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:25:03.015630+0000","last_undegraded":"2024-09-11T01:25:03.015630+0000","last_fullsized":"2024-09-11T01:25:03.015630+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220797+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:25:03.220797+0000","last_peered":"2024-09-11T01:25:03.220797+0000","last_clean":"2024-09-11T01:25:03.220797+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:25:03.220797+0000","last_undegraded":"2024-09-11T01:25:03.220797+0000","last_fullsized":"2024-09-11T01:25:03.220797+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24626,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015423+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:25:03.015423+0000","last_peered":"2024-09-11T01:25:03.015423+0000","last_clean":"2024-09-11T01:25:03.015423+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:25:03.015423+0000","last_undegraded":"2024-09-11T01:25:03.015423+0000","last_fullsized":"2024-09-11T01:25:03.015423+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015540+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:25:03.015540+0000","last_peered":"2024-09-11T01:25:03.015540+0000","last_clean":"2024-09-11T01:25:03.015540+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:25:03.015540+0000","last_undegraded":"2024-09-11T01:25:03.015540+0000","last_fullsized":"2024-09-11T01:25:03.015540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015405+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:25:03.015405+0000","last_peered":"2024-09-11T01:25:03.015405+0000","last_clean":"2024-09-11T01:25:03.015405+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:25:03.015405+0000","last_undegraded":"2024-09-11T01:25:03.015405+0000","last_fullsized":"2024-09-11T01:25:03.015405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20118,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:03.013826+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:25:03.013826+0000","last_peered":"2024-09-11T01:25:03.013826+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:25:03.013826+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":14896,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":32743,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176775168,"data_stored":176740378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182563,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518648,"kb_used_data":63592,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853192,"statfs":{"total":96636764160,"available":96105668608,"internally_reserved":0,"allocated":65118208,"data_stored":64784829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,1],"upper_bound":8},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986336,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":471108,"kb_used_data":110452,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93900732,"statfs":{"total":96636764160,"available":96154349568,"internally_reserved":0,"allocated":113102848,"data_stored":112773546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659280,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479188,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93892652,"statfs":{"total":96636764160,"available":96146075648,"internally_reserved":0,"allocated":528384,"data_stored":170652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":490139311},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":112119808,"data_stored":112116329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:15.904 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:16.257 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:16.258 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:25:16.280 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":478,"stamp":"2024-09-11T01:25:14.855802+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13997,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33643,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1477264,"kb_used_data":174560,"kb_used_omap":12,"kb_used_meta":1302643,"kb_avail":281638256,"statfs":{"total":289910292480,"available":288397574144,"internally_reserved":0,"allocated":178749440,"data_stored":177729027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1333906436},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2380,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2380,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.843167"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28527,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.017457+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:25:03.017457+0000","last_peered":"2024-09-11T01:25:03.017457+0000","last_clean":"2024-09-11T01:25:03.017457+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:25:03.017457+0000","last_undegraded":"2024-09-11T01:25:03.017457+0000","last_fullsized":"2024-09-11T01:25:03.017457+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20485,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220829+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:25:03.220829+0000","last_peered":"2024-09-11T01:25:03.220829+0000","last_clean":"2024-09-11T01:25:03.220829+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:25:03.220829+0000","last_undegraded":"2024-09-11T01:25:03.220829+0000","last_fullsized":"2024-09-11T01:25:03.220829+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20309,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015416+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:25:03.015416+0000","last_peered":"2024-09-11T01:25:03.015416+0000","last_clean":"2024-09-11T01:25:03.015416+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:25:03.015416+0000","last_undegraded":"2024-09-11T01:25:03.015416+0000","last_fullsized":"2024-09-11T01:25:03.015416+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20235,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220654+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:25:03.220654+0000","last_peered":"2024-09-11T01:25:03.220654+0000","last_clean":"2024-09-11T01:25:03.220654+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:25:03.220654+0000","last_undegraded":"2024-09-11T01:25:03.220654+0000","last_fullsized":"2024-09-11T01:25:03.220654+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20168,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.013822+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:25:03.013822+0000","last_peered":"2024-09-11T01:25:03.013822+0000","last_clean":"2024-09-11T01:25:03.013822+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:25:03.013822+0000","last_undegraded":"2024-09-11T01:25:03.013822+0000","last_fullsized":"2024-09-11T01:25:03.013822+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":24130,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:13.278783+0000","last_change":"2024-09-11T01:25:01.602361+0000","last_active":"2024-09-11T01:25:13.278783+0000","last_peered":"2024-09-11T01:25:13.278783+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:25:13.278783+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2103,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1850,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2103}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20092,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:03.013655+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:25:03.013655+0000","last_peered":"2024-09-11T01:25:03.013655+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:25:03.013655+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20416,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.013473+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:25:03.013473+0000","last_peered":"2024-09-11T01:25:03.013473+0000","last_clean":"2024-09-11T01:25:03.013473+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:25:03.013473+0000","last_undegraded":"2024-09-11T01:25:03.013473+0000","last_fullsized":"2024-09-11T01:25:03.013473+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.014944+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:25:03.014944+0000","last_peered":"2024-09-11T01:25:03.014944+0000","last_clean":"2024-09-11T01:25:03.014944+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:25:03.014944+0000","last_undegraded":"2024-09-11T01:25:03.014944+0000","last_fullsized":"2024-09-11T01:25:03.014944+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28013,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015230+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:25:03.015230+0000","last_peered":"2024-09-11T01:25:03.015230+0000","last_clean":"2024-09-11T01:25:03.015230+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:25:03.015230+0000","last_undegraded":"2024-09-11T01:25:03.015230+0000","last_fullsized":"2024-09-11T01:25:03.015230+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.217689+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:25:03.217689+0000","last_peered":"2024-09-11T01:25:03.217689+0000","last_clean":"2024-09-11T01:25:03.217689+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:25:03.217689+0000","last_undegraded":"2024-09-11T01:25:03.217689+0000","last_fullsized":"2024-09-11T01:25:03.217689+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20028,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:03.016969+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:25:03.016969+0000","last_peered":"2024-09-11T01:25:03.016969+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:25:03.016969+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":133,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.013350+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:25:03.013350+0000","last_peered":"2024-09-11T01:25:03.013350+0000","last_clean":"2024-09-11T01:25:03.013350+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:25:03.013350+0000","last_undegraded":"2024-09-11T01:25:03.013350+0000","last_fullsized":"2024-09-11T01:25:03.013350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28418,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220421+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:25:03.220421+0000","last_peered":"2024-09-11T01:25:03.220421+0000","last_clean":"2024-09-11T01:25:03.220421+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:25:03.220421+0000","last_undegraded":"2024-09-11T01:25:03.220421+0000","last_fullsized":"2024-09-11T01:25:03.220421+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015139+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:25:03.015139+0000","last_peered":"2024-09-11T01:25:03.015139+0000","last_clean":"2024-09-11T01:25:03.015139+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:25:03.015139+0000","last_undegraded":"2024-09-11T01:25:03.015139+0000","last_fullsized":"2024-09-11T01:25:03.015139+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27809,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.217608+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:25:03.217608+0000","last_peered":"2024-09-11T01:25:03.217608+0000","last_clean":"2024-09-11T01:25:03.217608+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:25:03.217608+0000","last_undegraded":"2024-09-11T01:25:03.217608+0000","last_fullsized":"2024-09-11T01:25:03.217608+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20419,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.116247+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:25:03.116247+0000","last_peered":"2024-09-11T01:25:03.116247+0000","last_clean":"2024-09-11T01:25:03.116247+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:25:03.116247+0000","last_undegraded":"2024-09-11T01:25:03.116247+0000","last_fullsized":"2024-09-11T01:25:03.116247+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015749+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:25:03.015749+0000","last_peered":"2024-09-11T01:25:03.015749+0000","last_clean":"2024-09-11T01:25:03.015749+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:25:03.015749+0000","last_undegraded":"2024-09-11T01:25:03.015749+0000","last_fullsized":"2024-09-11T01:25:03.015749+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27947,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015630+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:25:03.015630+0000","last_peered":"2024-09-11T01:25:03.015630+0000","last_clean":"2024-09-11T01:25:03.015630+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:25:03.015630+0000","last_undegraded":"2024-09-11T01:25:03.015630+0000","last_fullsized":"2024-09-11T01:25:03.015630+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220797+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:25:03.220797+0000","last_peered":"2024-09-11T01:25:03.220797+0000","last_clean":"2024-09-11T01:25:03.220797+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:25:03.220797+0000","last_undegraded":"2024-09-11T01:25:03.220797+0000","last_fullsized":"2024-09-11T01:25:03.220797+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24626,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015423+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:25:03.015423+0000","last_peered":"2024-09-11T01:25:03.015423+0000","last_clean":"2024-09-11T01:25:03.015423+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:25:03.015423+0000","last_undegraded":"2024-09-11T01:25:03.015423+0000","last_fullsized":"2024-09-11T01:25:03.015423+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015540+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:25:03.015540+0000","last_peered":"2024-09-11T01:25:03.015540+0000","last_clean":"2024-09-11T01:25:03.015540+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:25:03.015540+0000","last_undegraded":"2024-09-11T01:25:03.015540+0000","last_fullsized":"2024-09-11T01:25:03.015540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015405+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:25:03.015405+0000","last_peered":"2024-09-11T01:25:03.015405+0000","last_clean":"2024-09-11T01:25:03.015405+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:25:03.015405+0000","last_undegraded":"2024-09-11T01:25:03.015405+0000","last_fullsized":"2024-09-11T01:25:03.015405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20118,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:03.013826+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:25:03.013826+0000","last_peered":"2024-09-11T01:25:03.013826+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:25:03.013826+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13997,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33642,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176775168,"data_stored":176740378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182564,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518648,"kb_used_data":63592,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853192,"statfs":{"total":96636764160,"available":96105668608,"internally_reserved":0,"allocated":65118208,"data_stored":64784829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986336,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":471108,"kb_used_data":110452,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93900732,"statfs":{"total":96636764160,"available":96154349568,"internally_reserved":0,"allocated":113102848,"data_stored":112773546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659281,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487508,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":486971,"kb_avail":93884332,"statfs":{"total":96636764160,"available":96137555968,"internally_reserved":0,"allocated":528384,"data_stored":170652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":498658991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":112119808,"data_stored":112116329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:16.283 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:16.620 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:16.621 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:25:16.639 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":478,"stamp":"2024-09-11T01:25:14.855802+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13997,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33643,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1477264,"kb_used_data":174560,"kb_used_omap":12,"kb_used_meta":1302643,"kb_avail":281638256,"statfs":{"total":289910292480,"available":288397574144,"internally_reserved":0,"allocated":178749440,"data_stored":177729027,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1333906436},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2380,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2380,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.843167"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28527,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.017457+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:25:03.017457+0000","last_peered":"2024-09-11T01:25:03.017457+0000","last_clean":"2024-09-11T01:25:03.017457+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:25:03.017457+0000","last_undegraded":"2024-09-11T01:25:03.017457+0000","last_fullsized":"2024-09-11T01:25:03.017457+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20485,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220829+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:25:03.220829+0000","last_peered":"2024-09-11T01:25:03.220829+0000","last_clean":"2024-09-11T01:25:03.220829+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:25:03.220829+0000","last_undegraded":"2024-09-11T01:25:03.220829+0000","last_fullsized":"2024-09-11T01:25:03.220829+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20309,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015416+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:25:03.015416+0000","last_peered":"2024-09-11T01:25:03.015416+0000","last_clean":"2024-09-11T01:25:03.015416+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:25:03.015416+0000","last_undegraded":"2024-09-11T01:25:03.015416+0000","last_fullsized":"2024-09-11T01:25:03.015416+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20235,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220654+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:25:03.220654+0000","last_peered":"2024-09-11T01:25:03.220654+0000","last_clean":"2024-09-11T01:25:03.220654+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:25:03.220654+0000","last_undegraded":"2024-09-11T01:25:03.220654+0000","last_fullsized":"2024-09-11T01:25:03.220654+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20168,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.013822+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:25:03.013822+0000","last_peered":"2024-09-11T01:25:03.013822+0000","last_clean":"2024-09-11T01:25:03.013822+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:25:03.013822+0000","last_undegraded":"2024-09-11T01:25:03.013822+0000","last_fullsized":"2024-09-11T01:25:03.013822+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":24130,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:13.278783+0000","last_change":"2024-09-11T01:25:01.602361+0000","last_active":"2024-09-11T01:25:13.278783+0000","last_peered":"2024-09-11T01:25:13.278783+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:25:13.278783+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2103,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1850,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2103}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20092,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:03.013655+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:25:03.013655+0000","last_peered":"2024-09-11T01:25:03.013655+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:25:03.013655+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20416,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.013473+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:25:03.013473+0000","last_peered":"2024-09-11T01:25:03.013473+0000","last_clean":"2024-09-11T01:25:03.013473+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:25:03.013473+0000","last_undegraded":"2024-09-11T01:25:03.013473+0000","last_fullsized":"2024-09-11T01:25:03.013473+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.014944+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:25:03.014944+0000","last_peered":"2024-09-11T01:25:03.014944+0000","last_clean":"2024-09-11T01:25:03.014944+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:25:03.014944+0000","last_undegraded":"2024-09-11T01:25:03.014944+0000","last_fullsized":"2024-09-11T01:25:03.014944+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28013,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015230+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:25:03.015230+0000","last_peered":"2024-09-11T01:25:03.015230+0000","last_clean":"2024-09-11T01:25:03.015230+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:25:03.015230+0000","last_undegraded":"2024-09-11T01:25:03.015230+0000","last_fullsized":"2024-09-11T01:25:03.015230+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.217689+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:25:03.217689+0000","last_peered":"2024-09-11T01:25:03.217689+0000","last_clean":"2024-09-11T01:25:03.217689+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:25:03.217689+0000","last_undegraded":"2024-09-11T01:25:03.217689+0000","last_fullsized":"2024-09-11T01:25:03.217689+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20028,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:03.016969+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:25:03.016969+0000","last_peered":"2024-09-11T01:25:03.016969+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:25:03.016969+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":133,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.013350+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:25:03.013350+0000","last_peered":"2024-09-11T01:25:03.013350+0000","last_clean":"2024-09-11T01:25:03.013350+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:25:03.013350+0000","last_undegraded":"2024-09-11T01:25:03.013350+0000","last_fullsized":"2024-09-11T01:25:03.013350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28418,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220421+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:25:03.220421+0000","last_peered":"2024-09-11T01:25:03.220421+0000","last_clean":"2024-09-11T01:25:03.220421+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:25:03.220421+0000","last_undegraded":"2024-09-11T01:25:03.220421+0000","last_fullsized":"2024-09-11T01:25:03.220421+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015139+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:25:03.015139+0000","last_peered":"2024-09-11T01:25:03.015139+0000","last_clean":"2024-09-11T01:25:03.015139+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:25:03.015139+0000","last_undegraded":"2024-09-11T01:25:03.015139+0000","last_fullsized":"2024-09-11T01:25:03.015139+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27809,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.217608+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:25:03.217608+0000","last_peered":"2024-09-11T01:25:03.217608+0000","last_clean":"2024-09-11T01:25:03.217608+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:25:03.217608+0000","last_undegraded":"2024-09-11T01:25:03.217608+0000","last_fullsized":"2024-09-11T01:25:03.217608+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20419,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.116247+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:25:03.116247+0000","last_peered":"2024-09-11T01:25:03.116247+0000","last_clean":"2024-09-11T01:25:03.116247+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:25:03.116247+0000","last_undegraded":"2024-09-11T01:25:03.116247+0000","last_fullsized":"2024-09-11T01:25:03.116247+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015749+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:25:03.015749+0000","last_peered":"2024-09-11T01:25:03.015749+0000","last_clean":"2024-09-11T01:25:03.015749+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:25:03.015749+0000","last_undegraded":"2024-09-11T01:25:03.015749+0000","last_fullsized":"2024-09-11T01:25:03.015749+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27947,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015630+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:25:03.015630+0000","last_peered":"2024-09-11T01:25:03.015630+0000","last_clean":"2024-09-11T01:25:03.015630+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:25:03.015630+0000","last_undegraded":"2024-09-11T01:25:03.015630+0000","last_fullsized":"2024-09-11T01:25:03.015630+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220797+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:25:03.220797+0000","last_peered":"2024-09-11T01:25:03.220797+0000","last_clean":"2024-09-11T01:25:03.220797+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:25:03.220797+0000","last_undegraded":"2024-09-11T01:25:03.220797+0000","last_fullsized":"2024-09-11T01:25:03.220797+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24626,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015423+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:25:03.015423+0000","last_peered":"2024-09-11T01:25:03.015423+0000","last_clean":"2024-09-11T01:25:03.015423+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:25:03.015423+0000","last_undegraded":"2024-09-11T01:25:03.015423+0000","last_fullsized":"2024-09-11T01:25:03.015423+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015540+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:25:03.015540+0000","last_peered":"2024-09-11T01:25:03.015540+0000","last_clean":"2024-09-11T01:25:03.015540+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:25:03.015540+0000","last_undegraded":"2024-09-11T01:25:03.015540+0000","last_fullsized":"2024-09-11T01:25:03.015540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015405+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:25:03.015405+0000","last_peered":"2024-09-11T01:25:03.015405+0000","last_clean":"2024-09-11T01:25:03.015405+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:25:03.015405+0000","last_undegraded":"2024-09-11T01:25:03.015405+0000","last_fullsized":"2024-09-11T01:25:03.015405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20118,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:03.013826+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:25:03.013826+0000","last_peered":"2024-09-11T01:25:03.013826+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:25:03.013826+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13997,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33642,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":176775168,"data_stored":176740378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182564,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518648,"kb_used_data":63592,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853192,"statfs":{"total":96636764160,"available":96105668608,"internally_reserved":0,"allocated":65118208,"data_stored":64784829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986336,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":471108,"kb_used_data":110452,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93900732,"statfs":{"total":96636764160,"available":96154349568,"internally_reserved":0,"allocated":113102848,"data_stored":112773546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659281,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487508,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":486971,"kb_avail":93884332,"statfs":{"total":96636764160,"available":96137555968,"internally_reserved":0,"allocated":528384,"data_stored":170652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":498658991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":112119808,"data_stored":112116329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:16.640 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:25:16.641 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'20227", 'reported_seq': 24130, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:13.278783+0000', 'last_change': '2024-09-11T01:25:01.602361+0000', 'last_active': '2024-09-11T01:25:13.278783+0000', 'last_peered': '2024-09-11T01:25:13.278783+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:25:13.278783+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2103, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16568, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1850, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2103}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:16.641 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:25:16.641 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20092, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:03.013655+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:25:03.013655+0000', 'last_peered': '2024-09-11T01:25:03.013655+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:25:03.013655+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:16.641 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:25:16.641 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20028, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:03.016969+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:25:03.016969+0000', 'last_peered': '2024-09-11T01:25:03.016969+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:25:03.016969+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:16.641 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:25:16.642 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 20118, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:03.013826+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:25:03.013826+0000', 'last_peered': '2024-09-11T01:25:03.013826+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:25:03.013826+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:16.642 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:25:17.050 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:17.051 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 13997/6 objects degraded (233283.333%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":853,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1510260736,"bytes_avail":288400031744,"bytes_total":289910292480,"degraded_objects":13997,"degraded_total":6,"degraded_ratio":2332.8333333333335,"recovering_objects_per_sec":137,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1650,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:25:17.070 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:25:17.070 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:17.399 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:17.400 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:25:17.417 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":479,"stamp":"2024-09-11T01:25:16.856374+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13997,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33643,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1474864,"kb_used_data":172160,"kb_used_omap":12,"kb_used_meta":1302643,"kb_avail":281640656,"statfs":{"total":289910292480,"available":288400031744,"internally_reserved":0,"allocated":176291840,"data_stored":175271427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1333906436},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1650,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1650,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002985"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28527,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.017457+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:25:03.017457+0000","last_peered":"2024-09-11T01:25:03.017457+0000","last_clean":"2024-09-11T01:25:03.017457+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:25:03.017457+0000","last_undegraded":"2024-09-11T01:25:03.017457+0000","last_fullsized":"2024-09-11T01:25:03.017457+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20485,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220829+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:25:03.220829+0000","last_peered":"2024-09-11T01:25:03.220829+0000","last_clean":"2024-09-11T01:25:03.220829+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:25:03.220829+0000","last_undegraded":"2024-09-11T01:25:03.220829+0000","last_fullsized":"2024-09-11T01:25:03.220829+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20309,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015416+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:25:03.015416+0000","last_peered":"2024-09-11T01:25:03.015416+0000","last_clean":"2024-09-11T01:25:03.015416+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:25:03.015416+0000","last_undegraded":"2024-09-11T01:25:03.015416+0000","last_fullsized":"2024-09-11T01:25:03.015416+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20235,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220654+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:25:03.220654+0000","last_peered":"2024-09-11T01:25:03.220654+0000","last_clean":"2024-09-11T01:25:03.220654+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:25:03.220654+0000","last_undegraded":"2024-09-11T01:25:03.220654+0000","last_fullsized":"2024-09-11T01:25:03.220654+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20168,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.013822+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:25:03.013822+0000","last_peered":"2024-09-11T01:25:03.013822+0000","last_clean":"2024-09-11T01:25:03.013822+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:25:03.013822+0000","last_undegraded":"2024-09-11T01:25:03.013822+0000","last_fullsized":"2024-09-11T01:25:03.013822+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":24130,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:13.278783+0000","last_change":"2024-09-11T01:25:01.602361+0000","last_active":"2024-09-11T01:25:13.278783+0000","last_peered":"2024-09-11T01:25:13.278783+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:25:13.278783+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2103,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1850,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2103}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20092,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:03.013655+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:25:03.013655+0000","last_peered":"2024-09-11T01:25:03.013655+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:25:03.013655+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20416,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.013473+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:25:03.013473+0000","last_peered":"2024-09-11T01:25:03.013473+0000","last_clean":"2024-09-11T01:25:03.013473+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:25:03.013473+0000","last_undegraded":"2024-09-11T01:25:03.013473+0000","last_fullsized":"2024-09-11T01:25:03.013473+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.014944+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:25:03.014944+0000","last_peered":"2024-09-11T01:25:03.014944+0000","last_clean":"2024-09-11T01:25:03.014944+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:25:03.014944+0000","last_undegraded":"2024-09-11T01:25:03.014944+0000","last_fullsized":"2024-09-11T01:25:03.014944+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28013,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015230+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:25:03.015230+0000","last_peered":"2024-09-11T01:25:03.015230+0000","last_clean":"2024-09-11T01:25:03.015230+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:25:03.015230+0000","last_undegraded":"2024-09-11T01:25:03.015230+0000","last_fullsized":"2024-09-11T01:25:03.015230+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.217689+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:25:03.217689+0000","last_peered":"2024-09-11T01:25:03.217689+0000","last_clean":"2024-09-11T01:25:03.217689+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:25:03.217689+0000","last_undegraded":"2024-09-11T01:25:03.217689+0000","last_fullsized":"2024-09-11T01:25:03.217689+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20028,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:03.016969+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:25:03.016969+0000","last_peered":"2024-09-11T01:25:03.016969+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:25:03.016969+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":133,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.013350+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:25:03.013350+0000","last_peered":"2024-09-11T01:25:03.013350+0000","last_clean":"2024-09-11T01:25:03.013350+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:25:03.013350+0000","last_undegraded":"2024-09-11T01:25:03.013350+0000","last_fullsized":"2024-09-11T01:25:03.013350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28418,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220421+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:25:03.220421+0000","last_peered":"2024-09-11T01:25:03.220421+0000","last_clean":"2024-09-11T01:25:03.220421+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:25:03.220421+0000","last_undegraded":"2024-09-11T01:25:03.220421+0000","last_fullsized":"2024-09-11T01:25:03.220421+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015139+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:25:03.015139+0000","last_peered":"2024-09-11T01:25:03.015139+0000","last_clean":"2024-09-11T01:25:03.015139+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:25:03.015139+0000","last_undegraded":"2024-09-11T01:25:03.015139+0000","last_fullsized":"2024-09-11T01:25:03.015139+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27809,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.217608+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:25:03.217608+0000","last_peered":"2024-09-11T01:25:03.217608+0000","last_clean":"2024-09-11T01:25:03.217608+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:25:03.217608+0000","last_undegraded":"2024-09-11T01:25:03.217608+0000","last_fullsized":"2024-09-11T01:25:03.217608+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20419,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.116247+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:25:03.116247+0000","last_peered":"2024-09-11T01:25:03.116247+0000","last_clean":"2024-09-11T01:25:03.116247+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:25:03.116247+0000","last_undegraded":"2024-09-11T01:25:03.116247+0000","last_fullsized":"2024-09-11T01:25:03.116247+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015749+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:25:03.015749+0000","last_peered":"2024-09-11T01:25:03.015749+0000","last_clean":"2024-09-11T01:25:03.015749+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:25:03.015749+0000","last_undegraded":"2024-09-11T01:25:03.015749+0000","last_fullsized":"2024-09-11T01:25:03.015749+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27947,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015630+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:25:03.015630+0000","last_peered":"2024-09-11T01:25:03.015630+0000","last_clean":"2024-09-11T01:25:03.015630+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:25:03.015630+0000","last_undegraded":"2024-09-11T01:25:03.015630+0000","last_fullsized":"2024-09-11T01:25:03.015630+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220797+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:25:03.220797+0000","last_peered":"2024-09-11T01:25:03.220797+0000","last_clean":"2024-09-11T01:25:03.220797+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:25:03.220797+0000","last_undegraded":"2024-09-11T01:25:03.220797+0000","last_fullsized":"2024-09-11T01:25:03.220797+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24626,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015423+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:25:03.015423+0000","last_peered":"2024-09-11T01:25:03.015423+0000","last_clean":"2024-09-11T01:25:03.015423+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:25:03.015423+0000","last_undegraded":"2024-09-11T01:25:03.015423+0000","last_fullsized":"2024-09-11T01:25:03.015423+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015540+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:25:03.015540+0000","last_peered":"2024-09-11T01:25:03.015540+0000","last_clean":"2024-09-11T01:25:03.015540+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:25:03.015540+0000","last_undegraded":"2024-09-11T01:25:03.015540+0000","last_fullsized":"2024-09-11T01:25:03.015540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015405+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:25:03.015405+0000","last_peered":"2024-09-11T01:25:03.015405+0000","last_clean":"2024-09-11T01:25:03.015405+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:25:03.015405+0000","last_undegraded":"2024-09-11T01:25:03.015405+0000","last_fullsized":"2024-09-11T01:25:03.015405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20118,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:03.013826+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:25:03.013826+0000","last_peered":"2024-09-11T01:25:03.013826+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:25:03.013826+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":13997,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":33642,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174317568,"data_stored":174282778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182564,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518648,"kb_used_data":63592,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853192,"statfs":{"total":96636764160,"available":96105668608,"internally_reserved":0,"allocated":65118208,"data_stored":64784829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986337,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":468708,"kb_used_data":108052,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93903132,"statfs":{"total":96636764160,"available":96156807168,"internally_reserved":0,"allocated":110645248,"data_stored":110315946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659281,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487508,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":486971,"kb_avail":93884332,"statfs":{"total":96636764160,"available":96137555968,"internally_reserved":0,"allocated":528384,"data_stored":170652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":498658991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":109662208,"data_stored":109658729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:20.419 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:20.765 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:20.765 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:25:20.785 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":480,"stamp":"2024-09-11T01:25:18.856703+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12887,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34753,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1466544,"kb_used_data":172160,"kb_used_omap":12,"kb_used_meta":1294323,"kb_avail":281648976,"statfs":{"total":289910292480,"available":288408551424,"internally_reserved":0,"allocated":176291840,"data_stored":175271427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1325386756},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2760,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2760,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002848"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28527,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.017457+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:25:03.017457+0000","last_peered":"2024-09-11T01:25:03.017457+0000","last_clean":"2024-09-11T01:25:03.017457+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:25:03.017457+0000","last_undegraded":"2024-09-11T01:25:03.017457+0000","last_fullsized":"2024-09-11T01:25:03.017457+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20485,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220829+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:25:03.220829+0000","last_peered":"2024-09-11T01:25:03.220829+0000","last_clean":"2024-09-11T01:25:03.220829+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:25:03.220829+0000","last_undegraded":"2024-09-11T01:25:03.220829+0000","last_fullsized":"2024-09-11T01:25:03.220829+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20309,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015416+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:25:03.015416+0000","last_peered":"2024-09-11T01:25:03.015416+0000","last_clean":"2024-09-11T01:25:03.015416+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:25:03.015416+0000","last_undegraded":"2024-09-11T01:25:03.015416+0000","last_fullsized":"2024-09-11T01:25:03.015416+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20235,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220654+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:25:03.220654+0000","last_peered":"2024-09-11T01:25:03.220654+0000","last_clean":"2024-09-11T01:25:03.220654+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:25:03.220654+0000","last_undegraded":"2024-09-11T01:25:03.220654+0000","last_fullsized":"2024-09-11T01:25:03.220654+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20169,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.013872+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:25:03.013872+0000","last_peered":"2024-09-11T01:25:03.013872+0000","last_clean":"2024-09-11T01:25:03.013872+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:25:03.013872+0000","last_undegraded":"2024-09-11T01:25:03.013872+0000","last_fullsized":"2024-09-11T01:25:03.013872+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":26350,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:18.280743+0000","last_change":"2024-09-11T01:25:01.602361+0000","last_active":"2024-09-11T01:25:18.280743+0000","last_peered":"2024-09-11T01:25:18.280743+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:25:18.280743+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":993,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2960,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-993}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20092,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:03.013655+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:25:03.013655+0000","last_peered":"2024-09-11T01:25:03.013655+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:25:03.013655+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20416,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.013473+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:25:03.013473+0000","last_peered":"2024-09-11T01:25:03.013473+0000","last_clean":"2024-09-11T01:25:03.013473+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:25:03.013473+0000","last_undegraded":"2024-09-11T01:25:03.013473+0000","last_fullsized":"2024-09-11T01:25:03.013473+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.014944+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:25:03.014944+0000","last_peered":"2024-09-11T01:25:03.014944+0000","last_clean":"2024-09-11T01:25:03.014944+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:25:03.014944+0000","last_undegraded":"2024-09-11T01:25:03.014944+0000","last_fullsized":"2024-09-11T01:25:03.014944+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28013,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015230+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:25:03.015230+0000","last_peered":"2024-09-11T01:25:03.015230+0000","last_clean":"2024-09-11T01:25:03.015230+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:25:03.015230+0000","last_undegraded":"2024-09-11T01:25:03.015230+0000","last_fullsized":"2024-09-11T01:25:03.015230+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.217689+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:25:03.217689+0000","last_peered":"2024-09-11T01:25:03.217689+0000","last_clean":"2024-09-11T01:25:03.217689+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:25:03.217689+0000","last_undegraded":"2024-09-11T01:25:03.217689+0000","last_fullsized":"2024-09-11T01:25:03.217689+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20028,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:03.016969+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:25:03.016969+0000","last_peered":"2024-09-11T01:25:03.016969+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:25:03.016969+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":133,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.013350+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:25:03.013350+0000","last_peered":"2024-09-11T01:25:03.013350+0000","last_clean":"2024-09-11T01:25:03.013350+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:25:03.013350+0000","last_undegraded":"2024-09-11T01:25:03.013350+0000","last_fullsized":"2024-09-11T01:25:03.013350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28418,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220421+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:25:03.220421+0000","last_peered":"2024-09-11T01:25:03.220421+0000","last_clean":"2024-09-11T01:25:03.220421+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:25:03.220421+0000","last_undegraded":"2024-09-11T01:25:03.220421+0000","last_fullsized":"2024-09-11T01:25:03.220421+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015139+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:25:03.015139+0000","last_peered":"2024-09-11T01:25:03.015139+0000","last_clean":"2024-09-11T01:25:03.015139+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:25:03.015139+0000","last_undegraded":"2024-09-11T01:25:03.015139+0000","last_fullsized":"2024-09-11T01:25:03.015139+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27809,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.217608+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:25:03.217608+0000","last_peered":"2024-09-11T01:25:03.217608+0000","last_clean":"2024-09-11T01:25:03.217608+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:25:03.217608+0000","last_undegraded":"2024-09-11T01:25:03.217608+0000","last_fullsized":"2024-09-11T01:25:03.217608+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20419,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.116247+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:25:03.116247+0000","last_peered":"2024-09-11T01:25:03.116247+0000","last_clean":"2024-09-11T01:25:03.116247+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:25:03.116247+0000","last_undegraded":"2024-09-11T01:25:03.116247+0000","last_fullsized":"2024-09-11T01:25:03.116247+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015749+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:25:03.015749+0000","last_peered":"2024-09-11T01:25:03.015749+0000","last_clean":"2024-09-11T01:25:03.015749+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:25:03.015749+0000","last_undegraded":"2024-09-11T01:25:03.015749+0000","last_fullsized":"2024-09-11T01:25:03.015749+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27947,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015630+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:25:03.015630+0000","last_peered":"2024-09-11T01:25:03.015630+0000","last_clean":"2024-09-11T01:25:03.015630+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:25:03.015630+0000","last_undegraded":"2024-09-11T01:25:03.015630+0000","last_fullsized":"2024-09-11T01:25:03.015630+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220797+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:25:03.220797+0000","last_peered":"2024-09-11T01:25:03.220797+0000","last_clean":"2024-09-11T01:25:03.220797+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:25:03.220797+0000","last_undegraded":"2024-09-11T01:25:03.220797+0000","last_fullsized":"2024-09-11T01:25:03.220797+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24626,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015423+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:25:03.015423+0000","last_peered":"2024-09-11T01:25:03.015423+0000","last_clean":"2024-09-11T01:25:03.015423+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:25:03.015423+0000","last_undegraded":"2024-09-11T01:25:03.015423+0000","last_fullsized":"2024-09-11T01:25:03.015423+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015540+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:25:03.015540+0000","last_peered":"2024-09-11T01:25:03.015540+0000","last_clean":"2024-09-11T01:25:03.015540+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:25:03.015540+0000","last_undegraded":"2024-09-11T01:25:03.015540+0000","last_fullsized":"2024-09-11T01:25:03.015540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015405+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:25:03.015405+0000","last_peered":"2024-09-11T01:25:03.015405+0000","last_clean":"2024-09-11T01:25:03.015405+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:25:03.015405+0000","last_undegraded":"2024-09-11T01:25:03.015405+0000","last_fullsized":"2024-09-11T01:25:03.015405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20118,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:03.013826+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:25:03.013826+0000","last_peered":"2024-09-11T01:25:03.013826+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:25:03.013826+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12887,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34752,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":174317568,"data_stored":174282778,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182565,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518648,"kb_used_data":63592,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853192,"statfs":{"total":96636764160,"available":96105668608,"internally_reserved":0,"allocated":65118208,"data_stored":64784829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986337,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":468708,"kb_used_data":108052,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93903132,"statfs":{"total":96636764160,"available":96156807168,"internally_reserved":0,"allocated":110645248,"data_stored":110315946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659282,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479188,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93892652,"statfs":{"total":96636764160,"available":96146075648,"internally_reserved":0,"allocated":528384,"data_stored":170652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":490139311},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":109662208,"data_stored":109658729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:20.786 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:21.101 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:21.102 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:25:21.119 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":481,"stamp":"2024-09-11T01:25:20.857315+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12887,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34753,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1464144,"kb_used_data":169760,"kb_used_omap":12,"kb_used_meta":1294323,"kb_avail":281651376,"statfs":{"total":289910292480,"available":288411009024,"internally_reserved":0,"allocated":173834240,"data_stored":172813827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1325386756},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2009,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2009,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002791"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28527,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.017457+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:25:03.017457+0000","last_peered":"2024-09-11T01:25:03.017457+0000","last_clean":"2024-09-11T01:25:03.017457+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:25:03.017457+0000","last_undegraded":"2024-09-11T01:25:03.017457+0000","last_fullsized":"2024-09-11T01:25:03.017457+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20485,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220829+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:25:03.220829+0000","last_peered":"2024-09-11T01:25:03.220829+0000","last_clean":"2024-09-11T01:25:03.220829+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:25:03.220829+0000","last_undegraded":"2024-09-11T01:25:03.220829+0000","last_fullsized":"2024-09-11T01:25:03.220829+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20309,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015416+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:25:03.015416+0000","last_peered":"2024-09-11T01:25:03.015416+0000","last_clean":"2024-09-11T01:25:03.015416+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:25:03.015416+0000","last_undegraded":"2024-09-11T01:25:03.015416+0000","last_fullsized":"2024-09-11T01:25:03.015416+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20235,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220654+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:25:03.220654+0000","last_peered":"2024-09-11T01:25:03.220654+0000","last_clean":"2024-09-11T01:25:03.220654+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:25:03.220654+0000","last_undegraded":"2024-09-11T01:25:03.220654+0000","last_fullsized":"2024-09-11T01:25:03.220654+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20169,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.013872+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:25:03.013872+0000","last_peered":"2024-09-11T01:25:03.013872+0000","last_clean":"2024-09-11T01:25:03.013872+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:25:03.013872+0000","last_undegraded":"2024-09-11T01:25:03.013872+0000","last_fullsized":"2024-09-11T01:25:03.013872+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":26350,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:18.280743+0000","last_change":"2024-09-11T01:25:01.602361+0000","last_active":"2024-09-11T01:25:18.280743+0000","last_peered":"2024-09-11T01:25:18.280743+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:25:18.280743+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":993,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2960,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-993}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20092,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:03.013655+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:25:03.013655+0000","last_peered":"2024-09-11T01:25:03.013655+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:25:03.013655+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20416,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.013473+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:25:03.013473+0000","last_peered":"2024-09-11T01:25:03.013473+0000","last_clean":"2024-09-11T01:25:03.013473+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:25:03.013473+0000","last_undegraded":"2024-09-11T01:25:03.013473+0000","last_fullsized":"2024-09-11T01:25:03.013473+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.014944+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:25:03.014944+0000","last_peered":"2024-09-11T01:25:03.014944+0000","last_clean":"2024-09-11T01:25:03.014944+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:25:03.014944+0000","last_undegraded":"2024-09-11T01:25:03.014944+0000","last_fullsized":"2024-09-11T01:25:03.014944+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28013,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015230+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:25:03.015230+0000","last_peered":"2024-09-11T01:25:03.015230+0000","last_clean":"2024-09-11T01:25:03.015230+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:25:03.015230+0000","last_undegraded":"2024-09-11T01:25:03.015230+0000","last_fullsized":"2024-09-11T01:25:03.015230+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.217689+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:25:03.217689+0000","last_peered":"2024-09-11T01:25:03.217689+0000","last_clean":"2024-09-11T01:25:03.217689+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:25:03.217689+0000","last_undegraded":"2024-09-11T01:25:03.217689+0000","last_fullsized":"2024-09-11T01:25:03.217689+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20028,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:03.016969+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:25:03.016969+0000","last_peered":"2024-09-11T01:25:03.016969+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:25:03.016969+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":133,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.013350+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:25:03.013350+0000","last_peered":"2024-09-11T01:25:03.013350+0000","last_clean":"2024-09-11T01:25:03.013350+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:25:03.013350+0000","last_undegraded":"2024-09-11T01:25:03.013350+0000","last_fullsized":"2024-09-11T01:25:03.013350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28418,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220421+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:25:03.220421+0000","last_peered":"2024-09-11T01:25:03.220421+0000","last_clean":"2024-09-11T01:25:03.220421+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:25:03.220421+0000","last_undegraded":"2024-09-11T01:25:03.220421+0000","last_fullsized":"2024-09-11T01:25:03.220421+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015139+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:25:03.015139+0000","last_peered":"2024-09-11T01:25:03.015139+0000","last_clean":"2024-09-11T01:25:03.015139+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:25:03.015139+0000","last_undegraded":"2024-09-11T01:25:03.015139+0000","last_fullsized":"2024-09-11T01:25:03.015139+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27809,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.217608+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:25:03.217608+0000","last_peered":"2024-09-11T01:25:03.217608+0000","last_clean":"2024-09-11T01:25:03.217608+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:25:03.217608+0000","last_undegraded":"2024-09-11T01:25:03.217608+0000","last_fullsized":"2024-09-11T01:25:03.217608+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20419,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.116247+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:25:03.116247+0000","last_peered":"2024-09-11T01:25:03.116247+0000","last_clean":"2024-09-11T01:25:03.116247+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:25:03.116247+0000","last_undegraded":"2024-09-11T01:25:03.116247+0000","last_fullsized":"2024-09-11T01:25:03.116247+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015749+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:25:03.015749+0000","last_peered":"2024-09-11T01:25:03.015749+0000","last_clean":"2024-09-11T01:25:03.015749+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:25:03.015749+0000","last_undegraded":"2024-09-11T01:25:03.015749+0000","last_fullsized":"2024-09-11T01:25:03.015749+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27947,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015630+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:25:03.015630+0000","last_peered":"2024-09-11T01:25:03.015630+0000","last_clean":"2024-09-11T01:25:03.015630+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:25:03.015630+0000","last_undegraded":"2024-09-11T01:25:03.015630+0000","last_fullsized":"2024-09-11T01:25:03.015630+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220797+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:25:03.220797+0000","last_peered":"2024-09-11T01:25:03.220797+0000","last_clean":"2024-09-11T01:25:03.220797+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:25:03.220797+0000","last_undegraded":"2024-09-11T01:25:03.220797+0000","last_fullsized":"2024-09-11T01:25:03.220797+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24626,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015423+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:25:03.015423+0000","last_peered":"2024-09-11T01:25:03.015423+0000","last_clean":"2024-09-11T01:25:03.015423+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:25:03.015423+0000","last_undegraded":"2024-09-11T01:25:03.015423+0000","last_fullsized":"2024-09-11T01:25:03.015423+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015540+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:25:03.015540+0000","last_peered":"2024-09-11T01:25:03.015540+0000","last_clean":"2024-09-11T01:25:03.015540+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:25:03.015540+0000","last_undegraded":"2024-09-11T01:25:03.015540+0000","last_fullsized":"2024-09-11T01:25:03.015540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015405+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:25:03.015405+0000","last_peered":"2024-09-11T01:25:03.015405+0000","last_clean":"2024-09-11T01:25:03.015405+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:25:03.015405+0000","last_undegraded":"2024-09-11T01:25:03.015405+0000","last_fullsized":"2024-09-11T01:25:03.015405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20118,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:03.013826+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:25:03.013826+0000","last_peered":"2024-09-11T01:25:03.013826+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:25:03.013826+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12887,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34752,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":171859968,"data_stored":171825178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182565,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518648,"kb_used_data":63592,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853192,"statfs":{"total":96636764160,"available":96105668608,"internally_reserved":0,"allocated":65118208,"data_stored":64784829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986338,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":466308,"kb_used_data":105652,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93905532,"statfs":{"total":96636764160,"available":96159264768,"internally_reserved":0,"allocated":108187648,"data_stored":107858346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659282,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479188,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93892652,"statfs":{"total":96636764160,"available":96146075648,"internally_reserved":0,"allocated":528384,"data_stored":170652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":490139311},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":107204608,"data_stored":107201129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:21.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.a is not active+clean 2024-09-11T01:25:21.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.a', 'version': "49'20227", 'reported_seq': 26350, 'reported_epoch': 59, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:18.280743+0000', 'last_change': '2024-09-11T01:25:01.602361+0000', 'last_active': '2024-09-11T01:25:18.280743+0000', 'last_peered': '2024-09-11T01:25:18.280743+0000', 'last_clean': '2024-09-11T01:13:07.619558+0000', 'last_became_active': '2024-09-11T01:13:10.266680+0000', 'last_became_peered': '2024-09-11T01:13:10.266680+0000', 'last_unstale': '2024-09-11T01:25:18.280743+0000', 'last_undegraded': '2024-09-11T01:13:09.790043+0000', 'last_fullsized': '2024-09-11T01:13:09.656166+0000', 'mapping_epoch': 42, 'log_start': "45'10200", 'ondisk_log_start': "45'10200", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "39'3723", 'last_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:13:04.086089+0000', 'objects_scrubbed': 3654, 'log_size': 10027, 'log_dups_size': 0, 'ondisk_log_size': 10027, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1831, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 993, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16568, 'num_write_kb': 25828, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2960, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -993}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:21.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:25:21.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20092, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:03.013655+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:25:03.013655+0000', 'last_peered': '2024-09-11T01:25:03.013655+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:25:03.013655+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:21.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:25:21.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20028, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:03.016969+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:25:03.016969+0000', 'last_peered': '2024-09-11T01:25:03.016969+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:25:03.016969+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:21.120 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:25:21.121 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 20118, 'reported_epoch': 59, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:03.013826+0000', 'last_change': '2024-09-11T01:13:10.280944+0000', 'last_active': '2024-09-11T01:25:03.013826+0000', 'last_peered': '2024-09-11T01:25:03.013826+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:25:03.013826+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3920, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3916}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:21.121 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:25:21.501 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:21.501 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 12887/6 objects degraded (214783.333%), 4 pgs degraded, 4 pgs undersized","count":8},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":858,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":59,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":4},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":20},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":3},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1499283456,"bytes_avail":288411009024,"bytes_total":289910292480,"degraded_objects":12887,"degraded_total":6,"degraded_ratio":2147.8333333333335,"recovering_objects_per_sec":167,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2009,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:25:21.519 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:25:21.519 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:21.812 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:21.812 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:25:21.830 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":481,"stamp":"2024-09-11T01:25:20.857315+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12887,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34753,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":44,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1464144,"kb_used_data":169760,"kb_used_omap":12,"kb_used_meta":1294323,"kb_avail":281651376,"statfs":{"total":289910292480,"available":288411009024,"internally_reserved":0,"allocated":173834240,"data_stored":172813827,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1325386756},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2009,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2009,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002791"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28527,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.017457+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:25:03.017457+0000","last_peered":"2024-09-11T01:25:03.017457+0000","last_clean":"2024-09-11T01:25:03.017457+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:25:03.017457+0000","last_undegraded":"2024-09-11T01:25:03.017457+0000","last_fullsized":"2024-09-11T01:25:03.017457+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20485,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220829+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:25:03.220829+0000","last_peered":"2024-09-11T01:25:03.220829+0000","last_clean":"2024-09-11T01:25:03.220829+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:25:03.220829+0000","last_undegraded":"2024-09-11T01:25:03.220829+0000","last_fullsized":"2024-09-11T01:25:03.220829+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20309,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015416+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:25:03.015416+0000","last_peered":"2024-09-11T01:25:03.015416+0000","last_clean":"2024-09-11T01:25:03.015416+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:25:03.015416+0000","last_undegraded":"2024-09-11T01:25:03.015416+0000","last_fullsized":"2024-09-11T01:25:03.015416+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20235,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220654+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:25:03.220654+0000","last_peered":"2024-09-11T01:25:03.220654+0000","last_clean":"2024-09-11T01:25:03.220654+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:25:03.220654+0000","last_undegraded":"2024-09-11T01:25:03.220654+0000","last_fullsized":"2024-09-11T01:25:03.220654+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20169,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.013872+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:25:03.013872+0000","last_peered":"2024-09-11T01:25:03.013872+0000","last_clean":"2024-09-11T01:25:03.013872+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:25:03.013872+0000","last_undegraded":"2024-09-11T01:25:03.013872+0000","last_fullsized":"2024-09-11T01:25:03.013872+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":26350,"reported_epoch":59,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:18.280743+0000","last_change":"2024-09-11T01:25:01.602361+0000","last_active":"2024-09-11T01:25:18.280743+0000","last_peered":"2024-09-11T01:25:18.280743+0000","last_clean":"2024-09-11T01:13:07.619558+0000","last_became_active":"2024-09-11T01:13:10.266680+0000","last_became_peered":"2024-09-11T01:13:10.266680+0000","last_unstale":"2024-09-11T01:25:18.280743+0000","last_undegraded":"2024-09-11T01:13:09.790043+0000","last_fullsized":"2024-09-11T01:13:09.656166+0000","mapping_epoch":42,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":993,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2960,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-993}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20092,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:03.013655+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:25:03.013655+0000","last_peered":"2024-09-11T01:25:03.013655+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:25:03.013655+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20416,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.013473+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:25:03.013473+0000","last_peered":"2024-09-11T01:25:03.013473+0000","last_clean":"2024-09-11T01:25:03.013473+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:25:03.013473+0000","last_undegraded":"2024-09-11T01:25:03.013473+0000","last_fullsized":"2024-09-11T01:25:03.013473+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.014944+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:25:03.014944+0000","last_peered":"2024-09-11T01:25:03.014944+0000","last_clean":"2024-09-11T01:25:03.014944+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:25:03.014944+0000","last_undegraded":"2024-09-11T01:25:03.014944+0000","last_fullsized":"2024-09-11T01:25:03.014944+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28013,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015230+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:25:03.015230+0000","last_peered":"2024-09-11T01:25:03.015230+0000","last_clean":"2024-09-11T01:25:03.015230+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:25:03.015230+0000","last_undegraded":"2024-09-11T01:25:03.015230+0000","last_fullsized":"2024-09-11T01:25:03.015230+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.217689+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:25:03.217689+0000","last_peered":"2024-09-11T01:25:03.217689+0000","last_clean":"2024-09-11T01:25:03.217689+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:25:03.217689+0000","last_undegraded":"2024-09-11T01:25:03.217689+0000","last_fullsized":"2024-09-11T01:25:03.217689+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20028,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:03.016969+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:25:03.016969+0000","last_peered":"2024-09-11T01:25:03.016969+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:25:03.016969+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":133,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.013350+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:25:03.013350+0000","last_peered":"2024-09-11T01:25:03.013350+0000","last_clean":"2024-09-11T01:25:03.013350+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:25:03.013350+0000","last_undegraded":"2024-09-11T01:25:03.013350+0000","last_fullsized":"2024-09-11T01:25:03.013350+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28418,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220421+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:25:03.220421+0000","last_peered":"2024-09-11T01:25:03.220421+0000","last_clean":"2024-09-11T01:25:03.220421+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:25:03.220421+0000","last_undegraded":"2024-09-11T01:25:03.220421+0000","last_fullsized":"2024-09-11T01:25:03.220421+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":111,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015139+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:25:03.015139+0000","last_peered":"2024-09-11T01:25:03.015139+0000","last_clean":"2024-09-11T01:25:03.015139+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:25:03.015139+0000","last_undegraded":"2024-09-11T01:25:03.015139+0000","last_fullsized":"2024-09-11T01:25:03.015139+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27809,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.217608+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:25:03.217608+0000","last_peered":"2024-09-11T01:25:03.217608+0000","last_clean":"2024-09-11T01:25:03.217608+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:25:03.217608+0000","last_undegraded":"2024-09-11T01:25:03.217608+0000","last_fullsized":"2024-09-11T01:25:03.217608+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20419,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.116247+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:25:03.116247+0000","last_peered":"2024-09-11T01:25:03.116247+0000","last_clean":"2024-09-11T01:25:03.116247+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:25:03.116247+0000","last_undegraded":"2024-09-11T01:25:03.116247+0000","last_fullsized":"2024-09-11T01:25:03.116247+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015749+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:25:03.015749+0000","last_peered":"2024-09-11T01:25:03.015749+0000","last_clean":"2024-09-11T01:25:03.015749+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:25:03.015749+0000","last_undegraded":"2024-09-11T01:25:03.015749+0000","last_fullsized":"2024-09-11T01:25:03.015749+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27947,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015630+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:25:03.015630+0000","last_peered":"2024-09-11T01:25:03.015630+0000","last_clean":"2024-09-11T01:25:03.015630+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:25:03.015630+0000","last_undegraded":"2024-09-11T01:25:03.015630+0000","last_fullsized":"2024-09-11T01:25:03.015630+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220797+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:25:03.220797+0000","last_peered":"2024-09-11T01:25:03.220797+0000","last_clean":"2024-09-11T01:25:03.220797+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:25:03.220797+0000","last_undegraded":"2024-09-11T01:25:03.220797+0000","last_fullsized":"2024-09-11T01:25:03.220797+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24626,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015423+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:25:03.015423+0000","last_peered":"2024-09-11T01:25:03.015423+0000","last_clean":"2024-09-11T01:25:03.015423+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:25:03.015423+0000","last_undegraded":"2024-09-11T01:25:03.015423+0000","last_fullsized":"2024-09-11T01:25:03.015423+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":101,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015540+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:25:03.015540+0000","last_peered":"2024-09-11T01:25:03.015540+0000","last_clean":"2024-09-11T01:25:03.015540+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:25:03.015540+0000","last_undegraded":"2024-09-11T01:25:03.015540+0000","last_fullsized":"2024-09-11T01:25:03.015540+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.015405+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:25:03.015405+0000","last_peered":"2024-09-11T01:25:03.015405+0000","last_clean":"2024-09-11T01:25:03.015405+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:25:03.015405+0000","last_undegraded":"2024-09-11T01:25:03.015405+0000","last_fullsized":"2024-09-11T01:25:03.015405+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20118,"reported_epoch":59,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:03.013826+0000","last_change":"2024-09-11T01:13:10.280944+0000","last_active":"2024-09-11T01:25:03.013826+0000","last_peered":"2024-09-11T01:25:03.013826+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:25:03.013826+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3920,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3916}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":12887,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":34752,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":171859968,"data_stored":171825178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":28,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182565,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518648,"kb_used_data":63592,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853192,"statfs":{"total":96636764160,"available":96105668608,"internally_reserved":0,"allocated":65118208,"data_stored":64784829,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986338,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":466308,"kb_used_data":105652,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93905532,"statfs":{"total":96636764160,"available":96159264768,"internally_reserved":0,"allocated":108187648,"data_stored":107858346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659282,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479188,"kb_used_data":516,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93892652,"statfs":{"total":96636764160,"available":96146075648,"internally_reserved":0,"allocated":528384,"data_stored":170652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":490139311},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":107204608,"data_stored":107201129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:24.832 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:25.178 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:25.179 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:25:25.198 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":485,"stamp":"2024-09-11T01:25:24.858206+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11694,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35946,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472468,"kb_used_data":169764,"kb_used_omap":12,"kb_used_meta":1302643,"kb_avail":281643052,"statfs":{"total":289910292480,"available":288402485248,"internally_reserved":0,"allocated":173838336,"data_stored":172817586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1333906436},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2303,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001831"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28531,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085798+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:25:23.085798+0000","last_peered":"2024-09-11T01:25:23.085798+0000","last_clean":"2024-09-11T01:25:23.085798+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:25:23.085798+0000","last_undegraded":"2024-09-11T01:25:23.085798+0000","last_fullsized":"2024-09-11T01:25:23.085798+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20485,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220829+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:25:03.220829+0000","last_peered":"2024-09-11T01:25:03.220829+0000","last_clean":"2024-09-11T01:25:03.220829+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:25:03.220829+0000","last_undegraded":"2024-09-11T01:25:03.220829+0000","last_fullsized":"2024-09-11T01:25:03.220829+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20313,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085337+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:25:23.085337+0000","last_peered":"2024-09-11T01:25:23.085337+0000","last_clean":"2024-09-11T01:25:23.085337+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:25:23.085337+0000","last_undegraded":"2024-09-11T01:25:23.085337+0000","last_fullsized":"2024-09-11T01:25:23.085337+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20235,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220654+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:25:03.220654+0000","last_peered":"2024-09-11T01:25:03.220654+0000","last_clean":"2024-09-11T01:25:03.220654+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:25:03.220654+0000","last_undegraded":"2024-09-11T01:25:03.220654+0000","last_fullsized":"2024-09-11T01:25:03.220654+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20173,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.086787+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:25:23.086787+0000","last_peered":"2024-09-11T01:25:23.086787+0000","last_clean":"2024-09-11T01:25:23.086787+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:25:23.086787+0000","last_undegraded":"2024-09-11T01:25:23.086787+0000","last_fullsized":"2024-09-11T01:25:23.086787+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20419,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.194847+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:25:23.194847+0000","last_peered":"2024-09-11T01:25:23.194847+0000","last_clean":"2024-09-11T01:25:23.194847+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:25:23.194847+0000","last_undegraded":"2024-09-11T01:25:23.194847+0000","last_fullsized":"2024-09-11T01:25:23.194847+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20096,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.084087+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:25:23.084087+0000","last_peered":"2024-09-11T01:25:23.084087+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:25:23.084087+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20420,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083638+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:25:23.083638+0000","last_peered":"2024-09-11T01:25:23.083638+0000","last_clean":"2024-09-11T01:25:23.083638+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:25:23.083638+0000","last_undegraded":"2024-09-11T01:25:23.083638+0000","last_fullsized":"2024-09-11T01:25:23.083638+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083251+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:25:23.083251+0000","last_peered":"2024-09-11T01:25:23.083251+0000","last_clean":"2024-09-11T01:25:23.083251+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:25:23.083251+0000","last_undegraded":"2024-09-11T01:25:23.083251+0000","last_fullsized":"2024-09-11T01:25:23.083251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28017,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084352+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:25:23.084352+0000","last_peered":"2024-09-11T01:25:23.084352+0000","last_clean":"2024-09-11T01:25:23.084352+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:25:23.084352+0000","last_undegraded":"2024-09-11T01:25:23.084352+0000","last_fullsized":"2024-09-11T01:25:23.084352+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.217689+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:25:03.217689+0000","last_peered":"2024-09-11T01:25:03.217689+0000","last_clean":"2024-09-11T01:25:03.217689+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:25:03.217689+0000","last_undegraded":"2024-09-11T01:25:03.217689+0000","last_fullsized":"2024-09-11T01:25:03.217689+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20032,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.085615+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:25:23.085615+0000","last_peered":"2024-09-11T01:25:23.085615+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:25:23.085615+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083438+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:25:23.083438+0000","last_peered":"2024-09-11T01:25:23.083438+0000","last_clean":"2024-09-11T01:25:23.083438+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:25:23.083438+0000","last_undegraded":"2024-09-11T01:25:23.083438+0000","last_fullsized":"2024-09-11T01:25:23.083438+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28418,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220421+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:25:03.220421+0000","last_peered":"2024-09-11T01:25:03.220421+0000","last_clean":"2024-09-11T01:25:03.220421+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:25:03.220421+0000","last_undegraded":"2024-09-11T01:25:03.220421+0000","last_fullsized":"2024-09-11T01:25:03.220421+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084173+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:25:23.084173+0000","last_peered":"2024-09-11T01:25:23.084173+0000","last_clean":"2024-09-11T01:25:23.084173+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:25:23.084173+0000","last_undegraded":"2024-09-11T01:25:23.084173+0000","last_fullsized":"2024-09-11T01:25:23.084173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27809,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.217608+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:25:03.217608+0000","last_peered":"2024-09-11T01:25:03.217608+0000","last_clean":"2024-09-11T01:25:03.217608+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:25:03.217608+0000","last_undegraded":"2024-09-11T01:25:03.217608+0000","last_fullsized":"2024-09-11T01:25:03.217608+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20423,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085593+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:25:23.085593+0000","last_peered":"2024-09-11T01:25:23.085593+0000","last_clean":"2024-09-11T01:25:23.085593+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:25:23.085593+0000","last_undegraded":"2024-09-11T01:25:23.085593+0000","last_fullsized":"2024-09-11T01:25:23.085593+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085867+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:25:23.085867+0000","last_peered":"2024-09-11T01:25:23.085867+0000","last_clean":"2024-09-11T01:25:23.085867+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:25:23.085867+0000","last_undegraded":"2024-09-11T01:25:23.085867+0000","last_fullsized":"2024-09-11T01:25:23.085867+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27951,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085470+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:25:23.085470+0000","last_peered":"2024-09-11T01:25:23.085470+0000","last_clean":"2024-09-11T01:25:23.085470+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:25:23.085470+0000","last_undegraded":"2024-09-11T01:25:23.085470+0000","last_fullsized":"2024-09-11T01:25:23.085470+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220797+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:25:03.220797+0000","last_peered":"2024-09-11T01:25:03.220797+0000","last_clean":"2024-09-11T01:25:03.220797+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:25:03.220797+0000","last_undegraded":"2024-09-11T01:25:03.220797+0000","last_fullsized":"2024-09-11T01:25:03.220797+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24630,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085345+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:25:23.085345+0000","last_peered":"2024-09-11T01:25:23.085345+0000","last_clean":"2024-09-11T01:25:23.085345+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:25:23.085345+0000","last_undegraded":"2024-09-11T01:25:23.085345+0000","last_fullsized":"2024-09-11T01:25:23.085345+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084154+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:25:23.084154+0000","last_peered":"2024-09-11T01:25:23.084154+0000","last_clean":"2024-09-11T01:25:23.084154+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:25:23.084154+0000","last_undegraded":"2024-09-11T01:25:23.084154+0000","last_fullsized":"2024-09-11T01:25:23.084154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085458+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:25:23.085458+0000","last_peered":"2024-09-11T01:25:23.085458+0000","last_clean":"2024-09-11T01:25:23.085458+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:25:23.085458+0000","last_undegraded":"2024-09-11T01:25:23.085458+0000","last_fullsized":"2024-09-11T01:25:23.085458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20523,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.287261+0000","last_change":"2024-09-11T01:25:21.628441+0000","last_active":"2024-09-11T01:25:23.287261+0000","last_peered":"2024-09-11T01:25:23.287261+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:25:23.287261+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3720,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":200,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3719}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11694,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35945,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":171847680,"data_stored":171823327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182566,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518656,"kb_used_data":63600,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853184,"statfs":{"total":96636764160,"available":96105660416,"internally_reserved":0,"allocated":65126400,"data_stored":64787634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986338,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":466308,"kb_used_data":105652,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93905532,"statfs":{"total":96636764160,"available":96159264768,"internally_reserved":0,"allocated":108187648,"data_stored":107858346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659283,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487504,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":486971,"kb_avail":93884336,"statfs":{"total":96636764160,"available":96137560064,"internally_reserved":0,"allocated":524288,"data_stored":171606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":498658991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":107204608,"data_stored":107201129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:25.200 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:25.534 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:25.535 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:25:25.557 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":485,"stamp":"2024-09-11T01:25:24.858206+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11694,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35946,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472468,"kb_used_data":169764,"kb_used_omap":12,"kb_used_meta":1302643,"kb_avail":281643052,"statfs":{"total":289910292480,"available":288402485248,"internally_reserved":0,"allocated":173838336,"data_stored":172817586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1333906436},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2303,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001831"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28531,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085798+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:25:23.085798+0000","last_peered":"2024-09-11T01:25:23.085798+0000","last_clean":"2024-09-11T01:25:23.085798+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:25:23.085798+0000","last_undegraded":"2024-09-11T01:25:23.085798+0000","last_fullsized":"2024-09-11T01:25:23.085798+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20485,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220829+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:25:03.220829+0000","last_peered":"2024-09-11T01:25:03.220829+0000","last_clean":"2024-09-11T01:25:03.220829+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:25:03.220829+0000","last_undegraded":"2024-09-11T01:25:03.220829+0000","last_fullsized":"2024-09-11T01:25:03.220829+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20313,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085337+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:25:23.085337+0000","last_peered":"2024-09-11T01:25:23.085337+0000","last_clean":"2024-09-11T01:25:23.085337+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:25:23.085337+0000","last_undegraded":"2024-09-11T01:25:23.085337+0000","last_fullsized":"2024-09-11T01:25:23.085337+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20235,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220654+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:25:03.220654+0000","last_peered":"2024-09-11T01:25:03.220654+0000","last_clean":"2024-09-11T01:25:03.220654+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:25:03.220654+0000","last_undegraded":"2024-09-11T01:25:03.220654+0000","last_fullsized":"2024-09-11T01:25:03.220654+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20173,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.086787+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:25:23.086787+0000","last_peered":"2024-09-11T01:25:23.086787+0000","last_clean":"2024-09-11T01:25:23.086787+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:25:23.086787+0000","last_undegraded":"2024-09-11T01:25:23.086787+0000","last_fullsized":"2024-09-11T01:25:23.086787+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20419,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.194847+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:25:23.194847+0000","last_peered":"2024-09-11T01:25:23.194847+0000","last_clean":"2024-09-11T01:25:23.194847+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:25:23.194847+0000","last_undegraded":"2024-09-11T01:25:23.194847+0000","last_fullsized":"2024-09-11T01:25:23.194847+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20096,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.084087+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:25:23.084087+0000","last_peered":"2024-09-11T01:25:23.084087+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:25:23.084087+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20420,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083638+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:25:23.083638+0000","last_peered":"2024-09-11T01:25:23.083638+0000","last_clean":"2024-09-11T01:25:23.083638+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:25:23.083638+0000","last_undegraded":"2024-09-11T01:25:23.083638+0000","last_fullsized":"2024-09-11T01:25:23.083638+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083251+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:25:23.083251+0000","last_peered":"2024-09-11T01:25:23.083251+0000","last_clean":"2024-09-11T01:25:23.083251+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:25:23.083251+0000","last_undegraded":"2024-09-11T01:25:23.083251+0000","last_fullsized":"2024-09-11T01:25:23.083251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28017,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084352+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:25:23.084352+0000","last_peered":"2024-09-11T01:25:23.084352+0000","last_clean":"2024-09-11T01:25:23.084352+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:25:23.084352+0000","last_undegraded":"2024-09-11T01:25:23.084352+0000","last_fullsized":"2024-09-11T01:25:23.084352+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.217689+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:25:03.217689+0000","last_peered":"2024-09-11T01:25:03.217689+0000","last_clean":"2024-09-11T01:25:03.217689+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:25:03.217689+0000","last_undegraded":"2024-09-11T01:25:03.217689+0000","last_fullsized":"2024-09-11T01:25:03.217689+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20032,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.085615+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:25:23.085615+0000","last_peered":"2024-09-11T01:25:23.085615+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:25:23.085615+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083438+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:25:23.083438+0000","last_peered":"2024-09-11T01:25:23.083438+0000","last_clean":"2024-09-11T01:25:23.083438+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:25:23.083438+0000","last_undegraded":"2024-09-11T01:25:23.083438+0000","last_fullsized":"2024-09-11T01:25:23.083438+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28418,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220421+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:25:03.220421+0000","last_peered":"2024-09-11T01:25:03.220421+0000","last_clean":"2024-09-11T01:25:03.220421+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:25:03.220421+0000","last_undegraded":"2024-09-11T01:25:03.220421+0000","last_fullsized":"2024-09-11T01:25:03.220421+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084173+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:25:23.084173+0000","last_peered":"2024-09-11T01:25:23.084173+0000","last_clean":"2024-09-11T01:25:23.084173+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:25:23.084173+0000","last_undegraded":"2024-09-11T01:25:23.084173+0000","last_fullsized":"2024-09-11T01:25:23.084173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27809,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.217608+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:25:03.217608+0000","last_peered":"2024-09-11T01:25:03.217608+0000","last_clean":"2024-09-11T01:25:03.217608+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:25:03.217608+0000","last_undegraded":"2024-09-11T01:25:03.217608+0000","last_fullsized":"2024-09-11T01:25:03.217608+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20423,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085593+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:25:23.085593+0000","last_peered":"2024-09-11T01:25:23.085593+0000","last_clean":"2024-09-11T01:25:23.085593+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:25:23.085593+0000","last_undegraded":"2024-09-11T01:25:23.085593+0000","last_fullsized":"2024-09-11T01:25:23.085593+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085867+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:25:23.085867+0000","last_peered":"2024-09-11T01:25:23.085867+0000","last_clean":"2024-09-11T01:25:23.085867+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:25:23.085867+0000","last_undegraded":"2024-09-11T01:25:23.085867+0000","last_fullsized":"2024-09-11T01:25:23.085867+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27951,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085470+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:25:23.085470+0000","last_peered":"2024-09-11T01:25:23.085470+0000","last_clean":"2024-09-11T01:25:23.085470+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:25:23.085470+0000","last_undegraded":"2024-09-11T01:25:23.085470+0000","last_fullsized":"2024-09-11T01:25:23.085470+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220797+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:25:03.220797+0000","last_peered":"2024-09-11T01:25:03.220797+0000","last_clean":"2024-09-11T01:25:03.220797+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:25:03.220797+0000","last_undegraded":"2024-09-11T01:25:03.220797+0000","last_fullsized":"2024-09-11T01:25:03.220797+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24630,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085345+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:25:23.085345+0000","last_peered":"2024-09-11T01:25:23.085345+0000","last_clean":"2024-09-11T01:25:23.085345+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:25:23.085345+0000","last_undegraded":"2024-09-11T01:25:23.085345+0000","last_fullsized":"2024-09-11T01:25:23.085345+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084154+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:25:23.084154+0000","last_peered":"2024-09-11T01:25:23.084154+0000","last_clean":"2024-09-11T01:25:23.084154+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:25:23.084154+0000","last_undegraded":"2024-09-11T01:25:23.084154+0000","last_fullsized":"2024-09-11T01:25:23.084154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085458+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:25:23.085458+0000","last_peered":"2024-09-11T01:25:23.085458+0000","last_clean":"2024-09-11T01:25:23.085458+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:25:23.085458+0000","last_undegraded":"2024-09-11T01:25:23.085458+0000","last_fullsized":"2024-09-11T01:25:23.085458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20523,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.287261+0000","last_change":"2024-09-11T01:25:21.628441+0000","last_active":"2024-09-11T01:25:23.287261+0000","last_peered":"2024-09-11T01:25:23.287261+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:25:23.287261+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3720,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":200,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3719}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11694,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35945,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":171847680,"data_stored":171823327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182566,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518656,"kb_used_data":63600,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853184,"statfs":{"total":96636764160,"available":96105660416,"internally_reserved":0,"allocated":65126400,"data_stored":64787634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986338,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":466308,"kb_used_data":105652,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93905532,"statfs":{"total":96636764160,"available":96159264768,"internally_reserved":0,"allocated":108187648,"data_stored":107858346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659283,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487504,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":486971,"kb_avail":93884336,"statfs":{"total":96636764160,"available":96137560064,"internally_reserved":0,"allocated":524288,"data_stored":171606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":498658991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":107204608,"data_stored":107201129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:25.559 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:25:25.559 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20096, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:23.084087+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:25:23.084087+0000', 'last_peered': '2024-09-11T01:25:23.084087+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:25:23.084087+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:25.559 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:25:25.559 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20032, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:23.085615+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:25:23.085615+0000', 'last_peered': '2024-09-11T01:25:23.085615+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:25:23.085615+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:25.559 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:25:25.559 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 20523, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:23.287261+0000', 'last_change': '2024-09-11T01:25:21.628441+0000', 'last_active': '2024-09-11T01:25:23.287261+0000', 'last_peered': '2024-09-11T01:25:23.287261+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:25:23.287261+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3720, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 200, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3719}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:25.560 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:25:26.000 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:26.001 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 11694/6 objects degraded (194900.000%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":862,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1507807232,"bytes_avail":288402485248,"bytes_total":289910292480,"degraded_objects":11694,"degraded_total":6,"degraded_ratio":1949,"recovering_objects_per_sec":287,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2303,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:25:26.022 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:25:26.022 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:26.401 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:26.402 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:25:26.419 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":485,"stamp":"2024-09-11T01:25:24.858206+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11694,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35946,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1472468,"kb_used_data":169764,"kb_used_omap":12,"kb_used_meta":1302643,"kb_avail":281643052,"statfs":{"total":289910292480,"available":288402485248,"internally_reserved":0,"allocated":173838336,"data_stored":172817586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1333906436},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2303,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2303,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001831"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28531,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085798+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:25:23.085798+0000","last_peered":"2024-09-11T01:25:23.085798+0000","last_clean":"2024-09-11T01:25:23.085798+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:25:23.085798+0000","last_undegraded":"2024-09-11T01:25:23.085798+0000","last_fullsized":"2024-09-11T01:25:23.085798+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20485,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220829+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:25:03.220829+0000","last_peered":"2024-09-11T01:25:03.220829+0000","last_clean":"2024-09-11T01:25:03.220829+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:25:03.220829+0000","last_undegraded":"2024-09-11T01:25:03.220829+0000","last_fullsized":"2024-09-11T01:25:03.220829+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20313,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085337+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:25:23.085337+0000","last_peered":"2024-09-11T01:25:23.085337+0000","last_clean":"2024-09-11T01:25:23.085337+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:25:23.085337+0000","last_undegraded":"2024-09-11T01:25:23.085337+0000","last_fullsized":"2024-09-11T01:25:23.085337+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20235,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220654+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:25:03.220654+0000","last_peered":"2024-09-11T01:25:03.220654+0000","last_clean":"2024-09-11T01:25:03.220654+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:25:03.220654+0000","last_undegraded":"2024-09-11T01:25:03.220654+0000","last_fullsized":"2024-09-11T01:25:03.220654+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20173,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.086787+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:25:23.086787+0000","last_peered":"2024-09-11T01:25:23.086787+0000","last_clean":"2024-09-11T01:25:23.086787+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:25:23.086787+0000","last_undegraded":"2024-09-11T01:25:23.086787+0000","last_fullsized":"2024-09-11T01:25:23.086787+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20419,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.194847+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:25:23.194847+0000","last_peered":"2024-09-11T01:25:23.194847+0000","last_clean":"2024-09-11T01:25:23.194847+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:25:23.194847+0000","last_undegraded":"2024-09-11T01:25:23.194847+0000","last_fullsized":"2024-09-11T01:25:23.194847+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20096,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.084087+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:25:23.084087+0000","last_peered":"2024-09-11T01:25:23.084087+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:25:23.084087+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20420,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083638+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:25:23.083638+0000","last_peered":"2024-09-11T01:25:23.083638+0000","last_clean":"2024-09-11T01:25:23.083638+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:25:23.083638+0000","last_undegraded":"2024-09-11T01:25:23.083638+0000","last_fullsized":"2024-09-11T01:25:23.083638+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083251+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:25:23.083251+0000","last_peered":"2024-09-11T01:25:23.083251+0000","last_clean":"2024-09-11T01:25:23.083251+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:25:23.083251+0000","last_undegraded":"2024-09-11T01:25:23.083251+0000","last_fullsized":"2024-09-11T01:25:23.083251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28017,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084352+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:25:23.084352+0000","last_peered":"2024-09-11T01:25:23.084352+0000","last_clean":"2024-09-11T01:25:23.084352+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:25:23.084352+0000","last_undegraded":"2024-09-11T01:25:23.084352+0000","last_fullsized":"2024-09-11T01:25:23.084352+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.217689+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:25:03.217689+0000","last_peered":"2024-09-11T01:25:03.217689+0000","last_clean":"2024-09-11T01:25:03.217689+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:25:03.217689+0000","last_undegraded":"2024-09-11T01:25:03.217689+0000","last_fullsized":"2024-09-11T01:25:03.217689+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20032,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.085615+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:25:23.085615+0000","last_peered":"2024-09-11T01:25:23.085615+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:25:23.085615+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083438+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:25:23.083438+0000","last_peered":"2024-09-11T01:25:23.083438+0000","last_clean":"2024-09-11T01:25:23.083438+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:25:23.083438+0000","last_undegraded":"2024-09-11T01:25:23.083438+0000","last_fullsized":"2024-09-11T01:25:23.083438+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28418,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220421+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:25:03.220421+0000","last_peered":"2024-09-11T01:25:03.220421+0000","last_clean":"2024-09-11T01:25:03.220421+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:25:03.220421+0000","last_undegraded":"2024-09-11T01:25:03.220421+0000","last_fullsized":"2024-09-11T01:25:03.220421+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084173+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:25:23.084173+0000","last_peered":"2024-09-11T01:25:23.084173+0000","last_clean":"2024-09-11T01:25:23.084173+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:25:23.084173+0000","last_undegraded":"2024-09-11T01:25:23.084173+0000","last_fullsized":"2024-09-11T01:25:23.084173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27809,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.217608+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:25:03.217608+0000","last_peered":"2024-09-11T01:25:03.217608+0000","last_clean":"2024-09-11T01:25:03.217608+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:25:03.217608+0000","last_undegraded":"2024-09-11T01:25:03.217608+0000","last_fullsized":"2024-09-11T01:25:03.217608+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20423,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085593+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:25:23.085593+0000","last_peered":"2024-09-11T01:25:23.085593+0000","last_clean":"2024-09-11T01:25:23.085593+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:25:23.085593+0000","last_undegraded":"2024-09-11T01:25:23.085593+0000","last_fullsized":"2024-09-11T01:25:23.085593+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085867+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:25:23.085867+0000","last_peered":"2024-09-11T01:25:23.085867+0000","last_clean":"2024-09-11T01:25:23.085867+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:25:23.085867+0000","last_undegraded":"2024-09-11T01:25:23.085867+0000","last_fullsized":"2024-09-11T01:25:23.085867+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27951,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085470+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:25:23.085470+0000","last_peered":"2024-09-11T01:25:23.085470+0000","last_clean":"2024-09-11T01:25:23.085470+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:25:23.085470+0000","last_undegraded":"2024-09-11T01:25:23.085470+0000","last_fullsized":"2024-09-11T01:25:23.085470+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":102,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-11T01:25:03.220797+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:25:03.220797+0000","last_peered":"2024-09-11T01:25:03.220797+0000","last_clean":"2024-09-11T01:25:03.220797+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:25:03.220797+0000","last_undegraded":"2024-09-11T01:25:03.220797+0000","last_fullsized":"2024-09-11T01:25:03.220797+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24630,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085345+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:25:23.085345+0000","last_peered":"2024-09-11T01:25:23.085345+0000","last_clean":"2024-09-11T01:25:23.085345+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:25:23.085345+0000","last_undegraded":"2024-09-11T01:25:23.085345+0000","last_fullsized":"2024-09-11T01:25:23.085345+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084154+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:25:23.084154+0000","last_peered":"2024-09-11T01:25:23.084154+0000","last_clean":"2024-09-11T01:25:23.084154+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:25:23.084154+0000","last_undegraded":"2024-09-11T01:25:23.084154+0000","last_fullsized":"2024-09-11T01:25:23.084154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085458+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:25:23.085458+0000","last_peered":"2024-09-11T01:25:23.085458+0000","last_clean":"2024-09-11T01:25:23.085458+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:25:23.085458+0000","last_undegraded":"2024-09-11T01:25:23.085458+0000","last_fullsized":"2024-09-11T01:25:23.085458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20523,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.287261+0000","last_change":"2024-09-11T01:25:21.628441+0000","last_active":"2024-09-11T01:25:23.287261+0000","last_peered":"2024-09-11T01:25:23.287261+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:25:23.287261+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3720,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":200,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3719}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":11694,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":35945,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":171847680,"data_stored":171823327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182566,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518656,"kb_used_data":63600,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853184,"statfs":{"total":96636764160,"available":96105660416,"internally_reserved":0,"allocated":65126400,"data_stored":64787634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986338,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":466308,"kb_used_data":105652,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93905532,"statfs":{"total":96636764160,"available":96159264768,"internally_reserved":0,"allocated":108187648,"data_stored":107858346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659283,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487504,"kb_used_data":512,"kb_used_omap":4,"kb_used_meta":486971,"kb_avail":93884336,"statfs":{"total":96636764160,"available":96137560064,"internally_reserved":0,"allocated":524288,"data_stored":171606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":498658991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":107204608,"data_stored":107201129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:29.421 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:29.751 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:29.752 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:25:29.770 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":487,"stamp":"2024-09-11T01:25:28.859076+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10971,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36669,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1461544,"kb_used_data":167160,"kb_used_omap":12,"kb_used_meta":1294323,"kb_avail":281653976,"statfs":{"total":289910292480,"available":288413671424,"internally_reserved":0,"allocated":171171840,"data_stored":170129479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1325386756},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1916,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1916,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001761"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28532,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085852+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:25:23.085852+0000","last_peered":"2024-09-11T01:25:23.085852+0000","last_clean":"2024-09-11T01:25:23.085852+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:25:23.085852+0000","last_undegraded":"2024-09-11T01:25:23.085852+0000","last_fullsized":"2024-09-11T01:25:23.085852+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20489,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255114+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:25:23.255114+0000","last_peered":"2024-09-11T01:25:23.255114+0000","last_clean":"2024-09-11T01:25:23.255114+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:25:23.255114+0000","last_undegraded":"2024-09-11T01:25:23.255114+0000","last_fullsized":"2024-09-11T01:25:23.255114+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20313,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085337+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:25:23.085337+0000","last_peered":"2024-09-11T01:25:23.085337+0000","last_clean":"2024-09-11T01:25:23.085337+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:25:23.085337+0000","last_undegraded":"2024-09-11T01:25:23.085337+0000","last_fullsized":"2024-09-11T01:25:23.085337+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20239,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255297+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:25:23.255297+0000","last_peered":"2024-09-11T01:25:23.255297+0000","last_clean":"2024-09-11T01:25:23.255297+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:25:23.255297+0000","last_undegraded":"2024-09-11T01:25:23.255297+0000","last_fullsized":"2024-09-11T01:25:23.255297+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20173,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.086787+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:25:23.086787+0000","last_peered":"2024-09-11T01:25:23.086787+0000","last_clean":"2024-09-11T01:25:23.086787+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:25:23.086787+0000","last_undegraded":"2024-09-11T01:25:23.086787+0000","last_fullsized":"2024-09-11T01:25:23.086787+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20419,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.194847+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:25:23.194847+0000","last_peered":"2024-09-11T01:25:23.194847+0000","last_clean":"2024-09-11T01:25:23.194847+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:25:23.194847+0000","last_undegraded":"2024-09-11T01:25:23.194847+0000","last_fullsized":"2024-09-11T01:25:23.194847+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20096,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.084087+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:25:23.084087+0000","last_peered":"2024-09-11T01:25:23.084087+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:25:23.084087+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20420,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083638+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:25:23.083638+0000","last_peered":"2024-09-11T01:25:23.083638+0000","last_clean":"2024-09-11T01:25:23.083638+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:25:23.083638+0000","last_undegraded":"2024-09-11T01:25:23.083638+0000","last_fullsized":"2024-09-11T01:25:23.083638+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083251+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:25:23.083251+0000","last_peered":"2024-09-11T01:25:23.083251+0000","last_clean":"2024-09-11T01:25:23.083251+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:25:23.083251+0000","last_undegraded":"2024-09-11T01:25:23.083251+0000","last_fullsized":"2024-09-11T01:25:23.083251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28017,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084352+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:25:23.084352+0000","last_peered":"2024-09-11T01:25:23.084352+0000","last_clean":"2024-09-11T01:25:23.084352+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:25:23.084352+0000","last_undegraded":"2024-09-11T01:25:23.084352+0000","last_fullsized":"2024-09-11T01:25:23.084352+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254905+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:25:23.254905+0000","last_peered":"2024-09-11T01:25:23.254905+0000","last_clean":"2024-09-11T01:25:23.254905+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:25:23.254905+0000","last_undegraded":"2024-09-11T01:25:23.254905+0000","last_fullsized":"2024-09-11T01:25:23.254905+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20032,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.085615+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:25:23.085615+0000","last_peered":"2024-09-11T01:25:23.085615+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:25:23.085615+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083438+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:25:23.083438+0000","last_peered":"2024-09-11T01:25:23.083438+0000","last_clean":"2024-09-11T01:25:23.083438+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:25:23.083438+0000","last_undegraded":"2024-09-11T01:25:23.083438+0000","last_fullsized":"2024-09-11T01:25:23.083438+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28422,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254938+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:25:23.254938+0000","last_peered":"2024-09-11T01:25:23.254938+0000","last_clean":"2024-09-11T01:25:23.254938+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:25:23.254938+0000","last_undegraded":"2024-09-11T01:25:23.254938+0000","last_fullsized":"2024-09-11T01:25:23.254938+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084173+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:25:23.084173+0000","last_peered":"2024-09-11T01:25:23.084173+0000","last_clean":"2024-09-11T01:25:23.084173+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:25:23.084173+0000","last_undegraded":"2024-09-11T01:25:23.084173+0000","last_fullsized":"2024-09-11T01:25:23.084173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27813,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255394+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:25:23.255394+0000","last_peered":"2024-09-11T01:25:23.255394+0000","last_clean":"2024-09-11T01:25:23.255394+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:25:23.255394+0000","last_undegraded":"2024-09-11T01:25:23.255394+0000","last_fullsized":"2024-09-11T01:25:23.255394+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20423,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085593+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:25:23.085593+0000","last_peered":"2024-09-11T01:25:23.085593+0000","last_clean":"2024-09-11T01:25:23.085593+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:25:23.085593+0000","last_undegraded":"2024-09-11T01:25:23.085593+0000","last_fullsized":"2024-09-11T01:25:23.085593+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085867+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:25:23.085867+0000","last_peered":"2024-09-11T01:25:23.085867+0000","last_clean":"2024-09-11T01:25:23.085867+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:25:23.085867+0000","last_undegraded":"2024-09-11T01:25:23.085867+0000","last_fullsized":"2024-09-11T01:25:23.085867+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27951,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085470+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:25:23.085470+0000","last_peered":"2024-09-11T01:25:23.085470+0000","last_clean":"2024-09-11T01:25:23.085470+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:25:23.085470+0000","last_undegraded":"2024-09-11T01:25:23.085470+0000","last_fullsized":"2024-09-11T01:25:23.085470+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255181+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:25:23.255181+0000","last_peered":"2024-09-11T01:25:23.255181+0000","last_clean":"2024-09-11T01:25:23.255181+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:25:23.255181+0000","last_undegraded":"2024-09-11T01:25:23.255181+0000","last_fullsized":"2024-09-11T01:25:23.255181+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24630,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085345+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:25:23.085345+0000","last_peered":"2024-09-11T01:25:23.085345+0000","last_clean":"2024-09-11T01:25:23.085345+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:25:23.085345+0000","last_undegraded":"2024-09-11T01:25:23.085345+0000","last_fullsized":"2024-09-11T01:25:23.085345+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084154+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:25:23.084154+0000","last_peered":"2024-09-11T01:25:23.084154+0000","last_clean":"2024-09-11T01:25:23.084154+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:25:23.084154+0000","last_undegraded":"2024-09-11T01:25:23.084154+0000","last_fullsized":"2024-09-11T01:25:23.084154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085458+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:25:23.085458+0000","last_peered":"2024-09-11T01:25:23.085458+0000","last_clean":"2024-09-11T01:25:23.085458+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:25:23.085458+0000","last_undegraded":"2024-09-11T01:25:23.085458+0000","last_fullsized":"2024-09-11T01:25:23.085458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":21969,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:28.278296+0000","last_change":"2024-09-11T01:25:21.628441+0000","last_active":"2024-09-11T01:25:28.278296+0000","last_peered":"2024-09-11T01:25:28.278296+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:25:28.278296+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2997,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":923,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2996}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10971,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36668,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":169025536,"data_stored":169001183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182567,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518664,"kb_used_data":63608,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853176,"statfs":{"total":96636764160,"available":96105652224,"internally_reserved":0,"allocated":65134592,"data_stored":64790418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986339,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":463688,"kb_used_data":103032,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93908152,"statfs":{"total":96636764160,"available":96161947648,"internally_reserved":0,"allocated":105504768,"data_stored":105164671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659284,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479192,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93892648,"statfs":{"total":96636764160,"available":96146071552,"internally_reserved":0,"allocated":532480,"data_stored":174390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":490139311},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":104382464,"data_stored":104378985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:29.771 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:30.073 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:30.074 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:25:30.093 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":487,"stamp":"2024-09-11T01:25:28.859076+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10971,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36669,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1461544,"kb_used_data":167160,"kb_used_omap":12,"kb_used_meta":1294323,"kb_avail":281653976,"statfs":{"total":289910292480,"available":288413671424,"internally_reserved":0,"allocated":171171840,"data_stored":170129479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1325386756},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1916,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1916,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001761"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28532,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085852+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:25:23.085852+0000","last_peered":"2024-09-11T01:25:23.085852+0000","last_clean":"2024-09-11T01:25:23.085852+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:25:23.085852+0000","last_undegraded":"2024-09-11T01:25:23.085852+0000","last_fullsized":"2024-09-11T01:25:23.085852+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20489,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255114+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:25:23.255114+0000","last_peered":"2024-09-11T01:25:23.255114+0000","last_clean":"2024-09-11T01:25:23.255114+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:25:23.255114+0000","last_undegraded":"2024-09-11T01:25:23.255114+0000","last_fullsized":"2024-09-11T01:25:23.255114+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20313,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085337+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:25:23.085337+0000","last_peered":"2024-09-11T01:25:23.085337+0000","last_clean":"2024-09-11T01:25:23.085337+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:25:23.085337+0000","last_undegraded":"2024-09-11T01:25:23.085337+0000","last_fullsized":"2024-09-11T01:25:23.085337+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20239,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255297+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:25:23.255297+0000","last_peered":"2024-09-11T01:25:23.255297+0000","last_clean":"2024-09-11T01:25:23.255297+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:25:23.255297+0000","last_undegraded":"2024-09-11T01:25:23.255297+0000","last_fullsized":"2024-09-11T01:25:23.255297+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20173,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.086787+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:25:23.086787+0000","last_peered":"2024-09-11T01:25:23.086787+0000","last_clean":"2024-09-11T01:25:23.086787+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:25:23.086787+0000","last_undegraded":"2024-09-11T01:25:23.086787+0000","last_fullsized":"2024-09-11T01:25:23.086787+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20419,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.194847+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:25:23.194847+0000","last_peered":"2024-09-11T01:25:23.194847+0000","last_clean":"2024-09-11T01:25:23.194847+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:25:23.194847+0000","last_undegraded":"2024-09-11T01:25:23.194847+0000","last_fullsized":"2024-09-11T01:25:23.194847+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20096,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.084087+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:25:23.084087+0000","last_peered":"2024-09-11T01:25:23.084087+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:25:23.084087+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20420,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083638+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:25:23.083638+0000","last_peered":"2024-09-11T01:25:23.083638+0000","last_clean":"2024-09-11T01:25:23.083638+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:25:23.083638+0000","last_undegraded":"2024-09-11T01:25:23.083638+0000","last_fullsized":"2024-09-11T01:25:23.083638+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083251+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:25:23.083251+0000","last_peered":"2024-09-11T01:25:23.083251+0000","last_clean":"2024-09-11T01:25:23.083251+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:25:23.083251+0000","last_undegraded":"2024-09-11T01:25:23.083251+0000","last_fullsized":"2024-09-11T01:25:23.083251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28017,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084352+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:25:23.084352+0000","last_peered":"2024-09-11T01:25:23.084352+0000","last_clean":"2024-09-11T01:25:23.084352+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:25:23.084352+0000","last_undegraded":"2024-09-11T01:25:23.084352+0000","last_fullsized":"2024-09-11T01:25:23.084352+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254905+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:25:23.254905+0000","last_peered":"2024-09-11T01:25:23.254905+0000","last_clean":"2024-09-11T01:25:23.254905+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:25:23.254905+0000","last_undegraded":"2024-09-11T01:25:23.254905+0000","last_fullsized":"2024-09-11T01:25:23.254905+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20032,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.085615+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:25:23.085615+0000","last_peered":"2024-09-11T01:25:23.085615+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:25:23.085615+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083438+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:25:23.083438+0000","last_peered":"2024-09-11T01:25:23.083438+0000","last_clean":"2024-09-11T01:25:23.083438+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:25:23.083438+0000","last_undegraded":"2024-09-11T01:25:23.083438+0000","last_fullsized":"2024-09-11T01:25:23.083438+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28422,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254938+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:25:23.254938+0000","last_peered":"2024-09-11T01:25:23.254938+0000","last_clean":"2024-09-11T01:25:23.254938+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:25:23.254938+0000","last_undegraded":"2024-09-11T01:25:23.254938+0000","last_fullsized":"2024-09-11T01:25:23.254938+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084173+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:25:23.084173+0000","last_peered":"2024-09-11T01:25:23.084173+0000","last_clean":"2024-09-11T01:25:23.084173+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:25:23.084173+0000","last_undegraded":"2024-09-11T01:25:23.084173+0000","last_fullsized":"2024-09-11T01:25:23.084173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27813,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255394+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:25:23.255394+0000","last_peered":"2024-09-11T01:25:23.255394+0000","last_clean":"2024-09-11T01:25:23.255394+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:25:23.255394+0000","last_undegraded":"2024-09-11T01:25:23.255394+0000","last_fullsized":"2024-09-11T01:25:23.255394+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20423,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085593+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:25:23.085593+0000","last_peered":"2024-09-11T01:25:23.085593+0000","last_clean":"2024-09-11T01:25:23.085593+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:25:23.085593+0000","last_undegraded":"2024-09-11T01:25:23.085593+0000","last_fullsized":"2024-09-11T01:25:23.085593+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085867+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:25:23.085867+0000","last_peered":"2024-09-11T01:25:23.085867+0000","last_clean":"2024-09-11T01:25:23.085867+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:25:23.085867+0000","last_undegraded":"2024-09-11T01:25:23.085867+0000","last_fullsized":"2024-09-11T01:25:23.085867+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27951,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085470+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:25:23.085470+0000","last_peered":"2024-09-11T01:25:23.085470+0000","last_clean":"2024-09-11T01:25:23.085470+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:25:23.085470+0000","last_undegraded":"2024-09-11T01:25:23.085470+0000","last_fullsized":"2024-09-11T01:25:23.085470+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255181+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:25:23.255181+0000","last_peered":"2024-09-11T01:25:23.255181+0000","last_clean":"2024-09-11T01:25:23.255181+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:25:23.255181+0000","last_undegraded":"2024-09-11T01:25:23.255181+0000","last_fullsized":"2024-09-11T01:25:23.255181+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24630,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085345+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:25:23.085345+0000","last_peered":"2024-09-11T01:25:23.085345+0000","last_clean":"2024-09-11T01:25:23.085345+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:25:23.085345+0000","last_undegraded":"2024-09-11T01:25:23.085345+0000","last_fullsized":"2024-09-11T01:25:23.085345+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084154+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:25:23.084154+0000","last_peered":"2024-09-11T01:25:23.084154+0000","last_clean":"2024-09-11T01:25:23.084154+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:25:23.084154+0000","last_undegraded":"2024-09-11T01:25:23.084154+0000","last_fullsized":"2024-09-11T01:25:23.084154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085458+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:25:23.085458+0000","last_peered":"2024-09-11T01:25:23.085458+0000","last_clean":"2024-09-11T01:25:23.085458+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:25:23.085458+0000","last_undegraded":"2024-09-11T01:25:23.085458+0000","last_fullsized":"2024-09-11T01:25:23.085458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":21969,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:28.278296+0000","last_change":"2024-09-11T01:25:21.628441+0000","last_active":"2024-09-11T01:25:28.278296+0000","last_peered":"2024-09-11T01:25:28.278296+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:25:28.278296+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2997,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":923,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2996}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10971,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36668,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":169025536,"data_stored":169001183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182567,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518664,"kb_used_data":63608,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853176,"statfs":{"total":96636764160,"available":96105652224,"internally_reserved":0,"allocated":65134592,"data_stored":64790418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986339,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":463688,"kb_used_data":103032,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93908152,"statfs":{"total":96636764160,"available":96161947648,"internally_reserved":0,"allocated":105504768,"data_stored":105164671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659284,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479192,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93892648,"statfs":{"total":96636764160,"available":96146071552,"internally_reserved":0,"allocated":532480,"data_stored":174390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":490139311},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":104382464,"data_stored":104378985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:30.096 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:25:30.096 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20096, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:23.084087+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:25:23.084087+0000', 'last_peered': '2024-09-11T01:25:23.084087+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:25:23.084087+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:30.096 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:25:30.097 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20032, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:23.085615+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:25:23.085615+0000', 'last_peered': '2024-09-11T01:25:23.085615+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:25:23.085615+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:30.097 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:25:30.097 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 21969, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:28.278296+0000', 'last_change': '2024-09-11T01:25:21.628441+0000', 'last_active': '2024-09-11T01:25:28.278296+0000', 'last_peered': '2024-09-11T01:25:28.278296+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:25:28.278296+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2997, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 923, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2996}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:30.097 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:25:30.499 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:30.499 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 10971/6 objects degraded (182850.000%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":867,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1496621056,"bytes_avail":288413671424,"bytes_total":289910292480,"degraded_objects":10971,"degraded_total":6,"degraded_ratio":1828.5,"recovering_objects_per_sec":239,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1916,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:25:30.519 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:25:30.520 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:30.849 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:30.849 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:25:30.867 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":487,"stamp":"2024-09-11T01:25:28.859076+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10971,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36669,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1461544,"kb_used_data":167160,"kb_used_omap":12,"kb_used_meta":1294323,"kb_avail":281653976,"statfs":{"total":289910292480,"available":288413671424,"internally_reserved":0,"allocated":171171840,"data_stored":170129479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1325386756},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1916,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1916,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001761"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28532,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085852+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:25:23.085852+0000","last_peered":"2024-09-11T01:25:23.085852+0000","last_clean":"2024-09-11T01:25:23.085852+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:25:23.085852+0000","last_undegraded":"2024-09-11T01:25:23.085852+0000","last_fullsized":"2024-09-11T01:25:23.085852+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20489,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255114+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:25:23.255114+0000","last_peered":"2024-09-11T01:25:23.255114+0000","last_clean":"2024-09-11T01:25:23.255114+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:25:23.255114+0000","last_undegraded":"2024-09-11T01:25:23.255114+0000","last_fullsized":"2024-09-11T01:25:23.255114+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20313,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085337+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:25:23.085337+0000","last_peered":"2024-09-11T01:25:23.085337+0000","last_clean":"2024-09-11T01:25:23.085337+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:25:23.085337+0000","last_undegraded":"2024-09-11T01:25:23.085337+0000","last_fullsized":"2024-09-11T01:25:23.085337+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20239,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255297+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:25:23.255297+0000","last_peered":"2024-09-11T01:25:23.255297+0000","last_clean":"2024-09-11T01:25:23.255297+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:25:23.255297+0000","last_undegraded":"2024-09-11T01:25:23.255297+0000","last_fullsized":"2024-09-11T01:25:23.255297+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20173,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.086787+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:25:23.086787+0000","last_peered":"2024-09-11T01:25:23.086787+0000","last_clean":"2024-09-11T01:25:23.086787+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:25:23.086787+0000","last_undegraded":"2024-09-11T01:25:23.086787+0000","last_fullsized":"2024-09-11T01:25:23.086787+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20419,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.194847+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:25:23.194847+0000","last_peered":"2024-09-11T01:25:23.194847+0000","last_clean":"2024-09-11T01:25:23.194847+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:25:23.194847+0000","last_undegraded":"2024-09-11T01:25:23.194847+0000","last_fullsized":"2024-09-11T01:25:23.194847+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20096,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.084087+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:25:23.084087+0000","last_peered":"2024-09-11T01:25:23.084087+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:25:23.084087+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20420,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083638+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:25:23.083638+0000","last_peered":"2024-09-11T01:25:23.083638+0000","last_clean":"2024-09-11T01:25:23.083638+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:25:23.083638+0000","last_undegraded":"2024-09-11T01:25:23.083638+0000","last_fullsized":"2024-09-11T01:25:23.083638+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083251+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:25:23.083251+0000","last_peered":"2024-09-11T01:25:23.083251+0000","last_clean":"2024-09-11T01:25:23.083251+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:25:23.083251+0000","last_undegraded":"2024-09-11T01:25:23.083251+0000","last_fullsized":"2024-09-11T01:25:23.083251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28017,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084352+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:25:23.084352+0000","last_peered":"2024-09-11T01:25:23.084352+0000","last_clean":"2024-09-11T01:25:23.084352+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:25:23.084352+0000","last_undegraded":"2024-09-11T01:25:23.084352+0000","last_fullsized":"2024-09-11T01:25:23.084352+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254905+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:25:23.254905+0000","last_peered":"2024-09-11T01:25:23.254905+0000","last_clean":"2024-09-11T01:25:23.254905+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:25:23.254905+0000","last_undegraded":"2024-09-11T01:25:23.254905+0000","last_fullsized":"2024-09-11T01:25:23.254905+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20032,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.085615+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:25:23.085615+0000","last_peered":"2024-09-11T01:25:23.085615+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:25:23.085615+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083438+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:25:23.083438+0000","last_peered":"2024-09-11T01:25:23.083438+0000","last_clean":"2024-09-11T01:25:23.083438+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:25:23.083438+0000","last_undegraded":"2024-09-11T01:25:23.083438+0000","last_fullsized":"2024-09-11T01:25:23.083438+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28422,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254938+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:25:23.254938+0000","last_peered":"2024-09-11T01:25:23.254938+0000","last_clean":"2024-09-11T01:25:23.254938+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:25:23.254938+0000","last_undegraded":"2024-09-11T01:25:23.254938+0000","last_fullsized":"2024-09-11T01:25:23.254938+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084173+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:25:23.084173+0000","last_peered":"2024-09-11T01:25:23.084173+0000","last_clean":"2024-09-11T01:25:23.084173+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:25:23.084173+0000","last_undegraded":"2024-09-11T01:25:23.084173+0000","last_fullsized":"2024-09-11T01:25:23.084173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27813,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255394+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:25:23.255394+0000","last_peered":"2024-09-11T01:25:23.255394+0000","last_clean":"2024-09-11T01:25:23.255394+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:25:23.255394+0000","last_undegraded":"2024-09-11T01:25:23.255394+0000","last_fullsized":"2024-09-11T01:25:23.255394+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20423,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085593+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:25:23.085593+0000","last_peered":"2024-09-11T01:25:23.085593+0000","last_clean":"2024-09-11T01:25:23.085593+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:25:23.085593+0000","last_undegraded":"2024-09-11T01:25:23.085593+0000","last_fullsized":"2024-09-11T01:25:23.085593+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085867+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:25:23.085867+0000","last_peered":"2024-09-11T01:25:23.085867+0000","last_clean":"2024-09-11T01:25:23.085867+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:25:23.085867+0000","last_undegraded":"2024-09-11T01:25:23.085867+0000","last_fullsized":"2024-09-11T01:25:23.085867+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27951,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085470+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:25:23.085470+0000","last_peered":"2024-09-11T01:25:23.085470+0000","last_clean":"2024-09-11T01:25:23.085470+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:25:23.085470+0000","last_undegraded":"2024-09-11T01:25:23.085470+0000","last_fullsized":"2024-09-11T01:25:23.085470+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255181+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:25:23.255181+0000","last_peered":"2024-09-11T01:25:23.255181+0000","last_clean":"2024-09-11T01:25:23.255181+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:25:23.255181+0000","last_undegraded":"2024-09-11T01:25:23.255181+0000","last_fullsized":"2024-09-11T01:25:23.255181+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24630,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085345+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:25:23.085345+0000","last_peered":"2024-09-11T01:25:23.085345+0000","last_clean":"2024-09-11T01:25:23.085345+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:25:23.085345+0000","last_undegraded":"2024-09-11T01:25:23.085345+0000","last_fullsized":"2024-09-11T01:25:23.085345+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084154+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:25:23.084154+0000","last_peered":"2024-09-11T01:25:23.084154+0000","last_clean":"2024-09-11T01:25:23.084154+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:25:23.084154+0000","last_undegraded":"2024-09-11T01:25:23.084154+0000","last_fullsized":"2024-09-11T01:25:23.084154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085458+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:25:23.085458+0000","last_peered":"2024-09-11T01:25:23.085458+0000","last_clean":"2024-09-11T01:25:23.085458+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:25:23.085458+0000","last_undegraded":"2024-09-11T01:25:23.085458+0000","last_fullsized":"2024-09-11T01:25:23.085458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":21969,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:28.278296+0000","last_change":"2024-09-11T01:25:21.628441+0000","last_active":"2024-09-11T01:25:28.278296+0000","last_peered":"2024-09-11T01:25:28.278296+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:25:28.278296+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2997,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":923,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2996}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10971,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36668,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":169025536,"data_stored":169001183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182567,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518664,"kb_used_data":63608,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853176,"statfs":{"total":96636764160,"available":96105652224,"internally_reserved":0,"allocated":65134592,"data_stored":64790418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986339,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":463688,"kb_used_data":103032,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93908152,"statfs":{"total":96636764160,"available":96161947648,"internally_reserved":0,"allocated":105504768,"data_stored":105164671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659284,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479192,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93892648,"statfs":{"total":96636764160,"available":96146071552,"internally_reserved":0,"allocated":532480,"data_stored":174390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":490139311},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":104382464,"data_stored":104378985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:33.869 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:34.208 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:34.209 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:25:34.226 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":489,"stamp":"2024-09-11T01:25:32.860356+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10971,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36669,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1458544,"kb_used_data":164160,"kb_used_omap":12,"kb_used_meta":1294323,"kb_avail":281656976,"statfs":{"total":289910292480,"available":288416743424,"internally_reserved":0,"allocated":168099840,"data_stored":167057479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1325386756},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1916,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1916,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002678"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28532,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085852+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:25:23.085852+0000","last_peered":"2024-09-11T01:25:23.085852+0000","last_clean":"2024-09-11T01:25:23.085852+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:25:23.085852+0000","last_undegraded":"2024-09-11T01:25:23.085852+0000","last_fullsized":"2024-09-11T01:25:23.085852+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20489,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255114+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:25:23.255114+0000","last_peered":"2024-09-11T01:25:23.255114+0000","last_clean":"2024-09-11T01:25:23.255114+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:25:23.255114+0000","last_undegraded":"2024-09-11T01:25:23.255114+0000","last_fullsized":"2024-09-11T01:25:23.255114+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20313,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085337+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:25:23.085337+0000","last_peered":"2024-09-11T01:25:23.085337+0000","last_clean":"2024-09-11T01:25:23.085337+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:25:23.085337+0000","last_undegraded":"2024-09-11T01:25:23.085337+0000","last_fullsized":"2024-09-11T01:25:23.085337+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20239,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255297+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:25:23.255297+0000","last_peered":"2024-09-11T01:25:23.255297+0000","last_clean":"2024-09-11T01:25:23.255297+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:25:23.255297+0000","last_undegraded":"2024-09-11T01:25:23.255297+0000","last_fullsized":"2024-09-11T01:25:23.255297+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20173,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.086787+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:25:23.086787+0000","last_peered":"2024-09-11T01:25:23.086787+0000","last_clean":"2024-09-11T01:25:23.086787+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:25:23.086787+0000","last_undegraded":"2024-09-11T01:25:23.086787+0000","last_fullsized":"2024-09-11T01:25:23.086787+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20419,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.194847+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:25:23.194847+0000","last_peered":"2024-09-11T01:25:23.194847+0000","last_clean":"2024-09-11T01:25:23.194847+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:25:23.194847+0000","last_undegraded":"2024-09-11T01:25:23.194847+0000","last_fullsized":"2024-09-11T01:25:23.194847+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20096,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.084087+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:25:23.084087+0000","last_peered":"2024-09-11T01:25:23.084087+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:25:23.084087+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20420,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083638+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:25:23.083638+0000","last_peered":"2024-09-11T01:25:23.083638+0000","last_clean":"2024-09-11T01:25:23.083638+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:25:23.083638+0000","last_undegraded":"2024-09-11T01:25:23.083638+0000","last_fullsized":"2024-09-11T01:25:23.083638+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083251+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:25:23.083251+0000","last_peered":"2024-09-11T01:25:23.083251+0000","last_clean":"2024-09-11T01:25:23.083251+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:25:23.083251+0000","last_undegraded":"2024-09-11T01:25:23.083251+0000","last_fullsized":"2024-09-11T01:25:23.083251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28017,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084352+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:25:23.084352+0000","last_peered":"2024-09-11T01:25:23.084352+0000","last_clean":"2024-09-11T01:25:23.084352+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:25:23.084352+0000","last_undegraded":"2024-09-11T01:25:23.084352+0000","last_fullsized":"2024-09-11T01:25:23.084352+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254905+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:25:23.254905+0000","last_peered":"2024-09-11T01:25:23.254905+0000","last_clean":"2024-09-11T01:25:23.254905+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:25:23.254905+0000","last_undegraded":"2024-09-11T01:25:23.254905+0000","last_fullsized":"2024-09-11T01:25:23.254905+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20032,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.085615+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:25:23.085615+0000","last_peered":"2024-09-11T01:25:23.085615+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:25:23.085615+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083438+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:25:23.083438+0000","last_peered":"2024-09-11T01:25:23.083438+0000","last_clean":"2024-09-11T01:25:23.083438+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:25:23.083438+0000","last_undegraded":"2024-09-11T01:25:23.083438+0000","last_fullsized":"2024-09-11T01:25:23.083438+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28422,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254938+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:25:23.254938+0000","last_peered":"2024-09-11T01:25:23.254938+0000","last_clean":"2024-09-11T01:25:23.254938+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:25:23.254938+0000","last_undegraded":"2024-09-11T01:25:23.254938+0000","last_fullsized":"2024-09-11T01:25:23.254938+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084173+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:25:23.084173+0000","last_peered":"2024-09-11T01:25:23.084173+0000","last_clean":"2024-09-11T01:25:23.084173+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:25:23.084173+0000","last_undegraded":"2024-09-11T01:25:23.084173+0000","last_fullsized":"2024-09-11T01:25:23.084173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27813,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255394+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:25:23.255394+0000","last_peered":"2024-09-11T01:25:23.255394+0000","last_clean":"2024-09-11T01:25:23.255394+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:25:23.255394+0000","last_undegraded":"2024-09-11T01:25:23.255394+0000","last_fullsized":"2024-09-11T01:25:23.255394+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20423,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085593+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:25:23.085593+0000","last_peered":"2024-09-11T01:25:23.085593+0000","last_clean":"2024-09-11T01:25:23.085593+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:25:23.085593+0000","last_undegraded":"2024-09-11T01:25:23.085593+0000","last_fullsized":"2024-09-11T01:25:23.085593+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085867+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:25:23.085867+0000","last_peered":"2024-09-11T01:25:23.085867+0000","last_clean":"2024-09-11T01:25:23.085867+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:25:23.085867+0000","last_undegraded":"2024-09-11T01:25:23.085867+0000","last_fullsized":"2024-09-11T01:25:23.085867+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27951,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085470+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:25:23.085470+0000","last_peered":"2024-09-11T01:25:23.085470+0000","last_clean":"2024-09-11T01:25:23.085470+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:25:23.085470+0000","last_undegraded":"2024-09-11T01:25:23.085470+0000","last_fullsized":"2024-09-11T01:25:23.085470+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255181+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:25:23.255181+0000","last_peered":"2024-09-11T01:25:23.255181+0000","last_clean":"2024-09-11T01:25:23.255181+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:25:23.255181+0000","last_undegraded":"2024-09-11T01:25:23.255181+0000","last_fullsized":"2024-09-11T01:25:23.255181+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24630,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085345+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:25:23.085345+0000","last_peered":"2024-09-11T01:25:23.085345+0000","last_clean":"2024-09-11T01:25:23.085345+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:25:23.085345+0000","last_undegraded":"2024-09-11T01:25:23.085345+0000","last_fullsized":"2024-09-11T01:25:23.085345+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084154+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:25:23.084154+0000","last_peered":"2024-09-11T01:25:23.084154+0000","last_clean":"2024-09-11T01:25:23.084154+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:25:23.084154+0000","last_undegraded":"2024-09-11T01:25:23.084154+0000","last_fullsized":"2024-09-11T01:25:23.084154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085458+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:25:23.085458+0000","last_peered":"2024-09-11T01:25:23.085458+0000","last_clean":"2024-09-11T01:25:23.085458+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:25:23.085458+0000","last_undegraded":"2024-09-11T01:25:23.085458+0000","last_fullsized":"2024-09-11T01:25:23.085458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":21969,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:28.278296+0000","last_change":"2024-09-11T01:25:21.628441+0000","last_active":"2024-09-11T01:25:28.278296+0000","last_peered":"2024-09-11T01:25:28.278296+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:25:28.278296+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2997,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":923,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2996}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10971,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36668,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":165953536,"data_stored":165929183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182567,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518664,"kb_used_data":63608,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853176,"statfs":{"total":96636764160,"available":96105652224,"internally_reserved":0,"allocated":65134592,"data_stored":64790418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986340,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":460688,"kb_used_data":100032,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93911152,"statfs":{"total":96636764160,"available":96165019648,"internally_reserved":0,"allocated":102432768,"data_stored":102092671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659284,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479192,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93892648,"statfs":{"total":96636764160,"available":96146071552,"internally_reserved":0,"allocated":532480,"data_stored":174390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":490139311},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":101310464,"data_stored":101306985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:34.227 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:34.548 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:34.548 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:25:34.568 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":489,"stamp":"2024-09-11T01:25:32.860356+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10971,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36669,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1458544,"kb_used_data":164160,"kb_used_omap":12,"kb_used_meta":1294323,"kb_avail":281656976,"statfs":{"total":289910292480,"available":288416743424,"internally_reserved":0,"allocated":168099840,"data_stored":167057479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1325386756},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1916,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1916,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002678"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28532,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085852+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:25:23.085852+0000","last_peered":"2024-09-11T01:25:23.085852+0000","last_clean":"2024-09-11T01:25:23.085852+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:25:23.085852+0000","last_undegraded":"2024-09-11T01:25:23.085852+0000","last_fullsized":"2024-09-11T01:25:23.085852+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20489,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255114+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:25:23.255114+0000","last_peered":"2024-09-11T01:25:23.255114+0000","last_clean":"2024-09-11T01:25:23.255114+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:25:23.255114+0000","last_undegraded":"2024-09-11T01:25:23.255114+0000","last_fullsized":"2024-09-11T01:25:23.255114+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20313,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085337+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:25:23.085337+0000","last_peered":"2024-09-11T01:25:23.085337+0000","last_clean":"2024-09-11T01:25:23.085337+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:25:23.085337+0000","last_undegraded":"2024-09-11T01:25:23.085337+0000","last_fullsized":"2024-09-11T01:25:23.085337+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20239,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255297+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:25:23.255297+0000","last_peered":"2024-09-11T01:25:23.255297+0000","last_clean":"2024-09-11T01:25:23.255297+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:25:23.255297+0000","last_undegraded":"2024-09-11T01:25:23.255297+0000","last_fullsized":"2024-09-11T01:25:23.255297+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20173,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.086787+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:25:23.086787+0000","last_peered":"2024-09-11T01:25:23.086787+0000","last_clean":"2024-09-11T01:25:23.086787+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:25:23.086787+0000","last_undegraded":"2024-09-11T01:25:23.086787+0000","last_fullsized":"2024-09-11T01:25:23.086787+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20419,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.194847+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:25:23.194847+0000","last_peered":"2024-09-11T01:25:23.194847+0000","last_clean":"2024-09-11T01:25:23.194847+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:25:23.194847+0000","last_undegraded":"2024-09-11T01:25:23.194847+0000","last_fullsized":"2024-09-11T01:25:23.194847+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20096,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.084087+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:25:23.084087+0000","last_peered":"2024-09-11T01:25:23.084087+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:25:23.084087+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20420,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083638+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:25:23.083638+0000","last_peered":"2024-09-11T01:25:23.083638+0000","last_clean":"2024-09-11T01:25:23.083638+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:25:23.083638+0000","last_undegraded":"2024-09-11T01:25:23.083638+0000","last_fullsized":"2024-09-11T01:25:23.083638+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083251+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:25:23.083251+0000","last_peered":"2024-09-11T01:25:23.083251+0000","last_clean":"2024-09-11T01:25:23.083251+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:25:23.083251+0000","last_undegraded":"2024-09-11T01:25:23.083251+0000","last_fullsized":"2024-09-11T01:25:23.083251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28017,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084352+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:25:23.084352+0000","last_peered":"2024-09-11T01:25:23.084352+0000","last_clean":"2024-09-11T01:25:23.084352+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:25:23.084352+0000","last_undegraded":"2024-09-11T01:25:23.084352+0000","last_fullsized":"2024-09-11T01:25:23.084352+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254905+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:25:23.254905+0000","last_peered":"2024-09-11T01:25:23.254905+0000","last_clean":"2024-09-11T01:25:23.254905+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:25:23.254905+0000","last_undegraded":"2024-09-11T01:25:23.254905+0000","last_fullsized":"2024-09-11T01:25:23.254905+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20032,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.085615+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:25:23.085615+0000","last_peered":"2024-09-11T01:25:23.085615+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:25:23.085615+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083438+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:25:23.083438+0000","last_peered":"2024-09-11T01:25:23.083438+0000","last_clean":"2024-09-11T01:25:23.083438+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:25:23.083438+0000","last_undegraded":"2024-09-11T01:25:23.083438+0000","last_fullsized":"2024-09-11T01:25:23.083438+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28422,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254938+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:25:23.254938+0000","last_peered":"2024-09-11T01:25:23.254938+0000","last_clean":"2024-09-11T01:25:23.254938+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:25:23.254938+0000","last_undegraded":"2024-09-11T01:25:23.254938+0000","last_fullsized":"2024-09-11T01:25:23.254938+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084173+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:25:23.084173+0000","last_peered":"2024-09-11T01:25:23.084173+0000","last_clean":"2024-09-11T01:25:23.084173+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:25:23.084173+0000","last_undegraded":"2024-09-11T01:25:23.084173+0000","last_fullsized":"2024-09-11T01:25:23.084173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27813,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255394+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:25:23.255394+0000","last_peered":"2024-09-11T01:25:23.255394+0000","last_clean":"2024-09-11T01:25:23.255394+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:25:23.255394+0000","last_undegraded":"2024-09-11T01:25:23.255394+0000","last_fullsized":"2024-09-11T01:25:23.255394+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20423,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085593+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:25:23.085593+0000","last_peered":"2024-09-11T01:25:23.085593+0000","last_clean":"2024-09-11T01:25:23.085593+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:25:23.085593+0000","last_undegraded":"2024-09-11T01:25:23.085593+0000","last_fullsized":"2024-09-11T01:25:23.085593+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085867+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:25:23.085867+0000","last_peered":"2024-09-11T01:25:23.085867+0000","last_clean":"2024-09-11T01:25:23.085867+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:25:23.085867+0000","last_undegraded":"2024-09-11T01:25:23.085867+0000","last_fullsized":"2024-09-11T01:25:23.085867+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27951,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085470+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:25:23.085470+0000","last_peered":"2024-09-11T01:25:23.085470+0000","last_clean":"2024-09-11T01:25:23.085470+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:25:23.085470+0000","last_undegraded":"2024-09-11T01:25:23.085470+0000","last_fullsized":"2024-09-11T01:25:23.085470+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255181+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:25:23.255181+0000","last_peered":"2024-09-11T01:25:23.255181+0000","last_clean":"2024-09-11T01:25:23.255181+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:25:23.255181+0000","last_undegraded":"2024-09-11T01:25:23.255181+0000","last_fullsized":"2024-09-11T01:25:23.255181+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24630,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085345+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:25:23.085345+0000","last_peered":"2024-09-11T01:25:23.085345+0000","last_clean":"2024-09-11T01:25:23.085345+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:25:23.085345+0000","last_undegraded":"2024-09-11T01:25:23.085345+0000","last_fullsized":"2024-09-11T01:25:23.085345+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084154+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:25:23.084154+0000","last_peered":"2024-09-11T01:25:23.084154+0000","last_clean":"2024-09-11T01:25:23.084154+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:25:23.084154+0000","last_undegraded":"2024-09-11T01:25:23.084154+0000","last_fullsized":"2024-09-11T01:25:23.084154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085458+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:25:23.085458+0000","last_peered":"2024-09-11T01:25:23.085458+0000","last_clean":"2024-09-11T01:25:23.085458+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:25:23.085458+0000","last_undegraded":"2024-09-11T01:25:23.085458+0000","last_fullsized":"2024-09-11T01:25:23.085458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":21969,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:28.278296+0000","last_change":"2024-09-11T01:25:21.628441+0000","last_active":"2024-09-11T01:25:28.278296+0000","last_peered":"2024-09-11T01:25:28.278296+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:25:28.278296+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2997,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":923,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2996}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10971,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36668,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":165953536,"data_stored":165929183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182567,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518664,"kb_used_data":63608,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853176,"statfs":{"total":96636764160,"available":96105652224,"internally_reserved":0,"allocated":65134592,"data_stored":64790418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986340,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":460688,"kb_used_data":100032,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93911152,"statfs":{"total":96636764160,"available":96165019648,"internally_reserved":0,"allocated":102432768,"data_stored":102092671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659284,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479192,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93892648,"statfs":{"total":96636764160,"available":96146071552,"internally_reserved":0,"allocated":532480,"data_stored":174390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":490139311},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":101310464,"data_stored":101306985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:34.569 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:25:34.569 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20096, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:23.084087+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:25:23.084087+0000', 'last_peered': '2024-09-11T01:25:23.084087+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:25:23.084087+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:34.569 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:25:34.569 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20032, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:23.085615+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:25:23.085615+0000', 'last_peered': '2024-09-11T01:25:23.085615+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:25:23.085615+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:34.570 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:25:34.570 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 21969, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:28.278296+0000', 'last_change': '2024-09-11T01:25:21.628441+0000', 'last_active': '2024-09-11T01:25:28.278296+0000', 'last_peered': '2024-09-11T01:25:28.278296+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:25:28.278296+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2997, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 923, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2996}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:34.570 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:25:34.948 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:34.948 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 10971/6 objects degraded (182850.000%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":871,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1493549056,"bytes_avail":288416743424,"bytes_total":289910292480,"degraded_objects":10971,"degraded_total":6,"degraded_ratio":1828.5,"recovering_objects_per_sec":191,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1916,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:25:34.967 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:25:34.968 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:35.292 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:35.292 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:25:35.309 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":490,"stamp":"2024-09-11T01:25:34.860968+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10644,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36996,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1458544,"kb_used_data":164160,"kb_used_omap":12,"kb_used_meta":1294323,"kb_avail":281656976,"statfs":{"total":289910292480,"available":288416743424,"internally_reserved":0,"allocated":168099840,"data_stored":167057479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1325386756},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2243,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2243,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.779191"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28532,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085852+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:25:23.085852+0000","last_peered":"2024-09-11T01:25:23.085852+0000","last_clean":"2024-09-11T01:25:23.085852+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:25:23.085852+0000","last_undegraded":"2024-09-11T01:25:23.085852+0000","last_fullsized":"2024-09-11T01:25:23.085852+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20489,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255114+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:25:23.255114+0000","last_peered":"2024-09-11T01:25:23.255114+0000","last_clean":"2024-09-11T01:25:23.255114+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:25:23.255114+0000","last_undegraded":"2024-09-11T01:25:23.255114+0000","last_fullsized":"2024-09-11T01:25:23.255114+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20313,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085337+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:25:23.085337+0000","last_peered":"2024-09-11T01:25:23.085337+0000","last_clean":"2024-09-11T01:25:23.085337+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:25:23.085337+0000","last_undegraded":"2024-09-11T01:25:23.085337+0000","last_fullsized":"2024-09-11T01:25:23.085337+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20239,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255297+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:25:23.255297+0000","last_peered":"2024-09-11T01:25:23.255297+0000","last_clean":"2024-09-11T01:25:23.255297+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:25:23.255297+0000","last_undegraded":"2024-09-11T01:25:23.255297+0000","last_fullsized":"2024-09-11T01:25:23.255297+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20173,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.086787+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:25:23.086787+0000","last_peered":"2024-09-11T01:25:23.086787+0000","last_clean":"2024-09-11T01:25:23.086787+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:25:23.086787+0000","last_undegraded":"2024-09-11T01:25:23.086787+0000","last_fullsized":"2024-09-11T01:25:23.086787+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20419,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.194847+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:25:23.194847+0000","last_peered":"2024-09-11T01:25:23.194847+0000","last_clean":"2024-09-11T01:25:23.194847+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:25:23.194847+0000","last_undegraded":"2024-09-11T01:25:23.194847+0000","last_fullsized":"2024-09-11T01:25:23.194847+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20096,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.084087+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:25:23.084087+0000","last_peered":"2024-09-11T01:25:23.084087+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:25:23.084087+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20420,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083638+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:25:23.083638+0000","last_peered":"2024-09-11T01:25:23.083638+0000","last_clean":"2024-09-11T01:25:23.083638+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:25:23.083638+0000","last_undegraded":"2024-09-11T01:25:23.083638+0000","last_fullsized":"2024-09-11T01:25:23.083638+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083251+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:25:23.083251+0000","last_peered":"2024-09-11T01:25:23.083251+0000","last_clean":"2024-09-11T01:25:23.083251+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:25:23.083251+0000","last_undegraded":"2024-09-11T01:25:23.083251+0000","last_fullsized":"2024-09-11T01:25:23.083251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28017,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084352+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:25:23.084352+0000","last_peered":"2024-09-11T01:25:23.084352+0000","last_clean":"2024-09-11T01:25:23.084352+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:25:23.084352+0000","last_undegraded":"2024-09-11T01:25:23.084352+0000","last_fullsized":"2024-09-11T01:25:23.084352+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254905+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:25:23.254905+0000","last_peered":"2024-09-11T01:25:23.254905+0000","last_clean":"2024-09-11T01:25:23.254905+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:25:23.254905+0000","last_undegraded":"2024-09-11T01:25:23.254905+0000","last_fullsized":"2024-09-11T01:25:23.254905+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20033,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.086560+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:25:23.086560+0000","last_peered":"2024-09-11T01:25:23.086560+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:25:23.086560+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083438+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:25:23.083438+0000","last_peered":"2024-09-11T01:25:23.083438+0000","last_clean":"2024-09-11T01:25:23.083438+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:25:23.083438+0000","last_undegraded":"2024-09-11T01:25:23.083438+0000","last_fullsized":"2024-09-11T01:25:23.083438+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28422,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254938+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:25:23.254938+0000","last_peered":"2024-09-11T01:25:23.254938+0000","last_clean":"2024-09-11T01:25:23.254938+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:25:23.254938+0000","last_undegraded":"2024-09-11T01:25:23.254938+0000","last_fullsized":"2024-09-11T01:25:23.254938+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084173+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:25:23.084173+0000","last_peered":"2024-09-11T01:25:23.084173+0000","last_clean":"2024-09-11T01:25:23.084173+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:25:23.084173+0000","last_undegraded":"2024-09-11T01:25:23.084173+0000","last_fullsized":"2024-09-11T01:25:23.084173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27813,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255394+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:25:23.255394+0000","last_peered":"2024-09-11T01:25:23.255394+0000","last_clean":"2024-09-11T01:25:23.255394+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:25:23.255394+0000","last_undegraded":"2024-09-11T01:25:23.255394+0000","last_fullsized":"2024-09-11T01:25:23.255394+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20423,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085593+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:25:23.085593+0000","last_peered":"2024-09-11T01:25:23.085593+0000","last_clean":"2024-09-11T01:25:23.085593+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:25:23.085593+0000","last_undegraded":"2024-09-11T01:25:23.085593+0000","last_fullsized":"2024-09-11T01:25:23.085593+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085867+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:25:23.085867+0000","last_peered":"2024-09-11T01:25:23.085867+0000","last_clean":"2024-09-11T01:25:23.085867+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:25:23.085867+0000","last_undegraded":"2024-09-11T01:25:23.085867+0000","last_fullsized":"2024-09-11T01:25:23.085867+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27951,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085470+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:25:23.085470+0000","last_peered":"2024-09-11T01:25:23.085470+0000","last_clean":"2024-09-11T01:25:23.085470+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:25:23.085470+0000","last_undegraded":"2024-09-11T01:25:23.085470+0000","last_fullsized":"2024-09-11T01:25:23.085470+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255181+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:25:23.255181+0000","last_peered":"2024-09-11T01:25:23.255181+0000","last_clean":"2024-09-11T01:25:23.255181+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:25:23.255181+0000","last_undegraded":"2024-09-11T01:25:23.255181+0000","last_fullsized":"2024-09-11T01:25:23.255181+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24630,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085345+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:25:23.085345+0000","last_peered":"2024-09-11T01:25:23.085345+0000","last_clean":"2024-09-11T01:25:23.085345+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:25:23.085345+0000","last_undegraded":"2024-09-11T01:25:23.085345+0000","last_fullsized":"2024-09-11T01:25:23.085345+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084154+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:25:23.084154+0000","last_peered":"2024-09-11T01:25:23.084154+0000","last_clean":"2024-09-11T01:25:23.084154+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:25:23.084154+0000","last_undegraded":"2024-09-11T01:25:23.084154+0000","last_fullsized":"2024-09-11T01:25:23.084154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085458+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:25:23.085458+0000","last_peered":"2024-09-11T01:25:23.085458+0000","last_clean":"2024-09-11T01:25:23.085458+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:25:23.085458+0000","last_undegraded":"2024-09-11T01:25:23.085458+0000","last_fullsized":"2024-09-11T01:25:23.085458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":22623,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:33.277252+0000","last_change":"2024-09-11T01:25:21.628441+0000","last_active":"2024-09-11T01:25:33.277252+0000","last_peered":"2024-09-11T01:25:33.277252+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:25:33.277252+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2670,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1250,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2669}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10644,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36995,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":165953536,"data_stored":165929183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182568,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518664,"kb_used_data":63608,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853176,"statfs":{"total":96636764160,"available":96105652224,"internally_reserved":0,"allocated":65134592,"data_stored":64790418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986340,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":460688,"kb_used_data":100032,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93911152,"statfs":{"total":96636764160,"available":96165019648,"internally_reserved":0,"allocated":102432768,"data_stored":102092671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659285,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479192,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93892648,"statfs":{"total":96636764160,"available":96146071552,"internally_reserved":0,"allocated":532480,"data_stored":174390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":490139311},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":101310464,"data_stored":101306985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:38.311 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:38.660 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:38.661 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:25:38.678 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":491,"stamp":"2024-09-11T01:25:36.861637+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10644,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36996,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1455544,"kb_used_data":161160,"kb_used_omap":12,"kb_used_meta":1294323,"kb_avail":281659976,"statfs":{"total":289910292480,"available":288419815424,"internally_reserved":0,"allocated":165027840,"data_stored":163985479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1325386756},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1050,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1050,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003431"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28532,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085852+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:25:23.085852+0000","last_peered":"2024-09-11T01:25:23.085852+0000","last_clean":"2024-09-11T01:25:23.085852+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:25:23.085852+0000","last_undegraded":"2024-09-11T01:25:23.085852+0000","last_fullsized":"2024-09-11T01:25:23.085852+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20489,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255114+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:25:23.255114+0000","last_peered":"2024-09-11T01:25:23.255114+0000","last_clean":"2024-09-11T01:25:23.255114+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:25:23.255114+0000","last_undegraded":"2024-09-11T01:25:23.255114+0000","last_fullsized":"2024-09-11T01:25:23.255114+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20313,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085337+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:25:23.085337+0000","last_peered":"2024-09-11T01:25:23.085337+0000","last_clean":"2024-09-11T01:25:23.085337+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:25:23.085337+0000","last_undegraded":"2024-09-11T01:25:23.085337+0000","last_fullsized":"2024-09-11T01:25:23.085337+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20239,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255297+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:25:23.255297+0000","last_peered":"2024-09-11T01:25:23.255297+0000","last_clean":"2024-09-11T01:25:23.255297+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:25:23.255297+0000","last_undegraded":"2024-09-11T01:25:23.255297+0000","last_fullsized":"2024-09-11T01:25:23.255297+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20173,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.086787+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:25:23.086787+0000","last_peered":"2024-09-11T01:25:23.086787+0000","last_clean":"2024-09-11T01:25:23.086787+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:25:23.086787+0000","last_undegraded":"2024-09-11T01:25:23.086787+0000","last_fullsized":"2024-09-11T01:25:23.086787+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20419,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.194847+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:25:23.194847+0000","last_peered":"2024-09-11T01:25:23.194847+0000","last_clean":"2024-09-11T01:25:23.194847+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:25:23.194847+0000","last_undegraded":"2024-09-11T01:25:23.194847+0000","last_fullsized":"2024-09-11T01:25:23.194847+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20096,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.084087+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:25:23.084087+0000","last_peered":"2024-09-11T01:25:23.084087+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:25:23.084087+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20420,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083638+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:25:23.083638+0000","last_peered":"2024-09-11T01:25:23.083638+0000","last_clean":"2024-09-11T01:25:23.083638+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:25:23.083638+0000","last_undegraded":"2024-09-11T01:25:23.083638+0000","last_fullsized":"2024-09-11T01:25:23.083638+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083251+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:25:23.083251+0000","last_peered":"2024-09-11T01:25:23.083251+0000","last_clean":"2024-09-11T01:25:23.083251+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:25:23.083251+0000","last_undegraded":"2024-09-11T01:25:23.083251+0000","last_fullsized":"2024-09-11T01:25:23.083251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28017,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084352+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:25:23.084352+0000","last_peered":"2024-09-11T01:25:23.084352+0000","last_clean":"2024-09-11T01:25:23.084352+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:25:23.084352+0000","last_undegraded":"2024-09-11T01:25:23.084352+0000","last_fullsized":"2024-09-11T01:25:23.084352+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254905+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:25:23.254905+0000","last_peered":"2024-09-11T01:25:23.254905+0000","last_clean":"2024-09-11T01:25:23.254905+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:25:23.254905+0000","last_undegraded":"2024-09-11T01:25:23.254905+0000","last_fullsized":"2024-09-11T01:25:23.254905+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20033,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.086560+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:25:23.086560+0000","last_peered":"2024-09-11T01:25:23.086560+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:25:23.086560+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083438+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:25:23.083438+0000","last_peered":"2024-09-11T01:25:23.083438+0000","last_clean":"2024-09-11T01:25:23.083438+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:25:23.083438+0000","last_undegraded":"2024-09-11T01:25:23.083438+0000","last_fullsized":"2024-09-11T01:25:23.083438+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28422,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254938+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:25:23.254938+0000","last_peered":"2024-09-11T01:25:23.254938+0000","last_clean":"2024-09-11T01:25:23.254938+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:25:23.254938+0000","last_undegraded":"2024-09-11T01:25:23.254938+0000","last_fullsized":"2024-09-11T01:25:23.254938+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084173+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:25:23.084173+0000","last_peered":"2024-09-11T01:25:23.084173+0000","last_clean":"2024-09-11T01:25:23.084173+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:25:23.084173+0000","last_undegraded":"2024-09-11T01:25:23.084173+0000","last_fullsized":"2024-09-11T01:25:23.084173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27813,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255394+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:25:23.255394+0000","last_peered":"2024-09-11T01:25:23.255394+0000","last_clean":"2024-09-11T01:25:23.255394+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:25:23.255394+0000","last_undegraded":"2024-09-11T01:25:23.255394+0000","last_fullsized":"2024-09-11T01:25:23.255394+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20423,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085593+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:25:23.085593+0000","last_peered":"2024-09-11T01:25:23.085593+0000","last_clean":"2024-09-11T01:25:23.085593+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:25:23.085593+0000","last_undegraded":"2024-09-11T01:25:23.085593+0000","last_fullsized":"2024-09-11T01:25:23.085593+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085867+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:25:23.085867+0000","last_peered":"2024-09-11T01:25:23.085867+0000","last_clean":"2024-09-11T01:25:23.085867+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:25:23.085867+0000","last_undegraded":"2024-09-11T01:25:23.085867+0000","last_fullsized":"2024-09-11T01:25:23.085867+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27951,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085470+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:25:23.085470+0000","last_peered":"2024-09-11T01:25:23.085470+0000","last_clean":"2024-09-11T01:25:23.085470+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:25:23.085470+0000","last_undegraded":"2024-09-11T01:25:23.085470+0000","last_fullsized":"2024-09-11T01:25:23.085470+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255181+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:25:23.255181+0000","last_peered":"2024-09-11T01:25:23.255181+0000","last_clean":"2024-09-11T01:25:23.255181+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:25:23.255181+0000","last_undegraded":"2024-09-11T01:25:23.255181+0000","last_fullsized":"2024-09-11T01:25:23.255181+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24630,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085345+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:25:23.085345+0000","last_peered":"2024-09-11T01:25:23.085345+0000","last_clean":"2024-09-11T01:25:23.085345+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:25:23.085345+0000","last_undegraded":"2024-09-11T01:25:23.085345+0000","last_fullsized":"2024-09-11T01:25:23.085345+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084154+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:25:23.084154+0000","last_peered":"2024-09-11T01:25:23.084154+0000","last_clean":"2024-09-11T01:25:23.084154+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:25:23.084154+0000","last_undegraded":"2024-09-11T01:25:23.084154+0000","last_fullsized":"2024-09-11T01:25:23.084154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085458+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:25:23.085458+0000","last_peered":"2024-09-11T01:25:23.085458+0000","last_clean":"2024-09-11T01:25:23.085458+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:25:23.085458+0000","last_undegraded":"2024-09-11T01:25:23.085458+0000","last_fullsized":"2024-09-11T01:25:23.085458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":22623,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:33.277252+0000","last_change":"2024-09-11T01:25:21.628441+0000","last_active":"2024-09-11T01:25:33.277252+0000","last_peered":"2024-09-11T01:25:33.277252+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:25:33.277252+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2670,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1250,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2669}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10644,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":36995,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":162881536,"data_stored":162857183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182568,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518664,"kb_used_data":63608,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853176,"statfs":{"total":96636764160,"available":96105652224,"internally_reserved":0,"allocated":65134592,"data_stored":64790418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986341,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":457688,"kb_used_data":97032,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93914152,"statfs":{"total":96636764160,"available":96168091648,"internally_reserved":0,"allocated":99360768,"data_stored":99020671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659285,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":479192,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":478651,"kb_avail":93892648,"statfs":{"total":96636764160,"available":96146071552,"internally_reserved":0,"allocated":532480,"data_stored":174390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":490139311},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98238464,"data_stored":98234985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:38.680 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:39.035 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:39.035 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:25:39.053 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":492,"stamp":"2024-09-11T01:25:38.862218+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10304,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37336,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1459704,"kb_used_data":161160,"kb_used_omap":12,"kb_used_meta":1298483,"kb_avail":281655816,"statfs":{"total":289910292480,"available":288415555584,"internally_reserved":0,"allocated":165027840,"data_stored":163985479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1329646596},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1390,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1390,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003575"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28532,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085852+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:25:23.085852+0000","last_peered":"2024-09-11T01:25:23.085852+0000","last_clean":"2024-09-11T01:25:23.085852+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:25:23.085852+0000","last_undegraded":"2024-09-11T01:25:23.085852+0000","last_fullsized":"2024-09-11T01:25:23.085852+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20489,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255114+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:25:23.255114+0000","last_peered":"2024-09-11T01:25:23.255114+0000","last_clean":"2024-09-11T01:25:23.255114+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:25:23.255114+0000","last_undegraded":"2024-09-11T01:25:23.255114+0000","last_fullsized":"2024-09-11T01:25:23.255114+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20313,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085337+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:25:23.085337+0000","last_peered":"2024-09-11T01:25:23.085337+0000","last_clean":"2024-09-11T01:25:23.085337+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:25:23.085337+0000","last_undegraded":"2024-09-11T01:25:23.085337+0000","last_fullsized":"2024-09-11T01:25:23.085337+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20239,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255297+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:25:23.255297+0000","last_peered":"2024-09-11T01:25:23.255297+0000","last_clean":"2024-09-11T01:25:23.255297+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:25:23.255297+0000","last_undegraded":"2024-09-11T01:25:23.255297+0000","last_fullsized":"2024-09-11T01:25:23.255297+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20173,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.086787+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:25:23.086787+0000","last_peered":"2024-09-11T01:25:23.086787+0000","last_clean":"2024-09-11T01:25:23.086787+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:25:23.086787+0000","last_undegraded":"2024-09-11T01:25:23.086787+0000","last_fullsized":"2024-09-11T01:25:23.086787+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20419,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.194847+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:25:23.194847+0000","last_peered":"2024-09-11T01:25:23.194847+0000","last_clean":"2024-09-11T01:25:23.194847+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:25:23.194847+0000","last_undegraded":"2024-09-11T01:25:23.194847+0000","last_fullsized":"2024-09-11T01:25:23.194847+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20096,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.084087+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:25:23.084087+0000","last_peered":"2024-09-11T01:25:23.084087+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:25:23.084087+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20420,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083638+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:25:23.083638+0000","last_peered":"2024-09-11T01:25:23.083638+0000","last_clean":"2024-09-11T01:25:23.083638+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:25:23.083638+0000","last_undegraded":"2024-09-11T01:25:23.083638+0000","last_fullsized":"2024-09-11T01:25:23.083638+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083251+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:25:23.083251+0000","last_peered":"2024-09-11T01:25:23.083251+0000","last_clean":"2024-09-11T01:25:23.083251+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:25:23.083251+0000","last_undegraded":"2024-09-11T01:25:23.083251+0000","last_fullsized":"2024-09-11T01:25:23.083251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28017,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084352+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:25:23.084352+0000","last_peered":"2024-09-11T01:25:23.084352+0000","last_clean":"2024-09-11T01:25:23.084352+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:25:23.084352+0000","last_undegraded":"2024-09-11T01:25:23.084352+0000","last_fullsized":"2024-09-11T01:25:23.084352+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254905+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:25:23.254905+0000","last_peered":"2024-09-11T01:25:23.254905+0000","last_clean":"2024-09-11T01:25:23.254905+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:25:23.254905+0000","last_undegraded":"2024-09-11T01:25:23.254905+0000","last_fullsized":"2024-09-11T01:25:23.254905+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20033,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.086560+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:25:23.086560+0000","last_peered":"2024-09-11T01:25:23.086560+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:25:23.086560+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083438+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:25:23.083438+0000","last_peered":"2024-09-11T01:25:23.083438+0000","last_clean":"2024-09-11T01:25:23.083438+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:25:23.083438+0000","last_undegraded":"2024-09-11T01:25:23.083438+0000","last_fullsized":"2024-09-11T01:25:23.083438+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28422,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254938+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:25:23.254938+0000","last_peered":"2024-09-11T01:25:23.254938+0000","last_clean":"2024-09-11T01:25:23.254938+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:25:23.254938+0000","last_undegraded":"2024-09-11T01:25:23.254938+0000","last_fullsized":"2024-09-11T01:25:23.254938+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084173+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:25:23.084173+0000","last_peered":"2024-09-11T01:25:23.084173+0000","last_clean":"2024-09-11T01:25:23.084173+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:25:23.084173+0000","last_undegraded":"2024-09-11T01:25:23.084173+0000","last_fullsized":"2024-09-11T01:25:23.084173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27813,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255394+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:25:23.255394+0000","last_peered":"2024-09-11T01:25:23.255394+0000","last_clean":"2024-09-11T01:25:23.255394+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:25:23.255394+0000","last_undegraded":"2024-09-11T01:25:23.255394+0000","last_fullsized":"2024-09-11T01:25:23.255394+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20423,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085593+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:25:23.085593+0000","last_peered":"2024-09-11T01:25:23.085593+0000","last_clean":"2024-09-11T01:25:23.085593+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:25:23.085593+0000","last_undegraded":"2024-09-11T01:25:23.085593+0000","last_fullsized":"2024-09-11T01:25:23.085593+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085867+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:25:23.085867+0000","last_peered":"2024-09-11T01:25:23.085867+0000","last_clean":"2024-09-11T01:25:23.085867+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:25:23.085867+0000","last_undegraded":"2024-09-11T01:25:23.085867+0000","last_fullsized":"2024-09-11T01:25:23.085867+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27951,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085470+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:25:23.085470+0000","last_peered":"2024-09-11T01:25:23.085470+0000","last_clean":"2024-09-11T01:25:23.085470+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:25:23.085470+0000","last_undegraded":"2024-09-11T01:25:23.085470+0000","last_fullsized":"2024-09-11T01:25:23.085470+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255181+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:25:23.255181+0000","last_peered":"2024-09-11T01:25:23.255181+0000","last_clean":"2024-09-11T01:25:23.255181+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:25:23.255181+0000","last_undegraded":"2024-09-11T01:25:23.255181+0000","last_fullsized":"2024-09-11T01:25:23.255181+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24630,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085345+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:25:23.085345+0000","last_peered":"2024-09-11T01:25:23.085345+0000","last_clean":"2024-09-11T01:25:23.085345+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:25:23.085345+0000","last_undegraded":"2024-09-11T01:25:23.085345+0000","last_fullsized":"2024-09-11T01:25:23.085345+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084154+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:25:23.084154+0000","last_peered":"2024-09-11T01:25:23.084154+0000","last_clean":"2024-09-11T01:25:23.084154+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:25:23.084154+0000","last_undegraded":"2024-09-11T01:25:23.084154+0000","last_fullsized":"2024-09-11T01:25:23.084154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085458+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:25:23.085458+0000","last_peered":"2024-09-11T01:25:23.085458+0000","last_clean":"2024-09-11T01:25:23.085458+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:25:23.085458+0000","last_undegraded":"2024-09-11T01:25:23.085458+0000","last_fullsized":"2024-09-11T01:25:23.085458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":23303,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:38.281225+0000","last_change":"2024-09-11T01:25:21.628441+0000","last_active":"2024-09-11T01:25:38.281225+0000","last_peered":"2024-09-11T01:25:38.281225+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:25:38.281225+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2330,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1590,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2329}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10304,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37335,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":162881536,"data_stored":162857183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182569,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518664,"kb_used_data":63608,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853176,"statfs":{"total":96636764160,"available":96105652224,"internally_reserved":0,"allocated":65134592,"data_stored":64790418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986341,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":457688,"kb_used_data":97032,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93914152,"statfs":{"total":96636764160,"available":96168091648,"internally_reserved":0,"allocated":99360768,"data_stored":99020671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659286,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483352,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":482811,"kb_avail":93888488,"statfs":{"total":96636764160,"available":96141811712,"internally_reserved":0,"allocated":532480,"data_stored":174390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494399151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98238464,"data_stored":98234985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:39.055 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:25:39.055 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20096, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:23.084087+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:25:23.084087+0000', 'last_peered': '2024-09-11T01:25:23.084087+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:25:23.084087+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:39.055 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:25:39.055 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20033, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:23.086560+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:25:23.086560+0000', 'last_peered': '2024-09-11T01:25:23.086560+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:25:23.086560+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:39.055 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:25:39.055 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 23303, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:38.281225+0000', 'last_change': '2024-09-11T01:25:21.628441+0000', 'last_active': '2024-09-11T01:25:38.281225+0000', 'last_peered': '2024-09-11T01:25:38.281225+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:25:38.281225+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2330, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1590, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2329}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:39.056 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:25:39.416 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:39.416 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 10304/6 objects degraded (171733.333%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":875,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1494736896,"bytes_avail":288415555584,"bytes_total":289910292480,"degraded_objects":10304,"degraded_total":6,"degraded_ratio":1717.333333333333,"recovering_objects_per_sec":115,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1390,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:25:39.439 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:25:39.439 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:39.736 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:39.736 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:25:39.753 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":492,"stamp":"2024-09-11T01:25:38.862218+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10304,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37336,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1459704,"kb_used_data":161160,"kb_used_omap":12,"kb_used_meta":1298483,"kb_avail":281655816,"statfs":{"total":289910292480,"available":288415555584,"internally_reserved":0,"allocated":165027840,"data_stored":163985479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1329646596},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1390,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1390,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003575"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28532,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085852+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:25:23.085852+0000","last_peered":"2024-09-11T01:25:23.085852+0000","last_clean":"2024-09-11T01:25:23.085852+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:25:23.085852+0000","last_undegraded":"2024-09-11T01:25:23.085852+0000","last_fullsized":"2024-09-11T01:25:23.085852+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20489,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255114+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:25:23.255114+0000","last_peered":"2024-09-11T01:25:23.255114+0000","last_clean":"2024-09-11T01:25:23.255114+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:25:23.255114+0000","last_undegraded":"2024-09-11T01:25:23.255114+0000","last_fullsized":"2024-09-11T01:25:23.255114+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20313,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085337+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:25:23.085337+0000","last_peered":"2024-09-11T01:25:23.085337+0000","last_clean":"2024-09-11T01:25:23.085337+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:25:23.085337+0000","last_undegraded":"2024-09-11T01:25:23.085337+0000","last_fullsized":"2024-09-11T01:25:23.085337+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20239,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255297+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:25:23.255297+0000","last_peered":"2024-09-11T01:25:23.255297+0000","last_clean":"2024-09-11T01:25:23.255297+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:25:23.255297+0000","last_undegraded":"2024-09-11T01:25:23.255297+0000","last_fullsized":"2024-09-11T01:25:23.255297+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20173,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.086787+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:25:23.086787+0000","last_peered":"2024-09-11T01:25:23.086787+0000","last_clean":"2024-09-11T01:25:23.086787+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:25:23.086787+0000","last_undegraded":"2024-09-11T01:25:23.086787+0000","last_fullsized":"2024-09-11T01:25:23.086787+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20419,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.194847+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:25:23.194847+0000","last_peered":"2024-09-11T01:25:23.194847+0000","last_clean":"2024-09-11T01:25:23.194847+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:25:23.194847+0000","last_undegraded":"2024-09-11T01:25:23.194847+0000","last_fullsized":"2024-09-11T01:25:23.194847+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20096,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.084087+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:25:23.084087+0000","last_peered":"2024-09-11T01:25:23.084087+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:25:23.084087+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20420,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083638+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:25:23.083638+0000","last_peered":"2024-09-11T01:25:23.083638+0000","last_clean":"2024-09-11T01:25:23.083638+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:25:23.083638+0000","last_undegraded":"2024-09-11T01:25:23.083638+0000","last_fullsized":"2024-09-11T01:25:23.083638+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083251+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:25:23.083251+0000","last_peered":"2024-09-11T01:25:23.083251+0000","last_clean":"2024-09-11T01:25:23.083251+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:25:23.083251+0000","last_undegraded":"2024-09-11T01:25:23.083251+0000","last_fullsized":"2024-09-11T01:25:23.083251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28017,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084352+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:25:23.084352+0000","last_peered":"2024-09-11T01:25:23.084352+0000","last_clean":"2024-09-11T01:25:23.084352+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:25:23.084352+0000","last_undegraded":"2024-09-11T01:25:23.084352+0000","last_fullsized":"2024-09-11T01:25:23.084352+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254905+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:25:23.254905+0000","last_peered":"2024-09-11T01:25:23.254905+0000","last_clean":"2024-09-11T01:25:23.254905+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:25:23.254905+0000","last_undegraded":"2024-09-11T01:25:23.254905+0000","last_fullsized":"2024-09-11T01:25:23.254905+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20033,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.086560+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:25:23.086560+0000","last_peered":"2024-09-11T01:25:23.086560+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:25:23.086560+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083438+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:25:23.083438+0000","last_peered":"2024-09-11T01:25:23.083438+0000","last_clean":"2024-09-11T01:25:23.083438+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:25:23.083438+0000","last_undegraded":"2024-09-11T01:25:23.083438+0000","last_fullsized":"2024-09-11T01:25:23.083438+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28422,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254938+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:25:23.254938+0000","last_peered":"2024-09-11T01:25:23.254938+0000","last_clean":"2024-09-11T01:25:23.254938+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:25:23.254938+0000","last_undegraded":"2024-09-11T01:25:23.254938+0000","last_fullsized":"2024-09-11T01:25:23.254938+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084173+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:25:23.084173+0000","last_peered":"2024-09-11T01:25:23.084173+0000","last_clean":"2024-09-11T01:25:23.084173+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:25:23.084173+0000","last_undegraded":"2024-09-11T01:25:23.084173+0000","last_fullsized":"2024-09-11T01:25:23.084173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27813,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255394+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:25:23.255394+0000","last_peered":"2024-09-11T01:25:23.255394+0000","last_clean":"2024-09-11T01:25:23.255394+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:25:23.255394+0000","last_undegraded":"2024-09-11T01:25:23.255394+0000","last_fullsized":"2024-09-11T01:25:23.255394+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20423,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085593+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:25:23.085593+0000","last_peered":"2024-09-11T01:25:23.085593+0000","last_clean":"2024-09-11T01:25:23.085593+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:25:23.085593+0000","last_undegraded":"2024-09-11T01:25:23.085593+0000","last_fullsized":"2024-09-11T01:25:23.085593+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085867+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:25:23.085867+0000","last_peered":"2024-09-11T01:25:23.085867+0000","last_clean":"2024-09-11T01:25:23.085867+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:25:23.085867+0000","last_undegraded":"2024-09-11T01:25:23.085867+0000","last_fullsized":"2024-09-11T01:25:23.085867+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27951,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085470+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:25:23.085470+0000","last_peered":"2024-09-11T01:25:23.085470+0000","last_clean":"2024-09-11T01:25:23.085470+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:25:23.085470+0000","last_undegraded":"2024-09-11T01:25:23.085470+0000","last_fullsized":"2024-09-11T01:25:23.085470+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255181+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:25:23.255181+0000","last_peered":"2024-09-11T01:25:23.255181+0000","last_clean":"2024-09-11T01:25:23.255181+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:25:23.255181+0000","last_undegraded":"2024-09-11T01:25:23.255181+0000","last_fullsized":"2024-09-11T01:25:23.255181+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24630,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085345+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:25:23.085345+0000","last_peered":"2024-09-11T01:25:23.085345+0000","last_clean":"2024-09-11T01:25:23.085345+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:25:23.085345+0000","last_undegraded":"2024-09-11T01:25:23.085345+0000","last_fullsized":"2024-09-11T01:25:23.085345+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084154+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:25:23.084154+0000","last_peered":"2024-09-11T01:25:23.084154+0000","last_clean":"2024-09-11T01:25:23.084154+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:25:23.084154+0000","last_undegraded":"2024-09-11T01:25:23.084154+0000","last_fullsized":"2024-09-11T01:25:23.084154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085458+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:25:23.085458+0000","last_peered":"2024-09-11T01:25:23.085458+0000","last_clean":"2024-09-11T01:25:23.085458+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:25:23.085458+0000","last_undegraded":"2024-09-11T01:25:23.085458+0000","last_fullsized":"2024-09-11T01:25:23.085458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":23303,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:38.281225+0000","last_change":"2024-09-11T01:25:21.628441+0000","last_active":"2024-09-11T01:25:38.281225+0000","last_peered":"2024-09-11T01:25:38.281225+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:25:38.281225+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2330,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1590,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2329}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10304,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37335,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":162881536,"data_stored":162857183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182569,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518664,"kb_used_data":63608,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853176,"statfs":{"total":96636764160,"available":96105652224,"internally_reserved":0,"allocated":65134592,"data_stored":64790418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986341,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":457688,"kb_used_data":97032,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93914152,"statfs":{"total":96636764160,"available":96168091648,"internally_reserved":0,"allocated":99360768,"data_stored":99020671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659286,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483352,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":482811,"kb_avail":93888488,"statfs":{"total":96636764160,"available":96141811712,"internally_reserved":0,"allocated":532480,"data_stored":174390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494399151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98238464,"data_stored":98234985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:42.755 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:43.083 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:43.083 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:25:43.101 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":494,"stamp":"2024-09-11T01:25:42.863255+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10304,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37336,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1456704,"kb_used_data":158160,"kb_used_omap":12,"kb_used_meta":1298483,"kb_avail":281658816,"statfs":{"total":289910292480,"available":288418627584,"internally_reserved":0,"allocated":161955840,"data_stored":160913479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1329646596},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-667,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":667,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003496"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28532,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085852+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:25:23.085852+0000","last_peered":"2024-09-11T01:25:23.085852+0000","last_clean":"2024-09-11T01:25:23.085852+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:25:23.085852+0000","last_undegraded":"2024-09-11T01:25:23.085852+0000","last_fullsized":"2024-09-11T01:25:23.085852+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20490,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255153+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:25:23.255153+0000","last_peered":"2024-09-11T01:25:23.255153+0000","last_clean":"2024-09-11T01:25:23.255153+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:25:23.255153+0000","last_undegraded":"2024-09-11T01:25:23.255153+0000","last_fullsized":"2024-09-11T01:25:23.255153+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20313,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085337+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:25:23.085337+0000","last_peered":"2024-09-11T01:25:23.085337+0000","last_clean":"2024-09-11T01:25:23.085337+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:25:23.085337+0000","last_undegraded":"2024-09-11T01:25:23.085337+0000","last_fullsized":"2024-09-11T01:25:23.085337+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20240,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255347+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:25:23.255347+0000","last_peered":"2024-09-11T01:25:23.255347+0000","last_clean":"2024-09-11T01:25:23.255347+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:25:23.255347+0000","last_undegraded":"2024-09-11T01:25:23.255347+0000","last_fullsized":"2024-09-11T01:25:23.255347+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20173,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.086787+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:25:23.086787+0000","last_peered":"2024-09-11T01:25:23.086787+0000","last_clean":"2024-09-11T01:25:23.086787+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:25:23.086787+0000","last_undegraded":"2024-09-11T01:25:23.086787+0000","last_fullsized":"2024-09-11T01:25:23.086787+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20419,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.194847+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:25:23.194847+0000","last_peered":"2024-09-11T01:25:23.194847+0000","last_clean":"2024-09-11T01:25:23.194847+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:25:23.194847+0000","last_undegraded":"2024-09-11T01:25:23.194847+0000","last_fullsized":"2024-09-11T01:25:23.194847+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20096,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.084087+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:25:23.084087+0000","last_peered":"2024-09-11T01:25:23.084087+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:25:23.084087+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20420,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083638+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:25:23.083638+0000","last_peered":"2024-09-11T01:25:23.083638+0000","last_clean":"2024-09-11T01:25:23.083638+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:25:23.083638+0000","last_undegraded":"2024-09-11T01:25:23.083638+0000","last_fullsized":"2024-09-11T01:25:23.083638+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083251+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:25:23.083251+0000","last_peered":"2024-09-11T01:25:23.083251+0000","last_clean":"2024-09-11T01:25:23.083251+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:25:23.083251+0000","last_undegraded":"2024-09-11T01:25:23.083251+0000","last_fullsized":"2024-09-11T01:25:23.083251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28017,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084352+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:25:23.084352+0000","last_peered":"2024-09-11T01:25:23.084352+0000","last_clean":"2024-09-11T01:25:23.084352+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:25:23.084352+0000","last_undegraded":"2024-09-11T01:25:23.084352+0000","last_fullsized":"2024-09-11T01:25:23.084352+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254905+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:25:23.254905+0000","last_peered":"2024-09-11T01:25:23.254905+0000","last_clean":"2024-09-11T01:25:23.254905+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:25:23.254905+0000","last_undegraded":"2024-09-11T01:25:23.254905+0000","last_fullsized":"2024-09-11T01:25:23.254905+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20033,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.086560+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:25:23.086560+0000","last_peered":"2024-09-11T01:25:23.086560+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:25:23.086560+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083438+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:25:23.083438+0000","last_peered":"2024-09-11T01:25:23.083438+0000","last_clean":"2024-09-11T01:25:23.083438+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:25:23.083438+0000","last_undegraded":"2024-09-11T01:25:23.083438+0000","last_fullsized":"2024-09-11T01:25:23.083438+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28422,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254938+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:25:23.254938+0000","last_peered":"2024-09-11T01:25:23.254938+0000","last_clean":"2024-09-11T01:25:23.254938+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:25:23.254938+0000","last_undegraded":"2024-09-11T01:25:23.254938+0000","last_fullsized":"2024-09-11T01:25:23.254938+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084173+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:25:23.084173+0000","last_peered":"2024-09-11T01:25:23.084173+0000","last_clean":"2024-09-11T01:25:23.084173+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:25:23.084173+0000","last_undegraded":"2024-09-11T01:25:23.084173+0000","last_fullsized":"2024-09-11T01:25:23.084173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27813,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255394+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:25:23.255394+0000","last_peered":"2024-09-11T01:25:23.255394+0000","last_clean":"2024-09-11T01:25:23.255394+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:25:23.255394+0000","last_undegraded":"2024-09-11T01:25:23.255394+0000","last_fullsized":"2024-09-11T01:25:23.255394+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20423,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085593+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:25:23.085593+0000","last_peered":"2024-09-11T01:25:23.085593+0000","last_clean":"2024-09-11T01:25:23.085593+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:25:23.085593+0000","last_undegraded":"2024-09-11T01:25:23.085593+0000","last_fullsized":"2024-09-11T01:25:23.085593+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085867+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:25:23.085867+0000","last_peered":"2024-09-11T01:25:23.085867+0000","last_clean":"2024-09-11T01:25:23.085867+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:25:23.085867+0000","last_undegraded":"2024-09-11T01:25:23.085867+0000","last_fullsized":"2024-09-11T01:25:23.085867+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27951,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085470+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:25:23.085470+0000","last_peered":"2024-09-11T01:25:23.085470+0000","last_clean":"2024-09-11T01:25:23.085470+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:25:23.085470+0000","last_undegraded":"2024-09-11T01:25:23.085470+0000","last_fullsized":"2024-09-11T01:25:23.085470+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255181+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:25:23.255181+0000","last_peered":"2024-09-11T01:25:23.255181+0000","last_clean":"2024-09-11T01:25:23.255181+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:25:23.255181+0000","last_undegraded":"2024-09-11T01:25:23.255181+0000","last_fullsized":"2024-09-11T01:25:23.255181+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24630,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085345+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:25:23.085345+0000","last_peered":"2024-09-11T01:25:23.085345+0000","last_clean":"2024-09-11T01:25:23.085345+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:25:23.085345+0000","last_undegraded":"2024-09-11T01:25:23.085345+0000","last_fullsized":"2024-09-11T01:25:23.085345+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084154+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:25:23.084154+0000","last_peered":"2024-09-11T01:25:23.084154+0000","last_clean":"2024-09-11T01:25:23.084154+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:25:23.084154+0000","last_undegraded":"2024-09-11T01:25:23.084154+0000","last_fullsized":"2024-09-11T01:25:23.084154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085458+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:25:23.085458+0000","last_peered":"2024-09-11T01:25:23.085458+0000","last_clean":"2024-09-11T01:25:23.085458+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:25:23.085458+0000","last_undegraded":"2024-09-11T01:25:23.085458+0000","last_fullsized":"2024-09-11T01:25:23.085458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":23303,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:38.281225+0000","last_change":"2024-09-11T01:25:21.628441+0000","last_active":"2024-09-11T01:25:38.281225+0000","last_peered":"2024-09-11T01:25:38.281225+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:25:38.281225+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2330,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1590,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2329}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10304,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37335,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":159809536,"data_stored":159785183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182569,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518664,"kb_used_data":63608,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853176,"statfs":{"total":96636764160,"available":96105652224,"internally_reserved":0,"allocated":65134592,"data_stored":64790418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986342,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":454688,"kb_used_data":94032,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93917152,"statfs":{"total":96636764160,"available":96171163648,"internally_reserved":0,"allocated":96288768,"data_stored":95948671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659286,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483352,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":482811,"kb_avail":93888488,"statfs":{"total":96636764160,"available":96141811712,"internally_reserved":0,"allocated":532480,"data_stored":174390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494399151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":95166464,"data_stored":95162985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:43.103 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:43.417 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:43.417 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:25:43.434 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":494,"stamp":"2024-09-11T01:25:42.863255+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10304,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37336,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1456704,"kb_used_data":158160,"kb_used_omap":12,"kb_used_meta":1298483,"kb_avail":281658816,"statfs":{"total":289910292480,"available":288418627584,"internally_reserved":0,"allocated":161955840,"data_stored":160913479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1329646596},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-667,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":667,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003496"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28532,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085852+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:25:23.085852+0000","last_peered":"2024-09-11T01:25:23.085852+0000","last_clean":"2024-09-11T01:25:23.085852+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:25:23.085852+0000","last_undegraded":"2024-09-11T01:25:23.085852+0000","last_fullsized":"2024-09-11T01:25:23.085852+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20490,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255153+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:25:23.255153+0000","last_peered":"2024-09-11T01:25:23.255153+0000","last_clean":"2024-09-11T01:25:23.255153+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:25:23.255153+0000","last_undegraded":"2024-09-11T01:25:23.255153+0000","last_fullsized":"2024-09-11T01:25:23.255153+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20313,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085337+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:25:23.085337+0000","last_peered":"2024-09-11T01:25:23.085337+0000","last_clean":"2024-09-11T01:25:23.085337+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:25:23.085337+0000","last_undegraded":"2024-09-11T01:25:23.085337+0000","last_fullsized":"2024-09-11T01:25:23.085337+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20240,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255347+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:25:23.255347+0000","last_peered":"2024-09-11T01:25:23.255347+0000","last_clean":"2024-09-11T01:25:23.255347+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:25:23.255347+0000","last_undegraded":"2024-09-11T01:25:23.255347+0000","last_fullsized":"2024-09-11T01:25:23.255347+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20173,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.086787+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:25:23.086787+0000","last_peered":"2024-09-11T01:25:23.086787+0000","last_clean":"2024-09-11T01:25:23.086787+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:25:23.086787+0000","last_undegraded":"2024-09-11T01:25:23.086787+0000","last_fullsized":"2024-09-11T01:25:23.086787+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20419,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.194847+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:25:23.194847+0000","last_peered":"2024-09-11T01:25:23.194847+0000","last_clean":"2024-09-11T01:25:23.194847+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:25:23.194847+0000","last_undegraded":"2024-09-11T01:25:23.194847+0000","last_fullsized":"2024-09-11T01:25:23.194847+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20096,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.084087+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:25:23.084087+0000","last_peered":"2024-09-11T01:25:23.084087+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:25:23.084087+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20420,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083638+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:25:23.083638+0000","last_peered":"2024-09-11T01:25:23.083638+0000","last_clean":"2024-09-11T01:25:23.083638+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:25:23.083638+0000","last_undegraded":"2024-09-11T01:25:23.083638+0000","last_fullsized":"2024-09-11T01:25:23.083638+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083251+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:25:23.083251+0000","last_peered":"2024-09-11T01:25:23.083251+0000","last_clean":"2024-09-11T01:25:23.083251+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:25:23.083251+0000","last_undegraded":"2024-09-11T01:25:23.083251+0000","last_fullsized":"2024-09-11T01:25:23.083251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28017,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084352+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:25:23.084352+0000","last_peered":"2024-09-11T01:25:23.084352+0000","last_clean":"2024-09-11T01:25:23.084352+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:25:23.084352+0000","last_undegraded":"2024-09-11T01:25:23.084352+0000","last_fullsized":"2024-09-11T01:25:23.084352+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254905+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:25:23.254905+0000","last_peered":"2024-09-11T01:25:23.254905+0000","last_clean":"2024-09-11T01:25:23.254905+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:25:23.254905+0000","last_undegraded":"2024-09-11T01:25:23.254905+0000","last_fullsized":"2024-09-11T01:25:23.254905+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20033,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.086560+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:25:23.086560+0000","last_peered":"2024-09-11T01:25:23.086560+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:25:23.086560+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083438+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:25:23.083438+0000","last_peered":"2024-09-11T01:25:23.083438+0000","last_clean":"2024-09-11T01:25:23.083438+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:25:23.083438+0000","last_undegraded":"2024-09-11T01:25:23.083438+0000","last_fullsized":"2024-09-11T01:25:23.083438+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28422,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254938+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:25:23.254938+0000","last_peered":"2024-09-11T01:25:23.254938+0000","last_clean":"2024-09-11T01:25:23.254938+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:25:23.254938+0000","last_undegraded":"2024-09-11T01:25:23.254938+0000","last_fullsized":"2024-09-11T01:25:23.254938+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084173+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:25:23.084173+0000","last_peered":"2024-09-11T01:25:23.084173+0000","last_clean":"2024-09-11T01:25:23.084173+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:25:23.084173+0000","last_undegraded":"2024-09-11T01:25:23.084173+0000","last_fullsized":"2024-09-11T01:25:23.084173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27813,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255394+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:25:23.255394+0000","last_peered":"2024-09-11T01:25:23.255394+0000","last_clean":"2024-09-11T01:25:23.255394+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:25:23.255394+0000","last_undegraded":"2024-09-11T01:25:23.255394+0000","last_fullsized":"2024-09-11T01:25:23.255394+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20423,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085593+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:25:23.085593+0000","last_peered":"2024-09-11T01:25:23.085593+0000","last_clean":"2024-09-11T01:25:23.085593+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:25:23.085593+0000","last_undegraded":"2024-09-11T01:25:23.085593+0000","last_fullsized":"2024-09-11T01:25:23.085593+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085867+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:25:23.085867+0000","last_peered":"2024-09-11T01:25:23.085867+0000","last_clean":"2024-09-11T01:25:23.085867+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:25:23.085867+0000","last_undegraded":"2024-09-11T01:25:23.085867+0000","last_fullsized":"2024-09-11T01:25:23.085867+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27951,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085470+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:25:23.085470+0000","last_peered":"2024-09-11T01:25:23.085470+0000","last_clean":"2024-09-11T01:25:23.085470+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:25:23.085470+0000","last_undegraded":"2024-09-11T01:25:23.085470+0000","last_fullsized":"2024-09-11T01:25:23.085470+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255181+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:25:23.255181+0000","last_peered":"2024-09-11T01:25:23.255181+0000","last_clean":"2024-09-11T01:25:23.255181+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:25:23.255181+0000","last_undegraded":"2024-09-11T01:25:23.255181+0000","last_fullsized":"2024-09-11T01:25:23.255181+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24630,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085345+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:25:23.085345+0000","last_peered":"2024-09-11T01:25:23.085345+0000","last_clean":"2024-09-11T01:25:23.085345+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:25:23.085345+0000","last_undegraded":"2024-09-11T01:25:23.085345+0000","last_fullsized":"2024-09-11T01:25:23.085345+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084154+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:25:23.084154+0000","last_peered":"2024-09-11T01:25:23.084154+0000","last_clean":"2024-09-11T01:25:23.084154+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:25:23.084154+0000","last_undegraded":"2024-09-11T01:25:23.084154+0000","last_fullsized":"2024-09-11T01:25:23.084154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085458+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:25:23.085458+0000","last_peered":"2024-09-11T01:25:23.085458+0000","last_clean":"2024-09-11T01:25:23.085458+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:25:23.085458+0000","last_undegraded":"2024-09-11T01:25:23.085458+0000","last_fullsized":"2024-09-11T01:25:23.085458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":23303,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:38.281225+0000","last_change":"2024-09-11T01:25:21.628441+0000","last_active":"2024-09-11T01:25:38.281225+0000","last_peered":"2024-09-11T01:25:38.281225+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:25:38.281225+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2330,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1590,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2329}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10304,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37335,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":159809536,"data_stored":159785183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182569,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518664,"kb_used_data":63608,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853176,"statfs":{"total":96636764160,"available":96105652224,"internally_reserved":0,"allocated":65134592,"data_stored":64790418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986342,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":454688,"kb_used_data":94032,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93917152,"statfs":{"total":96636764160,"available":96171163648,"internally_reserved":0,"allocated":96288768,"data_stored":95948671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659286,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483352,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":482811,"kb_avail":93888488,"statfs":{"total":96636764160,"available":96141811712,"internally_reserved":0,"allocated":532480,"data_stored":174390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494399151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":95166464,"data_stored":95162985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:43.436 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:25:43.436 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20096, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:23.084087+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:25:23.084087+0000', 'last_peered': '2024-09-11T01:25:23.084087+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:25:23.084087+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:43.436 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:25:43.436 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20033, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:23.086560+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:25:23.086560+0000', 'last_peered': '2024-09-11T01:25:23.086560+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:25:23.086560+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:43.436 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:25:43.436 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 23303, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:38.281225+0000', 'last_change': '2024-09-11T01:25:21.628441+0000', 'last_active': '2024-09-11T01:25:38.281225+0000', 'last_peered': '2024-09-11T01:25:38.281225+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:25:38.281225+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2330, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1590, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2329}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:43.436 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:25:43.804 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:43.804 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 10304/6 objects degraded (171733.333%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":880,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1491664896,"bytes_avail":288418627584,"bytes_total":289910292480,"degraded_objects":10304,"degraded_total":6,"degraded_ratio":1717.333333333333,"recovering_objects_per_sec":55,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":667,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:25:43.825 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:25:43.825 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:44.135 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:44.136 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:25:44.152 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":494,"stamp":"2024-09-11T01:25:42.863255+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10304,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37336,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1456704,"kb_used_data":158160,"kb_used_omap":12,"kb_used_meta":1298483,"kb_avail":281658816,"statfs":{"total":289910292480,"available":288418627584,"internally_reserved":0,"allocated":161955840,"data_stored":160913479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1329646596},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-667,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":667,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003496"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28532,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085852+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:25:23.085852+0000","last_peered":"2024-09-11T01:25:23.085852+0000","last_clean":"2024-09-11T01:25:23.085852+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:25:23.085852+0000","last_undegraded":"2024-09-11T01:25:23.085852+0000","last_fullsized":"2024-09-11T01:25:23.085852+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20490,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255153+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:25:23.255153+0000","last_peered":"2024-09-11T01:25:23.255153+0000","last_clean":"2024-09-11T01:25:23.255153+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:25:23.255153+0000","last_undegraded":"2024-09-11T01:25:23.255153+0000","last_fullsized":"2024-09-11T01:25:23.255153+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20313,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085337+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:25:23.085337+0000","last_peered":"2024-09-11T01:25:23.085337+0000","last_clean":"2024-09-11T01:25:23.085337+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:25:23.085337+0000","last_undegraded":"2024-09-11T01:25:23.085337+0000","last_fullsized":"2024-09-11T01:25:23.085337+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20240,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255347+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:25:23.255347+0000","last_peered":"2024-09-11T01:25:23.255347+0000","last_clean":"2024-09-11T01:25:23.255347+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:25:23.255347+0000","last_undegraded":"2024-09-11T01:25:23.255347+0000","last_fullsized":"2024-09-11T01:25:23.255347+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20173,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.086787+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:25:23.086787+0000","last_peered":"2024-09-11T01:25:23.086787+0000","last_clean":"2024-09-11T01:25:23.086787+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:25:23.086787+0000","last_undegraded":"2024-09-11T01:25:23.086787+0000","last_fullsized":"2024-09-11T01:25:23.086787+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20419,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.194847+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:25:23.194847+0000","last_peered":"2024-09-11T01:25:23.194847+0000","last_clean":"2024-09-11T01:25:23.194847+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:25:23.194847+0000","last_undegraded":"2024-09-11T01:25:23.194847+0000","last_fullsized":"2024-09-11T01:25:23.194847+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20096,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.084087+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:25:23.084087+0000","last_peered":"2024-09-11T01:25:23.084087+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:25:23.084087+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20420,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083638+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:25:23.083638+0000","last_peered":"2024-09-11T01:25:23.083638+0000","last_clean":"2024-09-11T01:25:23.083638+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:25:23.083638+0000","last_undegraded":"2024-09-11T01:25:23.083638+0000","last_fullsized":"2024-09-11T01:25:23.083638+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083251+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:25:23.083251+0000","last_peered":"2024-09-11T01:25:23.083251+0000","last_clean":"2024-09-11T01:25:23.083251+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:25:23.083251+0000","last_undegraded":"2024-09-11T01:25:23.083251+0000","last_fullsized":"2024-09-11T01:25:23.083251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28017,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084352+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:25:23.084352+0000","last_peered":"2024-09-11T01:25:23.084352+0000","last_clean":"2024-09-11T01:25:23.084352+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:25:23.084352+0000","last_undegraded":"2024-09-11T01:25:23.084352+0000","last_fullsized":"2024-09-11T01:25:23.084352+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254905+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:25:23.254905+0000","last_peered":"2024-09-11T01:25:23.254905+0000","last_clean":"2024-09-11T01:25:23.254905+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:25:23.254905+0000","last_undegraded":"2024-09-11T01:25:23.254905+0000","last_fullsized":"2024-09-11T01:25:23.254905+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20033,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.086560+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:25:23.086560+0000","last_peered":"2024-09-11T01:25:23.086560+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:25:23.086560+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083438+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:25:23.083438+0000","last_peered":"2024-09-11T01:25:23.083438+0000","last_clean":"2024-09-11T01:25:23.083438+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:25:23.083438+0000","last_undegraded":"2024-09-11T01:25:23.083438+0000","last_fullsized":"2024-09-11T01:25:23.083438+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28422,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254938+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:25:23.254938+0000","last_peered":"2024-09-11T01:25:23.254938+0000","last_clean":"2024-09-11T01:25:23.254938+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:25:23.254938+0000","last_undegraded":"2024-09-11T01:25:23.254938+0000","last_fullsized":"2024-09-11T01:25:23.254938+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":115,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084173+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:25:23.084173+0000","last_peered":"2024-09-11T01:25:23.084173+0000","last_clean":"2024-09-11T01:25:23.084173+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:25:23.084173+0000","last_undegraded":"2024-09-11T01:25:23.084173+0000","last_fullsized":"2024-09-11T01:25:23.084173+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27813,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255394+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:25:23.255394+0000","last_peered":"2024-09-11T01:25:23.255394+0000","last_clean":"2024-09-11T01:25:23.255394+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:25:23.255394+0000","last_undegraded":"2024-09-11T01:25:23.255394+0000","last_fullsized":"2024-09-11T01:25:23.255394+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20423,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085593+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:25:23.085593+0000","last_peered":"2024-09-11T01:25:23.085593+0000","last_clean":"2024-09-11T01:25:23.085593+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:25:23.085593+0000","last_undegraded":"2024-09-11T01:25:23.085593+0000","last_fullsized":"2024-09-11T01:25:23.085593+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085867+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:25:23.085867+0000","last_peered":"2024-09-11T01:25:23.085867+0000","last_clean":"2024-09-11T01:25:23.085867+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:25:23.085867+0000","last_undegraded":"2024-09-11T01:25:23.085867+0000","last_fullsized":"2024-09-11T01:25:23.085867+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27951,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085470+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:25:23.085470+0000","last_peered":"2024-09-11T01:25:23.085470+0000","last_clean":"2024-09-11T01:25:23.085470+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:25:23.085470+0000","last_undegraded":"2024-09-11T01:25:23.085470+0000","last_fullsized":"2024-09-11T01:25:23.085470+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255181+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:25:23.255181+0000","last_peered":"2024-09-11T01:25:23.255181+0000","last_clean":"2024-09-11T01:25:23.255181+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:25:23.255181+0000","last_undegraded":"2024-09-11T01:25:23.255181+0000","last_fullsized":"2024-09-11T01:25:23.255181+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24630,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085345+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:25:23.085345+0000","last_peered":"2024-09-11T01:25:23.085345+0000","last_clean":"2024-09-11T01:25:23.085345+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:25:23.085345+0000","last_undegraded":"2024-09-11T01:25:23.085345+0000","last_fullsized":"2024-09-11T01:25:23.085345+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084154+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:25:23.084154+0000","last_peered":"2024-09-11T01:25:23.084154+0000","last_clean":"2024-09-11T01:25:23.084154+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:25:23.084154+0000","last_undegraded":"2024-09-11T01:25:23.084154+0000","last_fullsized":"2024-09-11T01:25:23.084154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085458+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:25:23.085458+0000","last_peered":"2024-09-11T01:25:23.085458+0000","last_clean":"2024-09-11T01:25:23.085458+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:25:23.085458+0000","last_undegraded":"2024-09-11T01:25:23.085458+0000","last_fullsized":"2024-09-11T01:25:23.085458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":23303,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:38.281225+0000","last_change":"2024-09-11T01:25:21.628441+0000","last_active":"2024-09-11T01:25:38.281225+0000","last_peered":"2024-09-11T01:25:38.281225+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:25:38.281225+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2330,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1590,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2329}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":10304,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37335,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":159809536,"data_stored":159785183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182569,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518664,"kb_used_data":63608,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853176,"statfs":{"total":96636764160,"available":96105652224,"internally_reserved":0,"allocated":65134592,"data_stored":64790418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986342,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":454688,"kb_used_data":94032,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93917152,"statfs":{"total":96636764160,"available":96171163648,"internally_reserved":0,"allocated":96288768,"data_stored":95948671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659286,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":483352,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":482811,"kb_avail":93888488,"statfs":{"total":96636764160,"available":96141811712,"internally_reserved":0,"allocated":532480,"data_stored":174390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":494399151},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":95166464,"data_stored":95162985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:47.154 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:47.477 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:47.477 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:25:47.495 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":496,"stamp":"2024-09-11T01:25:46.864605+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9912,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37728,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1457864,"kb_used_data":155160,"kb_used_omap":12,"kb_used_meta":1302643,"kb_avail":281657656,"statfs":{"total":289910292480,"available":288417439744,"internally_reserved":0,"allocated":158883840,"data_stored":157841479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1333906436},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-732,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":732,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003636"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28532,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085852+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:25:23.085852+0000","last_peered":"2024-09-11T01:25:23.085852+0000","last_clean":"2024-09-11T01:25:23.085852+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:25:23.085852+0000","last_undegraded":"2024-09-11T01:25:23.085852+0000","last_fullsized":"2024-09-11T01:25:23.085852+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20490,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255153+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:25:23.255153+0000","last_peered":"2024-09-11T01:25:23.255153+0000","last_clean":"2024-09-11T01:25:23.255153+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:25:23.255153+0000","last_undegraded":"2024-09-11T01:25:23.255153+0000","last_fullsized":"2024-09-11T01:25:23.255153+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20313,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085337+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:25:23.085337+0000","last_peered":"2024-09-11T01:25:23.085337+0000","last_clean":"2024-09-11T01:25:23.085337+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:25:23.085337+0000","last_undegraded":"2024-09-11T01:25:23.085337+0000","last_fullsized":"2024-09-11T01:25:23.085337+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20240,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255347+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:25:23.255347+0000","last_peered":"2024-09-11T01:25:23.255347+0000","last_clean":"2024-09-11T01:25:23.255347+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:25:23.255347+0000","last_undegraded":"2024-09-11T01:25:23.255347+0000","last_fullsized":"2024-09-11T01:25:23.255347+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20173,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.086787+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:25:23.086787+0000","last_peered":"2024-09-11T01:25:23.086787+0000","last_clean":"2024-09-11T01:25:23.086787+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:25:23.086787+0000","last_undegraded":"2024-09-11T01:25:23.086787+0000","last_fullsized":"2024-09-11T01:25:23.086787+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20419,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.194847+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:25:23.194847+0000","last_peered":"2024-09-11T01:25:23.194847+0000","last_clean":"2024-09-11T01:25:23.194847+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:25:23.194847+0000","last_undegraded":"2024-09-11T01:25:23.194847+0000","last_fullsized":"2024-09-11T01:25:23.194847+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20096,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.084087+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:25:23.084087+0000","last_peered":"2024-09-11T01:25:23.084087+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:25:23.084087+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20420,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083638+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:25:23.083638+0000","last_peered":"2024-09-11T01:25:23.083638+0000","last_clean":"2024-09-11T01:25:23.083638+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:25:23.083638+0000","last_undegraded":"2024-09-11T01:25:23.083638+0000","last_fullsized":"2024-09-11T01:25:23.083638+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083251+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:25:23.083251+0000","last_peered":"2024-09-11T01:25:23.083251+0000","last_clean":"2024-09-11T01:25:23.083251+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:25:23.083251+0000","last_undegraded":"2024-09-11T01:25:23.083251+0000","last_fullsized":"2024-09-11T01:25:23.083251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28017,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084352+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:25:23.084352+0000","last_peered":"2024-09-11T01:25:23.084352+0000","last_clean":"2024-09-11T01:25:23.084352+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:25:23.084352+0000","last_undegraded":"2024-09-11T01:25:23.084352+0000","last_fullsized":"2024-09-11T01:25:23.084352+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254905+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:25:23.254905+0000","last_peered":"2024-09-11T01:25:23.254905+0000","last_clean":"2024-09-11T01:25:23.254905+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:25:23.254905+0000","last_undegraded":"2024-09-11T01:25:23.254905+0000","last_fullsized":"2024-09-11T01:25:23.254905+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20033,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.086560+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:25:23.086560+0000","last_peered":"2024-09-11T01:25:23.086560+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:25:23.086560+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083438+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:25:23.083438+0000","last_peered":"2024-09-11T01:25:23.083438+0000","last_clean":"2024-09-11T01:25:23.083438+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:25:23.083438+0000","last_undegraded":"2024-09-11T01:25:23.083438+0000","last_fullsized":"2024-09-11T01:25:23.083438+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28422,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254938+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:25:23.254938+0000","last_peered":"2024-09-11T01:25:23.254938+0000","last_clean":"2024-09-11T01:25:23.254938+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:25:23.254938+0000","last_undegraded":"2024-09-11T01:25:23.254938+0000","last_fullsized":"2024-09-11T01:25:23.254938+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":116,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084248+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:25:23.084248+0000","last_peered":"2024-09-11T01:25:23.084248+0000","last_clean":"2024-09-11T01:25:23.084248+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:25:23.084248+0000","last_undegraded":"2024-09-11T01:25:23.084248+0000","last_fullsized":"2024-09-11T01:25:23.084248+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27813,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255394+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:25:23.255394+0000","last_peered":"2024-09-11T01:25:23.255394+0000","last_clean":"2024-09-11T01:25:23.255394+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:25:23.255394+0000","last_undegraded":"2024-09-11T01:25:23.255394+0000","last_fullsized":"2024-09-11T01:25:23.255394+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20423,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085593+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:25:23.085593+0000","last_peered":"2024-09-11T01:25:23.085593+0000","last_clean":"2024-09-11T01:25:23.085593+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:25:23.085593+0000","last_undegraded":"2024-09-11T01:25:23.085593+0000","last_fullsized":"2024-09-11T01:25:23.085593+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085903+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:25:23.085903+0000","last_peered":"2024-09-11T01:25:23.085903+0000","last_clean":"2024-09-11T01:25:23.085903+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:25:23.085903+0000","last_undegraded":"2024-09-11T01:25:23.085903+0000","last_fullsized":"2024-09-11T01:25:23.085903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27951,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085470+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:25:23.085470+0000","last_peered":"2024-09-11T01:25:23.085470+0000","last_clean":"2024-09-11T01:25:23.085470+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:25:23.085470+0000","last_undegraded":"2024-09-11T01:25:23.085470+0000","last_fullsized":"2024-09-11T01:25:23.085470+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255181+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:25:23.255181+0000","last_peered":"2024-09-11T01:25:23.255181+0000","last_clean":"2024-09-11T01:25:23.255181+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:25:23.255181+0000","last_undegraded":"2024-09-11T01:25:23.255181+0000","last_fullsized":"2024-09-11T01:25:23.255181+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24630,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085345+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:25:23.085345+0000","last_peered":"2024-09-11T01:25:23.085345+0000","last_clean":"2024-09-11T01:25:23.085345+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:25:23.085345+0000","last_undegraded":"2024-09-11T01:25:23.085345+0000","last_fullsized":"2024-09-11T01:25:23.085345+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084154+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:25:23.084154+0000","last_peered":"2024-09-11T01:25:23.084154+0000","last_clean":"2024-09-11T01:25:23.084154+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:25:23.084154+0000","last_undegraded":"2024-09-11T01:25:23.084154+0000","last_fullsized":"2024-09-11T01:25:23.084154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085458+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:25:23.085458+0000","last_peered":"2024-09-11T01:25:23.085458+0000","last_clean":"2024-09-11T01:25:23.085458+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:25:23.085458+0000","last_undegraded":"2024-09-11T01:25:23.085458+0000","last_fullsized":"2024-09-11T01:25:23.085458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":24087,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:43.286789+0000","last_change":"2024-09-11T01:25:21.628441+0000","last_active":"2024-09-11T01:25:43.286789+0000","last_peered":"2024-09-11T01:25:43.286789+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:25:43.286789+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1938,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1982,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1937}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9912,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37727,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":156737536,"data_stored":156713183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182570,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518664,"kb_used_data":63608,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853176,"statfs":{"total":96636764160,"available":96105652224,"internally_reserved":0,"allocated":65134592,"data_stored":64790418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986343,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451688,"kb_used_data":91032,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93920152,"statfs":{"total":96636764160,"available":96174235648,"internally_reserved":0,"allocated":93216768,"data_stored":92876671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659287,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487512,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":486971,"kb_avail":93884328,"statfs":{"total":96636764160,"available":96137551872,"internally_reserved":0,"allocated":532480,"data_stored":174390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":498658991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":92094464,"data_stored":92090985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:47.497 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:47.823 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:47.823 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:25:47.841 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":496,"stamp":"2024-09-11T01:25:46.864605+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9912,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37728,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1457864,"kb_used_data":155160,"kb_used_omap":12,"kb_used_meta":1302643,"kb_avail":281657656,"statfs":{"total":289910292480,"available":288417439744,"internally_reserved":0,"allocated":158883840,"data_stored":157841479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1333906436},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-732,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":732,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003636"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28532,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085852+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:25:23.085852+0000","last_peered":"2024-09-11T01:25:23.085852+0000","last_clean":"2024-09-11T01:25:23.085852+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:25:23.085852+0000","last_undegraded":"2024-09-11T01:25:23.085852+0000","last_fullsized":"2024-09-11T01:25:23.085852+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20490,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255153+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:25:23.255153+0000","last_peered":"2024-09-11T01:25:23.255153+0000","last_clean":"2024-09-11T01:25:23.255153+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:25:23.255153+0000","last_undegraded":"2024-09-11T01:25:23.255153+0000","last_fullsized":"2024-09-11T01:25:23.255153+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20313,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085337+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:25:23.085337+0000","last_peered":"2024-09-11T01:25:23.085337+0000","last_clean":"2024-09-11T01:25:23.085337+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:25:23.085337+0000","last_undegraded":"2024-09-11T01:25:23.085337+0000","last_fullsized":"2024-09-11T01:25:23.085337+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20240,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255347+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:25:23.255347+0000","last_peered":"2024-09-11T01:25:23.255347+0000","last_clean":"2024-09-11T01:25:23.255347+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:25:23.255347+0000","last_undegraded":"2024-09-11T01:25:23.255347+0000","last_fullsized":"2024-09-11T01:25:23.255347+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20173,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.086787+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:25:23.086787+0000","last_peered":"2024-09-11T01:25:23.086787+0000","last_clean":"2024-09-11T01:25:23.086787+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:25:23.086787+0000","last_undegraded":"2024-09-11T01:25:23.086787+0000","last_fullsized":"2024-09-11T01:25:23.086787+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20419,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.194847+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:25:23.194847+0000","last_peered":"2024-09-11T01:25:23.194847+0000","last_clean":"2024-09-11T01:25:23.194847+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:25:23.194847+0000","last_undegraded":"2024-09-11T01:25:23.194847+0000","last_fullsized":"2024-09-11T01:25:23.194847+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20096,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.084087+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:25:23.084087+0000","last_peered":"2024-09-11T01:25:23.084087+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:25:23.084087+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20420,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083638+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:25:23.083638+0000","last_peered":"2024-09-11T01:25:23.083638+0000","last_clean":"2024-09-11T01:25:23.083638+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:25:23.083638+0000","last_undegraded":"2024-09-11T01:25:23.083638+0000","last_fullsized":"2024-09-11T01:25:23.083638+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083251+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:25:23.083251+0000","last_peered":"2024-09-11T01:25:23.083251+0000","last_clean":"2024-09-11T01:25:23.083251+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:25:23.083251+0000","last_undegraded":"2024-09-11T01:25:23.083251+0000","last_fullsized":"2024-09-11T01:25:23.083251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28017,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084352+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:25:23.084352+0000","last_peered":"2024-09-11T01:25:23.084352+0000","last_clean":"2024-09-11T01:25:23.084352+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:25:23.084352+0000","last_undegraded":"2024-09-11T01:25:23.084352+0000","last_fullsized":"2024-09-11T01:25:23.084352+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254905+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:25:23.254905+0000","last_peered":"2024-09-11T01:25:23.254905+0000","last_clean":"2024-09-11T01:25:23.254905+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:25:23.254905+0000","last_undegraded":"2024-09-11T01:25:23.254905+0000","last_fullsized":"2024-09-11T01:25:23.254905+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20033,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.086560+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:25:23.086560+0000","last_peered":"2024-09-11T01:25:23.086560+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:25:23.086560+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083438+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:25:23.083438+0000","last_peered":"2024-09-11T01:25:23.083438+0000","last_clean":"2024-09-11T01:25:23.083438+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:25:23.083438+0000","last_undegraded":"2024-09-11T01:25:23.083438+0000","last_fullsized":"2024-09-11T01:25:23.083438+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28422,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254938+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:25:23.254938+0000","last_peered":"2024-09-11T01:25:23.254938+0000","last_clean":"2024-09-11T01:25:23.254938+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:25:23.254938+0000","last_undegraded":"2024-09-11T01:25:23.254938+0000","last_fullsized":"2024-09-11T01:25:23.254938+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":116,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084248+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:25:23.084248+0000","last_peered":"2024-09-11T01:25:23.084248+0000","last_clean":"2024-09-11T01:25:23.084248+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:25:23.084248+0000","last_undegraded":"2024-09-11T01:25:23.084248+0000","last_fullsized":"2024-09-11T01:25:23.084248+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27813,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255394+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:25:23.255394+0000","last_peered":"2024-09-11T01:25:23.255394+0000","last_clean":"2024-09-11T01:25:23.255394+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:25:23.255394+0000","last_undegraded":"2024-09-11T01:25:23.255394+0000","last_fullsized":"2024-09-11T01:25:23.255394+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20423,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085593+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:25:23.085593+0000","last_peered":"2024-09-11T01:25:23.085593+0000","last_clean":"2024-09-11T01:25:23.085593+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:25:23.085593+0000","last_undegraded":"2024-09-11T01:25:23.085593+0000","last_fullsized":"2024-09-11T01:25:23.085593+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085903+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:25:23.085903+0000","last_peered":"2024-09-11T01:25:23.085903+0000","last_clean":"2024-09-11T01:25:23.085903+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:25:23.085903+0000","last_undegraded":"2024-09-11T01:25:23.085903+0000","last_fullsized":"2024-09-11T01:25:23.085903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27951,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085470+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:25:23.085470+0000","last_peered":"2024-09-11T01:25:23.085470+0000","last_clean":"2024-09-11T01:25:23.085470+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:25:23.085470+0000","last_undegraded":"2024-09-11T01:25:23.085470+0000","last_fullsized":"2024-09-11T01:25:23.085470+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255181+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:25:23.255181+0000","last_peered":"2024-09-11T01:25:23.255181+0000","last_clean":"2024-09-11T01:25:23.255181+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:25:23.255181+0000","last_undegraded":"2024-09-11T01:25:23.255181+0000","last_fullsized":"2024-09-11T01:25:23.255181+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24630,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085345+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:25:23.085345+0000","last_peered":"2024-09-11T01:25:23.085345+0000","last_clean":"2024-09-11T01:25:23.085345+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:25:23.085345+0000","last_undegraded":"2024-09-11T01:25:23.085345+0000","last_fullsized":"2024-09-11T01:25:23.085345+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084154+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:25:23.084154+0000","last_peered":"2024-09-11T01:25:23.084154+0000","last_clean":"2024-09-11T01:25:23.084154+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:25:23.084154+0000","last_undegraded":"2024-09-11T01:25:23.084154+0000","last_fullsized":"2024-09-11T01:25:23.084154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085458+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:25:23.085458+0000","last_peered":"2024-09-11T01:25:23.085458+0000","last_clean":"2024-09-11T01:25:23.085458+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:25:23.085458+0000","last_undegraded":"2024-09-11T01:25:23.085458+0000","last_fullsized":"2024-09-11T01:25:23.085458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":24087,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:43.286789+0000","last_change":"2024-09-11T01:25:21.628441+0000","last_active":"2024-09-11T01:25:43.286789+0000","last_peered":"2024-09-11T01:25:43.286789+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:25:43.286789+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1938,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1982,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1937}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9912,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37727,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":156737536,"data_stored":156713183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182570,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518664,"kb_used_data":63608,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853176,"statfs":{"total":96636764160,"available":96105652224,"internally_reserved":0,"allocated":65134592,"data_stored":64790418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986343,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451688,"kb_used_data":91032,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93920152,"statfs":{"total":96636764160,"available":96174235648,"internally_reserved":0,"allocated":93216768,"data_stored":92876671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659287,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487512,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":486971,"kb_avail":93884328,"statfs":{"total":96636764160,"available":96137551872,"internally_reserved":0,"allocated":532480,"data_stored":174390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":498658991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":92094464,"data_stored":92090985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:47.842 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:25:47.842 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20096, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:23.084087+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:25:23.084087+0000', 'last_peered': '2024-09-11T01:25:23.084087+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:25:23.084087+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:47.843 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:25:47.843 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20033, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:23.086560+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:25:23.086560+0000', 'last_peered': '2024-09-11T01:25:23.086560+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:25:23.086560+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:47.843 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:25:47.843 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 24087, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:43.286789+0000', 'last_change': '2024-09-11T01:25:21.628441+0000', 'last_active': '2024-09-11T01:25:43.286789+0000', 'last_peered': '2024-09-11T01:25:43.286789+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:25:43.286789+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1938, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1982, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1937}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:47.843 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:25:48.233 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:48.233 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 9912/6 objects degraded (165200.000%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":884,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1492852736,"bytes_avail":288417439744,"bytes_total":289910292480,"degraded_objects":9912,"degraded_total":6,"degraded_ratio":1652,"recovering_objects_per_sec":60,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":732,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:25:48.252 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:25:48.252 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:48.544 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:48.545 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:25:48.562 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":496,"stamp":"2024-09-11T01:25:46.864605+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9912,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37728,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1457864,"kb_used_data":155160,"kb_used_omap":12,"kb_used_meta":1302643,"kb_avail":281657656,"statfs":{"total":289910292480,"available":288417439744,"internally_reserved":0,"allocated":158883840,"data_stored":157841479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1333906436},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-732,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":732,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003636"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28532,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085852+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:25:23.085852+0000","last_peered":"2024-09-11T01:25:23.085852+0000","last_clean":"2024-09-11T01:25:23.085852+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:25:23.085852+0000","last_undegraded":"2024-09-11T01:25:23.085852+0000","last_fullsized":"2024-09-11T01:25:23.085852+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20490,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255153+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:25:23.255153+0000","last_peered":"2024-09-11T01:25:23.255153+0000","last_clean":"2024-09-11T01:25:23.255153+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:25:23.255153+0000","last_undegraded":"2024-09-11T01:25:23.255153+0000","last_fullsized":"2024-09-11T01:25:23.255153+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20313,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085337+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:25:23.085337+0000","last_peered":"2024-09-11T01:25:23.085337+0000","last_clean":"2024-09-11T01:25:23.085337+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:25:23.085337+0000","last_undegraded":"2024-09-11T01:25:23.085337+0000","last_fullsized":"2024-09-11T01:25:23.085337+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20240,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255347+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:25:23.255347+0000","last_peered":"2024-09-11T01:25:23.255347+0000","last_clean":"2024-09-11T01:25:23.255347+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:25:23.255347+0000","last_undegraded":"2024-09-11T01:25:23.255347+0000","last_fullsized":"2024-09-11T01:25:23.255347+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20173,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.086787+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:25:23.086787+0000","last_peered":"2024-09-11T01:25:23.086787+0000","last_clean":"2024-09-11T01:25:23.086787+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:25:23.086787+0000","last_undegraded":"2024-09-11T01:25:23.086787+0000","last_fullsized":"2024-09-11T01:25:23.086787+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20419,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.194847+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:25:23.194847+0000","last_peered":"2024-09-11T01:25:23.194847+0000","last_clean":"2024-09-11T01:25:23.194847+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:25:23.194847+0000","last_undegraded":"2024-09-11T01:25:23.194847+0000","last_fullsized":"2024-09-11T01:25:23.194847+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20096,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.084087+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:25:23.084087+0000","last_peered":"2024-09-11T01:25:23.084087+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:25:23.084087+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20420,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083638+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:25:23.083638+0000","last_peered":"2024-09-11T01:25:23.083638+0000","last_clean":"2024-09-11T01:25:23.083638+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:25:23.083638+0000","last_undegraded":"2024-09-11T01:25:23.083638+0000","last_fullsized":"2024-09-11T01:25:23.083638+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083251+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:25:23.083251+0000","last_peered":"2024-09-11T01:25:23.083251+0000","last_clean":"2024-09-11T01:25:23.083251+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:25:23.083251+0000","last_undegraded":"2024-09-11T01:25:23.083251+0000","last_fullsized":"2024-09-11T01:25:23.083251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28017,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084352+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:25:23.084352+0000","last_peered":"2024-09-11T01:25:23.084352+0000","last_clean":"2024-09-11T01:25:23.084352+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:25:23.084352+0000","last_undegraded":"2024-09-11T01:25:23.084352+0000","last_fullsized":"2024-09-11T01:25:23.084352+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254905+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:25:23.254905+0000","last_peered":"2024-09-11T01:25:23.254905+0000","last_clean":"2024-09-11T01:25:23.254905+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:25:23.254905+0000","last_undegraded":"2024-09-11T01:25:23.254905+0000","last_fullsized":"2024-09-11T01:25:23.254905+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20033,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.086560+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:25:23.086560+0000","last_peered":"2024-09-11T01:25:23.086560+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:25:23.086560+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083438+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:25:23.083438+0000","last_peered":"2024-09-11T01:25:23.083438+0000","last_clean":"2024-09-11T01:25:23.083438+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:25:23.083438+0000","last_undegraded":"2024-09-11T01:25:23.083438+0000","last_fullsized":"2024-09-11T01:25:23.083438+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28422,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254938+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:25:23.254938+0000","last_peered":"2024-09-11T01:25:23.254938+0000","last_clean":"2024-09-11T01:25:23.254938+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:25:23.254938+0000","last_undegraded":"2024-09-11T01:25:23.254938+0000","last_fullsized":"2024-09-11T01:25:23.254938+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":116,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084248+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:25:23.084248+0000","last_peered":"2024-09-11T01:25:23.084248+0000","last_clean":"2024-09-11T01:25:23.084248+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:25:23.084248+0000","last_undegraded":"2024-09-11T01:25:23.084248+0000","last_fullsized":"2024-09-11T01:25:23.084248+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27813,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255394+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:25:23.255394+0000","last_peered":"2024-09-11T01:25:23.255394+0000","last_clean":"2024-09-11T01:25:23.255394+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:25:23.255394+0000","last_undegraded":"2024-09-11T01:25:23.255394+0000","last_fullsized":"2024-09-11T01:25:23.255394+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20423,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085593+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:25:23.085593+0000","last_peered":"2024-09-11T01:25:23.085593+0000","last_clean":"2024-09-11T01:25:23.085593+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:25:23.085593+0000","last_undegraded":"2024-09-11T01:25:23.085593+0000","last_fullsized":"2024-09-11T01:25:23.085593+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085903+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:25:23.085903+0000","last_peered":"2024-09-11T01:25:23.085903+0000","last_clean":"2024-09-11T01:25:23.085903+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:25:23.085903+0000","last_undegraded":"2024-09-11T01:25:23.085903+0000","last_fullsized":"2024-09-11T01:25:23.085903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27951,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085470+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:25:23.085470+0000","last_peered":"2024-09-11T01:25:23.085470+0000","last_clean":"2024-09-11T01:25:23.085470+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:25:23.085470+0000","last_undegraded":"2024-09-11T01:25:23.085470+0000","last_fullsized":"2024-09-11T01:25:23.085470+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255181+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:25:23.255181+0000","last_peered":"2024-09-11T01:25:23.255181+0000","last_clean":"2024-09-11T01:25:23.255181+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:25:23.255181+0000","last_undegraded":"2024-09-11T01:25:23.255181+0000","last_fullsized":"2024-09-11T01:25:23.255181+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24630,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085345+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:25:23.085345+0000","last_peered":"2024-09-11T01:25:23.085345+0000","last_clean":"2024-09-11T01:25:23.085345+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:25:23.085345+0000","last_undegraded":"2024-09-11T01:25:23.085345+0000","last_fullsized":"2024-09-11T01:25:23.085345+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084154+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:25:23.084154+0000","last_peered":"2024-09-11T01:25:23.084154+0000","last_clean":"2024-09-11T01:25:23.084154+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:25:23.084154+0000","last_undegraded":"2024-09-11T01:25:23.084154+0000","last_fullsized":"2024-09-11T01:25:23.084154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085458+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:25:23.085458+0000","last_peered":"2024-09-11T01:25:23.085458+0000","last_clean":"2024-09-11T01:25:23.085458+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:25:23.085458+0000","last_undegraded":"2024-09-11T01:25:23.085458+0000","last_fullsized":"2024-09-11T01:25:23.085458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":24087,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:43.286789+0000","last_change":"2024-09-11T01:25:21.628441+0000","last_active":"2024-09-11T01:25:43.286789+0000","last_peered":"2024-09-11T01:25:43.286789+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:25:43.286789+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1938,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1982,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1937}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9912,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":37727,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":156737536,"data_stored":156713183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182570,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518664,"kb_used_data":63608,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853176,"statfs":{"total":96636764160,"available":96105652224,"internally_reserved":0,"allocated":65134592,"data_stored":64790418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986343,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":451688,"kb_used_data":91032,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93920152,"statfs":{"total":96636764160,"available":96174235648,"internally_reserved":0,"allocated":93216768,"data_stored":92876671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659287,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":487512,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":486971,"kb_avail":93884328,"statfs":{"total":96636764160,"available":96137551872,"internally_reserved":0,"allocated":532480,"data_stored":174390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":498658991},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":92094464,"data_stored":92090985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:51.565 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:51.883 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:51.884 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:25:51.901 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":498,"stamp":"2024-09-11T01:25:50.865903+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9482,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38158,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1459024,"kb_used_data":152160,"kb_used_omap":12,"kb_used_meta":1306803,"kb_avail":281656496,"statfs":{"total":289910292480,"available":288416251904,"internally_reserved":0,"allocated":155811840,"data_stored":154769479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1338166276},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-822,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":822,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003685"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28532,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085852+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:25:23.085852+0000","last_peered":"2024-09-11T01:25:23.085852+0000","last_clean":"2024-09-11T01:25:23.085852+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:25:23.085852+0000","last_undegraded":"2024-09-11T01:25:23.085852+0000","last_fullsized":"2024-09-11T01:25:23.085852+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20490,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255153+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:25:23.255153+0000","last_peered":"2024-09-11T01:25:23.255153+0000","last_clean":"2024-09-11T01:25:23.255153+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:25:23.255153+0000","last_undegraded":"2024-09-11T01:25:23.255153+0000","last_fullsized":"2024-09-11T01:25:23.255153+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20313,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085337+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:25:23.085337+0000","last_peered":"2024-09-11T01:25:23.085337+0000","last_clean":"2024-09-11T01:25:23.085337+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:25:23.085337+0000","last_undegraded":"2024-09-11T01:25:23.085337+0000","last_fullsized":"2024-09-11T01:25:23.085337+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20240,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255347+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:25:23.255347+0000","last_peered":"2024-09-11T01:25:23.255347+0000","last_clean":"2024-09-11T01:25:23.255347+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:25:23.255347+0000","last_undegraded":"2024-09-11T01:25:23.255347+0000","last_fullsized":"2024-09-11T01:25:23.255347+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20173,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.086787+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:25:23.086787+0000","last_peered":"2024-09-11T01:25:23.086787+0000","last_clean":"2024-09-11T01:25:23.086787+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:25:23.086787+0000","last_undegraded":"2024-09-11T01:25:23.086787+0000","last_fullsized":"2024-09-11T01:25:23.086787+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20419,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.194847+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:25:23.194847+0000","last_peered":"2024-09-11T01:25:23.194847+0000","last_clean":"2024-09-11T01:25:23.194847+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:25:23.194847+0000","last_undegraded":"2024-09-11T01:25:23.194847+0000","last_fullsized":"2024-09-11T01:25:23.194847+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20096,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.084087+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:25:23.084087+0000","last_peered":"2024-09-11T01:25:23.084087+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:25:23.084087+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20420,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083638+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:25:23.083638+0000","last_peered":"2024-09-11T01:25:23.083638+0000","last_clean":"2024-09-11T01:25:23.083638+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:25:23.083638+0000","last_undegraded":"2024-09-11T01:25:23.083638+0000","last_fullsized":"2024-09-11T01:25:23.083638+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083251+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:25:23.083251+0000","last_peered":"2024-09-11T01:25:23.083251+0000","last_clean":"2024-09-11T01:25:23.083251+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:25:23.083251+0000","last_undegraded":"2024-09-11T01:25:23.083251+0000","last_fullsized":"2024-09-11T01:25:23.083251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28017,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084352+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:25:23.084352+0000","last_peered":"2024-09-11T01:25:23.084352+0000","last_clean":"2024-09-11T01:25:23.084352+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:25:23.084352+0000","last_undegraded":"2024-09-11T01:25:23.084352+0000","last_fullsized":"2024-09-11T01:25:23.084352+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254905+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:25:23.254905+0000","last_peered":"2024-09-11T01:25:23.254905+0000","last_clean":"2024-09-11T01:25:23.254905+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:25:23.254905+0000","last_undegraded":"2024-09-11T01:25:23.254905+0000","last_fullsized":"2024-09-11T01:25:23.254905+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20033,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.086560+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:25:23.086560+0000","last_peered":"2024-09-11T01:25:23.086560+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:25:23.086560+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083438+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:25:23.083438+0000","last_peered":"2024-09-11T01:25:23.083438+0000","last_clean":"2024-09-11T01:25:23.083438+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:25:23.083438+0000","last_undegraded":"2024-09-11T01:25:23.083438+0000","last_fullsized":"2024-09-11T01:25:23.083438+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28422,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254938+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:25:23.254938+0000","last_peered":"2024-09-11T01:25:23.254938+0000","last_clean":"2024-09-11T01:25:23.254938+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:25:23.254938+0000","last_undegraded":"2024-09-11T01:25:23.254938+0000","last_fullsized":"2024-09-11T01:25:23.254938+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":116,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084248+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:25:23.084248+0000","last_peered":"2024-09-11T01:25:23.084248+0000","last_clean":"2024-09-11T01:25:23.084248+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:25:23.084248+0000","last_undegraded":"2024-09-11T01:25:23.084248+0000","last_fullsized":"2024-09-11T01:25:23.084248+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27813,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255394+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:25:23.255394+0000","last_peered":"2024-09-11T01:25:23.255394+0000","last_clean":"2024-09-11T01:25:23.255394+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:25:23.255394+0000","last_undegraded":"2024-09-11T01:25:23.255394+0000","last_fullsized":"2024-09-11T01:25:23.255394+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20423,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085593+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:25:23.085593+0000","last_peered":"2024-09-11T01:25:23.085593+0000","last_clean":"2024-09-11T01:25:23.085593+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:25:23.085593+0000","last_undegraded":"2024-09-11T01:25:23.085593+0000","last_fullsized":"2024-09-11T01:25:23.085593+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085903+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:25:23.085903+0000","last_peered":"2024-09-11T01:25:23.085903+0000","last_clean":"2024-09-11T01:25:23.085903+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:25:23.085903+0000","last_undegraded":"2024-09-11T01:25:23.085903+0000","last_fullsized":"2024-09-11T01:25:23.085903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27951,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085470+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:25:23.085470+0000","last_peered":"2024-09-11T01:25:23.085470+0000","last_clean":"2024-09-11T01:25:23.085470+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:25:23.085470+0000","last_undegraded":"2024-09-11T01:25:23.085470+0000","last_fullsized":"2024-09-11T01:25:23.085470+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255181+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:25:23.255181+0000","last_peered":"2024-09-11T01:25:23.255181+0000","last_clean":"2024-09-11T01:25:23.255181+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:25:23.255181+0000","last_undegraded":"2024-09-11T01:25:23.255181+0000","last_fullsized":"2024-09-11T01:25:23.255181+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24630,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085345+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:25:23.085345+0000","last_peered":"2024-09-11T01:25:23.085345+0000","last_clean":"2024-09-11T01:25:23.085345+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:25:23.085345+0000","last_undegraded":"2024-09-11T01:25:23.085345+0000","last_fullsized":"2024-09-11T01:25:23.085345+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084154+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:25:23.084154+0000","last_peered":"2024-09-11T01:25:23.084154+0000","last_clean":"2024-09-11T01:25:23.084154+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:25:23.084154+0000","last_undegraded":"2024-09-11T01:25:23.084154+0000","last_fullsized":"2024-09-11T01:25:23.084154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085458+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:25:23.085458+0000","last_peered":"2024-09-11T01:25:23.085458+0000","last_clean":"2024-09-11T01:25:23.085458+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:25:23.085458+0000","last_undegraded":"2024-09-11T01:25:23.085458+0000","last_fullsized":"2024-09-11T01:25:23.085458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":24947,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:48.291128+0000","last_change":"2024-09-11T01:25:21.628441+0000","last_active":"2024-09-11T01:25:48.291128+0000","last_peered":"2024-09-11T01:25:48.291128+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:25:48.291128+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1508,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2412,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1507}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9482,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38157,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":153665536,"data_stored":153641183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182571,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518664,"kb_used_data":63608,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853176,"statfs":{"total":96636764160,"available":96105652224,"internally_reserved":0,"allocated":65134592,"data_stored":64790418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986344,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":448688,"kb_used_data":88032,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93923152,"statfs":{"total":96636764160,"available":96177307648,"internally_reserved":0,"allocated":90144768,"data_stored":89804671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659288,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491672,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93880168,"statfs":{"total":96636764160,"available":96133292032,"internally_reserved":0,"allocated":532480,"data_stored":174390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502918831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":89022464,"data_stored":89018985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:51.903 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:52.241 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:52.242 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:25:52.258 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":498,"stamp":"2024-09-11T01:25:50.865903+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9482,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38158,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1459024,"kb_used_data":152160,"kb_used_omap":12,"kb_used_meta":1306803,"kb_avail":281656496,"statfs":{"total":289910292480,"available":288416251904,"internally_reserved":0,"allocated":155811840,"data_stored":154769479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1338166276},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-822,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":822,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003685"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28532,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085852+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:25:23.085852+0000","last_peered":"2024-09-11T01:25:23.085852+0000","last_clean":"2024-09-11T01:25:23.085852+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:25:23.085852+0000","last_undegraded":"2024-09-11T01:25:23.085852+0000","last_fullsized":"2024-09-11T01:25:23.085852+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20490,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255153+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:25:23.255153+0000","last_peered":"2024-09-11T01:25:23.255153+0000","last_clean":"2024-09-11T01:25:23.255153+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:25:23.255153+0000","last_undegraded":"2024-09-11T01:25:23.255153+0000","last_fullsized":"2024-09-11T01:25:23.255153+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20313,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085337+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:25:23.085337+0000","last_peered":"2024-09-11T01:25:23.085337+0000","last_clean":"2024-09-11T01:25:23.085337+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:25:23.085337+0000","last_undegraded":"2024-09-11T01:25:23.085337+0000","last_fullsized":"2024-09-11T01:25:23.085337+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20240,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255347+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:25:23.255347+0000","last_peered":"2024-09-11T01:25:23.255347+0000","last_clean":"2024-09-11T01:25:23.255347+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:25:23.255347+0000","last_undegraded":"2024-09-11T01:25:23.255347+0000","last_fullsized":"2024-09-11T01:25:23.255347+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20173,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.086787+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:25:23.086787+0000","last_peered":"2024-09-11T01:25:23.086787+0000","last_clean":"2024-09-11T01:25:23.086787+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:25:23.086787+0000","last_undegraded":"2024-09-11T01:25:23.086787+0000","last_fullsized":"2024-09-11T01:25:23.086787+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20419,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.194847+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:25:23.194847+0000","last_peered":"2024-09-11T01:25:23.194847+0000","last_clean":"2024-09-11T01:25:23.194847+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:25:23.194847+0000","last_undegraded":"2024-09-11T01:25:23.194847+0000","last_fullsized":"2024-09-11T01:25:23.194847+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20096,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.084087+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:25:23.084087+0000","last_peered":"2024-09-11T01:25:23.084087+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:25:23.084087+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20420,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083638+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:25:23.083638+0000","last_peered":"2024-09-11T01:25:23.083638+0000","last_clean":"2024-09-11T01:25:23.083638+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:25:23.083638+0000","last_undegraded":"2024-09-11T01:25:23.083638+0000","last_fullsized":"2024-09-11T01:25:23.083638+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083251+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:25:23.083251+0000","last_peered":"2024-09-11T01:25:23.083251+0000","last_clean":"2024-09-11T01:25:23.083251+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:25:23.083251+0000","last_undegraded":"2024-09-11T01:25:23.083251+0000","last_fullsized":"2024-09-11T01:25:23.083251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28017,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084352+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:25:23.084352+0000","last_peered":"2024-09-11T01:25:23.084352+0000","last_clean":"2024-09-11T01:25:23.084352+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:25:23.084352+0000","last_undegraded":"2024-09-11T01:25:23.084352+0000","last_fullsized":"2024-09-11T01:25:23.084352+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254905+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:25:23.254905+0000","last_peered":"2024-09-11T01:25:23.254905+0000","last_clean":"2024-09-11T01:25:23.254905+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:25:23.254905+0000","last_undegraded":"2024-09-11T01:25:23.254905+0000","last_fullsized":"2024-09-11T01:25:23.254905+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20033,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.086560+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:25:23.086560+0000","last_peered":"2024-09-11T01:25:23.086560+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:25:23.086560+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083438+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:25:23.083438+0000","last_peered":"2024-09-11T01:25:23.083438+0000","last_clean":"2024-09-11T01:25:23.083438+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:25:23.083438+0000","last_undegraded":"2024-09-11T01:25:23.083438+0000","last_fullsized":"2024-09-11T01:25:23.083438+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28422,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254938+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:25:23.254938+0000","last_peered":"2024-09-11T01:25:23.254938+0000","last_clean":"2024-09-11T01:25:23.254938+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:25:23.254938+0000","last_undegraded":"2024-09-11T01:25:23.254938+0000","last_fullsized":"2024-09-11T01:25:23.254938+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":116,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084248+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:25:23.084248+0000","last_peered":"2024-09-11T01:25:23.084248+0000","last_clean":"2024-09-11T01:25:23.084248+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:25:23.084248+0000","last_undegraded":"2024-09-11T01:25:23.084248+0000","last_fullsized":"2024-09-11T01:25:23.084248+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27813,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255394+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:25:23.255394+0000","last_peered":"2024-09-11T01:25:23.255394+0000","last_clean":"2024-09-11T01:25:23.255394+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:25:23.255394+0000","last_undegraded":"2024-09-11T01:25:23.255394+0000","last_fullsized":"2024-09-11T01:25:23.255394+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20423,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085593+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:25:23.085593+0000","last_peered":"2024-09-11T01:25:23.085593+0000","last_clean":"2024-09-11T01:25:23.085593+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:25:23.085593+0000","last_undegraded":"2024-09-11T01:25:23.085593+0000","last_fullsized":"2024-09-11T01:25:23.085593+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085903+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:25:23.085903+0000","last_peered":"2024-09-11T01:25:23.085903+0000","last_clean":"2024-09-11T01:25:23.085903+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:25:23.085903+0000","last_undegraded":"2024-09-11T01:25:23.085903+0000","last_fullsized":"2024-09-11T01:25:23.085903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27951,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085470+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:25:23.085470+0000","last_peered":"2024-09-11T01:25:23.085470+0000","last_clean":"2024-09-11T01:25:23.085470+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:25:23.085470+0000","last_undegraded":"2024-09-11T01:25:23.085470+0000","last_fullsized":"2024-09-11T01:25:23.085470+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255181+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:25:23.255181+0000","last_peered":"2024-09-11T01:25:23.255181+0000","last_clean":"2024-09-11T01:25:23.255181+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:25:23.255181+0000","last_undegraded":"2024-09-11T01:25:23.255181+0000","last_fullsized":"2024-09-11T01:25:23.255181+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24630,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085345+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:25:23.085345+0000","last_peered":"2024-09-11T01:25:23.085345+0000","last_clean":"2024-09-11T01:25:23.085345+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:25:23.085345+0000","last_undegraded":"2024-09-11T01:25:23.085345+0000","last_fullsized":"2024-09-11T01:25:23.085345+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084154+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:25:23.084154+0000","last_peered":"2024-09-11T01:25:23.084154+0000","last_clean":"2024-09-11T01:25:23.084154+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:25:23.084154+0000","last_undegraded":"2024-09-11T01:25:23.084154+0000","last_fullsized":"2024-09-11T01:25:23.084154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085458+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:25:23.085458+0000","last_peered":"2024-09-11T01:25:23.085458+0000","last_clean":"2024-09-11T01:25:23.085458+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:25:23.085458+0000","last_undegraded":"2024-09-11T01:25:23.085458+0000","last_fullsized":"2024-09-11T01:25:23.085458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":24947,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:48.291128+0000","last_change":"2024-09-11T01:25:21.628441+0000","last_active":"2024-09-11T01:25:48.291128+0000","last_peered":"2024-09-11T01:25:48.291128+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:25:48.291128+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1508,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2412,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1507}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9482,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38157,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":153665536,"data_stored":153641183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182571,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518664,"kb_used_data":63608,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853176,"statfs":{"total":96636764160,"available":96105652224,"internally_reserved":0,"allocated":65134592,"data_stored":64790418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986344,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":448688,"kb_used_data":88032,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93923152,"statfs":{"total":96636764160,"available":96177307648,"internally_reserved":0,"allocated":90144768,"data_stored":89804671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659288,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491672,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93880168,"statfs":{"total":96636764160,"available":96133292032,"internally_reserved":0,"allocated":532480,"data_stored":174390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502918831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":89022464,"data_stored":89018985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:52.260 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:25:52.260 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20096, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:23.084087+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:25:23.084087+0000', 'last_peered': '2024-09-11T01:25:23.084087+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:25:23.084087+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:52.260 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:25:52.260 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20033, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:23.086560+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:25:23.086560+0000', 'last_peered': '2024-09-11T01:25:23.086560+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:25:23.086560+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:52.261 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:25:52.261 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 24947, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:48.291128+0000', 'last_change': '2024-09-11T01:25:21.628441+0000', 'last_active': '2024-09-11T01:25:48.291128+0000', 'last_peered': '2024-09-11T01:25:48.291128+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:25:48.291128+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1508, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2412, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1507}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:52.261 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:25:52.640 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:52.640 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 9482/6 objects degraded (158033.333%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":889,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1494040576,"bytes_avail":288416251904,"bytes_total":289910292480,"degraded_objects":9482,"degraded_total":6,"degraded_ratio":1580.333333333333,"recovering_objects_per_sec":68,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":822,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:25:52.657 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:25:52.657 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:52.967 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:52.968 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:25:52.985 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":499,"stamp":"2024-09-11T01:25:52.866470+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9482,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38158,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1459024,"kb_used_data":152160,"kb_used_omap":12,"kb_used_meta":1306803,"kb_avail":281656496,"statfs":{"total":289910292480,"available":288416251904,"internally_reserved":0,"allocated":155811840,"data_stored":154769479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1338166276},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-822,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":822,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003836"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28532,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085852+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:25:23.085852+0000","last_peered":"2024-09-11T01:25:23.085852+0000","last_clean":"2024-09-11T01:25:23.085852+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:25:23.085852+0000","last_undegraded":"2024-09-11T01:25:23.085852+0000","last_fullsized":"2024-09-11T01:25:23.085852+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20490,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255153+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:25:23.255153+0000","last_peered":"2024-09-11T01:25:23.255153+0000","last_clean":"2024-09-11T01:25:23.255153+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:25:23.255153+0000","last_undegraded":"2024-09-11T01:25:23.255153+0000","last_fullsized":"2024-09-11T01:25:23.255153+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20313,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085337+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:25:23.085337+0000","last_peered":"2024-09-11T01:25:23.085337+0000","last_clean":"2024-09-11T01:25:23.085337+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:25:23.085337+0000","last_undegraded":"2024-09-11T01:25:23.085337+0000","last_fullsized":"2024-09-11T01:25:23.085337+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20240,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255347+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:25:23.255347+0000","last_peered":"2024-09-11T01:25:23.255347+0000","last_clean":"2024-09-11T01:25:23.255347+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:25:23.255347+0000","last_undegraded":"2024-09-11T01:25:23.255347+0000","last_fullsized":"2024-09-11T01:25:23.255347+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20173,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.086787+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:25:23.086787+0000","last_peered":"2024-09-11T01:25:23.086787+0000","last_clean":"2024-09-11T01:25:23.086787+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:25:23.086787+0000","last_undegraded":"2024-09-11T01:25:23.086787+0000","last_fullsized":"2024-09-11T01:25:23.086787+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20419,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.194847+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:25:23.194847+0000","last_peered":"2024-09-11T01:25:23.194847+0000","last_clean":"2024-09-11T01:25:23.194847+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:25:23.194847+0000","last_undegraded":"2024-09-11T01:25:23.194847+0000","last_fullsized":"2024-09-11T01:25:23.194847+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20096,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.084087+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:25:23.084087+0000","last_peered":"2024-09-11T01:25:23.084087+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:25:23.084087+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20420,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083638+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:25:23.083638+0000","last_peered":"2024-09-11T01:25:23.083638+0000","last_clean":"2024-09-11T01:25:23.083638+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:25:23.083638+0000","last_undegraded":"2024-09-11T01:25:23.083638+0000","last_fullsized":"2024-09-11T01:25:23.083638+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083251+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:25:23.083251+0000","last_peered":"2024-09-11T01:25:23.083251+0000","last_clean":"2024-09-11T01:25:23.083251+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:25:23.083251+0000","last_undegraded":"2024-09-11T01:25:23.083251+0000","last_fullsized":"2024-09-11T01:25:23.083251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28017,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084352+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:25:23.084352+0000","last_peered":"2024-09-11T01:25:23.084352+0000","last_clean":"2024-09-11T01:25:23.084352+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:25:23.084352+0000","last_undegraded":"2024-09-11T01:25:23.084352+0000","last_fullsized":"2024-09-11T01:25:23.084352+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254905+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:25:23.254905+0000","last_peered":"2024-09-11T01:25:23.254905+0000","last_clean":"2024-09-11T01:25:23.254905+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:25:23.254905+0000","last_undegraded":"2024-09-11T01:25:23.254905+0000","last_fullsized":"2024-09-11T01:25:23.254905+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20033,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.086560+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:25:23.086560+0000","last_peered":"2024-09-11T01:25:23.086560+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:25:23.086560+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083438+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:25:23.083438+0000","last_peered":"2024-09-11T01:25:23.083438+0000","last_clean":"2024-09-11T01:25:23.083438+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:25:23.083438+0000","last_undegraded":"2024-09-11T01:25:23.083438+0000","last_fullsized":"2024-09-11T01:25:23.083438+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28422,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254938+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:25:23.254938+0000","last_peered":"2024-09-11T01:25:23.254938+0000","last_clean":"2024-09-11T01:25:23.254938+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:25:23.254938+0000","last_undegraded":"2024-09-11T01:25:23.254938+0000","last_fullsized":"2024-09-11T01:25:23.254938+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":116,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084248+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:25:23.084248+0000","last_peered":"2024-09-11T01:25:23.084248+0000","last_clean":"2024-09-11T01:25:23.084248+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:25:23.084248+0000","last_undegraded":"2024-09-11T01:25:23.084248+0000","last_fullsized":"2024-09-11T01:25:23.084248+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27813,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255394+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:25:23.255394+0000","last_peered":"2024-09-11T01:25:23.255394+0000","last_clean":"2024-09-11T01:25:23.255394+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:25:23.255394+0000","last_undegraded":"2024-09-11T01:25:23.255394+0000","last_fullsized":"2024-09-11T01:25:23.255394+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20423,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085593+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:25:23.085593+0000","last_peered":"2024-09-11T01:25:23.085593+0000","last_clean":"2024-09-11T01:25:23.085593+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:25:23.085593+0000","last_undegraded":"2024-09-11T01:25:23.085593+0000","last_fullsized":"2024-09-11T01:25:23.085593+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085903+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:25:23.085903+0000","last_peered":"2024-09-11T01:25:23.085903+0000","last_clean":"2024-09-11T01:25:23.085903+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:25:23.085903+0000","last_undegraded":"2024-09-11T01:25:23.085903+0000","last_fullsized":"2024-09-11T01:25:23.085903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27951,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085470+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:25:23.085470+0000","last_peered":"2024-09-11T01:25:23.085470+0000","last_clean":"2024-09-11T01:25:23.085470+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:25:23.085470+0000","last_undegraded":"2024-09-11T01:25:23.085470+0000","last_fullsized":"2024-09-11T01:25:23.085470+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255181+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:25:23.255181+0000","last_peered":"2024-09-11T01:25:23.255181+0000","last_clean":"2024-09-11T01:25:23.255181+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:25:23.255181+0000","last_undegraded":"2024-09-11T01:25:23.255181+0000","last_fullsized":"2024-09-11T01:25:23.255181+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24630,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085345+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:25:23.085345+0000","last_peered":"2024-09-11T01:25:23.085345+0000","last_clean":"2024-09-11T01:25:23.085345+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:25:23.085345+0000","last_undegraded":"2024-09-11T01:25:23.085345+0000","last_fullsized":"2024-09-11T01:25:23.085345+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":105,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084154+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:25:23.084154+0000","last_peered":"2024-09-11T01:25:23.084154+0000","last_clean":"2024-09-11T01:25:23.084154+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:25:23.084154+0000","last_undegraded":"2024-09-11T01:25:23.084154+0000","last_fullsized":"2024-09-11T01:25:23.084154+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085458+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:25:23.085458+0000","last_peered":"2024-09-11T01:25:23.085458+0000","last_clean":"2024-09-11T01:25:23.085458+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:25:23.085458+0000","last_undegraded":"2024-09-11T01:25:23.085458+0000","last_fullsized":"2024-09-11T01:25:23.085458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":24947,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:48.291128+0000","last_change":"2024-09-11T01:25:21.628441+0000","last_active":"2024-09-11T01:25:48.291128+0000","last_peered":"2024-09-11T01:25:48.291128+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:25:48.291128+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1508,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2412,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1507}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":9482,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38157,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":153665536,"data_stored":153641183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182571,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518664,"kb_used_data":63608,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853176,"statfs":{"total":96636764160,"available":96105652224,"internally_reserved":0,"allocated":65134592,"data_stored":64790418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986344,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":448688,"kb_used_data":88032,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93923152,"statfs":{"total":96636764160,"available":96177307648,"internally_reserved":0,"allocated":90144768,"data_stored":89804671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659288,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":491672,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":491131,"kb_avail":93880168,"statfs":{"total":96636764160,"available":96133292032,"internally_reserved":0,"allocated":532480,"data_stored":174390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":502918831},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":89022464,"data_stored":89018985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:55.987 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:56.305 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:56.305 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:25:56.322 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":500,"stamp":"2024-09-11T01:25:54.866896+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8983,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38657,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1481232,"kb_used_data":152160,"kb_used_omap":12,"kb_used_meta":1329011,"kb_avail":281634288,"statfs":{"total":289910292480,"available":288393510912,"internally_reserved":0,"allocated":155811840,"data_stored":154769479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1360907268},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1321,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1321,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003640"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28532,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085852+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:25:23.085852+0000","last_peered":"2024-09-11T01:25:23.085852+0000","last_clean":"2024-09-11T01:25:23.085852+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:25:23.085852+0000","last_undegraded":"2024-09-11T01:25:23.085852+0000","last_fullsized":"2024-09-11T01:25:23.085852+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20490,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255153+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:25:23.255153+0000","last_peered":"2024-09-11T01:25:23.255153+0000","last_clean":"2024-09-11T01:25:23.255153+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:25:23.255153+0000","last_undegraded":"2024-09-11T01:25:23.255153+0000","last_fullsized":"2024-09-11T01:25:23.255153+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20313,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085337+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:25:23.085337+0000","last_peered":"2024-09-11T01:25:23.085337+0000","last_clean":"2024-09-11T01:25:23.085337+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:25:23.085337+0000","last_undegraded":"2024-09-11T01:25:23.085337+0000","last_fullsized":"2024-09-11T01:25:23.085337+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20240,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255347+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:25:23.255347+0000","last_peered":"2024-09-11T01:25:23.255347+0000","last_clean":"2024-09-11T01:25:23.255347+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:25:23.255347+0000","last_undegraded":"2024-09-11T01:25:23.255347+0000","last_fullsized":"2024-09-11T01:25:23.255347+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20173,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.086787+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:25:23.086787+0000","last_peered":"2024-09-11T01:25:23.086787+0000","last_clean":"2024-09-11T01:25:23.086787+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:25:23.086787+0000","last_undegraded":"2024-09-11T01:25:23.086787+0000","last_fullsized":"2024-09-11T01:25:23.086787+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20419,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.194847+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:25:23.194847+0000","last_peered":"2024-09-11T01:25:23.194847+0000","last_clean":"2024-09-11T01:25:23.194847+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:25:23.194847+0000","last_undegraded":"2024-09-11T01:25:23.194847+0000","last_fullsized":"2024-09-11T01:25:23.194847+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20096,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.084087+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:25:23.084087+0000","last_peered":"2024-09-11T01:25:23.084087+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:25:23.084087+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20420,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083638+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:25:23.083638+0000","last_peered":"2024-09-11T01:25:23.083638+0000","last_clean":"2024-09-11T01:25:23.083638+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:25:23.083638+0000","last_undegraded":"2024-09-11T01:25:23.083638+0000","last_fullsized":"2024-09-11T01:25:23.083638+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083251+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:25:23.083251+0000","last_peered":"2024-09-11T01:25:23.083251+0000","last_clean":"2024-09-11T01:25:23.083251+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:25:23.083251+0000","last_undegraded":"2024-09-11T01:25:23.083251+0000","last_fullsized":"2024-09-11T01:25:23.083251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28017,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084352+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:25:23.084352+0000","last_peered":"2024-09-11T01:25:23.084352+0000","last_clean":"2024-09-11T01:25:23.084352+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:25:23.084352+0000","last_undegraded":"2024-09-11T01:25:23.084352+0000","last_fullsized":"2024-09-11T01:25:23.084352+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254905+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:25:23.254905+0000","last_peered":"2024-09-11T01:25:23.254905+0000","last_clean":"2024-09-11T01:25:23.254905+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:25:23.254905+0000","last_undegraded":"2024-09-11T01:25:23.254905+0000","last_fullsized":"2024-09-11T01:25:23.254905+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20033,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.086560+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:25:23.086560+0000","last_peered":"2024-09-11T01:25:23.086560+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:25:23.086560+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083438+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:25:23.083438+0000","last_peered":"2024-09-11T01:25:23.083438+0000","last_clean":"2024-09-11T01:25:23.083438+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:25:23.083438+0000","last_undegraded":"2024-09-11T01:25:23.083438+0000","last_fullsized":"2024-09-11T01:25:23.083438+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28422,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254938+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:25:23.254938+0000","last_peered":"2024-09-11T01:25:23.254938+0000","last_clean":"2024-09-11T01:25:23.254938+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:25:23.254938+0000","last_undegraded":"2024-09-11T01:25:23.254938+0000","last_fullsized":"2024-09-11T01:25:23.254938+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":116,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084248+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:25:23.084248+0000","last_peered":"2024-09-11T01:25:23.084248+0000","last_clean":"2024-09-11T01:25:23.084248+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:25:23.084248+0000","last_undegraded":"2024-09-11T01:25:23.084248+0000","last_fullsized":"2024-09-11T01:25:23.084248+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27813,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255394+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:25:23.255394+0000","last_peered":"2024-09-11T01:25:23.255394+0000","last_clean":"2024-09-11T01:25:23.255394+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:25:23.255394+0000","last_undegraded":"2024-09-11T01:25:23.255394+0000","last_fullsized":"2024-09-11T01:25:23.255394+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20423,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085593+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:25:23.085593+0000","last_peered":"2024-09-11T01:25:23.085593+0000","last_clean":"2024-09-11T01:25:23.085593+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:25:23.085593+0000","last_undegraded":"2024-09-11T01:25:23.085593+0000","last_fullsized":"2024-09-11T01:25:23.085593+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085903+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:25:23.085903+0000","last_peered":"2024-09-11T01:25:23.085903+0000","last_clean":"2024-09-11T01:25:23.085903+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:25:23.085903+0000","last_undegraded":"2024-09-11T01:25:23.085903+0000","last_fullsized":"2024-09-11T01:25:23.085903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27951,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085470+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:25:23.085470+0000","last_peered":"2024-09-11T01:25:23.085470+0000","last_clean":"2024-09-11T01:25:23.085470+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:25:23.085470+0000","last_undegraded":"2024-09-11T01:25:23.085470+0000","last_fullsized":"2024-09-11T01:25:23.085470+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255181+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:25:23.255181+0000","last_peered":"2024-09-11T01:25:23.255181+0000","last_clean":"2024-09-11T01:25:23.255181+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:25:23.255181+0000","last_undegraded":"2024-09-11T01:25:23.255181+0000","last_fullsized":"2024-09-11T01:25:23.255181+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24630,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085345+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:25:23.085345+0000","last_peered":"2024-09-11T01:25:23.085345+0000","last_clean":"2024-09-11T01:25:23.085345+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:25:23.085345+0000","last_undegraded":"2024-09-11T01:25:23.085345+0000","last_fullsized":"2024-09-11T01:25:23.085345+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084199+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:25:23.084199+0000","last_peered":"2024-09-11T01:25:23.084199+0000","last_clean":"2024-09-11T01:25:23.084199+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:25:23.084199+0000","last_undegraded":"2024-09-11T01:25:23.084199+0000","last_fullsized":"2024-09-11T01:25:23.084199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085458+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:25:23.085458+0000","last_peered":"2024-09-11T01:25:23.085458+0000","last_clean":"2024-09-11T01:25:23.085458+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:25:23.085458+0000","last_undegraded":"2024-09-11T01:25:23.085458+0000","last_fullsized":"2024-09-11T01:25:23.085458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":25945,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:53.292484+0000","last_change":"2024-09-11T01:25:21.628441+0000","last_active":"2024-09-11T01:25:53.292484+0000","last_peered":"2024-09-11T01:25:53.292484+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:25:53.292484+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1009,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2911,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1008}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8983,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38656,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":153665536,"data_stored":153641183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182572,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518664,"kb_used_data":63608,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853176,"statfs":{"total":96636764160,"available":96105652224,"internally_reserved":0,"allocated":65134592,"data_stored":64790418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986344,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":448688,"kb_used_data":88032,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93923152,"statfs":{"total":96636764160,"available":96177307648,"internally_reserved":0,"allocated":90144768,"data_stored":89804671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659289,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":513880,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":513339,"kb_avail":93857960,"statfs":{"total":96636764160,"available":96110551040,"internally_reserved":0,"allocated":532480,"data_stored":174390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":525659823},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":89022464,"data_stored":89018985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:56.324 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:56.650 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:56.650 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:25:56.668 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":500,"stamp":"2024-09-11T01:25:54.866896+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8983,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38657,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1481232,"kb_used_data":152160,"kb_used_omap":12,"kb_used_meta":1329011,"kb_avail":281634288,"statfs":{"total":289910292480,"available":288393510912,"internally_reserved":0,"allocated":155811840,"data_stored":154769479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1360907268},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1321,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1321,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003640"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28532,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085852+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:25:23.085852+0000","last_peered":"2024-09-11T01:25:23.085852+0000","last_clean":"2024-09-11T01:25:23.085852+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:25:23.085852+0000","last_undegraded":"2024-09-11T01:25:23.085852+0000","last_fullsized":"2024-09-11T01:25:23.085852+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20490,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255153+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:25:23.255153+0000","last_peered":"2024-09-11T01:25:23.255153+0000","last_clean":"2024-09-11T01:25:23.255153+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:25:23.255153+0000","last_undegraded":"2024-09-11T01:25:23.255153+0000","last_fullsized":"2024-09-11T01:25:23.255153+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20313,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085337+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:25:23.085337+0000","last_peered":"2024-09-11T01:25:23.085337+0000","last_clean":"2024-09-11T01:25:23.085337+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:25:23.085337+0000","last_undegraded":"2024-09-11T01:25:23.085337+0000","last_fullsized":"2024-09-11T01:25:23.085337+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20240,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255347+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:25:23.255347+0000","last_peered":"2024-09-11T01:25:23.255347+0000","last_clean":"2024-09-11T01:25:23.255347+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:25:23.255347+0000","last_undegraded":"2024-09-11T01:25:23.255347+0000","last_fullsized":"2024-09-11T01:25:23.255347+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20173,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.086787+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:25:23.086787+0000","last_peered":"2024-09-11T01:25:23.086787+0000","last_clean":"2024-09-11T01:25:23.086787+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:25:23.086787+0000","last_undegraded":"2024-09-11T01:25:23.086787+0000","last_fullsized":"2024-09-11T01:25:23.086787+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20419,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.194847+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:25:23.194847+0000","last_peered":"2024-09-11T01:25:23.194847+0000","last_clean":"2024-09-11T01:25:23.194847+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:25:23.194847+0000","last_undegraded":"2024-09-11T01:25:23.194847+0000","last_fullsized":"2024-09-11T01:25:23.194847+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20096,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.084087+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:25:23.084087+0000","last_peered":"2024-09-11T01:25:23.084087+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:25:23.084087+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20420,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083638+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:25:23.083638+0000","last_peered":"2024-09-11T01:25:23.083638+0000","last_clean":"2024-09-11T01:25:23.083638+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:25:23.083638+0000","last_undegraded":"2024-09-11T01:25:23.083638+0000","last_fullsized":"2024-09-11T01:25:23.083638+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083251+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:25:23.083251+0000","last_peered":"2024-09-11T01:25:23.083251+0000","last_clean":"2024-09-11T01:25:23.083251+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:25:23.083251+0000","last_undegraded":"2024-09-11T01:25:23.083251+0000","last_fullsized":"2024-09-11T01:25:23.083251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28017,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084352+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:25:23.084352+0000","last_peered":"2024-09-11T01:25:23.084352+0000","last_clean":"2024-09-11T01:25:23.084352+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:25:23.084352+0000","last_undegraded":"2024-09-11T01:25:23.084352+0000","last_fullsized":"2024-09-11T01:25:23.084352+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254905+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:25:23.254905+0000","last_peered":"2024-09-11T01:25:23.254905+0000","last_clean":"2024-09-11T01:25:23.254905+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:25:23.254905+0000","last_undegraded":"2024-09-11T01:25:23.254905+0000","last_fullsized":"2024-09-11T01:25:23.254905+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20033,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.086560+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:25:23.086560+0000","last_peered":"2024-09-11T01:25:23.086560+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:25:23.086560+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083438+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:25:23.083438+0000","last_peered":"2024-09-11T01:25:23.083438+0000","last_clean":"2024-09-11T01:25:23.083438+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:25:23.083438+0000","last_undegraded":"2024-09-11T01:25:23.083438+0000","last_fullsized":"2024-09-11T01:25:23.083438+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28422,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254938+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:25:23.254938+0000","last_peered":"2024-09-11T01:25:23.254938+0000","last_clean":"2024-09-11T01:25:23.254938+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:25:23.254938+0000","last_undegraded":"2024-09-11T01:25:23.254938+0000","last_fullsized":"2024-09-11T01:25:23.254938+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":116,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084248+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:25:23.084248+0000","last_peered":"2024-09-11T01:25:23.084248+0000","last_clean":"2024-09-11T01:25:23.084248+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:25:23.084248+0000","last_undegraded":"2024-09-11T01:25:23.084248+0000","last_fullsized":"2024-09-11T01:25:23.084248+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27813,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255394+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:25:23.255394+0000","last_peered":"2024-09-11T01:25:23.255394+0000","last_clean":"2024-09-11T01:25:23.255394+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:25:23.255394+0000","last_undegraded":"2024-09-11T01:25:23.255394+0000","last_fullsized":"2024-09-11T01:25:23.255394+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20423,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085593+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:25:23.085593+0000","last_peered":"2024-09-11T01:25:23.085593+0000","last_clean":"2024-09-11T01:25:23.085593+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:25:23.085593+0000","last_undegraded":"2024-09-11T01:25:23.085593+0000","last_fullsized":"2024-09-11T01:25:23.085593+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085903+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:25:23.085903+0000","last_peered":"2024-09-11T01:25:23.085903+0000","last_clean":"2024-09-11T01:25:23.085903+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:25:23.085903+0000","last_undegraded":"2024-09-11T01:25:23.085903+0000","last_fullsized":"2024-09-11T01:25:23.085903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27951,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085470+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:25:23.085470+0000","last_peered":"2024-09-11T01:25:23.085470+0000","last_clean":"2024-09-11T01:25:23.085470+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:25:23.085470+0000","last_undegraded":"2024-09-11T01:25:23.085470+0000","last_fullsized":"2024-09-11T01:25:23.085470+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255181+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:25:23.255181+0000","last_peered":"2024-09-11T01:25:23.255181+0000","last_clean":"2024-09-11T01:25:23.255181+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:25:23.255181+0000","last_undegraded":"2024-09-11T01:25:23.255181+0000","last_fullsized":"2024-09-11T01:25:23.255181+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24630,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085345+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:25:23.085345+0000","last_peered":"2024-09-11T01:25:23.085345+0000","last_clean":"2024-09-11T01:25:23.085345+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:25:23.085345+0000","last_undegraded":"2024-09-11T01:25:23.085345+0000","last_fullsized":"2024-09-11T01:25:23.085345+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084199+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:25:23.084199+0000","last_peered":"2024-09-11T01:25:23.084199+0000","last_clean":"2024-09-11T01:25:23.084199+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:25:23.084199+0000","last_undegraded":"2024-09-11T01:25:23.084199+0000","last_fullsized":"2024-09-11T01:25:23.084199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085458+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:25:23.085458+0000","last_peered":"2024-09-11T01:25:23.085458+0000","last_clean":"2024-09-11T01:25:23.085458+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:25:23.085458+0000","last_undegraded":"2024-09-11T01:25:23.085458+0000","last_fullsized":"2024-09-11T01:25:23.085458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":25945,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:53.292484+0000","last_change":"2024-09-11T01:25:21.628441+0000","last_active":"2024-09-11T01:25:53.292484+0000","last_peered":"2024-09-11T01:25:53.292484+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:25:53.292484+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1009,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2911,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1008}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8983,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38656,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":153665536,"data_stored":153641183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182572,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518664,"kb_used_data":63608,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853176,"statfs":{"total":96636764160,"available":96105652224,"internally_reserved":0,"allocated":65134592,"data_stored":64790418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986344,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":448688,"kb_used_data":88032,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93923152,"statfs":{"total":96636764160,"available":96177307648,"internally_reserved":0,"allocated":90144768,"data_stored":89804671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659289,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":513880,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":513339,"kb_avail":93857960,"statfs":{"total":96636764160,"available":96110551040,"internally_reserved":0,"allocated":532480,"data_stored":174390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":525659823},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":89022464,"data_stored":89018985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:25:56.670 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:25:56.670 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20096, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:23.084087+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:25:23.084087+0000', 'last_peered': '2024-09-11T01:25:23.084087+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:25:23.084087+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:56.670 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:25:56.670 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20033, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:23.086560+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:25:23.086560+0000', 'last_peered': '2024-09-11T01:25:23.086560+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:25:23.086560+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:56.670 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:25:56.670 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 25945, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:53.292484+0000', 'last_change': '2024-09-11T01:25:21.628441+0000', 'last_active': '2024-09-11T01:25:53.292484+0000', 'last_peered': '2024-09-11T01:25:53.292484+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:25:53.292484+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1009, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2911, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1008}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:25:56.671 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:25:57.060 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:57.061 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 8983/6 objects degraded (149716.667%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":893,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1513709568,"bytes_avail":288396582912,"bytes_total":289910292480,"degraded_objects":8983,"degraded_total":6,"degraded_ratio":1497.166666666667,"recovering_objects_per_sec":77,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":929,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:25:57.081 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:25:57.081 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:25:57.394 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:25:57.394 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:25:57.412 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":501,"stamp":"2024-09-11T01:25:56.867663+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8983,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38657,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1478232,"kb_used_data":149160,"kb_used_omap":12,"kb_used_meta":1329011,"kb_avail":281637288,"statfs":{"total":289910292480,"available":288396582912,"internally_reserved":0,"allocated":152739840,"data_stored":151697479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1360907268},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-929,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":929,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003858"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28532,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085852+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:25:23.085852+0000","last_peered":"2024-09-11T01:25:23.085852+0000","last_clean":"2024-09-11T01:25:23.085852+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:25:23.085852+0000","last_undegraded":"2024-09-11T01:25:23.085852+0000","last_fullsized":"2024-09-11T01:25:23.085852+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20490,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255153+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:25:23.255153+0000","last_peered":"2024-09-11T01:25:23.255153+0000","last_clean":"2024-09-11T01:25:23.255153+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:25:23.255153+0000","last_undegraded":"2024-09-11T01:25:23.255153+0000","last_fullsized":"2024-09-11T01:25:23.255153+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20313,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085337+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:25:23.085337+0000","last_peered":"2024-09-11T01:25:23.085337+0000","last_clean":"2024-09-11T01:25:23.085337+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:25:23.085337+0000","last_undegraded":"2024-09-11T01:25:23.085337+0000","last_fullsized":"2024-09-11T01:25:23.085337+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20240,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255347+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:25:23.255347+0000","last_peered":"2024-09-11T01:25:23.255347+0000","last_clean":"2024-09-11T01:25:23.255347+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:25:23.255347+0000","last_undegraded":"2024-09-11T01:25:23.255347+0000","last_fullsized":"2024-09-11T01:25:23.255347+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20173,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.086787+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:25:23.086787+0000","last_peered":"2024-09-11T01:25:23.086787+0000","last_clean":"2024-09-11T01:25:23.086787+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:25:23.086787+0000","last_undegraded":"2024-09-11T01:25:23.086787+0000","last_fullsized":"2024-09-11T01:25:23.086787+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20419,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.194847+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:25:23.194847+0000","last_peered":"2024-09-11T01:25:23.194847+0000","last_clean":"2024-09-11T01:25:23.194847+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:25:23.194847+0000","last_undegraded":"2024-09-11T01:25:23.194847+0000","last_fullsized":"2024-09-11T01:25:23.194847+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20096,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.084087+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:25:23.084087+0000","last_peered":"2024-09-11T01:25:23.084087+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:25:23.084087+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20420,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083638+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:25:23.083638+0000","last_peered":"2024-09-11T01:25:23.083638+0000","last_clean":"2024-09-11T01:25:23.083638+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:25:23.083638+0000","last_undegraded":"2024-09-11T01:25:23.083638+0000","last_fullsized":"2024-09-11T01:25:23.083638+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083251+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:25:23.083251+0000","last_peered":"2024-09-11T01:25:23.083251+0000","last_clean":"2024-09-11T01:25:23.083251+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:25:23.083251+0000","last_undegraded":"2024-09-11T01:25:23.083251+0000","last_fullsized":"2024-09-11T01:25:23.083251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28017,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084352+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:25:23.084352+0000","last_peered":"2024-09-11T01:25:23.084352+0000","last_clean":"2024-09-11T01:25:23.084352+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:25:23.084352+0000","last_undegraded":"2024-09-11T01:25:23.084352+0000","last_fullsized":"2024-09-11T01:25:23.084352+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254905+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:25:23.254905+0000","last_peered":"2024-09-11T01:25:23.254905+0000","last_clean":"2024-09-11T01:25:23.254905+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:25:23.254905+0000","last_undegraded":"2024-09-11T01:25:23.254905+0000","last_fullsized":"2024-09-11T01:25:23.254905+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20033,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.086560+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:25:23.086560+0000","last_peered":"2024-09-11T01:25:23.086560+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:25:23.086560+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083438+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:25:23.083438+0000","last_peered":"2024-09-11T01:25:23.083438+0000","last_clean":"2024-09-11T01:25:23.083438+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:25:23.083438+0000","last_undegraded":"2024-09-11T01:25:23.083438+0000","last_fullsized":"2024-09-11T01:25:23.083438+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28422,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254938+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:25:23.254938+0000","last_peered":"2024-09-11T01:25:23.254938+0000","last_clean":"2024-09-11T01:25:23.254938+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:25:23.254938+0000","last_undegraded":"2024-09-11T01:25:23.254938+0000","last_fullsized":"2024-09-11T01:25:23.254938+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":116,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084248+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:25:23.084248+0000","last_peered":"2024-09-11T01:25:23.084248+0000","last_clean":"2024-09-11T01:25:23.084248+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:25:23.084248+0000","last_undegraded":"2024-09-11T01:25:23.084248+0000","last_fullsized":"2024-09-11T01:25:23.084248+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27813,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255394+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:25:23.255394+0000","last_peered":"2024-09-11T01:25:23.255394+0000","last_clean":"2024-09-11T01:25:23.255394+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:25:23.255394+0000","last_undegraded":"2024-09-11T01:25:23.255394+0000","last_fullsized":"2024-09-11T01:25:23.255394+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20423,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085593+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:25:23.085593+0000","last_peered":"2024-09-11T01:25:23.085593+0000","last_clean":"2024-09-11T01:25:23.085593+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:25:23.085593+0000","last_undegraded":"2024-09-11T01:25:23.085593+0000","last_fullsized":"2024-09-11T01:25:23.085593+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085903+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:25:23.085903+0000","last_peered":"2024-09-11T01:25:23.085903+0000","last_clean":"2024-09-11T01:25:23.085903+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:25:23.085903+0000","last_undegraded":"2024-09-11T01:25:23.085903+0000","last_fullsized":"2024-09-11T01:25:23.085903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27951,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085470+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:25:23.085470+0000","last_peered":"2024-09-11T01:25:23.085470+0000","last_clean":"2024-09-11T01:25:23.085470+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:25:23.085470+0000","last_undegraded":"2024-09-11T01:25:23.085470+0000","last_fullsized":"2024-09-11T01:25:23.085470+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255181+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:25:23.255181+0000","last_peered":"2024-09-11T01:25:23.255181+0000","last_clean":"2024-09-11T01:25:23.255181+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:25:23.255181+0000","last_undegraded":"2024-09-11T01:25:23.255181+0000","last_fullsized":"2024-09-11T01:25:23.255181+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24630,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085345+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:25:23.085345+0000","last_peered":"2024-09-11T01:25:23.085345+0000","last_clean":"2024-09-11T01:25:23.085345+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:25:23.085345+0000","last_undegraded":"2024-09-11T01:25:23.085345+0000","last_fullsized":"2024-09-11T01:25:23.085345+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084199+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:25:23.084199+0000","last_peered":"2024-09-11T01:25:23.084199+0000","last_clean":"2024-09-11T01:25:23.084199+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:25:23.084199+0000","last_undegraded":"2024-09-11T01:25:23.084199+0000","last_fullsized":"2024-09-11T01:25:23.084199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085458+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:25:23.085458+0000","last_peered":"2024-09-11T01:25:23.085458+0000","last_clean":"2024-09-11T01:25:23.085458+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:25:23.085458+0000","last_undegraded":"2024-09-11T01:25:23.085458+0000","last_fullsized":"2024-09-11T01:25:23.085458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":25945,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:53.292484+0000","last_change":"2024-09-11T01:25:21.628441+0000","last_active":"2024-09-11T01:25:53.292484+0000","last_peered":"2024-09-11T01:25:53.292484+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:25:53.292484+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1009,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2911,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1008}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8983,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":38656,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":150593536,"data_stored":150569183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182572,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518664,"kb_used_data":63608,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853176,"statfs":{"total":96636764160,"available":96105652224,"internally_reserved":0,"allocated":65134592,"data_stored":64790418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986345,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":445688,"kb_used_data":85032,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93926152,"statfs":{"total":96636764160,"available":96180379648,"internally_reserved":0,"allocated":87072768,"data_stored":86732671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659289,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":513880,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":513339,"kb_avail":93857960,"statfs":{"total":96636764160,"available":96110551040,"internally_reserved":0,"allocated":532480,"data_stored":174390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":525659823},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":85950464,"data_stored":85946985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:00.414 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:00.736 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:00.737 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:26:00.755 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":502,"stamp":"2024-09-11T01:25:58.868060+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8379,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39261,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1465752,"kb_used_data":149160,"kb_used_omap":12,"kb_used_meta":1316531,"kb_avail":281649768,"statfs":{"total":289910292480,"available":288409362432,"internally_reserved":0,"allocated":152739840,"data_stored":151697479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1348127748},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1533,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1533,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003454"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28532,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085852+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:25:23.085852+0000","last_peered":"2024-09-11T01:25:23.085852+0000","last_clean":"2024-09-11T01:25:23.085852+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:25:23.085852+0000","last_undegraded":"2024-09-11T01:25:23.085852+0000","last_fullsized":"2024-09-11T01:25:23.085852+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20490,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255153+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:25:23.255153+0000","last_peered":"2024-09-11T01:25:23.255153+0000","last_clean":"2024-09-11T01:25:23.255153+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:25:23.255153+0000","last_undegraded":"2024-09-11T01:25:23.255153+0000","last_fullsized":"2024-09-11T01:25:23.255153+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20313,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085337+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:25:23.085337+0000","last_peered":"2024-09-11T01:25:23.085337+0000","last_clean":"2024-09-11T01:25:23.085337+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:25:23.085337+0000","last_undegraded":"2024-09-11T01:25:23.085337+0000","last_fullsized":"2024-09-11T01:25:23.085337+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20240,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255347+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:25:23.255347+0000","last_peered":"2024-09-11T01:25:23.255347+0000","last_clean":"2024-09-11T01:25:23.255347+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:25:23.255347+0000","last_undegraded":"2024-09-11T01:25:23.255347+0000","last_fullsized":"2024-09-11T01:25:23.255347+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20173,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.086787+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:25:23.086787+0000","last_peered":"2024-09-11T01:25:23.086787+0000","last_clean":"2024-09-11T01:25:23.086787+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:25:23.086787+0000","last_undegraded":"2024-09-11T01:25:23.086787+0000","last_fullsized":"2024-09-11T01:25:23.086787+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20419,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.194847+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:25:23.194847+0000","last_peered":"2024-09-11T01:25:23.194847+0000","last_clean":"2024-09-11T01:25:23.194847+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:25:23.194847+0000","last_undegraded":"2024-09-11T01:25:23.194847+0000","last_fullsized":"2024-09-11T01:25:23.194847+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20097,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.085569+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:25:23.085569+0000","last_peered":"2024-09-11T01:25:23.085569+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:25:23.085569+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20420,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083638+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:25:23.083638+0000","last_peered":"2024-09-11T01:25:23.083638+0000","last_clean":"2024-09-11T01:25:23.083638+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:25:23.083638+0000","last_undegraded":"2024-09-11T01:25:23.083638+0000","last_fullsized":"2024-09-11T01:25:23.083638+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083251+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:25:23.083251+0000","last_peered":"2024-09-11T01:25:23.083251+0000","last_clean":"2024-09-11T01:25:23.083251+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:25:23.083251+0000","last_undegraded":"2024-09-11T01:25:23.083251+0000","last_fullsized":"2024-09-11T01:25:23.083251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28017,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084352+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:25:23.084352+0000","last_peered":"2024-09-11T01:25:23.084352+0000","last_clean":"2024-09-11T01:25:23.084352+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:25:23.084352+0000","last_undegraded":"2024-09-11T01:25:23.084352+0000","last_fullsized":"2024-09-11T01:25:23.084352+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254905+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:25:23.254905+0000","last_peered":"2024-09-11T01:25:23.254905+0000","last_clean":"2024-09-11T01:25:23.254905+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:25:23.254905+0000","last_undegraded":"2024-09-11T01:25:23.254905+0000","last_fullsized":"2024-09-11T01:25:23.254905+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20033,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.086560+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:25:23.086560+0000","last_peered":"2024-09-11T01:25:23.086560+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:25:23.086560+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083438+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:25:23.083438+0000","last_peered":"2024-09-11T01:25:23.083438+0000","last_clean":"2024-09-11T01:25:23.083438+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:25:23.083438+0000","last_undegraded":"2024-09-11T01:25:23.083438+0000","last_fullsized":"2024-09-11T01:25:23.083438+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28422,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254938+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:25:23.254938+0000","last_peered":"2024-09-11T01:25:23.254938+0000","last_clean":"2024-09-11T01:25:23.254938+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:25:23.254938+0000","last_undegraded":"2024-09-11T01:25:23.254938+0000","last_fullsized":"2024-09-11T01:25:23.254938+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":116,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084248+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:25:23.084248+0000","last_peered":"2024-09-11T01:25:23.084248+0000","last_clean":"2024-09-11T01:25:23.084248+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:25:23.084248+0000","last_undegraded":"2024-09-11T01:25:23.084248+0000","last_fullsized":"2024-09-11T01:25:23.084248+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27813,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255394+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:25:23.255394+0000","last_peered":"2024-09-11T01:25:23.255394+0000","last_clean":"2024-09-11T01:25:23.255394+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:25:23.255394+0000","last_undegraded":"2024-09-11T01:25:23.255394+0000","last_fullsized":"2024-09-11T01:25:23.255394+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20423,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085593+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:25:23.085593+0000","last_peered":"2024-09-11T01:25:23.085593+0000","last_clean":"2024-09-11T01:25:23.085593+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:25:23.085593+0000","last_undegraded":"2024-09-11T01:25:23.085593+0000","last_fullsized":"2024-09-11T01:25:23.085593+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085903+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:25:23.085903+0000","last_peered":"2024-09-11T01:25:23.085903+0000","last_clean":"2024-09-11T01:25:23.085903+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:25:23.085903+0000","last_undegraded":"2024-09-11T01:25:23.085903+0000","last_fullsized":"2024-09-11T01:25:23.085903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27951,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085470+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:25:23.085470+0000","last_peered":"2024-09-11T01:25:23.085470+0000","last_clean":"2024-09-11T01:25:23.085470+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:25:23.085470+0000","last_undegraded":"2024-09-11T01:25:23.085470+0000","last_fullsized":"2024-09-11T01:25:23.085470+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255181+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:25:23.255181+0000","last_peered":"2024-09-11T01:25:23.255181+0000","last_clean":"2024-09-11T01:25:23.255181+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:25:23.255181+0000","last_undegraded":"2024-09-11T01:25:23.255181+0000","last_fullsized":"2024-09-11T01:25:23.255181+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24630,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085345+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:25:23.085345+0000","last_peered":"2024-09-11T01:25:23.085345+0000","last_clean":"2024-09-11T01:25:23.085345+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:25:23.085345+0000","last_undegraded":"2024-09-11T01:25:23.085345+0000","last_fullsized":"2024-09-11T01:25:23.085345+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084199+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:25:23.084199+0000","last_peered":"2024-09-11T01:25:23.084199+0000","last_clean":"2024-09-11T01:25:23.084199+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:25:23.084199+0000","last_undegraded":"2024-09-11T01:25:23.084199+0000","last_fullsized":"2024-09-11T01:25:23.084199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085458+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:25:23.085458+0000","last_peered":"2024-09-11T01:25:23.085458+0000","last_clean":"2024-09-11T01:25:23.085458+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:25:23.085458+0000","last_undegraded":"2024-09-11T01:25:23.085458+0000","last_fullsized":"2024-09-11T01:25:23.085458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":27153,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:58.297503+0000","last_change":"2024-09-11T01:25:21.628441+0000","last_active":"2024-09-11T01:25:58.297503+0000","last_peered":"2024-09-11T01:25:58.297503+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:25:58.297503+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":405,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3515,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-404}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8379,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39260,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":150593536,"data_stored":150569183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182573,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518664,"kb_used_data":63608,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853176,"statfs":{"total":96636764160,"available":96105652224,"internally_reserved":0,"allocated":65134592,"data_stored":64790418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986345,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":445688,"kb_used_data":85032,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93926152,"statfs":{"total":96636764160,"available":96180379648,"internally_reserved":0,"allocated":87072768,"data_stored":86732671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659290,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501400,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":500859,"kb_avail":93870440,"statfs":{"total":96636764160,"available":96123330560,"internally_reserved":0,"allocated":532480,"data_stored":174390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512880303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":85950464,"data_stored":85946985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:00.756 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:01.087 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:01.088 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:26:01.107 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":503,"stamp":"2024-09-11T01:26:00.868734+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8379,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39261,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1462752,"kb_used_data":146160,"kb_used_omap":12,"kb_used_meta":1316531,"kb_avail":281652768,"statfs":{"total":289910292480,"available":288412434432,"internally_reserved":0,"allocated":149667840,"data_stored":148625479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1348127748},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1103,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1103,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003528"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28532,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085852+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:25:23.085852+0000","last_peered":"2024-09-11T01:25:23.085852+0000","last_clean":"2024-09-11T01:25:23.085852+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:25:23.085852+0000","last_undegraded":"2024-09-11T01:25:23.085852+0000","last_fullsized":"2024-09-11T01:25:23.085852+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20490,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255153+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:25:23.255153+0000","last_peered":"2024-09-11T01:25:23.255153+0000","last_clean":"2024-09-11T01:25:23.255153+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:25:23.255153+0000","last_undegraded":"2024-09-11T01:25:23.255153+0000","last_fullsized":"2024-09-11T01:25:23.255153+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20313,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085337+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:25:23.085337+0000","last_peered":"2024-09-11T01:25:23.085337+0000","last_clean":"2024-09-11T01:25:23.085337+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:25:23.085337+0000","last_undegraded":"2024-09-11T01:25:23.085337+0000","last_fullsized":"2024-09-11T01:25:23.085337+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20240,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255347+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:25:23.255347+0000","last_peered":"2024-09-11T01:25:23.255347+0000","last_clean":"2024-09-11T01:25:23.255347+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:25:23.255347+0000","last_undegraded":"2024-09-11T01:25:23.255347+0000","last_fullsized":"2024-09-11T01:25:23.255347+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20173,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.086787+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:25:23.086787+0000","last_peered":"2024-09-11T01:25:23.086787+0000","last_clean":"2024-09-11T01:25:23.086787+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:25:23.086787+0000","last_undegraded":"2024-09-11T01:25:23.086787+0000","last_fullsized":"2024-09-11T01:25:23.086787+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20419,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.194847+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:25:23.194847+0000","last_peered":"2024-09-11T01:25:23.194847+0000","last_clean":"2024-09-11T01:25:23.194847+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:25:23.194847+0000","last_undegraded":"2024-09-11T01:25:23.194847+0000","last_fullsized":"2024-09-11T01:25:23.194847+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20097,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.085569+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:25:23.085569+0000","last_peered":"2024-09-11T01:25:23.085569+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:25:23.085569+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20420,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083638+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:25:23.083638+0000","last_peered":"2024-09-11T01:25:23.083638+0000","last_clean":"2024-09-11T01:25:23.083638+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:25:23.083638+0000","last_undegraded":"2024-09-11T01:25:23.083638+0000","last_fullsized":"2024-09-11T01:25:23.083638+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083251+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:25:23.083251+0000","last_peered":"2024-09-11T01:25:23.083251+0000","last_clean":"2024-09-11T01:25:23.083251+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:25:23.083251+0000","last_undegraded":"2024-09-11T01:25:23.083251+0000","last_fullsized":"2024-09-11T01:25:23.083251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28017,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084352+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:25:23.084352+0000","last_peered":"2024-09-11T01:25:23.084352+0000","last_clean":"2024-09-11T01:25:23.084352+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:25:23.084352+0000","last_undegraded":"2024-09-11T01:25:23.084352+0000","last_fullsized":"2024-09-11T01:25:23.084352+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254905+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:25:23.254905+0000","last_peered":"2024-09-11T01:25:23.254905+0000","last_clean":"2024-09-11T01:25:23.254905+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:25:23.254905+0000","last_undegraded":"2024-09-11T01:25:23.254905+0000","last_fullsized":"2024-09-11T01:25:23.254905+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20033,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.086560+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:25:23.086560+0000","last_peered":"2024-09-11T01:25:23.086560+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:25:23.086560+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083438+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:25:23.083438+0000","last_peered":"2024-09-11T01:25:23.083438+0000","last_clean":"2024-09-11T01:25:23.083438+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:25:23.083438+0000","last_undegraded":"2024-09-11T01:25:23.083438+0000","last_fullsized":"2024-09-11T01:25:23.083438+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28422,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254938+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:25:23.254938+0000","last_peered":"2024-09-11T01:25:23.254938+0000","last_clean":"2024-09-11T01:25:23.254938+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:25:23.254938+0000","last_undegraded":"2024-09-11T01:25:23.254938+0000","last_fullsized":"2024-09-11T01:25:23.254938+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":116,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084248+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:25:23.084248+0000","last_peered":"2024-09-11T01:25:23.084248+0000","last_clean":"2024-09-11T01:25:23.084248+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:25:23.084248+0000","last_undegraded":"2024-09-11T01:25:23.084248+0000","last_fullsized":"2024-09-11T01:25:23.084248+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27813,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255394+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:25:23.255394+0000","last_peered":"2024-09-11T01:25:23.255394+0000","last_clean":"2024-09-11T01:25:23.255394+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:25:23.255394+0000","last_undegraded":"2024-09-11T01:25:23.255394+0000","last_fullsized":"2024-09-11T01:25:23.255394+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20423,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085593+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:25:23.085593+0000","last_peered":"2024-09-11T01:25:23.085593+0000","last_clean":"2024-09-11T01:25:23.085593+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:25:23.085593+0000","last_undegraded":"2024-09-11T01:25:23.085593+0000","last_fullsized":"2024-09-11T01:25:23.085593+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085903+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:25:23.085903+0000","last_peered":"2024-09-11T01:25:23.085903+0000","last_clean":"2024-09-11T01:25:23.085903+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:25:23.085903+0000","last_undegraded":"2024-09-11T01:25:23.085903+0000","last_fullsized":"2024-09-11T01:25:23.085903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27951,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085470+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:25:23.085470+0000","last_peered":"2024-09-11T01:25:23.085470+0000","last_clean":"2024-09-11T01:25:23.085470+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:25:23.085470+0000","last_undegraded":"2024-09-11T01:25:23.085470+0000","last_fullsized":"2024-09-11T01:25:23.085470+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255181+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:25:23.255181+0000","last_peered":"2024-09-11T01:25:23.255181+0000","last_clean":"2024-09-11T01:25:23.255181+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:25:23.255181+0000","last_undegraded":"2024-09-11T01:25:23.255181+0000","last_fullsized":"2024-09-11T01:25:23.255181+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24630,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085345+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:25:23.085345+0000","last_peered":"2024-09-11T01:25:23.085345+0000","last_clean":"2024-09-11T01:25:23.085345+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:25:23.085345+0000","last_undegraded":"2024-09-11T01:25:23.085345+0000","last_fullsized":"2024-09-11T01:25:23.085345+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084199+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:25:23.084199+0000","last_peered":"2024-09-11T01:25:23.084199+0000","last_clean":"2024-09-11T01:25:23.084199+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:25:23.084199+0000","last_undegraded":"2024-09-11T01:25:23.084199+0000","last_fullsized":"2024-09-11T01:25:23.084199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085458+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:25:23.085458+0000","last_peered":"2024-09-11T01:25:23.085458+0000","last_clean":"2024-09-11T01:25:23.085458+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:25:23.085458+0000","last_undegraded":"2024-09-11T01:25:23.085458+0000","last_fullsized":"2024-09-11T01:25:23.085458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":27153,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:58.297503+0000","last_change":"2024-09-11T01:25:21.628441+0000","last_active":"2024-09-11T01:25:58.297503+0000","last_peered":"2024-09-11T01:25:58.297503+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:25:58.297503+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":405,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3515,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-404}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8379,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39260,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147521536,"data_stored":147497183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182573,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518664,"kb_used_data":63608,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853176,"statfs":{"total":96636764160,"available":96105652224,"internally_reserved":0,"allocated":65134592,"data_stored":64790418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986346,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442688,"kb_used_data":82032,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93929152,"statfs":{"total":96636764160,"available":96183451648,"internally_reserved":0,"allocated":84000768,"data_stored":83660671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659290,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501400,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":500859,"kb_avail":93870440,"statfs":{"total":96636764160,"available":96123330560,"internally_reserved":0,"allocated":532480,"data_stored":174390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512880303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":82878464,"data_stored":82874985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:01.108 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:26:01.108 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20097, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:23.085569+0000', 'last_change': '2024-09-11T01:13:10.282982+0000', 'last_active': '2024-09-11T01:25:23.085569+0000', 'last_peered': '2024-09-11T01:25:23.085569+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:25:23.085569+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3988, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3986}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:01.109 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:26:01.109 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20033, 'reported_epoch': 61, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:23.086560+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:25:23.086560+0000', 'last_peered': '2024-09-11T01:25:23.086560+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:25:23.086560+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:01.109 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.4 is not active+clean 2024-09-11T01:26:01.109 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.4', 'version': "49'19952", 'reported_seq': 27153, 'reported_epoch': 61, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:25:58.297503+0000', 'last_change': '2024-09-11T01:25:21.628441+0000', 'last_active': '2024-09-11T01:25:58.297503+0000', 'last_peered': '2024-09-11T01:25:58.297503+0000', 'last_clean': '2024-09-11T01:13:07.636621+0000', 'last_became_active': '2024-09-11T01:13:10.271396+0000', 'last_became_peered': '2024-09-11T01:13:10.271396+0000', 'last_unstale': '2024-09-11T01:25:58.297503+0000', 'last_undegraded': '2024-09-11T01:13:09.792569+0000', 'last_fullsized': '2024-09-11T01:13:09.656159+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'2993", 'last_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:51.738891+0000', 'objects_scrubbed': 2919, 'log_size': 10052, 'log_dups_size': 0, 'ondisk_log_size': 10052, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1627, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 617, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 405, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 17022, 'num_write_kb': 28208, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3515, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -404}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:01.109 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:26:01.498 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:01.498 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 8379/6 objects degraded (139650.000%), 3 pgs degraded, 3 pgs undersized","count":6},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":897,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":61,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":3},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":21},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":2},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1497858048,"bytes_avail":288412434432,"bytes_total":289910292480,"degraded_objects":8379,"degraded_total":6,"degraded_ratio":1396.5,"recovering_objects_per_sec":91,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1103,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:26:01.518 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:26:01.518 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:01.835 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:01.835 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:26:01.854 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":503,"stamp":"2024-09-11T01:26:00.868734+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8379,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39261,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":45,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1462752,"kb_used_data":146160,"kb_used_omap":12,"kb_used_meta":1316531,"kb_avail":281652768,"statfs":{"total":289910292480,"available":288412434432,"internally_reserved":0,"allocated":149667840,"data_stored":148625479,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1348127748},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1103,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1103,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003528"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28532,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085852+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:25:23.085852+0000","last_peered":"2024-09-11T01:25:23.085852+0000","last_clean":"2024-09-11T01:25:23.085852+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:25:23.085852+0000","last_undegraded":"2024-09-11T01:25:23.085852+0000","last_fullsized":"2024-09-11T01:25:23.085852+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20490,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255153+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:25:23.255153+0000","last_peered":"2024-09-11T01:25:23.255153+0000","last_clean":"2024-09-11T01:25:23.255153+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:25:23.255153+0000","last_undegraded":"2024-09-11T01:25:23.255153+0000","last_fullsized":"2024-09-11T01:25:23.255153+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20313,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085337+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:25:23.085337+0000","last_peered":"2024-09-11T01:25:23.085337+0000","last_clean":"2024-09-11T01:25:23.085337+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:25:23.085337+0000","last_undegraded":"2024-09-11T01:25:23.085337+0000","last_fullsized":"2024-09-11T01:25:23.085337+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20240,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255347+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:25:23.255347+0000","last_peered":"2024-09-11T01:25:23.255347+0000","last_clean":"2024-09-11T01:25:23.255347+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:25:23.255347+0000","last_undegraded":"2024-09-11T01:25:23.255347+0000","last_fullsized":"2024-09-11T01:25:23.255347+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20173,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.086787+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:25:23.086787+0000","last_peered":"2024-09-11T01:25:23.086787+0000","last_clean":"2024-09-11T01:25:23.086787+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:25:23.086787+0000","last_undegraded":"2024-09-11T01:25:23.086787+0000","last_fullsized":"2024-09-11T01:25:23.086787+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20419,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.194847+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:25:23.194847+0000","last_peered":"2024-09-11T01:25:23.194847+0000","last_clean":"2024-09-11T01:25:23.194847+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:25:23.194847+0000","last_undegraded":"2024-09-11T01:25:23.194847+0000","last_fullsized":"2024-09-11T01:25:23.194847+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20097,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.085569+0000","last_change":"2024-09-11T01:13:10.282982+0000","last_active":"2024-09-11T01:25:23.085569+0000","last_peered":"2024-09-11T01:25:23.085569+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:25:23.085569+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3988,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3986}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20420,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083638+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:25:23.083638+0000","last_peered":"2024-09-11T01:25:23.083638+0000","last_clean":"2024-09-11T01:25:23.083638+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:25:23.083638+0000","last_undegraded":"2024-09-11T01:25:23.083638+0000","last_fullsized":"2024-09-11T01:25:23.083638+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083251+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:25:23.083251+0000","last_peered":"2024-09-11T01:25:23.083251+0000","last_clean":"2024-09-11T01:25:23.083251+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:25:23.083251+0000","last_undegraded":"2024-09-11T01:25:23.083251+0000","last_fullsized":"2024-09-11T01:25:23.083251+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28017,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084352+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:25:23.084352+0000","last_peered":"2024-09-11T01:25:23.084352+0000","last_clean":"2024-09-11T01:25:23.084352+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:25:23.084352+0000","last_undegraded":"2024-09-11T01:25:23.084352+0000","last_fullsized":"2024-09-11T01:25:23.084352+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254905+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:25:23.254905+0000","last_peered":"2024-09-11T01:25:23.254905+0000","last_clean":"2024-09-11T01:25:23.254905+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:25:23.254905+0000","last_undegraded":"2024-09-11T01:25:23.254905+0000","last_fullsized":"2024-09-11T01:25:23.254905+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20033,"reported_epoch":61,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:23.086560+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:25:23.086560+0000","last_peered":"2024-09-11T01:25:23.086560+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:25:23.086560+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":137,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.083438+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:25:23.083438+0000","last_peered":"2024-09-11T01:25:23.083438+0000","last_clean":"2024-09-11T01:25:23.083438+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:25:23.083438+0000","last_undegraded":"2024-09-11T01:25:23.083438+0000","last_fullsized":"2024-09-11T01:25:23.083438+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28422,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254938+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:25:23.254938+0000","last_peered":"2024-09-11T01:25:23.254938+0000","last_clean":"2024-09-11T01:25:23.254938+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:25:23.254938+0000","last_undegraded":"2024-09-11T01:25:23.254938+0000","last_fullsized":"2024-09-11T01:25:23.254938+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":116,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084248+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:25:23.084248+0000","last_peered":"2024-09-11T01:25:23.084248+0000","last_clean":"2024-09-11T01:25:23.084248+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:25:23.084248+0000","last_undegraded":"2024-09-11T01:25:23.084248+0000","last_fullsized":"2024-09-11T01:25:23.084248+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27813,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255394+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:25:23.255394+0000","last_peered":"2024-09-11T01:25:23.255394+0000","last_clean":"2024-09-11T01:25:23.255394+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:25:23.255394+0000","last_undegraded":"2024-09-11T01:25:23.255394+0000","last_fullsized":"2024-09-11T01:25:23.255394+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20423,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085593+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:25:23.085593+0000","last_peered":"2024-09-11T01:25:23.085593+0000","last_clean":"2024-09-11T01:25:23.085593+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:25:23.085593+0000","last_undegraded":"2024-09-11T01:25:23.085593+0000","last_fullsized":"2024-09-11T01:25:23.085593+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085903+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:25:23.085903+0000","last_peered":"2024-09-11T01:25:23.085903+0000","last_clean":"2024-09-11T01:25:23.085903+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:25:23.085903+0000","last_undegraded":"2024-09-11T01:25:23.085903+0000","last_fullsized":"2024-09-11T01:25:23.085903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27951,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085470+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:25:23.085470+0000","last_peered":"2024-09-11T01:25:23.085470+0000","last_clean":"2024-09-11T01:25:23.085470+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:25:23.085470+0000","last_undegraded":"2024-09-11T01:25:23.085470+0000","last_fullsized":"2024-09-11T01:25:23.085470+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255181+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:25:23.255181+0000","last_peered":"2024-09-11T01:25:23.255181+0000","last_clean":"2024-09-11T01:25:23.255181+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:25:23.255181+0000","last_undegraded":"2024-09-11T01:25:23.255181+0000","last_fullsized":"2024-09-11T01:25:23.255181+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24630,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085345+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:25:23.085345+0000","last_peered":"2024-09-11T01:25:23.085345+0000","last_clean":"2024-09-11T01:25:23.085345+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:25:23.085345+0000","last_undegraded":"2024-09-11T01:25:23.085345+0000","last_fullsized":"2024-09-11T01:25:23.085345+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.084199+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:25:23.084199+0000","last_peered":"2024-09-11T01:25:23.084199+0000","last_clean":"2024-09-11T01:25:23.084199+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:25:23.084199+0000","last_undegraded":"2024-09-11T01:25:23.084199+0000","last_fullsized":"2024-09-11T01:25:23.084199+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":110,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.085458+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:25:23.085458+0000","last_peered":"2024-09-11T01:25:23.085458+0000","last_clean":"2024-09-11T01:25:23.085458+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:25:23.085458+0000","last_undegraded":"2024-09-11T01:25:23.085458+0000","last_fullsized":"2024-09-11T01:25:23.085458+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":27153,"reported_epoch":61,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:25:58.297503+0000","last_change":"2024-09-11T01:25:21.628441+0000","last_active":"2024-09-11T01:25:58.297503+0000","last_peered":"2024-09-11T01:25:58.297503+0000","last_clean":"2024-09-11T01:13:07.636621+0000","last_became_active":"2024-09-11T01:13:10.271396+0000","last_became_peered":"2024-09-11T01:13:10.271396+0000","last_unstale":"2024-09-11T01:25:58.297503+0000","last_undegraded":"2024-09-11T01:13:09.792569+0000","last_fullsized":"2024-09-11T01:13:09.656159+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":405,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3515,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-404}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":8379,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39260,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147521536,"data_stored":147497183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":29,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182573,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518664,"kb_used_data":63608,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853176,"statfs":{"total":96636764160,"available":96105652224,"internally_reserved":0,"allocated":65134592,"data_stored":64790418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986346,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442688,"kb_used_data":82032,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93929152,"statfs":{"total":96636764160,"available":96183451648,"internally_reserved":0,"allocated":84000768,"data_stored":83660671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659290,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501400,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":500859,"kb_avail":93870440,"statfs":{"total":96636764160,"available":96123330560,"internally_reserved":0,"allocated":532480,"data_stored":174390,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512880303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":82878464,"data_stored":82874985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:04.857 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:05.198 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:05.199 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:26:05.219 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":507,"stamp":"2024-09-11T01:26:04.869422+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7744,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39896,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1466920,"kb_used_data":146168,"kb_used_omap":12,"kb_used_meta":1320691,"kb_avail":281648600,"statfs":{"total":289910292480,"available":288408166400,"internally_reserved":0,"allocated":149676032,"data_stored":148629805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1352387588},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1239,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1239,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001758"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28536,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985357+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:02.985357+0000","last_peered":"2024-09-11T01:26:02.985357+0000","last_clean":"2024-09-11T01:26:02.985357+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:02.985357+0000","last_undegraded":"2024-09-11T01:26:02.985357+0000","last_fullsized":"2024-09-11T01:26:02.985357+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20490,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255153+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:25:23.255153+0000","last_peered":"2024-09-11T01:25:23.255153+0000","last_clean":"2024-09-11T01:25:23.255153+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:25:23.255153+0000","last_undegraded":"2024-09-11T01:25:23.255153+0000","last_fullsized":"2024-09-11T01:25:23.255153+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20317,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20240,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255347+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:25:23.255347+0000","last_peered":"2024-09-11T01:25:23.255347+0000","last_clean":"2024-09-11T01:25:23.255347+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:25:23.255347+0000","last_undegraded":"2024-09-11T01:25:23.255347+0000","last_fullsized":"2024-09-11T01:25:23.255347+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20177,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984547+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:02.984547+0000","last_peered":"2024-09-11T01:26:02.984547+0000","last_clean":"2024-09-11T01:26:02.984547+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:02.984547+0000","last_undegraded":"2024-09-11T01:26:02.984547+0000","last_fullsized":"2024-09-11T01:26:02.984547+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20423,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986863+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:02.986863+0000","last_peered":"2024-09-11T01:26:02.986863+0000","last_clean":"2024-09-11T01:26:02.986863+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:02.986863+0000","last_undegraded":"2024-09-11T01:26:02.986863+0000","last_fullsized":"2024-09-11T01:26:02.986863+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20562,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:03.298296+0000","last_change":"2024-09-11T01:26:01.184322+0000","last_active":"2024-09-11T01:26:03.298296+0000","last_peered":"2024-09-11T01:26:03.298296+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:26:03.298296+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3758,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3758}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982464+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:02.982464+0000","last_peered":"2024-09-11T01:26:02.982464+0000","last_clean":"2024-09-11T01:26:02.982464+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:02.982464+0000","last_undegraded":"2024-09-11T01:26:02.982464+0000","last_fullsized":"2024-09-11T01:26:02.982464+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986745+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:02.986745+0000","last_peered":"2024-09-11T01:26:02.986745+0000","last_clean":"2024-09-11T01:26:02.986745+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:02.986745+0000","last_undegraded":"2024-09-11T01:26:02.986745+0000","last_fullsized":"2024-09-11T01:26:02.986745+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28021,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989204+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:02.989204+0000","last_peered":"2024-09-11T01:26:02.989204+0000","last_clean":"2024-09-11T01:26:02.989204+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:02.989204+0000","last_undegraded":"2024-09-11T01:26:02.989204+0000","last_fullsized":"2024-09-11T01:26:02.989204+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254905+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:25:23.254905+0000","last_peered":"2024-09-11T01:25:23.254905+0000","last_clean":"2024-09-11T01:25:23.254905+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:25:23.254905+0000","last_undegraded":"2024-09-11T01:25:23.254905+0000","last_fullsized":"2024-09-11T01:25:23.254905+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20037,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:02.982813+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:26:02.982813+0000","last_peered":"2024-09-11T01:26:02.982813+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:26:02.982813+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982616+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:02.982616+0000","last_peered":"2024-09-11T01:26:02.982616+0000","last_clean":"2024-09-11T01:26:02.982616+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:02.982616+0000","last_undegraded":"2024-09-11T01:26:02.982616+0000","last_fullsized":"2024-09-11T01:26:02.982616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28422,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254938+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:25:23.254938+0000","last_peered":"2024-09-11T01:25:23.254938+0000","last_clean":"2024-09-11T01:25:23.254938+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:25:23.254938+0000","last_undegraded":"2024-09-11T01:25:23.254938+0000","last_fullsized":"2024-09-11T01:25:23.254938+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989042+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:02.989042+0000","last_peered":"2024-09-11T01:26:02.989042+0000","last_clean":"2024-09-11T01:26:02.989042+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:02.989042+0000","last_undegraded":"2024-09-11T01:26:02.989042+0000","last_fullsized":"2024-09-11T01:26:02.989042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27813,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255394+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:25:23.255394+0000","last_peered":"2024-09-11T01:25:23.255394+0000","last_clean":"2024-09-11T01:25:23.255394+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:25:23.255394+0000","last_undegraded":"2024-09-11T01:25:23.255394+0000","last_fullsized":"2024-09-11T01:25:23.255394+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20427,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989519+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:02.989519+0000","last_peered":"2024-09-11T01:26:02.989519+0000","last_clean":"2024-09-11T01:26:02.989519+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:02.989519+0000","last_undegraded":"2024-09-11T01:26:02.989519+0000","last_fullsized":"2024-09-11T01:26:02.989519+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989097+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:02.989097+0000","last_peered":"2024-09-11T01:26:02.989097+0000","last_clean":"2024-09-11T01:26:02.989097+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:02.989097+0000","last_undegraded":"2024-09-11T01:26:02.989097+0000","last_fullsized":"2024-09-11T01:26:02.989097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27955,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989371+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:02.989371+0000","last_peered":"2024-09-11T01:26:02.989371+0000","last_clean":"2024-09-11T01:26:02.989371+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:02.989371+0000","last_undegraded":"2024-09-11T01:26:02.989371+0000","last_fullsized":"2024-09-11T01:26:02.989371+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255181+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:25:23.255181+0000","last_peered":"2024-09-11T01:25:23.255181+0000","last_clean":"2024-09-11T01:25:23.255181+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:25:23.255181+0000","last_undegraded":"2024-09-11T01:25:23.255181+0000","last_fullsized":"2024-09-11T01:25:23.255181+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24634,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989242+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:02.989242+0000","last_peered":"2024-09-11T01:26:02.989242+0000","last_clean":"2024-09-11T01:26:02.989242+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:02.989242+0000","last_undegraded":"2024-09-11T01:26:02.989242+0000","last_fullsized":"2024-09-11T01:26:02.989242+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989374+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:02.989374+0000","last_peered":"2024-09-11T01:26:02.989374+0000","last_clean":"2024-09-11T01:26:02.989374+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:02.989374+0000","last_undegraded":"2024-09-11T01:26:02.989374+0000","last_fullsized":"2024-09-11T01:26:02.989374+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20107,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:03.078750+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:03.078750+0000","last_peered":"2024-09-11T01:26:03.078750+0000","last_clean":"2024-09-11T01:26:03.078750+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:03.078750+0000","last_undegraded":"2024-09-11T01:26:03.078750+0000","last_fullsized":"2024-09-11T01:26:03.078750+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7744,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39895,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147513344,"data_stored":147495949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182574,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518672,"kb_used_data":63616,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853168,"statfs":{"total":96636764160,"available":96105644032,"internally_reserved":0,"allocated":65142784,"data_stored":64793198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986346,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442688,"kb_used_data":82032,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93929152,"statfs":{"total":96636764160,"available":96183451648,"internally_reserved":0,"allocated":84000768,"data_stored":83660671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659291,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505560,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":505019,"kb_avail":93866280,"statfs":{"total":96636764160,"available":96119070720,"internally_reserved":0,"allocated":532480,"data_stored":175936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517140143},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":82878464,"data_stored":82874985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:05.221 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:05.553 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:05.553 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:26:05.572 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":507,"stamp":"2024-09-11T01:26:04.869422+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7744,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39896,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1466920,"kb_used_data":146168,"kb_used_omap":12,"kb_used_meta":1320691,"kb_avail":281648600,"statfs":{"total":289910292480,"available":288408166400,"internally_reserved":0,"allocated":149676032,"data_stored":148629805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1352387588},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1239,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1239,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001758"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28536,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985357+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:02.985357+0000","last_peered":"2024-09-11T01:26:02.985357+0000","last_clean":"2024-09-11T01:26:02.985357+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:02.985357+0000","last_undegraded":"2024-09-11T01:26:02.985357+0000","last_fullsized":"2024-09-11T01:26:02.985357+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20490,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255153+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:25:23.255153+0000","last_peered":"2024-09-11T01:25:23.255153+0000","last_clean":"2024-09-11T01:25:23.255153+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:25:23.255153+0000","last_undegraded":"2024-09-11T01:25:23.255153+0000","last_fullsized":"2024-09-11T01:25:23.255153+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20317,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20240,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255347+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:25:23.255347+0000","last_peered":"2024-09-11T01:25:23.255347+0000","last_clean":"2024-09-11T01:25:23.255347+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:25:23.255347+0000","last_undegraded":"2024-09-11T01:25:23.255347+0000","last_fullsized":"2024-09-11T01:25:23.255347+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20177,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984547+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:02.984547+0000","last_peered":"2024-09-11T01:26:02.984547+0000","last_clean":"2024-09-11T01:26:02.984547+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:02.984547+0000","last_undegraded":"2024-09-11T01:26:02.984547+0000","last_fullsized":"2024-09-11T01:26:02.984547+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20423,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986863+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:02.986863+0000","last_peered":"2024-09-11T01:26:02.986863+0000","last_clean":"2024-09-11T01:26:02.986863+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:02.986863+0000","last_undegraded":"2024-09-11T01:26:02.986863+0000","last_fullsized":"2024-09-11T01:26:02.986863+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20562,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:03.298296+0000","last_change":"2024-09-11T01:26:01.184322+0000","last_active":"2024-09-11T01:26:03.298296+0000","last_peered":"2024-09-11T01:26:03.298296+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:26:03.298296+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3758,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3758}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982464+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:02.982464+0000","last_peered":"2024-09-11T01:26:02.982464+0000","last_clean":"2024-09-11T01:26:02.982464+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:02.982464+0000","last_undegraded":"2024-09-11T01:26:02.982464+0000","last_fullsized":"2024-09-11T01:26:02.982464+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986745+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:02.986745+0000","last_peered":"2024-09-11T01:26:02.986745+0000","last_clean":"2024-09-11T01:26:02.986745+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:02.986745+0000","last_undegraded":"2024-09-11T01:26:02.986745+0000","last_fullsized":"2024-09-11T01:26:02.986745+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28021,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989204+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:02.989204+0000","last_peered":"2024-09-11T01:26:02.989204+0000","last_clean":"2024-09-11T01:26:02.989204+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:02.989204+0000","last_undegraded":"2024-09-11T01:26:02.989204+0000","last_fullsized":"2024-09-11T01:26:02.989204+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254905+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:25:23.254905+0000","last_peered":"2024-09-11T01:25:23.254905+0000","last_clean":"2024-09-11T01:25:23.254905+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:25:23.254905+0000","last_undegraded":"2024-09-11T01:25:23.254905+0000","last_fullsized":"2024-09-11T01:25:23.254905+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20037,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:02.982813+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:26:02.982813+0000","last_peered":"2024-09-11T01:26:02.982813+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:26:02.982813+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982616+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:02.982616+0000","last_peered":"2024-09-11T01:26:02.982616+0000","last_clean":"2024-09-11T01:26:02.982616+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:02.982616+0000","last_undegraded":"2024-09-11T01:26:02.982616+0000","last_fullsized":"2024-09-11T01:26:02.982616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28422,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254938+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:25:23.254938+0000","last_peered":"2024-09-11T01:25:23.254938+0000","last_clean":"2024-09-11T01:25:23.254938+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:25:23.254938+0000","last_undegraded":"2024-09-11T01:25:23.254938+0000","last_fullsized":"2024-09-11T01:25:23.254938+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989042+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:02.989042+0000","last_peered":"2024-09-11T01:26:02.989042+0000","last_clean":"2024-09-11T01:26:02.989042+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:02.989042+0000","last_undegraded":"2024-09-11T01:26:02.989042+0000","last_fullsized":"2024-09-11T01:26:02.989042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27813,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255394+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:25:23.255394+0000","last_peered":"2024-09-11T01:25:23.255394+0000","last_clean":"2024-09-11T01:25:23.255394+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:25:23.255394+0000","last_undegraded":"2024-09-11T01:25:23.255394+0000","last_fullsized":"2024-09-11T01:25:23.255394+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20427,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989519+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:02.989519+0000","last_peered":"2024-09-11T01:26:02.989519+0000","last_clean":"2024-09-11T01:26:02.989519+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:02.989519+0000","last_undegraded":"2024-09-11T01:26:02.989519+0000","last_fullsized":"2024-09-11T01:26:02.989519+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989097+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:02.989097+0000","last_peered":"2024-09-11T01:26:02.989097+0000","last_clean":"2024-09-11T01:26:02.989097+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:02.989097+0000","last_undegraded":"2024-09-11T01:26:02.989097+0000","last_fullsized":"2024-09-11T01:26:02.989097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27955,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989371+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:02.989371+0000","last_peered":"2024-09-11T01:26:02.989371+0000","last_clean":"2024-09-11T01:26:02.989371+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:02.989371+0000","last_undegraded":"2024-09-11T01:26:02.989371+0000","last_fullsized":"2024-09-11T01:26:02.989371+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255181+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:25:23.255181+0000","last_peered":"2024-09-11T01:25:23.255181+0000","last_clean":"2024-09-11T01:25:23.255181+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:25:23.255181+0000","last_undegraded":"2024-09-11T01:25:23.255181+0000","last_fullsized":"2024-09-11T01:25:23.255181+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24634,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989242+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:02.989242+0000","last_peered":"2024-09-11T01:26:02.989242+0000","last_clean":"2024-09-11T01:26:02.989242+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:02.989242+0000","last_undegraded":"2024-09-11T01:26:02.989242+0000","last_fullsized":"2024-09-11T01:26:02.989242+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989374+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:02.989374+0000","last_peered":"2024-09-11T01:26:02.989374+0000","last_clean":"2024-09-11T01:26:02.989374+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:02.989374+0000","last_undegraded":"2024-09-11T01:26:02.989374+0000","last_fullsized":"2024-09-11T01:26:02.989374+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20107,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:03.078750+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:03.078750+0000","last_peered":"2024-09-11T01:26:03.078750+0000","last_clean":"2024-09-11T01:26:03.078750+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:03.078750+0000","last_undegraded":"2024-09-11T01:26:03.078750+0000","last_fullsized":"2024-09-11T01:26:03.078750+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7744,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39895,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147513344,"data_stored":147495949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182574,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518672,"kb_used_data":63616,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853168,"statfs":{"total":96636764160,"available":96105644032,"internally_reserved":0,"allocated":65142784,"data_stored":64793198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986346,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442688,"kb_used_data":82032,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93929152,"statfs":{"total":96636764160,"available":96183451648,"internally_reserved":0,"allocated":84000768,"data_stored":83660671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659291,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505560,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":505019,"kb_avail":93866280,"statfs":{"total":96636764160,"available":96119070720,"internally_reserved":0,"allocated":532480,"data_stored":175936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517140143},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":82878464,"data_stored":82874985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:05.573 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:26:05.573 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20562, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:03.298296+0000', 'last_change': '2024-09-11T01:26:01.184322+0000', 'last_active': '2024-09-11T01:26:03.298296+0000', 'last_peered': '2024-09-11T01:26:03.298296+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:26:03.298296+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3758, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 230, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3758}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:05.574 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:26:05.574 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20037, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:02.982813+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:26:02.982813+0000', 'last_peered': '2024-09-11T01:26:02.982813+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:26:02.982813+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:05.574 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:26:05.967 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:05.968 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 7744/6 objects degraded (129066.667%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":902,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1502126080,"bytes_avail":288408166400,"bytes_total":289910292480,"degraded_objects":7744,"degraded_total":6,"degraded_ratio":1290.6666666666667,"recovering_objects_per_sec":154,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1239,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:26:05.991 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:26:05.991 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:06.339 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:06.339 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:26:06.357 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":507,"stamp":"2024-09-11T01:26:04.869422+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7744,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39896,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1466920,"kb_used_data":146168,"kb_used_omap":12,"kb_used_meta":1320691,"kb_avail":281648600,"statfs":{"total":289910292480,"available":288408166400,"internally_reserved":0,"allocated":149676032,"data_stored":148629805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1352387588},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1239,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1239,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001758"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28536,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985357+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:02.985357+0000","last_peered":"2024-09-11T01:26:02.985357+0000","last_clean":"2024-09-11T01:26:02.985357+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:02.985357+0000","last_undegraded":"2024-09-11T01:26:02.985357+0000","last_fullsized":"2024-09-11T01:26:02.985357+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20490,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255153+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:25:23.255153+0000","last_peered":"2024-09-11T01:25:23.255153+0000","last_clean":"2024-09-11T01:25:23.255153+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:25:23.255153+0000","last_undegraded":"2024-09-11T01:25:23.255153+0000","last_fullsized":"2024-09-11T01:25:23.255153+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20317,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20240,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255347+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:25:23.255347+0000","last_peered":"2024-09-11T01:25:23.255347+0000","last_clean":"2024-09-11T01:25:23.255347+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:25:23.255347+0000","last_undegraded":"2024-09-11T01:25:23.255347+0000","last_fullsized":"2024-09-11T01:25:23.255347+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20177,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984547+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:02.984547+0000","last_peered":"2024-09-11T01:26:02.984547+0000","last_clean":"2024-09-11T01:26:02.984547+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:02.984547+0000","last_undegraded":"2024-09-11T01:26:02.984547+0000","last_fullsized":"2024-09-11T01:26:02.984547+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20423,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986863+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:02.986863+0000","last_peered":"2024-09-11T01:26:02.986863+0000","last_clean":"2024-09-11T01:26:02.986863+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:02.986863+0000","last_undegraded":"2024-09-11T01:26:02.986863+0000","last_fullsized":"2024-09-11T01:26:02.986863+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20562,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:03.298296+0000","last_change":"2024-09-11T01:26:01.184322+0000","last_active":"2024-09-11T01:26:03.298296+0000","last_peered":"2024-09-11T01:26:03.298296+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:26:03.298296+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3758,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":230,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3758}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982464+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:02.982464+0000","last_peered":"2024-09-11T01:26:02.982464+0000","last_clean":"2024-09-11T01:26:02.982464+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:02.982464+0000","last_undegraded":"2024-09-11T01:26:02.982464+0000","last_fullsized":"2024-09-11T01:26:02.982464+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986745+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:02.986745+0000","last_peered":"2024-09-11T01:26:02.986745+0000","last_clean":"2024-09-11T01:26:02.986745+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:02.986745+0000","last_undegraded":"2024-09-11T01:26:02.986745+0000","last_fullsized":"2024-09-11T01:26:02.986745+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28021,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989204+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:02.989204+0000","last_peered":"2024-09-11T01:26:02.989204+0000","last_clean":"2024-09-11T01:26:02.989204+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:02.989204+0000","last_undegraded":"2024-09-11T01:26:02.989204+0000","last_fullsized":"2024-09-11T01:26:02.989204+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254905+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:25:23.254905+0000","last_peered":"2024-09-11T01:25:23.254905+0000","last_clean":"2024-09-11T01:25:23.254905+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:25:23.254905+0000","last_undegraded":"2024-09-11T01:25:23.254905+0000","last_fullsized":"2024-09-11T01:25:23.254905+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20037,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:02.982813+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:26:02.982813+0000","last_peered":"2024-09-11T01:26:02.982813+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:26:02.982813+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982616+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:02.982616+0000","last_peered":"2024-09-11T01:26:02.982616+0000","last_clean":"2024-09-11T01:26:02.982616+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:02.982616+0000","last_undegraded":"2024-09-11T01:26:02.982616+0000","last_fullsized":"2024-09-11T01:26:02.982616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28422,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.254938+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:25:23.254938+0000","last_peered":"2024-09-11T01:25:23.254938+0000","last_clean":"2024-09-11T01:25:23.254938+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:25:23.254938+0000","last_undegraded":"2024-09-11T01:25:23.254938+0000","last_fullsized":"2024-09-11T01:25:23.254938+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989042+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:02.989042+0000","last_peered":"2024-09-11T01:26:02.989042+0000","last_clean":"2024-09-11T01:26:02.989042+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:02.989042+0000","last_undegraded":"2024-09-11T01:26:02.989042+0000","last_fullsized":"2024-09-11T01:26:02.989042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27813,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255394+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:25:23.255394+0000","last_peered":"2024-09-11T01:25:23.255394+0000","last_clean":"2024-09-11T01:25:23.255394+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:25:23.255394+0000","last_undegraded":"2024-09-11T01:25:23.255394+0000","last_fullsized":"2024-09-11T01:25:23.255394+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20427,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989519+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:02.989519+0000","last_peered":"2024-09-11T01:26:02.989519+0000","last_clean":"2024-09-11T01:26:02.989519+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:02.989519+0000","last_undegraded":"2024-09-11T01:26:02.989519+0000","last_fullsized":"2024-09-11T01:26:02.989519+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989097+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:02.989097+0000","last_peered":"2024-09-11T01:26:02.989097+0000","last_clean":"2024-09-11T01:26:02.989097+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:02.989097+0000","last_undegraded":"2024-09-11T01:26:02.989097+0000","last_fullsized":"2024-09-11T01:26:02.989097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27955,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989371+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:02.989371+0000","last_peered":"2024-09-11T01:26:02.989371+0000","last_clean":"2024-09-11T01:26:02.989371+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:02.989371+0000","last_undegraded":"2024-09-11T01:26:02.989371+0000","last_fullsized":"2024-09-11T01:26:02.989371+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":106,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-11T01:25:23.255181+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:25:23.255181+0000","last_peered":"2024-09-11T01:25:23.255181+0000","last_clean":"2024-09-11T01:25:23.255181+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:25:23.255181+0000","last_undegraded":"2024-09-11T01:25:23.255181+0000","last_fullsized":"2024-09-11T01:25:23.255181+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24634,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989242+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:02.989242+0000","last_peered":"2024-09-11T01:26:02.989242+0000","last_clean":"2024-09-11T01:26:02.989242+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:02.989242+0000","last_undegraded":"2024-09-11T01:26:02.989242+0000","last_fullsized":"2024-09-11T01:26:02.989242+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989374+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:02.989374+0000","last_peered":"2024-09-11T01:26:02.989374+0000","last_clean":"2024-09-11T01:26:02.989374+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:02.989374+0000","last_undegraded":"2024-09-11T01:26:02.989374+0000","last_fullsized":"2024-09-11T01:26:02.989374+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20107,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:03.078750+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:03.078750+0000","last_peered":"2024-09-11T01:26:03.078750+0000","last_clean":"2024-09-11T01:26:03.078750+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:03.078750+0000","last_undegraded":"2024-09-11T01:26:03.078750+0000","last_fullsized":"2024-09-11T01:26:03.078750+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7744,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":39895,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147513344,"data_stored":147495949,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182574,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518672,"kb_used_data":63616,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853168,"statfs":{"total":96636764160,"available":96105644032,"internally_reserved":0,"allocated":65142784,"data_stored":64793198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986346,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":442688,"kb_used_data":82032,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93929152,"statfs":{"total":96636764160,"available":96183451648,"internally_reserved":0,"allocated":84000768,"data_stored":83660671,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659291,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505560,"kb_used_data":520,"kb_used_omap":4,"kb_used_meta":505019,"kb_avail":93866280,"statfs":{"total":96636764160,"available":96119070720,"internally_reserved":0,"allocated":532480,"data_stored":175936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517140143},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":82878464,"data_stored":82874985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:09.360 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:09.677 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:09.678 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:26:09.699 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":509,"stamp":"2024-09-11T01:26:08.870384+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7122,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40518,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1467876,"kb_used_data":142964,"kb_used_omap":12,"kb_used_meta":1324851,"kb_avail":281647644,"statfs":{"total":289910292480,"available":288407187456,"internally_reserved":0,"allocated":146395136,"data_stored":145327198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1356647428},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001649"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28536,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985357+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:02.985357+0000","last_peered":"2024-09-11T01:26:02.985357+0000","last_clean":"2024-09-11T01:26:02.985357+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:02.985357+0000","last_undegraded":"2024-09-11T01:26:02.985357+0000","last_fullsized":"2024-09-11T01:26:02.985357+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20494,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984919+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:26:02.984919+0000","last_peered":"2024-09-11T01:26:02.984919+0000","last_clean":"2024-09-11T01:26:02.984919+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:26:02.984919+0000","last_undegraded":"2024-09-11T01:26:02.984919+0000","last_fullsized":"2024-09-11T01:26:02.984919+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20317,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20244,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984629+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:26:02.984629+0000","last_peered":"2024-09-11T01:26:02.984629+0000","last_clean":"2024-09-11T01:26:02.984629+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:26:02.984629+0000","last_undegraded":"2024-09-11T01:26:02.984629+0000","last_fullsized":"2024-09-11T01:26:02.984629+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20177,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984547+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:02.984547+0000","last_peered":"2024-09-11T01:26:02.984547+0000","last_clean":"2024-09-11T01:26:02.984547+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:02.984547+0000","last_undegraded":"2024-09-11T01:26:02.984547+0000","last_fullsized":"2024-09-11T01:26:02.984547+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20423,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986863+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:02.986863+0000","last_peered":"2024-09-11T01:26:02.986863+0000","last_clean":"2024-09-11T01:26:02.986863+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:02.986863+0000","last_undegraded":"2024-09-11T01:26:02.986863+0000","last_fullsized":"2024-09-11T01:26:02.986863+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":21806,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:08.288889+0000","last_change":"2024-09-11T01:26:01.184322+0000","last_active":"2024-09-11T01:26:08.288889+0000","last_peered":"2024-09-11T01:26:08.288889+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:26:08.288889+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3136,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":852,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3136}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982464+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:02.982464+0000","last_peered":"2024-09-11T01:26:02.982464+0000","last_clean":"2024-09-11T01:26:02.982464+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:02.982464+0000","last_undegraded":"2024-09-11T01:26:02.982464+0000","last_fullsized":"2024-09-11T01:26:02.982464+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986745+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:02.986745+0000","last_peered":"2024-09-11T01:26:02.986745+0000","last_clean":"2024-09-11T01:26:02.986745+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:02.986745+0000","last_undegraded":"2024-09-11T01:26:02.986745+0000","last_fullsized":"2024-09-11T01:26:02.986745+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28021,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989204+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:02.989204+0000","last_peered":"2024-09-11T01:26:02.989204+0000","last_clean":"2024-09-11T01:26:02.989204+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:02.989204+0000","last_undegraded":"2024-09-11T01:26:02.989204+0000","last_fullsized":"2024-09-11T01:26:02.989204+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982959+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:26:02.982959+0000","last_peered":"2024-09-11T01:26:02.982959+0000","last_clean":"2024-09-11T01:26:02.982959+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:26:02.982959+0000","last_undegraded":"2024-09-11T01:26:02.982959+0000","last_fullsized":"2024-09-11T01:26:02.982959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20037,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:02.982813+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:26:02.982813+0000","last_peered":"2024-09-11T01:26:02.982813+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:26:02.982813+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982616+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:02.982616+0000","last_peered":"2024-09-11T01:26:02.982616+0000","last_clean":"2024-09-11T01:26:02.982616+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:02.982616+0000","last_undegraded":"2024-09-11T01:26:02.982616+0000","last_fullsized":"2024-09-11T01:26:02.982616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28426,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.983065+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:26:02.983065+0000","last_peered":"2024-09-11T01:26:02.983065+0000","last_clean":"2024-09-11T01:26:02.983065+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:26:02.983065+0000","last_undegraded":"2024-09-11T01:26:02.983065+0000","last_fullsized":"2024-09-11T01:26:02.983065+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989042+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:02.989042+0000","last_peered":"2024-09-11T01:26:02.989042+0000","last_clean":"2024-09-11T01:26:02.989042+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:02.989042+0000","last_undegraded":"2024-09-11T01:26:02.989042+0000","last_fullsized":"2024-09-11T01:26:02.989042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27817,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982725+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:26:02.982725+0000","last_peered":"2024-09-11T01:26:02.982725+0000","last_clean":"2024-09-11T01:26:02.982725+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:26:02.982725+0000","last_undegraded":"2024-09-11T01:26:02.982725+0000","last_fullsized":"2024-09-11T01:26:02.982725+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20427,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989519+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:02.989519+0000","last_peered":"2024-09-11T01:26:02.989519+0000","last_clean":"2024-09-11T01:26:02.989519+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:02.989519+0000","last_undegraded":"2024-09-11T01:26:02.989519+0000","last_fullsized":"2024-09-11T01:26:02.989519+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989097+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:02.989097+0000","last_peered":"2024-09-11T01:26:02.989097+0000","last_clean":"2024-09-11T01:26:02.989097+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:02.989097+0000","last_undegraded":"2024-09-11T01:26:02.989097+0000","last_fullsized":"2024-09-11T01:26:02.989097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27955,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989371+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:02.989371+0000","last_peered":"2024-09-11T01:26:02.989371+0000","last_clean":"2024-09-11T01:26:02.989371+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:02.989371+0000","last_undegraded":"2024-09-11T01:26:02.989371+0000","last_fullsized":"2024-09-11T01:26:02.989371+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985710+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:26:02.985710+0000","last_peered":"2024-09-11T01:26:02.985710+0000","last_clean":"2024-09-11T01:26:02.985710+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:26:02.985710+0000","last_undegraded":"2024-09-11T01:26:02.985710+0000","last_fullsized":"2024-09-11T01:26:02.985710+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24634,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989242+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:02.989242+0000","last_peered":"2024-09-11T01:26:02.989242+0000","last_clean":"2024-09-11T01:26:02.989242+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:02.989242+0000","last_undegraded":"2024-09-11T01:26:02.989242+0000","last_fullsized":"2024-09-11T01:26:02.989242+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989374+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:02.989374+0000","last_peered":"2024-09-11T01:26:02.989374+0000","last_clean":"2024-09-11T01:26:02.989374+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:02.989374+0000","last_undegraded":"2024-09-11T01:26:02.989374+0000","last_fullsized":"2024-09-11T01:26:02.989374+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20107,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:03.078750+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:03.078750+0000","last_peered":"2024-09-11T01:26:03.078750+0000","last_clean":"2024-09-11T01:26:03.078750+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:03.078750+0000","last_undegraded":"2024-09-11T01:26:03.078750+0000","last_fullsized":"2024-09-11T01:26:03.078750+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7122,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40517,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":144076800,"data_stored":144059405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182575,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518680,"kb_used_data":63624,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853160,"statfs":{"total":96636764160,"available":96105635840,"internally_reserved":0,"allocated":65150976,"data_stored":64795957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986347,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":439468,"kb_used_data":78812,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93932372,"statfs":{"total":96636764160,"available":96186748928,"internally_reserved":0,"allocated":80703488,"data_stored":80352546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659292,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509728,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":509179,"kb_avail":93862112,"statfs":{"total":96636764160,"available":96114802688,"internally_reserved":0,"allocated":540672,"data_stored":178695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521399983},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":79441920,"data_stored":79438441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:09.700 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:10.020 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:10.020 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:26:10.040 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":509,"stamp":"2024-09-11T01:26:08.870384+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7122,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40518,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1467876,"kb_used_data":142964,"kb_used_omap":12,"kb_used_meta":1324851,"kb_avail":281647644,"statfs":{"total":289910292480,"available":288407187456,"internally_reserved":0,"allocated":146395136,"data_stored":145327198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1356647428},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001649"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28536,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985357+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:02.985357+0000","last_peered":"2024-09-11T01:26:02.985357+0000","last_clean":"2024-09-11T01:26:02.985357+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:02.985357+0000","last_undegraded":"2024-09-11T01:26:02.985357+0000","last_fullsized":"2024-09-11T01:26:02.985357+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20494,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984919+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:26:02.984919+0000","last_peered":"2024-09-11T01:26:02.984919+0000","last_clean":"2024-09-11T01:26:02.984919+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:26:02.984919+0000","last_undegraded":"2024-09-11T01:26:02.984919+0000","last_fullsized":"2024-09-11T01:26:02.984919+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20317,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20244,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984629+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:26:02.984629+0000","last_peered":"2024-09-11T01:26:02.984629+0000","last_clean":"2024-09-11T01:26:02.984629+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:26:02.984629+0000","last_undegraded":"2024-09-11T01:26:02.984629+0000","last_fullsized":"2024-09-11T01:26:02.984629+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20177,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984547+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:02.984547+0000","last_peered":"2024-09-11T01:26:02.984547+0000","last_clean":"2024-09-11T01:26:02.984547+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:02.984547+0000","last_undegraded":"2024-09-11T01:26:02.984547+0000","last_fullsized":"2024-09-11T01:26:02.984547+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20423,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986863+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:02.986863+0000","last_peered":"2024-09-11T01:26:02.986863+0000","last_clean":"2024-09-11T01:26:02.986863+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:02.986863+0000","last_undegraded":"2024-09-11T01:26:02.986863+0000","last_fullsized":"2024-09-11T01:26:02.986863+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":21806,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:08.288889+0000","last_change":"2024-09-11T01:26:01.184322+0000","last_active":"2024-09-11T01:26:08.288889+0000","last_peered":"2024-09-11T01:26:08.288889+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:26:08.288889+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3136,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":852,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3136}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982464+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:02.982464+0000","last_peered":"2024-09-11T01:26:02.982464+0000","last_clean":"2024-09-11T01:26:02.982464+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:02.982464+0000","last_undegraded":"2024-09-11T01:26:02.982464+0000","last_fullsized":"2024-09-11T01:26:02.982464+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986745+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:02.986745+0000","last_peered":"2024-09-11T01:26:02.986745+0000","last_clean":"2024-09-11T01:26:02.986745+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:02.986745+0000","last_undegraded":"2024-09-11T01:26:02.986745+0000","last_fullsized":"2024-09-11T01:26:02.986745+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28021,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989204+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:02.989204+0000","last_peered":"2024-09-11T01:26:02.989204+0000","last_clean":"2024-09-11T01:26:02.989204+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:02.989204+0000","last_undegraded":"2024-09-11T01:26:02.989204+0000","last_fullsized":"2024-09-11T01:26:02.989204+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982959+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:26:02.982959+0000","last_peered":"2024-09-11T01:26:02.982959+0000","last_clean":"2024-09-11T01:26:02.982959+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:26:02.982959+0000","last_undegraded":"2024-09-11T01:26:02.982959+0000","last_fullsized":"2024-09-11T01:26:02.982959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20037,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:02.982813+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:26:02.982813+0000","last_peered":"2024-09-11T01:26:02.982813+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:26:02.982813+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982616+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:02.982616+0000","last_peered":"2024-09-11T01:26:02.982616+0000","last_clean":"2024-09-11T01:26:02.982616+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:02.982616+0000","last_undegraded":"2024-09-11T01:26:02.982616+0000","last_fullsized":"2024-09-11T01:26:02.982616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28426,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.983065+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:26:02.983065+0000","last_peered":"2024-09-11T01:26:02.983065+0000","last_clean":"2024-09-11T01:26:02.983065+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:26:02.983065+0000","last_undegraded":"2024-09-11T01:26:02.983065+0000","last_fullsized":"2024-09-11T01:26:02.983065+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989042+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:02.989042+0000","last_peered":"2024-09-11T01:26:02.989042+0000","last_clean":"2024-09-11T01:26:02.989042+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:02.989042+0000","last_undegraded":"2024-09-11T01:26:02.989042+0000","last_fullsized":"2024-09-11T01:26:02.989042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27817,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982725+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:26:02.982725+0000","last_peered":"2024-09-11T01:26:02.982725+0000","last_clean":"2024-09-11T01:26:02.982725+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:26:02.982725+0000","last_undegraded":"2024-09-11T01:26:02.982725+0000","last_fullsized":"2024-09-11T01:26:02.982725+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20427,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989519+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:02.989519+0000","last_peered":"2024-09-11T01:26:02.989519+0000","last_clean":"2024-09-11T01:26:02.989519+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:02.989519+0000","last_undegraded":"2024-09-11T01:26:02.989519+0000","last_fullsized":"2024-09-11T01:26:02.989519+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989097+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:02.989097+0000","last_peered":"2024-09-11T01:26:02.989097+0000","last_clean":"2024-09-11T01:26:02.989097+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:02.989097+0000","last_undegraded":"2024-09-11T01:26:02.989097+0000","last_fullsized":"2024-09-11T01:26:02.989097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27955,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989371+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:02.989371+0000","last_peered":"2024-09-11T01:26:02.989371+0000","last_clean":"2024-09-11T01:26:02.989371+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:02.989371+0000","last_undegraded":"2024-09-11T01:26:02.989371+0000","last_fullsized":"2024-09-11T01:26:02.989371+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985710+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:26:02.985710+0000","last_peered":"2024-09-11T01:26:02.985710+0000","last_clean":"2024-09-11T01:26:02.985710+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:26:02.985710+0000","last_undegraded":"2024-09-11T01:26:02.985710+0000","last_fullsized":"2024-09-11T01:26:02.985710+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24634,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989242+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:02.989242+0000","last_peered":"2024-09-11T01:26:02.989242+0000","last_clean":"2024-09-11T01:26:02.989242+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:02.989242+0000","last_undegraded":"2024-09-11T01:26:02.989242+0000","last_fullsized":"2024-09-11T01:26:02.989242+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989374+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:02.989374+0000","last_peered":"2024-09-11T01:26:02.989374+0000","last_clean":"2024-09-11T01:26:02.989374+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:02.989374+0000","last_undegraded":"2024-09-11T01:26:02.989374+0000","last_fullsized":"2024-09-11T01:26:02.989374+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20107,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:03.078750+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:03.078750+0000","last_peered":"2024-09-11T01:26:03.078750+0000","last_clean":"2024-09-11T01:26:03.078750+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:03.078750+0000","last_undegraded":"2024-09-11T01:26:03.078750+0000","last_fullsized":"2024-09-11T01:26:03.078750+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7122,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40517,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":144076800,"data_stored":144059405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182575,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518680,"kb_used_data":63624,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853160,"statfs":{"total":96636764160,"available":96105635840,"internally_reserved":0,"allocated":65150976,"data_stored":64795957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986347,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":439468,"kb_used_data":78812,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93932372,"statfs":{"total":96636764160,"available":96186748928,"internally_reserved":0,"allocated":80703488,"data_stored":80352546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659292,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509728,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":509179,"kb_avail":93862112,"statfs":{"total":96636764160,"available":96114802688,"internally_reserved":0,"allocated":540672,"data_stored":178695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521399983},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":79441920,"data_stored":79438441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:10.041 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:26:10.041 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 21806, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:08.288889+0000', 'last_change': '2024-09-11T01:26:01.184322+0000', 'last_active': '2024-09-11T01:26:08.288889+0000', 'last_peered': '2024-09-11T01:26:08.288889+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:26:08.288889+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3136, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 852, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3136}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:10.041 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:26:10.042 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20037, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:02.982813+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:26:02.982813+0000', 'last_peered': '2024-09-11T01:26:02.982813+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:26:02.982813+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:10.042 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:26:10.440 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:10.440 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 7122/6 objects degraded (118700.000%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":906,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1503105024,"bytes_avail":288407187456,"bytes_total":289910292480,"degraded_objects":7122,"degraded_total":6,"degraded_ratio":1187,"recovering_objects_per_sec":157,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1257,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:26:10.459 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:26:10.460 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:10.796 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:10.796 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:26:10.816 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":509,"stamp":"2024-09-11T01:26:08.870384+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7122,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40518,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1467876,"kb_used_data":142964,"kb_used_omap":12,"kb_used_meta":1324851,"kb_avail":281647644,"statfs":{"total":289910292480,"available":288407187456,"internally_reserved":0,"allocated":146395136,"data_stored":145327198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1356647428},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001649"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28536,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985357+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:02.985357+0000","last_peered":"2024-09-11T01:26:02.985357+0000","last_clean":"2024-09-11T01:26:02.985357+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:02.985357+0000","last_undegraded":"2024-09-11T01:26:02.985357+0000","last_fullsized":"2024-09-11T01:26:02.985357+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20494,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984919+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:26:02.984919+0000","last_peered":"2024-09-11T01:26:02.984919+0000","last_clean":"2024-09-11T01:26:02.984919+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:26:02.984919+0000","last_undegraded":"2024-09-11T01:26:02.984919+0000","last_fullsized":"2024-09-11T01:26:02.984919+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20317,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20244,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984629+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:26:02.984629+0000","last_peered":"2024-09-11T01:26:02.984629+0000","last_clean":"2024-09-11T01:26:02.984629+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:26:02.984629+0000","last_undegraded":"2024-09-11T01:26:02.984629+0000","last_fullsized":"2024-09-11T01:26:02.984629+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20177,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984547+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:02.984547+0000","last_peered":"2024-09-11T01:26:02.984547+0000","last_clean":"2024-09-11T01:26:02.984547+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:02.984547+0000","last_undegraded":"2024-09-11T01:26:02.984547+0000","last_fullsized":"2024-09-11T01:26:02.984547+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20423,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986863+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:02.986863+0000","last_peered":"2024-09-11T01:26:02.986863+0000","last_clean":"2024-09-11T01:26:02.986863+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:02.986863+0000","last_undegraded":"2024-09-11T01:26:02.986863+0000","last_fullsized":"2024-09-11T01:26:02.986863+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":21806,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:08.288889+0000","last_change":"2024-09-11T01:26:01.184322+0000","last_active":"2024-09-11T01:26:08.288889+0000","last_peered":"2024-09-11T01:26:08.288889+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:26:08.288889+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3136,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":852,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3136}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982464+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:02.982464+0000","last_peered":"2024-09-11T01:26:02.982464+0000","last_clean":"2024-09-11T01:26:02.982464+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:02.982464+0000","last_undegraded":"2024-09-11T01:26:02.982464+0000","last_fullsized":"2024-09-11T01:26:02.982464+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986745+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:02.986745+0000","last_peered":"2024-09-11T01:26:02.986745+0000","last_clean":"2024-09-11T01:26:02.986745+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:02.986745+0000","last_undegraded":"2024-09-11T01:26:02.986745+0000","last_fullsized":"2024-09-11T01:26:02.986745+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28021,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989204+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:02.989204+0000","last_peered":"2024-09-11T01:26:02.989204+0000","last_clean":"2024-09-11T01:26:02.989204+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:02.989204+0000","last_undegraded":"2024-09-11T01:26:02.989204+0000","last_fullsized":"2024-09-11T01:26:02.989204+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982959+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:26:02.982959+0000","last_peered":"2024-09-11T01:26:02.982959+0000","last_clean":"2024-09-11T01:26:02.982959+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:26:02.982959+0000","last_undegraded":"2024-09-11T01:26:02.982959+0000","last_fullsized":"2024-09-11T01:26:02.982959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20037,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:02.982813+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:26:02.982813+0000","last_peered":"2024-09-11T01:26:02.982813+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:26:02.982813+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982616+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:02.982616+0000","last_peered":"2024-09-11T01:26:02.982616+0000","last_clean":"2024-09-11T01:26:02.982616+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:02.982616+0000","last_undegraded":"2024-09-11T01:26:02.982616+0000","last_fullsized":"2024-09-11T01:26:02.982616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28426,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.983065+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:26:02.983065+0000","last_peered":"2024-09-11T01:26:02.983065+0000","last_clean":"2024-09-11T01:26:02.983065+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:26:02.983065+0000","last_undegraded":"2024-09-11T01:26:02.983065+0000","last_fullsized":"2024-09-11T01:26:02.983065+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989042+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:02.989042+0000","last_peered":"2024-09-11T01:26:02.989042+0000","last_clean":"2024-09-11T01:26:02.989042+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:02.989042+0000","last_undegraded":"2024-09-11T01:26:02.989042+0000","last_fullsized":"2024-09-11T01:26:02.989042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27817,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982725+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:26:02.982725+0000","last_peered":"2024-09-11T01:26:02.982725+0000","last_clean":"2024-09-11T01:26:02.982725+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:26:02.982725+0000","last_undegraded":"2024-09-11T01:26:02.982725+0000","last_fullsized":"2024-09-11T01:26:02.982725+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20427,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989519+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:02.989519+0000","last_peered":"2024-09-11T01:26:02.989519+0000","last_clean":"2024-09-11T01:26:02.989519+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:02.989519+0000","last_undegraded":"2024-09-11T01:26:02.989519+0000","last_fullsized":"2024-09-11T01:26:02.989519+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989097+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:02.989097+0000","last_peered":"2024-09-11T01:26:02.989097+0000","last_clean":"2024-09-11T01:26:02.989097+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:02.989097+0000","last_undegraded":"2024-09-11T01:26:02.989097+0000","last_fullsized":"2024-09-11T01:26:02.989097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27955,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989371+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:02.989371+0000","last_peered":"2024-09-11T01:26:02.989371+0000","last_clean":"2024-09-11T01:26:02.989371+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:02.989371+0000","last_undegraded":"2024-09-11T01:26:02.989371+0000","last_fullsized":"2024-09-11T01:26:02.989371+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985710+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:26:02.985710+0000","last_peered":"2024-09-11T01:26:02.985710+0000","last_clean":"2024-09-11T01:26:02.985710+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:26:02.985710+0000","last_undegraded":"2024-09-11T01:26:02.985710+0000","last_fullsized":"2024-09-11T01:26:02.985710+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24634,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989242+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:02.989242+0000","last_peered":"2024-09-11T01:26:02.989242+0000","last_clean":"2024-09-11T01:26:02.989242+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:02.989242+0000","last_undegraded":"2024-09-11T01:26:02.989242+0000","last_fullsized":"2024-09-11T01:26:02.989242+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989374+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:02.989374+0000","last_peered":"2024-09-11T01:26:02.989374+0000","last_clean":"2024-09-11T01:26:02.989374+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:02.989374+0000","last_undegraded":"2024-09-11T01:26:02.989374+0000","last_fullsized":"2024-09-11T01:26:02.989374+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20107,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:03.078750+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:03.078750+0000","last_peered":"2024-09-11T01:26:03.078750+0000","last_clean":"2024-09-11T01:26:03.078750+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:03.078750+0000","last_undegraded":"2024-09-11T01:26:03.078750+0000","last_fullsized":"2024-09-11T01:26:03.078750+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7122,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40517,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":144076800,"data_stored":144059405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182575,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518680,"kb_used_data":63624,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853160,"statfs":{"total":96636764160,"available":96105635840,"internally_reserved":0,"allocated":65150976,"data_stored":64795957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986347,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":439468,"kb_used_data":78812,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93932372,"statfs":{"total":96636764160,"available":96186748928,"internally_reserved":0,"allocated":80703488,"data_stored":80352546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659292,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509728,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":509179,"kb_avail":93862112,"statfs":{"total":96636764160,"available":96114802688,"internally_reserved":0,"allocated":540672,"data_stored":178695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521399983},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":79441920,"data_stored":79438441,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:13.820 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:14.150 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:14.151 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:26:14.173 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":511,"stamp":"2024-09-11T01:26:12.871513+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7122,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40518,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1464276,"kb_used_data":139364,"kb_used_omap":12,"kb_used_meta":1324851,"kb_avail":281651244,"statfs":{"total":289910292480,"available":288410873856,"internally_reserved":0,"allocated":142708736,"data_stored":141640798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1356647428},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002463"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28536,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985357+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:02.985357+0000","last_peered":"2024-09-11T01:26:02.985357+0000","last_clean":"2024-09-11T01:26:02.985357+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:02.985357+0000","last_undegraded":"2024-09-11T01:26:02.985357+0000","last_fullsized":"2024-09-11T01:26:02.985357+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20494,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984919+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:26:02.984919+0000","last_peered":"2024-09-11T01:26:02.984919+0000","last_clean":"2024-09-11T01:26:02.984919+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:26:02.984919+0000","last_undegraded":"2024-09-11T01:26:02.984919+0000","last_fullsized":"2024-09-11T01:26:02.984919+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20317,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20244,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984629+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:26:02.984629+0000","last_peered":"2024-09-11T01:26:02.984629+0000","last_clean":"2024-09-11T01:26:02.984629+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:26:02.984629+0000","last_undegraded":"2024-09-11T01:26:02.984629+0000","last_fullsized":"2024-09-11T01:26:02.984629+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20177,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984547+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:02.984547+0000","last_peered":"2024-09-11T01:26:02.984547+0000","last_clean":"2024-09-11T01:26:02.984547+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:02.984547+0000","last_undegraded":"2024-09-11T01:26:02.984547+0000","last_fullsized":"2024-09-11T01:26:02.984547+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20423,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986863+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:02.986863+0000","last_peered":"2024-09-11T01:26:02.986863+0000","last_clean":"2024-09-11T01:26:02.986863+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:02.986863+0000","last_undegraded":"2024-09-11T01:26:02.986863+0000","last_fullsized":"2024-09-11T01:26:02.986863+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":21806,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:08.288889+0000","last_change":"2024-09-11T01:26:01.184322+0000","last_active":"2024-09-11T01:26:08.288889+0000","last_peered":"2024-09-11T01:26:08.288889+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:26:08.288889+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3136,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":852,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3136}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982464+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:02.982464+0000","last_peered":"2024-09-11T01:26:02.982464+0000","last_clean":"2024-09-11T01:26:02.982464+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:02.982464+0000","last_undegraded":"2024-09-11T01:26:02.982464+0000","last_fullsized":"2024-09-11T01:26:02.982464+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986745+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:02.986745+0000","last_peered":"2024-09-11T01:26:02.986745+0000","last_clean":"2024-09-11T01:26:02.986745+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:02.986745+0000","last_undegraded":"2024-09-11T01:26:02.986745+0000","last_fullsized":"2024-09-11T01:26:02.986745+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28021,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989204+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:02.989204+0000","last_peered":"2024-09-11T01:26:02.989204+0000","last_clean":"2024-09-11T01:26:02.989204+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:02.989204+0000","last_undegraded":"2024-09-11T01:26:02.989204+0000","last_fullsized":"2024-09-11T01:26:02.989204+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982959+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:26:02.982959+0000","last_peered":"2024-09-11T01:26:02.982959+0000","last_clean":"2024-09-11T01:26:02.982959+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:26:02.982959+0000","last_undegraded":"2024-09-11T01:26:02.982959+0000","last_fullsized":"2024-09-11T01:26:02.982959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20037,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:02.982813+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:26:02.982813+0000","last_peered":"2024-09-11T01:26:02.982813+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:26:02.982813+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982616+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:02.982616+0000","last_peered":"2024-09-11T01:26:02.982616+0000","last_clean":"2024-09-11T01:26:02.982616+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:02.982616+0000","last_undegraded":"2024-09-11T01:26:02.982616+0000","last_fullsized":"2024-09-11T01:26:02.982616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28426,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.983065+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:26:02.983065+0000","last_peered":"2024-09-11T01:26:02.983065+0000","last_clean":"2024-09-11T01:26:02.983065+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:26:02.983065+0000","last_undegraded":"2024-09-11T01:26:02.983065+0000","last_fullsized":"2024-09-11T01:26:02.983065+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989042+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:02.989042+0000","last_peered":"2024-09-11T01:26:02.989042+0000","last_clean":"2024-09-11T01:26:02.989042+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:02.989042+0000","last_undegraded":"2024-09-11T01:26:02.989042+0000","last_fullsized":"2024-09-11T01:26:02.989042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982786+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:26:02.982786+0000","last_peered":"2024-09-11T01:26:02.982786+0000","last_clean":"2024-09-11T01:26:02.982786+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:26:02.982786+0000","last_undegraded":"2024-09-11T01:26:02.982786+0000","last_fullsized":"2024-09-11T01:26:02.982786+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20427,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989519+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:02.989519+0000","last_peered":"2024-09-11T01:26:02.989519+0000","last_clean":"2024-09-11T01:26:02.989519+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:02.989519+0000","last_undegraded":"2024-09-11T01:26:02.989519+0000","last_fullsized":"2024-09-11T01:26:02.989519+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989097+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:02.989097+0000","last_peered":"2024-09-11T01:26:02.989097+0000","last_clean":"2024-09-11T01:26:02.989097+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:02.989097+0000","last_undegraded":"2024-09-11T01:26:02.989097+0000","last_fullsized":"2024-09-11T01:26:02.989097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27955,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989371+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:02.989371+0000","last_peered":"2024-09-11T01:26:02.989371+0000","last_clean":"2024-09-11T01:26:02.989371+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:02.989371+0000","last_undegraded":"2024-09-11T01:26:02.989371+0000","last_fullsized":"2024-09-11T01:26:02.989371+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985710+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:26:02.985710+0000","last_peered":"2024-09-11T01:26:02.985710+0000","last_clean":"2024-09-11T01:26:02.985710+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:26:02.985710+0000","last_undegraded":"2024-09-11T01:26:02.985710+0000","last_fullsized":"2024-09-11T01:26:02.985710+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24634,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989242+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:02.989242+0000","last_peered":"2024-09-11T01:26:02.989242+0000","last_clean":"2024-09-11T01:26:02.989242+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:02.989242+0000","last_undegraded":"2024-09-11T01:26:02.989242+0000","last_fullsized":"2024-09-11T01:26:02.989242+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989374+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:02.989374+0000","last_peered":"2024-09-11T01:26:02.989374+0000","last_clean":"2024-09-11T01:26:02.989374+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:02.989374+0000","last_undegraded":"2024-09-11T01:26:02.989374+0000","last_fullsized":"2024-09-11T01:26:02.989374+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20107,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:03.078750+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:03.078750+0000","last_peered":"2024-09-11T01:26:03.078750+0000","last_clean":"2024-09-11T01:26:03.078750+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:03.078750+0000","last_undegraded":"2024-09-11T01:26:03.078750+0000","last_fullsized":"2024-09-11T01:26:03.078750+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7122,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40517,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":140390400,"data_stored":140373005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182575,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518680,"kb_used_data":63624,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853160,"statfs":{"total":96636764160,"available":96105635840,"internally_reserved":0,"allocated":65150976,"data_stored":64795957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986348,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":435868,"kb_used_data":75212,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93935972,"statfs":{"total":96636764160,"available":96190435328,"internally_reserved":0,"allocated":77017088,"data_stored":76666146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659292,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509728,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":509179,"kb_avail":93862112,"statfs":{"total":96636764160,"available":96114802688,"internally_reserved":0,"allocated":540672,"data_stored":178695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521399983},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":75755520,"data_stored":75752041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:14.174 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:14.504 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:14.504 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:26:14.522 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":511,"stamp":"2024-09-11T01:26:12.871513+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7122,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40518,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1464276,"kb_used_data":139364,"kb_used_omap":12,"kb_used_meta":1324851,"kb_avail":281651244,"statfs":{"total":289910292480,"available":288410873856,"internally_reserved":0,"allocated":142708736,"data_stored":141640798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1356647428},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1257,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1257,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002463"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28536,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985357+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:02.985357+0000","last_peered":"2024-09-11T01:26:02.985357+0000","last_clean":"2024-09-11T01:26:02.985357+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:02.985357+0000","last_undegraded":"2024-09-11T01:26:02.985357+0000","last_fullsized":"2024-09-11T01:26:02.985357+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20494,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984919+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:26:02.984919+0000","last_peered":"2024-09-11T01:26:02.984919+0000","last_clean":"2024-09-11T01:26:02.984919+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:26:02.984919+0000","last_undegraded":"2024-09-11T01:26:02.984919+0000","last_fullsized":"2024-09-11T01:26:02.984919+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20317,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20244,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984629+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:26:02.984629+0000","last_peered":"2024-09-11T01:26:02.984629+0000","last_clean":"2024-09-11T01:26:02.984629+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:26:02.984629+0000","last_undegraded":"2024-09-11T01:26:02.984629+0000","last_fullsized":"2024-09-11T01:26:02.984629+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20177,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984547+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:02.984547+0000","last_peered":"2024-09-11T01:26:02.984547+0000","last_clean":"2024-09-11T01:26:02.984547+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:02.984547+0000","last_undegraded":"2024-09-11T01:26:02.984547+0000","last_fullsized":"2024-09-11T01:26:02.984547+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20423,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986863+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:02.986863+0000","last_peered":"2024-09-11T01:26:02.986863+0000","last_clean":"2024-09-11T01:26:02.986863+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:02.986863+0000","last_undegraded":"2024-09-11T01:26:02.986863+0000","last_fullsized":"2024-09-11T01:26:02.986863+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":21806,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:08.288889+0000","last_change":"2024-09-11T01:26:01.184322+0000","last_active":"2024-09-11T01:26:08.288889+0000","last_peered":"2024-09-11T01:26:08.288889+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:26:08.288889+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3136,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":852,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3136}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982464+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:02.982464+0000","last_peered":"2024-09-11T01:26:02.982464+0000","last_clean":"2024-09-11T01:26:02.982464+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:02.982464+0000","last_undegraded":"2024-09-11T01:26:02.982464+0000","last_fullsized":"2024-09-11T01:26:02.982464+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986745+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:02.986745+0000","last_peered":"2024-09-11T01:26:02.986745+0000","last_clean":"2024-09-11T01:26:02.986745+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:02.986745+0000","last_undegraded":"2024-09-11T01:26:02.986745+0000","last_fullsized":"2024-09-11T01:26:02.986745+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28021,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989204+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:02.989204+0000","last_peered":"2024-09-11T01:26:02.989204+0000","last_clean":"2024-09-11T01:26:02.989204+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:02.989204+0000","last_undegraded":"2024-09-11T01:26:02.989204+0000","last_fullsized":"2024-09-11T01:26:02.989204+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982959+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:26:02.982959+0000","last_peered":"2024-09-11T01:26:02.982959+0000","last_clean":"2024-09-11T01:26:02.982959+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:26:02.982959+0000","last_undegraded":"2024-09-11T01:26:02.982959+0000","last_fullsized":"2024-09-11T01:26:02.982959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20037,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:02.982813+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:26:02.982813+0000","last_peered":"2024-09-11T01:26:02.982813+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:26:02.982813+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982616+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:02.982616+0000","last_peered":"2024-09-11T01:26:02.982616+0000","last_clean":"2024-09-11T01:26:02.982616+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:02.982616+0000","last_undegraded":"2024-09-11T01:26:02.982616+0000","last_fullsized":"2024-09-11T01:26:02.982616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28426,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.983065+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:26:02.983065+0000","last_peered":"2024-09-11T01:26:02.983065+0000","last_clean":"2024-09-11T01:26:02.983065+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:26:02.983065+0000","last_undegraded":"2024-09-11T01:26:02.983065+0000","last_fullsized":"2024-09-11T01:26:02.983065+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989042+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:02.989042+0000","last_peered":"2024-09-11T01:26:02.989042+0000","last_clean":"2024-09-11T01:26:02.989042+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:02.989042+0000","last_undegraded":"2024-09-11T01:26:02.989042+0000","last_fullsized":"2024-09-11T01:26:02.989042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982786+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:26:02.982786+0000","last_peered":"2024-09-11T01:26:02.982786+0000","last_clean":"2024-09-11T01:26:02.982786+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:26:02.982786+0000","last_undegraded":"2024-09-11T01:26:02.982786+0000","last_fullsized":"2024-09-11T01:26:02.982786+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20427,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989519+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:02.989519+0000","last_peered":"2024-09-11T01:26:02.989519+0000","last_clean":"2024-09-11T01:26:02.989519+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:02.989519+0000","last_undegraded":"2024-09-11T01:26:02.989519+0000","last_fullsized":"2024-09-11T01:26:02.989519+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989097+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:02.989097+0000","last_peered":"2024-09-11T01:26:02.989097+0000","last_clean":"2024-09-11T01:26:02.989097+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:02.989097+0000","last_undegraded":"2024-09-11T01:26:02.989097+0000","last_fullsized":"2024-09-11T01:26:02.989097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27955,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989371+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:02.989371+0000","last_peered":"2024-09-11T01:26:02.989371+0000","last_clean":"2024-09-11T01:26:02.989371+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:02.989371+0000","last_undegraded":"2024-09-11T01:26:02.989371+0000","last_fullsized":"2024-09-11T01:26:02.989371+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985710+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:26:02.985710+0000","last_peered":"2024-09-11T01:26:02.985710+0000","last_clean":"2024-09-11T01:26:02.985710+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:26:02.985710+0000","last_undegraded":"2024-09-11T01:26:02.985710+0000","last_fullsized":"2024-09-11T01:26:02.985710+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24634,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989242+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:02.989242+0000","last_peered":"2024-09-11T01:26:02.989242+0000","last_clean":"2024-09-11T01:26:02.989242+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:02.989242+0000","last_undegraded":"2024-09-11T01:26:02.989242+0000","last_fullsized":"2024-09-11T01:26:02.989242+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":114,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989374+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:02.989374+0000","last_peered":"2024-09-11T01:26:02.989374+0000","last_clean":"2024-09-11T01:26:02.989374+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:02.989374+0000","last_undegraded":"2024-09-11T01:26:02.989374+0000","last_fullsized":"2024-09-11T01:26:02.989374+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20107,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:03.078750+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:03.078750+0000","last_peered":"2024-09-11T01:26:03.078750+0000","last_clean":"2024-09-11T01:26:03.078750+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:03.078750+0000","last_undegraded":"2024-09-11T01:26:03.078750+0000","last_fullsized":"2024-09-11T01:26:03.078750+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":7122,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40517,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":140390400,"data_stored":140373005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182575,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518680,"kb_used_data":63624,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853160,"statfs":{"total":96636764160,"available":96105635840,"internally_reserved":0,"allocated":65150976,"data_stored":64795957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986348,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":435868,"kb_used_data":75212,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93935972,"statfs":{"total":96636764160,"available":96190435328,"internally_reserved":0,"allocated":77017088,"data_stored":76666146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659292,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509728,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":509179,"kb_avail":93862112,"statfs":{"total":96636764160,"available":96114802688,"internally_reserved":0,"allocated":540672,"data_stored":178695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521399983},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":75755520,"data_stored":75752041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:14.524 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:26:14.524 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 21806, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:08.288889+0000', 'last_change': '2024-09-11T01:26:01.184322+0000', 'last_active': '2024-09-11T01:26:08.288889+0000', 'last_peered': '2024-09-11T01:26:08.288889+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:26:08.288889+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3136, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 852, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3136}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:14.524 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:26:14.524 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20037, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:02.982813+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:26:02.982813+0000', 'last_peered': '2024-09-11T01:26:02.982813+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:26:02.982813+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:14.525 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:26:14.920 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:14.920 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 7122/6 objects degraded (118700.000%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":911,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1499418624,"bytes_avail":288410873856,"bytes_total":289910292480,"degraded_objects":7122,"degraded_total":6,"degraded_ratio":1187,"recovering_objects_per_sec":125,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1257,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:26:14.937 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:26:14.938 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:15.245 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:15.246 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:26:15.264 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":512,"stamp":"2024-09-11T01:26:14.872229+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6818,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40822,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1451796,"kb_used_data":139364,"kb_used_omap":12,"kb_used_meta":1312371,"kb_avail":281663724,"statfs":{"total":289910292480,"available":288423653376,"internally_reserved":0,"allocated":142708736,"data_stored":141640798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1343867908},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1561,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1561,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.891728"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28536,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985357+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:02.985357+0000","last_peered":"2024-09-11T01:26:02.985357+0000","last_clean":"2024-09-11T01:26:02.985357+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:02.985357+0000","last_undegraded":"2024-09-11T01:26:02.985357+0000","last_fullsized":"2024-09-11T01:26:02.985357+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20494,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984919+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:26:02.984919+0000","last_peered":"2024-09-11T01:26:02.984919+0000","last_clean":"2024-09-11T01:26:02.984919+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:26:02.984919+0000","last_undegraded":"2024-09-11T01:26:02.984919+0000","last_fullsized":"2024-09-11T01:26:02.984919+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20317,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20244,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984629+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:26:02.984629+0000","last_peered":"2024-09-11T01:26:02.984629+0000","last_clean":"2024-09-11T01:26:02.984629+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:26:02.984629+0000","last_undegraded":"2024-09-11T01:26:02.984629+0000","last_fullsized":"2024-09-11T01:26:02.984629+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20177,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984547+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:02.984547+0000","last_peered":"2024-09-11T01:26:02.984547+0000","last_clean":"2024-09-11T01:26:02.984547+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:02.984547+0000","last_undegraded":"2024-09-11T01:26:02.984547+0000","last_fullsized":"2024-09-11T01:26:02.984547+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20423,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986863+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:02.986863+0000","last_peered":"2024-09-11T01:26:02.986863+0000","last_clean":"2024-09-11T01:26:02.986863+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:02.986863+0000","last_undegraded":"2024-09-11T01:26:02.986863+0000","last_fullsized":"2024-09-11T01:26:02.986863+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":22414,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:13.291085+0000","last_change":"2024-09-11T01:26:01.184322+0000","last_active":"2024-09-11T01:26:13.291085+0000","last_peered":"2024-09-11T01:26:13.291085+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:26:13.291085+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2832,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1156,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2832}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982464+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:02.982464+0000","last_peered":"2024-09-11T01:26:02.982464+0000","last_clean":"2024-09-11T01:26:02.982464+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:02.982464+0000","last_undegraded":"2024-09-11T01:26:02.982464+0000","last_fullsized":"2024-09-11T01:26:02.982464+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986745+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:02.986745+0000","last_peered":"2024-09-11T01:26:02.986745+0000","last_clean":"2024-09-11T01:26:02.986745+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:02.986745+0000","last_undegraded":"2024-09-11T01:26:02.986745+0000","last_fullsized":"2024-09-11T01:26:02.986745+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28021,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989204+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:02.989204+0000","last_peered":"2024-09-11T01:26:02.989204+0000","last_clean":"2024-09-11T01:26:02.989204+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:02.989204+0000","last_undegraded":"2024-09-11T01:26:02.989204+0000","last_fullsized":"2024-09-11T01:26:02.989204+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982959+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:26:02.982959+0000","last_peered":"2024-09-11T01:26:02.982959+0000","last_clean":"2024-09-11T01:26:02.982959+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:26:02.982959+0000","last_undegraded":"2024-09-11T01:26:02.982959+0000","last_fullsized":"2024-09-11T01:26:02.982959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20037,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:02.982813+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:26:02.982813+0000","last_peered":"2024-09-11T01:26:02.982813+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:26:02.982813+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982616+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:02.982616+0000","last_peered":"2024-09-11T01:26:02.982616+0000","last_clean":"2024-09-11T01:26:02.982616+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:02.982616+0000","last_undegraded":"2024-09-11T01:26:02.982616+0000","last_fullsized":"2024-09-11T01:26:02.982616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28426,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.983065+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:26:02.983065+0000","last_peered":"2024-09-11T01:26:02.983065+0000","last_clean":"2024-09-11T01:26:02.983065+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:26:02.983065+0000","last_undegraded":"2024-09-11T01:26:02.983065+0000","last_fullsized":"2024-09-11T01:26:02.983065+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989042+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:02.989042+0000","last_peered":"2024-09-11T01:26:02.989042+0000","last_clean":"2024-09-11T01:26:02.989042+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:02.989042+0000","last_undegraded":"2024-09-11T01:26:02.989042+0000","last_fullsized":"2024-09-11T01:26:02.989042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982786+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:26:02.982786+0000","last_peered":"2024-09-11T01:26:02.982786+0000","last_clean":"2024-09-11T01:26:02.982786+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:26:02.982786+0000","last_undegraded":"2024-09-11T01:26:02.982786+0000","last_fullsized":"2024-09-11T01:26:02.982786+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20427,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989519+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:02.989519+0000","last_peered":"2024-09-11T01:26:02.989519+0000","last_clean":"2024-09-11T01:26:02.989519+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:02.989519+0000","last_undegraded":"2024-09-11T01:26:02.989519+0000","last_fullsized":"2024-09-11T01:26:02.989519+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989097+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:02.989097+0000","last_peered":"2024-09-11T01:26:02.989097+0000","last_clean":"2024-09-11T01:26:02.989097+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:02.989097+0000","last_undegraded":"2024-09-11T01:26:02.989097+0000","last_fullsized":"2024-09-11T01:26:02.989097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27955,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989371+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:02.989371+0000","last_peered":"2024-09-11T01:26:02.989371+0000","last_clean":"2024-09-11T01:26:02.989371+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:02.989371+0000","last_undegraded":"2024-09-11T01:26:02.989371+0000","last_fullsized":"2024-09-11T01:26:02.989371+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985710+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:26:02.985710+0000","last_peered":"2024-09-11T01:26:02.985710+0000","last_clean":"2024-09-11T01:26:02.985710+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:26:02.985710+0000","last_undegraded":"2024-09-11T01:26:02.985710+0000","last_fullsized":"2024-09-11T01:26:02.985710+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24634,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989242+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:02.989242+0000","last_peered":"2024-09-11T01:26:02.989242+0000","last_clean":"2024-09-11T01:26:02.989242+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:02.989242+0000","last_undegraded":"2024-09-11T01:26:02.989242+0000","last_fullsized":"2024-09-11T01:26:02.989242+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989422+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:02.989422+0000","last_peered":"2024-09-11T01:26:02.989422+0000","last_clean":"2024-09-11T01:26:02.989422+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:02.989422+0000","last_undegraded":"2024-09-11T01:26:02.989422+0000","last_fullsized":"2024-09-11T01:26:02.989422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20107,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:03.078750+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:03.078750+0000","last_peered":"2024-09-11T01:26:03.078750+0000","last_clean":"2024-09-11T01:26:03.078750+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:03.078750+0000","last_undegraded":"2024-09-11T01:26:03.078750+0000","last_fullsized":"2024-09-11T01:26:03.078750+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6818,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40821,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":140390400,"data_stored":140373005,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182576,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518680,"kb_used_data":63624,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853160,"statfs":{"total":96636764160,"available":96105635840,"internally_reserved":0,"allocated":65150976,"data_stored":64795957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986348,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":435868,"kb_used_data":75212,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93935972,"statfs":{"total":96636764160,"available":96190435328,"internally_reserved":0,"allocated":77017088,"data_stored":76666146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659293,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497248,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":496699,"kb_avail":93874592,"statfs":{"total":96636764160,"available":96127582208,"internally_reserved":0,"allocated":540672,"data_stored":178695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508620463},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":75755520,"data_stored":75752041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:18.266 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:18.592 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:18.593 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:26:18.613 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":513,"stamp":"2024-09-11T01:26:16.872975+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6818,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40822,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1448196,"kb_used_data":135764,"kb_used_omap":12,"kb_used_meta":1312371,"kb_avail":281667324,"statfs":{"total":289910292480,"available":288427339776,"internally_reserved":0,"allocated":139022336,"data_stored":137954398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1343867908},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-926,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":926,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003552"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28536,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985357+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:02.985357+0000","last_peered":"2024-09-11T01:26:02.985357+0000","last_clean":"2024-09-11T01:26:02.985357+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:02.985357+0000","last_undegraded":"2024-09-11T01:26:02.985357+0000","last_fullsized":"2024-09-11T01:26:02.985357+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20494,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984919+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:26:02.984919+0000","last_peered":"2024-09-11T01:26:02.984919+0000","last_clean":"2024-09-11T01:26:02.984919+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:26:02.984919+0000","last_undegraded":"2024-09-11T01:26:02.984919+0000","last_fullsized":"2024-09-11T01:26:02.984919+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20317,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20244,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984629+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:26:02.984629+0000","last_peered":"2024-09-11T01:26:02.984629+0000","last_clean":"2024-09-11T01:26:02.984629+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:26:02.984629+0000","last_undegraded":"2024-09-11T01:26:02.984629+0000","last_fullsized":"2024-09-11T01:26:02.984629+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20177,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984547+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:02.984547+0000","last_peered":"2024-09-11T01:26:02.984547+0000","last_clean":"2024-09-11T01:26:02.984547+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:02.984547+0000","last_undegraded":"2024-09-11T01:26:02.984547+0000","last_fullsized":"2024-09-11T01:26:02.984547+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20423,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986863+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:02.986863+0000","last_peered":"2024-09-11T01:26:02.986863+0000","last_clean":"2024-09-11T01:26:02.986863+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:02.986863+0000","last_undegraded":"2024-09-11T01:26:02.986863+0000","last_fullsized":"2024-09-11T01:26:02.986863+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":22414,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:13.291085+0000","last_change":"2024-09-11T01:26:01.184322+0000","last_active":"2024-09-11T01:26:13.291085+0000","last_peered":"2024-09-11T01:26:13.291085+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:26:13.291085+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2832,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1156,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2832}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982464+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:02.982464+0000","last_peered":"2024-09-11T01:26:02.982464+0000","last_clean":"2024-09-11T01:26:02.982464+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:02.982464+0000","last_undegraded":"2024-09-11T01:26:02.982464+0000","last_fullsized":"2024-09-11T01:26:02.982464+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986745+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:02.986745+0000","last_peered":"2024-09-11T01:26:02.986745+0000","last_clean":"2024-09-11T01:26:02.986745+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:02.986745+0000","last_undegraded":"2024-09-11T01:26:02.986745+0000","last_fullsized":"2024-09-11T01:26:02.986745+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28021,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989204+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:02.989204+0000","last_peered":"2024-09-11T01:26:02.989204+0000","last_clean":"2024-09-11T01:26:02.989204+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:02.989204+0000","last_undegraded":"2024-09-11T01:26:02.989204+0000","last_fullsized":"2024-09-11T01:26:02.989204+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982959+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:26:02.982959+0000","last_peered":"2024-09-11T01:26:02.982959+0000","last_clean":"2024-09-11T01:26:02.982959+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:26:02.982959+0000","last_undegraded":"2024-09-11T01:26:02.982959+0000","last_fullsized":"2024-09-11T01:26:02.982959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20037,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:02.982813+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:26:02.982813+0000","last_peered":"2024-09-11T01:26:02.982813+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:26:02.982813+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982616+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:02.982616+0000","last_peered":"2024-09-11T01:26:02.982616+0000","last_clean":"2024-09-11T01:26:02.982616+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:02.982616+0000","last_undegraded":"2024-09-11T01:26:02.982616+0000","last_fullsized":"2024-09-11T01:26:02.982616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28426,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.983065+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:26:02.983065+0000","last_peered":"2024-09-11T01:26:02.983065+0000","last_clean":"2024-09-11T01:26:02.983065+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:26:02.983065+0000","last_undegraded":"2024-09-11T01:26:02.983065+0000","last_fullsized":"2024-09-11T01:26:02.983065+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989042+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:02.989042+0000","last_peered":"2024-09-11T01:26:02.989042+0000","last_clean":"2024-09-11T01:26:02.989042+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:02.989042+0000","last_undegraded":"2024-09-11T01:26:02.989042+0000","last_fullsized":"2024-09-11T01:26:02.989042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982786+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:26:02.982786+0000","last_peered":"2024-09-11T01:26:02.982786+0000","last_clean":"2024-09-11T01:26:02.982786+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:26:02.982786+0000","last_undegraded":"2024-09-11T01:26:02.982786+0000","last_fullsized":"2024-09-11T01:26:02.982786+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20427,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989519+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:02.989519+0000","last_peered":"2024-09-11T01:26:02.989519+0000","last_clean":"2024-09-11T01:26:02.989519+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:02.989519+0000","last_undegraded":"2024-09-11T01:26:02.989519+0000","last_fullsized":"2024-09-11T01:26:02.989519+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989097+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:02.989097+0000","last_peered":"2024-09-11T01:26:02.989097+0000","last_clean":"2024-09-11T01:26:02.989097+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:02.989097+0000","last_undegraded":"2024-09-11T01:26:02.989097+0000","last_fullsized":"2024-09-11T01:26:02.989097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27955,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989371+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:02.989371+0000","last_peered":"2024-09-11T01:26:02.989371+0000","last_clean":"2024-09-11T01:26:02.989371+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:02.989371+0000","last_undegraded":"2024-09-11T01:26:02.989371+0000","last_fullsized":"2024-09-11T01:26:02.989371+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985710+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:26:02.985710+0000","last_peered":"2024-09-11T01:26:02.985710+0000","last_clean":"2024-09-11T01:26:02.985710+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:26:02.985710+0000","last_undegraded":"2024-09-11T01:26:02.985710+0000","last_fullsized":"2024-09-11T01:26:02.985710+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24634,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989242+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:02.989242+0000","last_peered":"2024-09-11T01:26:02.989242+0000","last_clean":"2024-09-11T01:26:02.989242+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:02.989242+0000","last_undegraded":"2024-09-11T01:26:02.989242+0000","last_fullsized":"2024-09-11T01:26:02.989242+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989422+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:02.989422+0000","last_peered":"2024-09-11T01:26:02.989422+0000","last_clean":"2024-09-11T01:26:02.989422+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:02.989422+0000","last_undegraded":"2024-09-11T01:26:02.989422+0000","last_fullsized":"2024-09-11T01:26:02.989422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20107,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:03.078750+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:03.078750+0000","last_peered":"2024-09-11T01:26:03.078750+0000","last_clean":"2024-09-11T01:26:03.078750+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:03.078750+0000","last_undegraded":"2024-09-11T01:26:03.078750+0000","last_fullsized":"2024-09-11T01:26:03.078750+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6818,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":40821,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":136949760,"data_stored":136932365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182576,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518680,"kb_used_data":63624,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853160,"statfs":{"total":96636764160,"available":96105635840,"internally_reserved":0,"allocated":65150976,"data_stored":64795957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986349,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432268,"kb_used_data":71612,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93939572,"statfs":{"total":96636764160,"available":96194121728,"internally_reserved":0,"allocated":73330688,"data_stored":72979746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659293,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497248,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":496699,"kb_avail":93874592,"statfs":{"total":96636764160,"available":96127582208,"internally_reserved":0,"allocated":540672,"data_stored":178695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508620463},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":72314880,"data_stored":72311401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:18.614 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:18.920 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:18.920 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:26:18.939 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":514,"stamp":"2024-09-11T01:26:18.873680+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6485,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41155,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1452356,"kb_used_data":135764,"kb_used_omap":12,"kb_used_meta":1316531,"kb_avail":281663164,"statfs":{"total":289910292480,"available":288423079936,"internally_reserved":0,"allocated":139022336,"data_stored":137954398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1348127748},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003721"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28536,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985357+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:02.985357+0000","last_peered":"2024-09-11T01:26:02.985357+0000","last_clean":"2024-09-11T01:26:02.985357+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:02.985357+0000","last_undegraded":"2024-09-11T01:26:02.985357+0000","last_fullsized":"2024-09-11T01:26:02.985357+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20494,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984919+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:26:02.984919+0000","last_peered":"2024-09-11T01:26:02.984919+0000","last_clean":"2024-09-11T01:26:02.984919+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:26:02.984919+0000","last_undegraded":"2024-09-11T01:26:02.984919+0000","last_fullsized":"2024-09-11T01:26:02.984919+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20317,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20244,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984629+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:26:02.984629+0000","last_peered":"2024-09-11T01:26:02.984629+0000","last_clean":"2024-09-11T01:26:02.984629+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:26:02.984629+0000","last_undegraded":"2024-09-11T01:26:02.984629+0000","last_fullsized":"2024-09-11T01:26:02.984629+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20177,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984547+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:02.984547+0000","last_peered":"2024-09-11T01:26:02.984547+0000","last_clean":"2024-09-11T01:26:02.984547+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:02.984547+0000","last_undegraded":"2024-09-11T01:26:02.984547+0000","last_fullsized":"2024-09-11T01:26:02.984547+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20423,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986863+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:02.986863+0000","last_peered":"2024-09-11T01:26:02.986863+0000","last_clean":"2024-09-11T01:26:02.986863+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:02.986863+0000","last_undegraded":"2024-09-11T01:26:02.986863+0000","last_fullsized":"2024-09-11T01:26:02.986863+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":23080,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:18.298708+0000","last_change":"2024-09-11T01:26:01.184322+0000","last_active":"2024-09-11T01:26:18.298708+0000","last_peered":"2024-09-11T01:26:18.298708+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:26:18.298708+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1489,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2499}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982464+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:02.982464+0000","last_peered":"2024-09-11T01:26:02.982464+0000","last_clean":"2024-09-11T01:26:02.982464+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:02.982464+0000","last_undegraded":"2024-09-11T01:26:02.982464+0000","last_fullsized":"2024-09-11T01:26:02.982464+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986745+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:02.986745+0000","last_peered":"2024-09-11T01:26:02.986745+0000","last_clean":"2024-09-11T01:26:02.986745+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:02.986745+0000","last_undegraded":"2024-09-11T01:26:02.986745+0000","last_fullsized":"2024-09-11T01:26:02.986745+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28021,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989204+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:02.989204+0000","last_peered":"2024-09-11T01:26:02.989204+0000","last_clean":"2024-09-11T01:26:02.989204+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:02.989204+0000","last_undegraded":"2024-09-11T01:26:02.989204+0000","last_fullsized":"2024-09-11T01:26:02.989204+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982959+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:26:02.982959+0000","last_peered":"2024-09-11T01:26:02.982959+0000","last_clean":"2024-09-11T01:26:02.982959+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:26:02.982959+0000","last_undegraded":"2024-09-11T01:26:02.982959+0000","last_fullsized":"2024-09-11T01:26:02.982959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20037,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:02.982813+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:26:02.982813+0000","last_peered":"2024-09-11T01:26:02.982813+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:26:02.982813+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982616+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:02.982616+0000","last_peered":"2024-09-11T01:26:02.982616+0000","last_clean":"2024-09-11T01:26:02.982616+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:02.982616+0000","last_undegraded":"2024-09-11T01:26:02.982616+0000","last_fullsized":"2024-09-11T01:26:02.982616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28426,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.983065+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:26:02.983065+0000","last_peered":"2024-09-11T01:26:02.983065+0000","last_clean":"2024-09-11T01:26:02.983065+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:26:02.983065+0000","last_undegraded":"2024-09-11T01:26:02.983065+0000","last_fullsized":"2024-09-11T01:26:02.983065+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989042+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:02.989042+0000","last_peered":"2024-09-11T01:26:02.989042+0000","last_clean":"2024-09-11T01:26:02.989042+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:02.989042+0000","last_undegraded":"2024-09-11T01:26:02.989042+0000","last_fullsized":"2024-09-11T01:26:02.989042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982786+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:26:02.982786+0000","last_peered":"2024-09-11T01:26:02.982786+0000","last_clean":"2024-09-11T01:26:02.982786+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:26:02.982786+0000","last_undegraded":"2024-09-11T01:26:02.982786+0000","last_fullsized":"2024-09-11T01:26:02.982786+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20427,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989519+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:02.989519+0000","last_peered":"2024-09-11T01:26:02.989519+0000","last_clean":"2024-09-11T01:26:02.989519+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:02.989519+0000","last_undegraded":"2024-09-11T01:26:02.989519+0000","last_fullsized":"2024-09-11T01:26:02.989519+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989097+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:02.989097+0000","last_peered":"2024-09-11T01:26:02.989097+0000","last_clean":"2024-09-11T01:26:02.989097+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:02.989097+0000","last_undegraded":"2024-09-11T01:26:02.989097+0000","last_fullsized":"2024-09-11T01:26:02.989097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27955,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989371+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:02.989371+0000","last_peered":"2024-09-11T01:26:02.989371+0000","last_clean":"2024-09-11T01:26:02.989371+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:02.989371+0000","last_undegraded":"2024-09-11T01:26:02.989371+0000","last_fullsized":"2024-09-11T01:26:02.989371+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985710+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:26:02.985710+0000","last_peered":"2024-09-11T01:26:02.985710+0000","last_clean":"2024-09-11T01:26:02.985710+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:26:02.985710+0000","last_undegraded":"2024-09-11T01:26:02.985710+0000","last_fullsized":"2024-09-11T01:26:02.985710+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24634,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989242+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:02.989242+0000","last_peered":"2024-09-11T01:26:02.989242+0000","last_clean":"2024-09-11T01:26:02.989242+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:02.989242+0000","last_undegraded":"2024-09-11T01:26:02.989242+0000","last_fullsized":"2024-09-11T01:26:02.989242+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989422+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:02.989422+0000","last_peered":"2024-09-11T01:26:02.989422+0000","last_clean":"2024-09-11T01:26:02.989422+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:02.989422+0000","last_undegraded":"2024-09-11T01:26:02.989422+0000","last_fullsized":"2024-09-11T01:26:02.989422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20107,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:03.078750+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:03.078750+0000","last_peered":"2024-09-11T01:26:03.078750+0000","last_clean":"2024-09-11T01:26:03.078750+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:03.078750+0000","last_undegraded":"2024-09-11T01:26:03.078750+0000","last_fullsized":"2024-09-11T01:26:03.078750+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6485,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41154,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":136949760,"data_stored":136932365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182577,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518680,"kb_used_data":63624,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853160,"statfs":{"total":96636764160,"available":96105635840,"internally_reserved":0,"allocated":65150976,"data_stored":64795957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986349,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432268,"kb_used_data":71612,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93939572,"statfs":{"total":96636764160,"available":96194121728,"internally_reserved":0,"allocated":73330688,"data_stored":72979746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659294,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501408,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":500859,"kb_avail":93870432,"statfs":{"total":96636764160,"available":96123322368,"internally_reserved":0,"allocated":540672,"data_stored":178695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512880303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":72314880,"data_stored":72311401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:18.940 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:26:18.940 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 23080, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:18.298708+0000', 'last_change': '2024-09-11T01:26:01.184322+0000', 'last_active': '2024-09-11T01:26:18.298708+0000', 'last_peered': '2024-09-11T01:26:18.298708+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:26:18.298708+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2499, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1489, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2499}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:18.940 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:26:18.940 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20037, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:02.982813+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:26:02.982813+0000', 'last_peered': '2024-09-11T01:26:02.982813+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:26:02.982813+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:18.941 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:26:19.315 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:19.315 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 6485/6 objects degraded (108083.333%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":915,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1487212544,"bytes_avail":288423079936,"bytes_total":289910292480,"degraded_objects":6485,"degraded_total":6,"degraded_ratio":1080.8333333333333,"recovering_objects_per_sec":104,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1259,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:26:19.333 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:26:19.333 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:19.643 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:19.644 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:26:19.661 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":514,"stamp":"2024-09-11T01:26:18.873680+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6485,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41155,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1452356,"kb_used_data":135764,"kb_used_omap":12,"kb_used_meta":1316531,"kb_avail":281663164,"statfs":{"total":289910292480,"available":288423079936,"internally_reserved":0,"allocated":139022336,"data_stored":137954398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1348127748},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1259,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1259,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003721"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28536,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985357+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:02.985357+0000","last_peered":"2024-09-11T01:26:02.985357+0000","last_clean":"2024-09-11T01:26:02.985357+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:02.985357+0000","last_undegraded":"2024-09-11T01:26:02.985357+0000","last_fullsized":"2024-09-11T01:26:02.985357+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20494,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984919+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:26:02.984919+0000","last_peered":"2024-09-11T01:26:02.984919+0000","last_clean":"2024-09-11T01:26:02.984919+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:26:02.984919+0000","last_undegraded":"2024-09-11T01:26:02.984919+0000","last_fullsized":"2024-09-11T01:26:02.984919+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20317,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20244,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984629+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:26:02.984629+0000","last_peered":"2024-09-11T01:26:02.984629+0000","last_clean":"2024-09-11T01:26:02.984629+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:26:02.984629+0000","last_undegraded":"2024-09-11T01:26:02.984629+0000","last_fullsized":"2024-09-11T01:26:02.984629+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20177,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984547+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:02.984547+0000","last_peered":"2024-09-11T01:26:02.984547+0000","last_clean":"2024-09-11T01:26:02.984547+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:02.984547+0000","last_undegraded":"2024-09-11T01:26:02.984547+0000","last_fullsized":"2024-09-11T01:26:02.984547+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20423,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986863+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:02.986863+0000","last_peered":"2024-09-11T01:26:02.986863+0000","last_clean":"2024-09-11T01:26:02.986863+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:02.986863+0000","last_undegraded":"2024-09-11T01:26:02.986863+0000","last_fullsized":"2024-09-11T01:26:02.986863+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":23080,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:18.298708+0000","last_change":"2024-09-11T01:26:01.184322+0000","last_active":"2024-09-11T01:26:18.298708+0000","last_peered":"2024-09-11T01:26:18.298708+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:26:18.298708+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1489,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2499}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982464+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:02.982464+0000","last_peered":"2024-09-11T01:26:02.982464+0000","last_clean":"2024-09-11T01:26:02.982464+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:02.982464+0000","last_undegraded":"2024-09-11T01:26:02.982464+0000","last_fullsized":"2024-09-11T01:26:02.982464+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986745+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:02.986745+0000","last_peered":"2024-09-11T01:26:02.986745+0000","last_clean":"2024-09-11T01:26:02.986745+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:02.986745+0000","last_undegraded":"2024-09-11T01:26:02.986745+0000","last_fullsized":"2024-09-11T01:26:02.986745+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28021,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989204+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:02.989204+0000","last_peered":"2024-09-11T01:26:02.989204+0000","last_clean":"2024-09-11T01:26:02.989204+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:02.989204+0000","last_undegraded":"2024-09-11T01:26:02.989204+0000","last_fullsized":"2024-09-11T01:26:02.989204+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982959+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:26:02.982959+0000","last_peered":"2024-09-11T01:26:02.982959+0000","last_clean":"2024-09-11T01:26:02.982959+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:26:02.982959+0000","last_undegraded":"2024-09-11T01:26:02.982959+0000","last_fullsized":"2024-09-11T01:26:02.982959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20037,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:02.982813+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:26:02.982813+0000","last_peered":"2024-09-11T01:26:02.982813+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:26:02.982813+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982616+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:02.982616+0000","last_peered":"2024-09-11T01:26:02.982616+0000","last_clean":"2024-09-11T01:26:02.982616+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:02.982616+0000","last_undegraded":"2024-09-11T01:26:02.982616+0000","last_fullsized":"2024-09-11T01:26:02.982616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28426,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.983065+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:26:02.983065+0000","last_peered":"2024-09-11T01:26:02.983065+0000","last_clean":"2024-09-11T01:26:02.983065+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:26:02.983065+0000","last_undegraded":"2024-09-11T01:26:02.983065+0000","last_fullsized":"2024-09-11T01:26:02.983065+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989042+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:02.989042+0000","last_peered":"2024-09-11T01:26:02.989042+0000","last_clean":"2024-09-11T01:26:02.989042+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:02.989042+0000","last_undegraded":"2024-09-11T01:26:02.989042+0000","last_fullsized":"2024-09-11T01:26:02.989042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982786+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:26:02.982786+0000","last_peered":"2024-09-11T01:26:02.982786+0000","last_clean":"2024-09-11T01:26:02.982786+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:26:02.982786+0000","last_undegraded":"2024-09-11T01:26:02.982786+0000","last_fullsized":"2024-09-11T01:26:02.982786+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20427,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989519+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:02.989519+0000","last_peered":"2024-09-11T01:26:02.989519+0000","last_clean":"2024-09-11T01:26:02.989519+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:02.989519+0000","last_undegraded":"2024-09-11T01:26:02.989519+0000","last_fullsized":"2024-09-11T01:26:02.989519+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989097+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:02.989097+0000","last_peered":"2024-09-11T01:26:02.989097+0000","last_clean":"2024-09-11T01:26:02.989097+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:02.989097+0000","last_undegraded":"2024-09-11T01:26:02.989097+0000","last_fullsized":"2024-09-11T01:26:02.989097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27955,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989371+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:02.989371+0000","last_peered":"2024-09-11T01:26:02.989371+0000","last_clean":"2024-09-11T01:26:02.989371+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:02.989371+0000","last_undegraded":"2024-09-11T01:26:02.989371+0000","last_fullsized":"2024-09-11T01:26:02.989371+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985710+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:26:02.985710+0000","last_peered":"2024-09-11T01:26:02.985710+0000","last_clean":"2024-09-11T01:26:02.985710+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:26:02.985710+0000","last_undegraded":"2024-09-11T01:26:02.985710+0000","last_fullsized":"2024-09-11T01:26:02.985710+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24634,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989242+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:02.989242+0000","last_peered":"2024-09-11T01:26:02.989242+0000","last_clean":"2024-09-11T01:26:02.989242+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:02.989242+0000","last_undegraded":"2024-09-11T01:26:02.989242+0000","last_fullsized":"2024-09-11T01:26:02.989242+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989422+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:02.989422+0000","last_peered":"2024-09-11T01:26:02.989422+0000","last_clean":"2024-09-11T01:26:02.989422+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:02.989422+0000","last_undegraded":"2024-09-11T01:26:02.989422+0000","last_fullsized":"2024-09-11T01:26:02.989422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20107,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:03.078750+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:03.078750+0000","last_peered":"2024-09-11T01:26:03.078750+0000","last_clean":"2024-09-11T01:26:03.078750+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:03.078750+0000","last_undegraded":"2024-09-11T01:26:03.078750+0000","last_fullsized":"2024-09-11T01:26:03.078750+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6485,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41154,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":136949760,"data_stored":136932365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182577,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518680,"kb_used_data":63624,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853160,"statfs":{"total":96636764160,"available":96105635840,"internally_reserved":0,"allocated":65150976,"data_stored":64795957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986349,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432268,"kb_used_data":71612,"kb_used_omap":4,"kb_used_meta":360635,"kb_avail":93939572,"statfs":{"total":96636764160,"available":96194121728,"internally_reserved":0,"allocated":73330688,"data_stored":72979746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":369290922},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659294,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501408,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":500859,"kb_avail":93870432,"statfs":{"total":96636764160,"available":96123322368,"internally_reserved":0,"allocated":540672,"data_stored":178695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512880303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":72314880,"data_stored":72311401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:22.664 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:23.000 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:23.000 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:26:23.020 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":516,"stamp":"2024-09-11T01:26:22.874832+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6485,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41155,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1452916,"kb_used_data":132164,"kb_used_omap":12,"kb_used_meta":1320691,"kb_avail":281662604,"statfs":{"total":289910292480,"available":288422506496,"internally_reserved":0,"allocated":135335936,"data_stored":134267998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1352387588},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-637,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":637,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003769"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28536,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985357+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:02.985357+0000","last_peered":"2024-09-11T01:26:02.985357+0000","last_clean":"2024-09-11T01:26:02.985357+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:02.985357+0000","last_undegraded":"2024-09-11T01:26:02.985357+0000","last_fullsized":"2024-09-11T01:26:02.985357+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20494,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984919+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:26:02.984919+0000","last_peered":"2024-09-11T01:26:02.984919+0000","last_clean":"2024-09-11T01:26:02.984919+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:26:02.984919+0000","last_undegraded":"2024-09-11T01:26:02.984919+0000","last_fullsized":"2024-09-11T01:26:02.984919+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20317,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20244,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984629+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:26:02.984629+0000","last_peered":"2024-09-11T01:26:02.984629+0000","last_clean":"2024-09-11T01:26:02.984629+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:26:02.984629+0000","last_undegraded":"2024-09-11T01:26:02.984629+0000","last_fullsized":"2024-09-11T01:26:02.984629+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20177,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984547+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:02.984547+0000","last_peered":"2024-09-11T01:26:02.984547+0000","last_clean":"2024-09-11T01:26:02.984547+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:02.984547+0000","last_undegraded":"2024-09-11T01:26:02.984547+0000","last_fullsized":"2024-09-11T01:26:02.984547+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20423,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986863+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:02.986863+0000","last_peered":"2024-09-11T01:26:02.986863+0000","last_clean":"2024-09-11T01:26:02.986863+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:02.986863+0000","last_undegraded":"2024-09-11T01:26:02.986863+0000","last_fullsized":"2024-09-11T01:26:02.986863+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":23080,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:18.298708+0000","last_change":"2024-09-11T01:26:01.184322+0000","last_active":"2024-09-11T01:26:18.298708+0000","last_peered":"2024-09-11T01:26:18.298708+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:26:18.298708+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1489,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2499}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982464+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:02.982464+0000","last_peered":"2024-09-11T01:26:02.982464+0000","last_clean":"2024-09-11T01:26:02.982464+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:02.982464+0000","last_undegraded":"2024-09-11T01:26:02.982464+0000","last_fullsized":"2024-09-11T01:26:02.982464+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986745+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:02.986745+0000","last_peered":"2024-09-11T01:26:02.986745+0000","last_clean":"2024-09-11T01:26:02.986745+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:02.986745+0000","last_undegraded":"2024-09-11T01:26:02.986745+0000","last_fullsized":"2024-09-11T01:26:02.986745+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28021,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989204+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:02.989204+0000","last_peered":"2024-09-11T01:26:02.989204+0000","last_clean":"2024-09-11T01:26:02.989204+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:02.989204+0000","last_undegraded":"2024-09-11T01:26:02.989204+0000","last_fullsized":"2024-09-11T01:26:02.989204+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982959+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:26:02.982959+0000","last_peered":"2024-09-11T01:26:02.982959+0000","last_clean":"2024-09-11T01:26:02.982959+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:26:02.982959+0000","last_undegraded":"2024-09-11T01:26:02.982959+0000","last_fullsized":"2024-09-11T01:26:02.982959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20037,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:02.982813+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:26:02.982813+0000","last_peered":"2024-09-11T01:26:02.982813+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:26:02.982813+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982616+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:02.982616+0000","last_peered":"2024-09-11T01:26:02.982616+0000","last_clean":"2024-09-11T01:26:02.982616+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:02.982616+0000","last_undegraded":"2024-09-11T01:26:02.982616+0000","last_fullsized":"2024-09-11T01:26:02.982616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28426,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.983065+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:26:02.983065+0000","last_peered":"2024-09-11T01:26:02.983065+0000","last_clean":"2024-09-11T01:26:02.983065+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:26:02.983065+0000","last_undegraded":"2024-09-11T01:26:02.983065+0000","last_fullsized":"2024-09-11T01:26:02.983065+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989042+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:02.989042+0000","last_peered":"2024-09-11T01:26:02.989042+0000","last_clean":"2024-09-11T01:26:02.989042+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:02.989042+0000","last_undegraded":"2024-09-11T01:26:02.989042+0000","last_fullsized":"2024-09-11T01:26:02.989042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982786+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:26:02.982786+0000","last_peered":"2024-09-11T01:26:02.982786+0000","last_clean":"2024-09-11T01:26:02.982786+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:26:02.982786+0000","last_undegraded":"2024-09-11T01:26:02.982786+0000","last_fullsized":"2024-09-11T01:26:02.982786+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20427,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989519+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:02.989519+0000","last_peered":"2024-09-11T01:26:02.989519+0000","last_clean":"2024-09-11T01:26:02.989519+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:02.989519+0000","last_undegraded":"2024-09-11T01:26:02.989519+0000","last_fullsized":"2024-09-11T01:26:02.989519+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989097+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:02.989097+0000","last_peered":"2024-09-11T01:26:02.989097+0000","last_clean":"2024-09-11T01:26:02.989097+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:02.989097+0000","last_undegraded":"2024-09-11T01:26:02.989097+0000","last_fullsized":"2024-09-11T01:26:02.989097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27955,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989371+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:02.989371+0000","last_peered":"2024-09-11T01:26:02.989371+0000","last_clean":"2024-09-11T01:26:02.989371+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:02.989371+0000","last_undegraded":"2024-09-11T01:26:02.989371+0000","last_fullsized":"2024-09-11T01:26:02.989371+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985710+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:26:02.985710+0000","last_peered":"2024-09-11T01:26:02.985710+0000","last_clean":"2024-09-11T01:26:02.985710+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:26:02.985710+0000","last_undegraded":"2024-09-11T01:26:02.985710+0000","last_fullsized":"2024-09-11T01:26:02.985710+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24634,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989242+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:02.989242+0000","last_peered":"2024-09-11T01:26:02.989242+0000","last_clean":"2024-09-11T01:26:02.989242+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:02.989242+0000","last_undegraded":"2024-09-11T01:26:02.989242+0000","last_fullsized":"2024-09-11T01:26:02.989242+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989422+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:02.989422+0000","last_peered":"2024-09-11T01:26:02.989422+0000","last_clean":"2024-09-11T01:26:02.989422+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:02.989422+0000","last_undegraded":"2024-09-11T01:26:02.989422+0000","last_fullsized":"2024-09-11T01:26:02.989422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20107,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:03.078750+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:03.078750+0000","last_peered":"2024-09-11T01:26:03.078750+0000","last_clean":"2024-09-11T01:26:03.078750+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:03.078750+0000","last_undegraded":"2024-09-11T01:26:03.078750+0000","last_fullsized":"2024-09-11T01:26:03.078750+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6485,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41154,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":133263360,"data_stored":133245965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182577,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518680,"kb_used_data":63624,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853160,"statfs":{"total":96636764160,"available":96105635840,"internally_reserved":0,"allocated":65150976,"data_stored":64795957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986350,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432828,"kb_used_data":68012,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93939012,"statfs":{"total":96636764160,"available":96193548288,"internally_reserved":0,"allocated":69644288,"data_stored":69293346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659294,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501408,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":500859,"kb_avail":93870432,"statfs":{"total":96636764160,"available":96123322368,"internally_reserved":0,"allocated":540672,"data_stored":178695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512880303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":68628480,"data_stored":68625001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:23.022 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:23.340 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:23.340 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:26:23.358 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":516,"stamp":"2024-09-11T01:26:22.874832+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6485,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41155,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1452916,"kb_used_data":132164,"kb_used_omap":12,"kb_used_meta":1320691,"kb_avail":281662604,"statfs":{"total":289910292480,"available":288422506496,"internally_reserved":0,"allocated":135335936,"data_stored":134267998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1352387588},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-637,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":637,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003769"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28536,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985357+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:02.985357+0000","last_peered":"2024-09-11T01:26:02.985357+0000","last_clean":"2024-09-11T01:26:02.985357+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:02.985357+0000","last_undegraded":"2024-09-11T01:26:02.985357+0000","last_fullsized":"2024-09-11T01:26:02.985357+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20494,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984919+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:26:02.984919+0000","last_peered":"2024-09-11T01:26:02.984919+0000","last_clean":"2024-09-11T01:26:02.984919+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:26:02.984919+0000","last_undegraded":"2024-09-11T01:26:02.984919+0000","last_fullsized":"2024-09-11T01:26:02.984919+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20317,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20244,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984629+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:26:02.984629+0000","last_peered":"2024-09-11T01:26:02.984629+0000","last_clean":"2024-09-11T01:26:02.984629+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:26:02.984629+0000","last_undegraded":"2024-09-11T01:26:02.984629+0000","last_fullsized":"2024-09-11T01:26:02.984629+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20177,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984547+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:02.984547+0000","last_peered":"2024-09-11T01:26:02.984547+0000","last_clean":"2024-09-11T01:26:02.984547+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:02.984547+0000","last_undegraded":"2024-09-11T01:26:02.984547+0000","last_fullsized":"2024-09-11T01:26:02.984547+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20423,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986863+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:02.986863+0000","last_peered":"2024-09-11T01:26:02.986863+0000","last_clean":"2024-09-11T01:26:02.986863+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:02.986863+0000","last_undegraded":"2024-09-11T01:26:02.986863+0000","last_fullsized":"2024-09-11T01:26:02.986863+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":23080,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:18.298708+0000","last_change":"2024-09-11T01:26:01.184322+0000","last_active":"2024-09-11T01:26:18.298708+0000","last_peered":"2024-09-11T01:26:18.298708+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:26:18.298708+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1489,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2499}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982464+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:02.982464+0000","last_peered":"2024-09-11T01:26:02.982464+0000","last_clean":"2024-09-11T01:26:02.982464+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:02.982464+0000","last_undegraded":"2024-09-11T01:26:02.982464+0000","last_fullsized":"2024-09-11T01:26:02.982464+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986745+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:02.986745+0000","last_peered":"2024-09-11T01:26:02.986745+0000","last_clean":"2024-09-11T01:26:02.986745+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:02.986745+0000","last_undegraded":"2024-09-11T01:26:02.986745+0000","last_fullsized":"2024-09-11T01:26:02.986745+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28021,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989204+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:02.989204+0000","last_peered":"2024-09-11T01:26:02.989204+0000","last_clean":"2024-09-11T01:26:02.989204+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:02.989204+0000","last_undegraded":"2024-09-11T01:26:02.989204+0000","last_fullsized":"2024-09-11T01:26:02.989204+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982959+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:26:02.982959+0000","last_peered":"2024-09-11T01:26:02.982959+0000","last_clean":"2024-09-11T01:26:02.982959+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:26:02.982959+0000","last_undegraded":"2024-09-11T01:26:02.982959+0000","last_fullsized":"2024-09-11T01:26:02.982959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20037,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:02.982813+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:26:02.982813+0000","last_peered":"2024-09-11T01:26:02.982813+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:26:02.982813+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982616+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:02.982616+0000","last_peered":"2024-09-11T01:26:02.982616+0000","last_clean":"2024-09-11T01:26:02.982616+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:02.982616+0000","last_undegraded":"2024-09-11T01:26:02.982616+0000","last_fullsized":"2024-09-11T01:26:02.982616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28426,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.983065+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:26:02.983065+0000","last_peered":"2024-09-11T01:26:02.983065+0000","last_clean":"2024-09-11T01:26:02.983065+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:26:02.983065+0000","last_undegraded":"2024-09-11T01:26:02.983065+0000","last_fullsized":"2024-09-11T01:26:02.983065+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989042+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:02.989042+0000","last_peered":"2024-09-11T01:26:02.989042+0000","last_clean":"2024-09-11T01:26:02.989042+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:02.989042+0000","last_undegraded":"2024-09-11T01:26:02.989042+0000","last_fullsized":"2024-09-11T01:26:02.989042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982786+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:26:02.982786+0000","last_peered":"2024-09-11T01:26:02.982786+0000","last_clean":"2024-09-11T01:26:02.982786+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:26:02.982786+0000","last_undegraded":"2024-09-11T01:26:02.982786+0000","last_fullsized":"2024-09-11T01:26:02.982786+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20427,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989519+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:02.989519+0000","last_peered":"2024-09-11T01:26:02.989519+0000","last_clean":"2024-09-11T01:26:02.989519+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:02.989519+0000","last_undegraded":"2024-09-11T01:26:02.989519+0000","last_fullsized":"2024-09-11T01:26:02.989519+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989097+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:02.989097+0000","last_peered":"2024-09-11T01:26:02.989097+0000","last_clean":"2024-09-11T01:26:02.989097+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:02.989097+0000","last_undegraded":"2024-09-11T01:26:02.989097+0000","last_fullsized":"2024-09-11T01:26:02.989097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27955,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989371+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:02.989371+0000","last_peered":"2024-09-11T01:26:02.989371+0000","last_clean":"2024-09-11T01:26:02.989371+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:02.989371+0000","last_undegraded":"2024-09-11T01:26:02.989371+0000","last_fullsized":"2024-09-11T01:26:02.989371+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985710+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:26:02.985710+0000","last_peered":"2024-09-11T01:26:02.985710+0000","last_clean":"2024-09-11T01:26:02.985710+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:26:02.985710+0000","last_undegraded":"2024-09-11T01:26:02.985710+0000","last_fullsized":"2024-09-11T01:26:02.985710+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24634,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989242+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:02.989242+0000","last_peered":"2024-09-11T01:26:02.989242+0000","last_clean":"2024-09-11T01:26:02.989242+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:02.989242+0000","last_undegraded":"2024-09-11T01:26:02.989242+0000","last_fullsized":"2024-09-11T01:26:02.989242+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989422+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:02.989422+0000","last_peered":"2024-09-11T01:26:02.989422+0000","last_clean":"2024-09-11T01:26:02.989422+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:02.989422+0000","last_undegraded":"2024-09-11T01:26:02.989422+0000","last_fullsized":"2024-09-11T01:26:02.989422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20107,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:03.078750+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:03.078750+0000","last_peered":"2024-09-11T01:26:03.078750+0000","last_clean":"2024-09-11T01:26:03.078750+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:03.078750+0000","last_undegraded":"2024-09-11T01:26:03.078750+0000","last_fullsized":"2024-09-11T01:26:03.078750+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6485,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41154,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":133263360,"data_stored":133245965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182577,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518680,"kb_used_data":63624,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853160,"statfs":{"total":96636764160,"available":96105635840,"internally_reserved":0,"allocated":65150976,"data_stored":64795957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986350,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432828,"kb_used_data":68012,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93939012,"statfs":{"total":96636764160,"available":96193548288,"internally_reserved":0,"allocated":69644288,"data_stored":69293346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659294,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501408,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":500859,"kb_avail":93870432,"statfs":{"total":96636764160,"available":96123322368,"internally_reserved":0,"allocated":540672,"data_stored":178695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512880303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":68628480,"data_stored":68625001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:23.360 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:26:23.360 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 23080, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:18.298708+0000', 'last_change': '2024-09-11T01:26:01.184322+0000', 'last_active': '2024-09-11T01:26:18.298708+0000', 'last_peered': '2024-09-11T01:26:18.298708+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:26:18.298708+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2499, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1489, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2499}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:23.360 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:26:23.360 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20037, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:02.982813+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:26:02.982813+0000', 'last_peered': '2024-09-11T01:26:02.982813+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:26:02.982813+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:23.360 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:26:23.740 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:23.740 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 6485/6 objects degraded (108083.333%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":920,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1487785984,"bytes_avail":288422506496,"bytes_total":289910292480,"degraded_objects":6485,"degraded_total":6,"degraded_ratio":1080.8333333333333,"recovering_objects_per_sec":53,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":637,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:26:23.759 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:26:23.760 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:24.063 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:24.064 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:26:24.082 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":516,"stamp":"2024-09-11T01:26:22.874832+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6485,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41155,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1452916,"kb_used_data":132164,"kb_used_omap":12,"kb_used_meta":1320691,"kb_avail":281662604,"statfs":{"total":289910292480,"available":288422506496,"internally_reserved":0,"allocated":135335936,"data_stored":134267998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1352387588},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-637,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":637,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003769"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28536,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985357+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:02.985357+0000","last_peered":"2024-09-11T01:26:02.985357+0000","last_clean":"2024-09-11T01:26:02.985357+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:02.985357+0000","last_undegraded":"2024-09-11T01:26:02.985357+0000","last_fullsized":"2024-09-11T01:26:02.985357+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20494,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984919+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:26:02.984919+0000","last_peered":"2024-09-11T01:26:02.984919+0000","last_clean":"2024-09-11T01:26:02.984919+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:26:02.984919+0000","last_undegraded":"2024-09-11T01:26:02.984919+0000","last_fullsized":"2024-09-11T01:26:02.984919+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20317,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20244,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984629+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:26:02.984629+0000","last_peered":"2024-09-11T01:26:02.984629+0000","last_clean":"2024-09-11T01:26:02.984629+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:26:02.984629+0000","last_undegraded":"2024-09-11T01:26:02.984629+0000","last_fullsized":"2024-09-11T01:26:02.984629+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20177,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984547+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:02.984547+0000","last_peered":"2024-09-11T01:26:02.984547+0000","last_clean":"2024-09-11T01:26:02.984547+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:02.984547+0000","last_undegraded":"2024-09-11T01:26:02.984547+0000","last_fullsized":"2024-09-11T01:26:02.984547+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20423,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986863+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:02.986863+0000","last_peered":"2024-09-11T01:26:02.986863+0000","last_clean":"2024-09-11T01:26:02.986863+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:02.986863+0000","last_undegraded":"2024-09-11T01:26:02.986863+0000","last_fullsized":"2024-09-11T01:26:02.986863+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":23080,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:18.298708+0000","last_change":"2024-09-11T01:26:01.184322+0000","last_active":"2024-09-11T01:26:18.298708+0000","last_peered":"2024-09-11T01:26:18.298708+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:26:18.298708+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2499,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1489,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2499}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982464+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:02.982464+0000","last_peered":"2024-09-11T01:26:02.982464+0000","last_clean":"2024-09-11T01:26:02.982464+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:02.982464+0000","last_undegraded":"2024-09-11T01:26:02.982464+0000","last_fullsized":"2024-09-11T01:26:02.982464+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986745+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:02.986745+0000","last_peered":"2024-09-11T01:26:02.986745+0000","last_clean":"2024-09-11T01:26:02.986745+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:02.986745+0000","last_undegraded":"2024-09-11T01:26:02.986745+0000","last_fullsized":"2024-09-11T01:26:02.986745+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28021,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989204+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:02.989204+0000","last_peered":"2024-09-11T01:26:02.989204+0000","last_clean":"2024-09-11T01:26:02.989204+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:02.989204+0000","last_undegraded":"2024-09-11T01:26:02.989204+0000","last_fullsized":"2024-09-11T01:26:02.989204+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982959+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:26:02.982959+0000","last_peered":"2024-09-11T01:26:02.982959+0000","last_clean":"2024-09-11T01:26:02.982959+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:26:02.982959+0000","last_undegraded":"2024-09-11T01:26:02.982959+0000","last_fullsized":"2024-09-11T01:26:02.982959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20037,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:02.982813+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:26:02.982813+0000","last_peered":"2024-09-11T01:26:02.982813+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:26:02.982813+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982616+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:02.982616+0000","last_peered":"2024-09-11T01:26:02.982616+0000","last_clean":"2024-09-11T01:26:02.982616+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:02.982616+0000","last_undegraded":"2024-09-11T01:26:02.982616+0000","last_fullsized":"2024-09-11T01:26:02.982616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28426,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.983065+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:26:02.983065+0000","last_peered":"2024-09-11T01:26:02.983065+0000","last_clean":"2024-09-11T01:26:02.983065+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:26:02.983065+0000","last_undegraded":"2024-09-11T01:26:02.983065+0000","last_fullsized":"2024-09-11T01:26:02.983065+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989042+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:02.989042+0000","last_peered":"2024-09-11T01:26:02.989042+0000","last_clean":"2024-09-11T01:26:02.989042+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:02.989042+0000","last_undegraded":"2024-09-11T01:26:02.989042+0000","last_fullsized":"2024-09-11T01:26:02.989042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982786+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:26:02.982786+0000","last_peered":"2024-09-11T01:26:02.982786+0000","last_clean":"2024-09-11T01:26:02.982786+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:26:02.982786+0000","last_undegraded":"2024-09-11T01:26:02.982786+0000","last_fullsized":"2024-09-11T01:26:02.982786+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20427,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989519+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:02.989519+0000","last_peered":"2024-09-11T01:26:02.989519+0000","last_clean":"2024-09-11T01:26:02.989519+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:02.989519+0000","last_undegraded":"2024-09-11T01:26:02.989519+0000","last_fullsized":"2024-09-11T01:26:02.989519+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989097+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:02.989097+0000","last_peered":"2024-09-11T01:26:02.989097+0000","last_clean":"2024-09-11T01:26:02.989097+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:02.989097+0000","last_undegraded":"2024-09-11T01:26:02.989097+0000","last_fullsized":"2024-09-11T01:26:02.989097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27955,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989371+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:02.989371+0000","last_peered":"2024-09-11T01:26:02.989371+0000","last_clean":"2024-09-11T01:26:02.989371+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:02.989371+0000","last_undegraded":"2024-09-11T01:26:02.989371+0000","last_fullsized":"2024-09-11T01:26:02.989371+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985710+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:26:02.985710+0000","last_peered":"2024-09-11T01:26:02.985710+0000","last_clean":"2024-09-11T01:26:02.985710+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:26:02.985710+0000","last_undegraded":"2024-09-11T01:26:02.985710+0000","last_fullsized":"2024-09-11T01:26:02.985710+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24634,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989242+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:02.989242+0000","last_peered":"2024-09-11T01:26:02.989242+0000","last_clean":"2024-09-11T01:26:02.989242+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:02.989242+0000","last_undegraded":"2024-09-11T01:26:02.989242+0000","last_fullsized":"2024-09-11T01:26:02.989242+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989422+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:02.989422+0000","last_peered":"2024-09-11T01:26:02.989422+0000","last_clean":"2024-09-11T01:26:02.989422+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:02.989422+0000","last_undegraded":"2024-09-11T01:26:02.989422+0000","last_fullsized":"2024-09-11T01:26:02.989422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20107,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:03.078750+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:03.078750+0000","last_peered":"2024-09-11T01:26:03.078750+0000","last_clean":"2024-09-11T01:26:03.078750+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:03.078750+0000","last_undegraded":"2024-09-11T01:26:03.078750+0000","last_fullsized":"2024-09-11T01:26:03.078750+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6485,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41154,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":133263360,"data_stored":133245965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182577,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518680,"kb_used_data":63624,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853160,"statfs":{"total":96636764160,"available":96105635840,"internally_reserved":0,"allocated":65150976,"data_stored":64795957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986350,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":432828,"kb_used_data":68012,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93939012,"statfs":{"total":96636764160,"available":96193548288,"internally_reserved":0,"allocated":69644288,"data_stored":69293346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659294,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501408,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":500859,"kb_avail":93870432,"statfs":{"total":96636764160,"available":96123322368,"internally_reserved":0,"allocated":540672,"data_stored":178695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512880303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":68628480,"data_stored":68625001,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:27.085 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:27.408 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:27.409 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:26:27.427 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":518,"stamp":"2024-09-11T01:26:26.875899+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41542,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1449316,"kb_used_data":128564,"kb_used_omap":12,"kb_used_meta":1320691,"kb_avail":281666204,"statfs":{"total":289910292480,"available":288426192896,"internally_reserved":0,"allocated":131649536,"data_stored":130581598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1352387588},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-720,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":720,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003669"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28536,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985357+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:02.985357+0000","last_peered":"2024-09-11T01:26:02.985357+0000","last_clean":"2024-09-11T01:26:02.985357+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:02.985357+0000","last_undegraded":"2024-09-11T01:26:02.985357+0000","last_fullsized":"2024-09-11T01:26:02.985357+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20494,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984919+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:26:02.984919+0000","last_peered":"2024-09-11T01:26:02.984919+0000","last_clean":"2024-09-11T01:26:02.984919+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:26:02.984919+0000","last_undegraded":"2024-09-11T01:26:02.984919+0000","last_fullsized":"2024-09-11T01:26:02.984919+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20317,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20244,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984629+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:26:02.984629+0000","last_peered":"2024-09-11T01:26:02.984629+0000","last_clean":"2024-09-11T01:26:02.984629+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:26:02.984629+0000","last_undegraded":"2024-09-11T01:26:02.984629+0000","last_fullsized":"2024-09-11T01:26:02.984629+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20177,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984547+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:02.984547+0000","last_peered":"2024-09-11T01:26:02.984547+0000","last_clean":"2024-09-11T01:26:02.984547+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:02.984547+0000","last_undegraded":"2024-09-11T01:26:02.984547+0000","last_fullsized":"2024-09-11T01:26:02.984547+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20423,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986863+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:02.986863+0000","last_peered":"2024-09-11T01:26:02.986863+0000","last_clean":"2024-09-11T01:26:02.986863+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:02.986863+0000","last_undegraded":"2024-09-11T01:26:02.986863+0000","last_fullsized":"2024-09-11T01:26:02.986863+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":23854,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:23.304712+0000","last_change":"2024-09-11T01:26:01.184322+0000","last_active":"2024-09-11T01:26:23.304712+0000","last_peered":"2024-09-11T01:26:23.304712+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:26:23.304712+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2112,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1876,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2112}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982464+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:02.982464+0000","last_peered":"2024-09-11T01:26:02.982464+0000","last_clean":"2024-09-11T01:26:02.982464+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:02.982464+0000","last_undegraded":"2024-09-11T01:26:02.982464+0000","last_fullsized":"2024-09-11T01:26:02.982464+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986745+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:02.986745+0000","last_peered":"2024-09-11T01:26:02.986745+0000","last_clean":"2024-09-11T01:26:02.986745+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:02.986745+0000","last_undegraded":"2024-09-11T01:26:02.986745+0000","last_fullsized":"2024-09-11T01:26:02.986745+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28021,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989204+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:02.989204+0000","last_peered":"2024-09-11T01:26:02.989204+0000","last_clean":"2024-09-11T01:26:02.989204+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:02.989204+0000","last_undegraded":"2024-09-11T01:26:02.989204+0000","last_fullsized":"2024-09-11T01:26:02.989204+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982959+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:26:02.982959+0000","last_peered":"2024-09-11T01:26:02.982959+0000","last_clean":"2024-09-11T01:26:02.982959+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:26:02.982959+0000","last_undegraded":"2024-09-11T01:26:02.982959+0000","last_fullsized":"2024-09-11T01:26:02.982959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20037,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:02.982813+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:26:02.982813+0000","last_peered":"2024-09-11T01:26:02.982813+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:26:02.982813+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982616+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:02.982616+0000","last_peered":"2024-09-11T01:26:02.982616+0000","last_clean":"2024-09-11T01:26:02.982616+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:02.982616+0000","last_undegraded":"2024-09-11T01:26:02.982616+0000","last_fullsized":"2024-09-11T01:26:02.982616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28426,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.983065+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:26:02.983065+0000","last_peered":"2024-09-11T01:26:02.983065+0000","last_clean":"2024-09-11T01:26:02.983065+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:26:02.983065+0000","last_undegraded":"2024-09-11T01:26:02.983065+0000","last_fullsized":"2024-09-11T01:26:02.983065+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989042+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:02.989042+0000","last_peered":"2024-09-11T01:26:02.989042+0000","last_clean":"2024-09-11T01:26:02.989042+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:02.989042+0000","last_undegraded":"2024-09-11T01:26:02.989042+0000","last_fullsized":"2024-09-11T01:26:02.989042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982786+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:26:02.982786+0000","last_peered":"2024-09-11T01:26:02.982786+0000","last_clean":"2024-09-11T01:26:02.982786+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:26:02.982786+0000","last_undegraded":"2024-09-11T01:26:02.982786+0000","last_fullsized":"2024-09-11T01:26:02.982786+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20427,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989519+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:02.989519+0000","last_peered":"2024-09-11T01:26:02.989519+0000","last_clean":"2024-09-11T01:26:02.989519+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:02.989519+0000","last_undegraded":"2024-09-11T01:26:02.989519+0000","last_fullsized":"2024-09-11T01:26:02.989519+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989097+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:02.989097+0000","last_peered":"2024-09-11T01:26:02.989097+0000","last_clean":"2024-09-11T01:26:02.989097+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:02.989097+0000","last_undegraded":"2024-09-11T01:26:02.989097+0000","last_fullsized":"2024-09-11T01:26:02.989097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27955,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989371+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:02.989371+0000","last_peered":"2024-09-11T01:26:02.989371+0000","last_clean":"2024-09-11T01:26:02.989371+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:02.989371+0000","last_undegraded":"2024-09-11T01:26:02.989371+0000","last_fullsized":"2024-09-11T01:26:02.989371+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985710+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:26:02.985710+0000","last_peered":"2024-09-11T01:26:02.985710+0000","last_clean":"2024-09-11T01:26:02.985710+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:26:02.985710+0000","last_undegraded":"2024-09-11T01:26:02.985710+0000","last_fullsized":"2024-09-11T01:26:02.985710+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24634,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989242+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:02.989242+0000","last_peered":"2024-09-11T01:26:02.989242+0000","last_clean":"2024-09-11T01:26:02.989242+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:02.989242+0000","last_undegraded":"2024-09-11T01:26:02.989242+0000","last_fullsized":"2024-09-11T01:26:02.989242+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989422+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:02.989422+0000","last_peered":"2024-09-11T01:26:02.989422+0000","last_clean":"2024-09-11T01:26:02.989422+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:02.989422+0000","last_undegraded":"2024-09-11T01:26:02.989422+0000","last_fullsized":"2024-09-11T01:26:02.989422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20107,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:03.078750+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:03.078750+0000","last_peered":"2024-09-11T01:26:03.078750+0000","last_clean":"2024-09-11T01:26:03.078750+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:03.078750+0000","last_undegraded":"2024-09-11T01:26:03.078750+0000","last_fullsized":"2024-09-11T01:26:03.078750+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41541,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":129576960,"data_stored":129559565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182578,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518680,"kb_used_data":63624,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853160,"statfs":{"total":96636764160,"available":96105635840,"internally_reserved":0,"allocated":65150976,"data_stored":64795957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986351,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":429228,"kb_used_data":64412,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93942612,"statfs":{"total":96636764160,"available":96197234688,"internally_reserved":0,"allocated":65957888,"data_stored":65606946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659295,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501408,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":500859,"kb_avail":93870432,"statfs":{"total":96636764160,"available":96123322368,"internally_reserved":0,"allocated":540672,"data_stored":178695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512880303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":64942080,"data_stored":64938601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:27.429 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:27.740 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:27.741 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:26:27.757 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":518,"stamp":"2024-09-11T01:26:26.875899+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41542,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1449316,"kb_used_data":128564,"kb_used_omap":12,"kb_used_meta":1320691,"kb_avail":281666204,"statfs":{"total":289910292480,"available":288426192896,"internally_reserved":0,"allocated":131649536,"data_stored":130581598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1352387588},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-720,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":720,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003669"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28536,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985357+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:02.985357+0000","last_peered":"2024-09-11T01:26:02.985357+0000","last_clean":"2024-09-11T01:26:02.985357+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:02.985357+0000","last_undegraded":"2024-09-11T01:26:02.985357+0000","last_fullsized":"2024-09-11T01:26:02.985357+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20494,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984919+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:26:02.984919+0000","last_peered":"2024-09-11T01:26:02.984919+0000","last_clean":"2024-09-11T01:26:02.984919+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:26:02.984919+0000","last_undegraded":"2024-09-11T01:26:02.984919+0000","last_fullsized":"2024-09-11T01:26:02.984919+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20317,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20244,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984629+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:26:02.984629+0000","last_peered":"2024-09-11T01:26:02.984629+0000","last_clean":"2024-09-11T01:26:02.984629+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:26:02.984629+0000","last_undegraded":"2024-09-11T01:26:02.984629+0000","last_fullsized":"2024-09-11T01:26:02.984629+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20177,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984547+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:02.984547+0000","last_peered":"2024-09-11T01:26:02.984547+0000","last_clean":"2024-09-11T01:26:02.984547+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:02.984547+0000","last_undegraded":"2024-09-11T01:26:02.984547+0000","last_fullsized":"2024-09-11T01:26:02.984547+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20423,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986863+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:02.986863+0000","last_peered":"2024-09-11T01:26:02.986863+0000","last_clean":"2024-09-11T01:26:02.986863+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:02.986863+0000","last_undegraded":"2024-09-11T01:26:02.986863+0000","last_fullsized":"2024-09-11T01:26:02.986863+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":23854,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:23.304712+0000","last_change":"2024-09-11T01:26:01.184322+0000","last_active":"2024-09-11T01:26:23.304712+0000","last_peered":"2024-09-11T01:26:23.304712+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:26:23.304712+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2112,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1876,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2112}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982464+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:02.982464+0000","last_peered":"2024-09-11T01:26:02.982464+0000","last_clean":"2024-09-11T01:26:02.982464+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:02.982464+0000","last_undegraded":"2024-09-11T01:26:02.982464+0000","last_fullsized":"2024-09-11T01:26:02.982464+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986745+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:02.986745+0000","last_peered":"2024-09-11T01:26:02.986745+0000","last_clean":"2024-09-11T01:26:02.986745+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:02.986745+0000","last_undegraded":"2024-09-11T01:26:02.986745+0000","last_fullsized":"2024-09-11T01:26:02.986745+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28021,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989204+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:02.989204+0000","last_peered":"2024-09-11T01:26:02.989204+0000","last_clean":"2024-09-11T01:26:02.989204+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:02.989204+0000","last_undegraded":"2024-09-11T01:26:02.989204+0000","last_fullsized":"2024-09-11T01:26:02.989204+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982959+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:26:02.982959+0000","last_peered":"2024-09-11T01:26:02.982959+0000","last_clean":"2024-09-11T01:26:02.982959+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:26:02.982959+0000","last_undegraded":"2024-09-11T01:26:02.982959+0000","last_fullsized":"2024-09-11T01:26:02.982959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20037,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:02.982813+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:26:02.982813+0000","last_peered":"2024-09-11T01:26:02.982813+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:26:02.982813+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982616+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:02.982616+0000","last_peered":"2024-09-11T01:26:02.982616+0000","last_clean":"2024-09-11T01:26:02.982616+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:02.982616+0000","last_undegraded":"2024-09-11T01:26:02.982616+0000","last_fullsized":"2024-09-11T01:26:02.982616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28426,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.983065+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:26:02.983065+0000","last_peered":"2024-09-11T01:26:02.983065+0000","last_clean":"2024-09-11T01:26:02.983065+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:26:02.983065+0000","last_undegraded":"2024-09-11T01:26:02.983065+0000","last_fullsized":"2024-09-11T01:26:02.983065+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989042+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:02.989042+0000","last_peered":"2024-09-11T01:26:02.989042+0000","last_clean":"2024-09-11T01:26:02.989042+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:02.989042+0000","last_undegraded":"2024-09-11T01:26:02.989042+0000","last_fullsized":"2024-09-11T01:26:02.989042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982786+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:26:02.982786+0000","last_peered":"2024-09-11T01:26:02.982786+0000","last_clean":"2024-09-11T01:26:02.982786+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:26:02.982786+0000","last_undegraded":"2024-09-11T01:26:02.982786+0000","last_fullsized":"2024-09-11T01:26:02.982786+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20427,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989519+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:02.989519+0000","last_peered":"2024-09-11T01:26:02.989519+0000","last_clean":"2024-09-11T01:26:02.989519+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:02.989519+0000","last_undegraded":"2024-09-11T01:26:02.989519+0000","last_fullsized":"2024-09-11T01:26:02.989519+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989097+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:02.989097+0000","last_peered":"2024-09-11T01:26:02.989097+0000","last_clean":"2024-09-11T01:26:02.989097+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:02.989097+0000","last_undegraded":"2024-09-11T01:26:02.989097+0000","last_fullsized":"2024-09-11T01:26:02.989097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27955,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989371+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:02.989371+0000","last_peered":"2024-09-11T01:26:02.989371+0000","last_clean":"2024-09-11T01:26:02.989371+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:02.989371+0000","last_undegraded":"2024-09-11T01:26:02.989371+0000","last_fullsized":"2024-09-11T01:26:02.989371+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985710+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:26:02.985710+0000","last_peered":"2024-09-11T01:26:02.985710+0000","last_clean":"2024-09-11T01:26:02.985710+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:26:02.985710+0000","last_undegraded":"2024-09-11T01:26:02.985710+0000","last_fullsized":"2024-09-11T01:26:02.985710+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24634,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989242+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:02.989242+0000","last_peered":"2024-09-11T01:26:02.989242+0000","last_clean":"2024-09-11T01:26:02.989242+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:02.989242+0000","last_undegraded":"2024-09-11T01:26:02.989242+0000","last_fullsized":"2024-09-11T01:26:02.989242+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989422+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:02.989422+0000","last_peered":"2024-09-11T01:26:02.989422+0000","last_clean":"2024-09-11T01:26:02.989422+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:02.989422+0000","last_undegraded":"2024-09-11T01:26:02.989422+0000","last_fullsized":"2024-09-11T01:26:02.989422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20107,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:03.078750+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:03.078750+0000","last_peered":"2024-09-11T01:26:03.078750+0000","last_clean":"2024-09-11T01:26:03.078750+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:03.078750+0000","last_undegraded":"2024-09-11T01:26:03.078750+0000","last_fullsized":"2024-09-11T01:26:03.078750+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41541,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":129576960,"data_stored":129559565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182578,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518680,"kb_used_data":63624,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853160,"statfs":{"total":96636764160,"available":96105635840,"internally_reserved":0,"allocated":65150976,"data_stored":64795957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986351,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":429228,"kb_used_data":64412,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93942612,"statfs":{"total":96636764160,"available":96197234688,"internally_reserved":0,"allocated":65957888,"data_stored":65606946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659295,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501408,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":500859,"kb_avail":93870432,"statfs":{"total":96636764160,"available":96123322368,"internally_reserved":0,"allocated":540672,"data_stored":178695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512880303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":64942080,"data_stored":64938601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:27.759 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:26:27.759 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 23854, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:23.304712+0000', 'last_change': '2024-09-11T01:26:01.184322+0000', 'last_active': '2024-09-11T01:26:23.304712+0000', 'last_peered': '2024-09-11T01:26:23.304712+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:26:23.304712+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 2112, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1876, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -2112}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:27.759 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:26:27.759 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20037, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:02.982813+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:26:02.982813+0000', 'last_peered': '2024-09-11T01:26:02.982813+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:26:02.982813+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:27.759 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:26:28.145 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:28.145 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 6098/6 objects degraded (101633.333%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":924,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1484099584,"bytes_avail":288426192896,"bytes_total":289910292480,"degraded_objects":6098,"degraded_total":6,"degraded_ratio":1016.3333333333335,"recovering_objects_per_sec":59,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":720,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:26:28.163 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:26:28.163 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:28.474 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:28.474 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:26:28.491 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":518,"stamp":"2024-09-11T01:26:26.875899+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41542,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1449316,"kb_used_data":128564,"kb_used_omap":12,"kb_used_meta":1320691,"kb_avail":281666204,"statfs":{"total":289910292480,"available":288426192896,"internally_reserved":0,"allocated":131649536,"data_stored":130581598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1352387588},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-720,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":720,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003669"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28536,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985357+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:02.985357+0000","last_peered":"2024-09-11T01:26:02.985357+0000","last_clean":"2024-09-11T01:26:02.985357+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:02.985357+0000","last_undegraded":"2024-09-11T01:26:02.985357+0000","last_fullsized":"2024-09-11T01:26:02.985357+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20494,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984919+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:26:02.984919+0000","last_peered":"2024-09-11T01:26:02.984919+0000","last_clean":"2024-09-11T01:26:02.984919+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:26:02.984919+0000","last_undegraded":"2024-09-11T01:26:02.984919+0000","last_fullsized":"2024-09-11T01:26:02.984919+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20317,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20244,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984629+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:26:02.984629+0000","last_peered":"2024-09-11T01:26:02.984629+0000","last_clean":"2024-09-11T01:26:02.984629+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:26:02.984629+0000","last_undegraded":"2024-09-11T01:26:02.984629+0000","last_fullsized":"2024-09-11T01:26:02.984629+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20177,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984547+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:02.984547+0000","last_peered":"2024-09-11T01:26:02.984547+0000","last_clean":"2024-09-11T01:26:02.984547+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:02.984547+0000","last_undegraded":"2024-09-11T01:26:02.984547+0000","last_fullsized":"2024-09-11T01:26:02.984547+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20423,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986863+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:02.986863+0000","last_peered":"2024-09-11T01:26:02.986863+0000","last_clean":"2024-09-11T01:26:02.986863+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:02.986863+0000","last_undegraded":"2024-09-11T01:26:02.986863+0000","last_fullsized":"2024-09-11T01:26:02.986863+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":23854,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:23.304712+0000","last_change":"2024-09-11T01:26:01.184322+0000","last_active":"2024-09-11T01:26:23.304712+0000","last_peered":"2024-09-11T01:26:23.304712+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:26:23.304712+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2112,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1876,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2112}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982464+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:02.982464+0000","last_peered":"2024-09-11T01:26:02.982464+0000","last_clean":"2024-09-11T01:26:02.982464+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:02.982464+0000","last_undegraded":"2024-09-11T01:26:02.982464+0000","last_fullsized":"2024-09-11T01:26:02.982464+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986745+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:02.986745+0000","last_peered":"2024-09-11T01:26:02.986745+0000","last_clean":"2024-09-11T01:26:02.986745+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:02.986745+0000","last_undegraded":"2024-09-11T01:26:02.986745+0000","last_fullsized":"2024-09-11T01:26:02.986745+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28021,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989204+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:02.989204+0000","last_peered":"2024-09-11T01:26:02.989204+0000","last_clean":"2024-09-11T01:26:02.989204+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:02.989204+0000","last_undegraded":"2024-09-11T01:26:02.989204+0000","last_fullsized":"2024-09-11T01:26:02.989204+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982959+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:26:02.982959+0000","last_peered":"2024-09-11T01:26:02.982959+0000","last_clean":"2024-09-11T01:26:02.982959+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:26:02.982959+0000","last_undegraded":"2024-09-11T01:26:02.982959+0000","last_fullsized":"2024-09-11T01:26:02.982959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20037,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:02.982813+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:26:02.982813+0000","last_peered":"2024-09-11T01:26:02.982813+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:26:02.982813+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982616+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:02.982616+0000","last_peered":"2024-09-11T01:26:02.982616+0000","last_clean":"2024-09-11T01:26:02.982616+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:02.982616+0000","last_undegraded":"2024-09-11T01:26:02.982616+0000","last_fullsized":"2024-09-11T01:26:02.982616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28426,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.983065+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:26:02.983065+0000","last_peered":"2024-09-11T01:26:02.983065+0000","last_clean":"2024-09-11T01:26:02.983065+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:26:02.983065+0000","last_undegraded":"2024-09-11T01:26:02.983065+0000","last_fullsized":"2024-09-11T01:26:02.983065+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989042+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:02.989042+0000","last_peered":"2024-09-11T01:26:02.989042+0000","last_clean":"2024-09-11T01:26:02.989042+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:02.989042+0000","last_undegraded":"2024-09-11T01:26:02.989042+0000","last_fullsized":"2024-09-11T01:26:02.989042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982786+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:26:02.982786+0000","last_peered":"2024-09-11T01:26:02.982786+0000","last_clean":"2024-09-11T01:26:02.982786+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:26:02.982786+0000","last_undegraded":"2024-09-11T01:26:02.982786+0000","last_fullsized":"2024-09-11T01:26:02.982786+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20427,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989519+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:02.989519+0000","last_peered":"2024-09-11T01:26:02.989519+0000","last_clean":"2024-09-11T01:26:02.989519+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:02.989519+0000","last_undegraded":"2024-09-11T01:26:02.989519+0000","last_fullsized":"2024-09-11T01:26:02.989519+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989097+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:02.989097+0000","last_peered":"2024-09-11T01:26:02.989097+0000","last_clean":"2024-09-11T01:26:02.989097+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:02.989097+0000","last_undegraded":"2024-09-11T01:26:02.989097+0000","last_fullsized":"2024-09-11T01:26:02.989097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27955,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989371+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:02.989371+0000","last_peered":"2024-09-11T01:26:02.989371+0000","last_clean":"2024-09-11T01:26:02.989371+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:02.989371+0000","last_undegraded":"2024-09-11T01:26:02.989371+0000","last_fullsized":"2024-09-11T01:26:02.989371+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985710+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:26:02.985710+0000","last_peered":"2024-09-11T01:26:02.985710+0000","last_clean":"2024-09-11T01:26:02.985710+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:26:02.985710+0000","last_undegraded":"2024-09-11T01:26:02.985710+0000","last_fullsized":"2024-09-11T01:26:02.985710+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24634,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989242+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:02.989242+0000","last_peered":"2024-09-11T01:26:02.989242+0000","last_clean":"2024-09-11T01:26:02.989242+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:02.989242+0000","last_undegraded":"2024-09-11T01:26:02.989242+0000","last_fullsized":"2024-09-11T01:26:02.989242+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989422+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:02.989422+0000","last_peered":"2024-09-11T01:26:02.989422+0000","last_clean":"2024-09-11T01:26:02.989422+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:02.989422+0000","last_undegraded":"2024-09-11T01:26:02.989422+0000","last_fullsized":"2024-09-11T01:26:02.989422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20107,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:03.078750+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:03.078750+0000","last_peered":"2024-09-11T01:26:03.078750+0000","last_clean":"2024-09-11T01:26:03.078750+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:03.078750+0000","last_undegraded":"2024-09-11T01:26:03.078750+0000","last_fullsized":"2024-09-11T01:26:03.078750+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":6098,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41541,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":129576960,"data_stored":129559565,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182578,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518680,"kb_used_data":63624,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853160,"statfs":{"total":96636764160,"available":96105635840,"internally_reserved":0,"allocated":65150976,"data_stored":64795957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986351,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":429228,"kb_used_data":64412,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93942612,"statfs":{"total":96636764160,"available":96197234688,"internally_reserved":0,"allocated":65957888,"data_stored":65606946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659295,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501408,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":500859,"kb_avail":93870432,"statfs":{"total":96636764160,"available":96123322368,"internally_reserved":0,"allocated":540672,"data_stored":178695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512880303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":64942080,"data_stored":64938601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:31.495 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:31.833 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:31.833 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:26:31.851 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":520,"stamp":"2024-09-11T01:26:30.877325+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5662,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41978,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1449876,"kb_used_data":124964,"kb_used_omap":12,"kb_used_meta":1324851,"kb_avail":281665644,"statfs":{"total":289910292480,"available":288425619456,"internally_reserved":0,"allocated":127963136,"data_stored":126895198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1356647428},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-823,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":823,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003645"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28536,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985357+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:02.985357+0000","last_peered":"2024-09-11T01:26:02.985357+0000","last_clean":"2024-09-11T01:26:02.985357+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:02.985357+0000","last_undegraded":"2024-09-11T01:26:02.985357+0000","last_fullsized":"2024-09-11T01:26:02.985357+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20494,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984919+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:26:02.984919+0000","last_peered":"2024-09-11T01:26:02.984919+0000","last_clean":"2024-09-11T01:26:02.984919+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:26:02.984919+0000","last_undegraded":"2024-09-11T01:26:02.984919+0000","last_fullsized":"2024-09-11T01:26:02.984919+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20317,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20244,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984629+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:26:02.984629+0000","last_peered":"2024-09-11T01:26:02.984629+0000","last_clean":"2024-09-11T01:26:02.984629+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:26:02.984629+0000","last_undegraded":"2024-09-11T01:26:02.984629+0000","last_fullsized":"2024-09-11T01:26:02.984629+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20177,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984547+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:02.984547+0000","last_peered":"2024-09-11T01:26:02.984547+0000","last_clean":"2024-09-11T01:26:02.984547+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:02.984547+0000","last_undegraded":"2024-09-11T01:26:02.984547+0000","last_fullsized":"2024-09-11T01:26:02.984547+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20423,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986863+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:02.986863+0000","last_peered":"2024-09-11T01:26:02.986863+0000","last_clean":"2024-09-11T01:26:02.986863+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:02.986863+0000","last_undegraded":"2024-09-11T01:26:02.986863+0000","last_fullsized":"2024-09-11T01:26:02.986863+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":24726,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:28.304595+0000","last_change":"2024-09-11T01:26:01.184322+0000","last_active":"2024-09-11T01:26:28.304595+0000","last_peered":"2024-09-11T01:26:28.304595+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:26:28.304595+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1676,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2312,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1676}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982464+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:02.982464+0000","last_peered":"2024-09-11T01:26:02.982464+0000","last_clean":"2024-09-11T01:26:02.982464+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:02.982464+0000","last_undegraded":"2024-09-11T01:26:02.982464+0000","last_fullsized":"2024-09-11T01:26:02.982464+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986745+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:02.986745+0000","last_peered":"2024-09-11T01:26:02.986745+0000","last_clean":"2024-09-11T01:26:02.986745+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:02.986745+0000","last_undegraded":"2024-09-11T01:26:02.986745+0000","last_fullsized":"2024-09-11T01:26:02.986745+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28021,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989204+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:02.989204+0000","last_peered":"2024-09-11T01:26:02.989204+0000","last_clean":"2024-09-11T01:26:02.989204+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:02.989204+0000","last_undegraded":"2024-09-11T01:26:02.989204+0000","last_fullsized":"2024-09-11T01:26:02.989204+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982959+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:26:02.982959+0000","last_peered":"2024-09-11T01:26:02.982959+0000","last_clean":"2024-09-11T01:26:02.982959+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:26:02.982959+0000","last_undegraded":"2024-09-11T01:26:02.982959+0000","last_fullsized":"2024-09-11T01:26:02.982959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20037,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:02.982813+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:26:02.982813+0000","last_peered":"2024-09-11T01:26:02.982813+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:26:02.982813+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982616+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:02.982616+0000","last_peered":"2024-09-11T01:26:02.982616+0000","last_clean":"2024-09-11T01:26:02.982616+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:02.982616+0000","last_undegraded":"2024-09-11T01:26:02.982616+0000","last_fullsized":"2024-09-11T01:26:02.982616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28426,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.983065+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:26:02.983065+0000","last_peered":"2024-09-11T01:26:02.983065+0000","last_clean":"2024-09-11T01:26:02.983065+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:26:02.983065+0000","last_undegraded":"2024-09-11T01:26:02.983065+0000","last_fullsized":"2024-09-11T01:26:02.983065+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989042+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:02.989042+0000","last_peered":"2024-09-11T01:26:02.989042+0000","last_clean":"2024-09-11T01:26:02.989042+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:02.989042+0000","last_undegraded":"2024-09-11T01:26:02.989042+0000","last_fullsized":"2024-09-11T01:26:02.989042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982786+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:26:02.982786+0000","last_peered":"2024-09-11T01:26:02.982786+0000","last_clean":"2024-09-11T01:26:02.982786+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:26:02.982786+0000","last_undegraded":"2024-09-11T01:26:02.982786+0000","last_fullsized":"2024-09-11T01:26:02.982786+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20427,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989519+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:02.989519+0000","last_peered":"2024-09-11T01:26:02.989519+0000","last_clean":"2024-09-11T01:26:02.989519+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:02.989519+0000","last_undegraded":"2024-09-11T01:26:02.989519+0000","last_fullsized":"2024-09-11T01:26:02.989519+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989097+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:02.989097+0000","last_peered":"2024-09-11T01:26:02.989097+0000","last_clean":"2024-09-11T01:26:02.989097+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:02.989097+0000","last_undegraded":"2024-09-11T01:26:02.989097+0000","last_fullsized":"2024-09-11T01:26:02.989097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27955,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989371+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:02.989371+0000","last_peered":"2024-09-11T01:26:02.989371+0000","last_clean":"2024-09-11T01:26:02.989371+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:02.989371+0000","last_undegraded":"2024-09-11T01:26:02.989371+0000","last_fullsized":"2024-09-11T01:26:02.989371+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985710+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:26:02.985710+0000","last_peered":"2024-09-11T01:26:02.985710+0000","last_clean":"2024-09-11T01:26:02.985710+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:26:02.985710+0000","last_undegraded":"2024-09-11T01:26:02.985710+0000","last_fullsized":"2024-09-11T01:26:02.985710+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24634,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989242+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:02.989242+0000","last_peered":"2024-09-11T01:26:02.989242+0000","last_clean":"2024-09-11T01:26:02.989242+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:02.989242+0000","last_undegraded":"2024-09-11T01:26:02.989242+0000","last_fullsized":"2024-09-11T01:26:02.989242+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989422+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:02.989422+0000","last_peered":"2024-09-11T01:26:02.989422+0000","last_clean":"2024-09-11T01:26:02.989422+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:02.989422+0000","last_undegraded":"2024-09-11T01:26:02.989422+0000","last_fullsized":"2024-09-11T01:26:02.989422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20107,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:03.078750+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:03.078750+0000","last_peered":"2024-09-11T01:26:03.078750+0000","last_clean":"2024-09-11T01:26:03.078750+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:03.078750+0000","last_undegraded":"2024-09-11T01:26:03.078750+0000","last_fullsized":"2024-09-11T01:26:03.078750+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5662,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41977,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":125890560,"data_stored":125873165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182579,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518680,"kb_used_data":63624,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853160,"statfs":{"total":96636764160,"available":96105635840,"internally_reserved":0,"allocated":65150976,"data_stored":64795957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986352,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425628,"kb_used_data":60812,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93946212,"statfs":{"total":96636764160,"available":96200921088,"internally_reserved":0,"allocated":62271488,"data_stored":61920546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659296,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505568,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":505019,"kb_avail":93866272,"statfs":{"total":96636764160,"available":96119062528,"internally_reserved":0,"allocated":540672,"data_stored":178695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517140143},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61255680,"data_stored":61252201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:31.852 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:32.187 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:32.187 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:26:32.207 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":520,"stamp":"2024-09-11T01:26:30.877325+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5662,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41978,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1449876,"kb_used_data":124964,"kb_used_omap":12,"kb_used_meta":1324851,"kb_avail":281665644,"statfs":{"total":289910292480,"available":288425619456,"internally_reserved":0,"allocated":127963136,"data_stored":126895198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1356647428},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-823,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":823,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003645"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28536,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985357+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:02.985357+0000","last_peered":"2024-09-11T01:26:02.985357+0000","last_clean":"2024-09-11T01:26:02.985357+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:02.985357+0000","last_undegraded":"2024-09-11T01:26:02.985357+0000","last_fullsized":"2024-09-11T01:26:02.985357+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20494,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984919+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:26:02.984919+0000","last_peered":"2024-09-11T01:26:02.984919+0000","last_clean":"2024-09-11T01:26:02.984919+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:26:02.984919+0000","last_undegraded":"2024-09-11T01:26:02.984919+0000","last_fullsized":"2024-09-11T01:26:02.984919+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20317,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20244,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984629+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:26:02.984629+0000","last_peered":"2024-09-11T01:26:02.984629+0000","last_clean":"2024-09-11T01:26:02.984629+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:26:02.984629+0000","last_undegraded":"2024-09-11T01:26:02.984629+0000","last_fullsized":"2024-09-11T01:26:02.984629+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20177,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984547+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:02.984547+0000","last_peered":"2024-09-11T01:26:02.984547+0000","last_clean":"2024-09-11T01:26:02.984547+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:02.984547+0000","last_undegraded":"2024-09-11T01:26:02.984547+0000","last_fullsized":"2024-09-11T01:26:02.984547+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20423,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986863+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:02.986863+0000","last_peered":"2024-09-11T01:26:02.986863+0000","last_clean":"2024-09-11T01:26:02.986863+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:02.986863+0000","last_undegraded":"2024-09-11T01:26:02.986863+0000","last_fullsized":"2024-09-11T01:26:02.986863+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":24726,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:28.304595+0000","last_change":"2024-09-11T01:26:01.184322+0000","last_active":"2024-09-11T01:26:28.304595+0000","last_peered":"2024-09-11T01:26:28.304595+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:26:28.304595+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1676,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2312,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1676}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982464+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:02.982464+0000","last_peered":"2024-09-11T01:26:02.982464+0000","last_clean":"2024-09-11T01:26:02.982464+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:02.982464+0000","last_undegraded":"2024-09-11T01:26:02.982464+0000","last_fullsized":"2024-09-11T01:26:02.982464+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986745+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:02.986745+0000","last_peered":"2024-09-11T01:26:02.986745+0000","last_clean":"2024-09-11T01:26:02.986745+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:02.986745+0000","last_undegraded":"2024-09-11T01:26:02.986745+0000","last_fullsized":"2024-09-11T01:26:02.986745+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28021,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989204+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:02.989204+0000","last_peered":"2024-09-11T01:26:02.989204+0000","last_clean":"2024-09-11T01:26:02.989204+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:02.989204+0000","last_undegraded":"2024-09-11T01:26:02.989204+0000","last_fullsized":"2024-09-11T01:26:02.989204+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982959+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:26:02.982959+0000","last_peered":"2024-09-11T01:26:02.982959+0000","last_clean":"2024-09-11T01:26:02.982959+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:26:02.982959+0000","last_undegraded":"2024-09-11T01:26:02.982959+0000","last_fullsized":"2024-09-11T01:26:02.982959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20037,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:02.982813+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:26:02.982813+0000","last_peered":"2024-09-11T01:26:02.982813+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:26:02.982813+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982616+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:02.982616+0000","last_peered":"2024-09-11T01:26:02.982616+0000","last_clean":"2024-09-11T01:26:02.982616+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:02.982616+0000","last_undegraded":"2024-09-11T01:26:02.982616+0000","last_fullsized":"2024-09-11T01:26:02.982616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28426,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.983065+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:26:02.983065+0000","last_peered":"2024-09-11T01:26:02.983065+0000","last_clean":"2024-09-11T01:26:02.983065+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:26:02.983065+0000","last_undegraded":"2024-09-11T01:26:02.983065+0000","last_fullsized":"2024-09-11T01:26:02.983065+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989042+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:02.989042+0000","last_peered":"2024-09-11T01:26:02.989042+0000","last_clean":"2024-09-11T01:26:02.989042+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:02.989042+0000","last_undegraded":"2024-09-11T01:26:02.989042+0000","last_fullsized":"2024-09-11T01:26:02.989042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982786+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:26:02.982786+0000","last_peered":"2024-09-11T01:26:02.982786+0000","last_clean":"2024-09-11T01:26:02.982786+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:26:02.982786+0000","last_undegraded":"2024-09-11T01:26:02.982786+0000","last_fullsized":"2024-09-11T01:26:02.982786+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20427,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989519+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:02.989519+0000","last_peered":"2024-09-11T01:26:02.989519+0000","last_clean":"2024-09-11T01:26:02.989519+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:02.989519+0000","last_undegraded":"2024-09-11T01:26:02.989519+0000","last_fullsized":"2024-09-11T01:26:02.989519+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989097+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:02.989097+0000","last_peered":"2024-09-11T01:26:02.989097+0000","last_clean":"2024-09-11T01:26:02.989097+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:02.989097+0000","last_undegraded":"2024-09-11T01:26:02.989097+0000","last_fullsized":"2024-09-11T01:26:02.989097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27955,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989371+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:02.989371+0000","last_peered":"2024-09-11T01:26:02.989371+0000","last_clean":"2024-09-11T01:26:02.989371+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:02.989371+0000","last_undegraded":"2024-09-11T01:26:02.989371+0000","last_fullsized":"2024-09-11T01:26:02.989371+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985710+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:26:02.985710+0000","last_peered":"2024-09-11T01:26:02.985710+0000","last_clean":"2024-09-11T01:26:02.985710+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:26:02.985710+0000","last_undegraded":"2024-09-11T01:26:02.985710+0000","last_fullsized":"2024-09-11T01:26:02.985710+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24634,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989242+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:02.989242+0000","last_peered":"2024-09-11T01:26:02.989242+0000","last_clean":"2024-09-11T01:26:02.989242+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:02.989242+0000","last_undegraded":"2024-09-11T01:26:02.989242+0000","last_fullsized":"2024-09-11T01:26:02.989242+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989422+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:02.989422+0000","last_peered":"2024-09-11T01:26:02.989422+0000","last_clean":"2024-09-11T01:26:02.989422+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:02.989422+0000","last_undegraded":"2024-09-11T01:26:02.989422+0000","last_fullsized":"2024-09-11T01:26:02.989422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20107,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:03.078750+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:03.078750+0000","last_peered":"2024-09-11T01:26:03.078750+0000","last_clean":"2024-09-11T01:26:03.078750+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:03.078750+0000","last_undegraded":"2024-09-11T01:26:03.078750+0000","last_fullsized":"2024-09-11T01:26:03.078750+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5662,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41977,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":125890560,"data_stored":125873165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182579,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518680,"kb_used_data":63624,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853160,"statfs":{"total":96636764160,"available":96105635840,"internally_reserved":0,"allocated":65150976,"data_stored":64795957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986352,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425628,"kb_used_data":60812,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93946212,"statfs":{"total":96636764160,"available":96200921088,"internally_reserved":0,"allocated":62271488,"data_stored":61920546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659296,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505568,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":505019,"kb_avail":93866272,"statfs":{"total":96636764160,"available":96119062528,"internally_reserved":0,"allocated":540672,"data_stored":178695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517140143},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61255680,"data_stored":61252201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:32.208 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:26:32.208 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 24726, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:28.304595+0000', 'last_change': '2024-09-11T01:26:01.184322+0000', 'last_active': '2024-09-11T01:26:28.304595+0000', 'last_peered': '2024-09-11T01:26:28.304595+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:26:28.304595+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1676, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2312, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1676}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:32.208 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:26:32.208 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20037, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:02.982813+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:26:02.982813+0000', 'last_peered': '2024-09-11T01:26:02.982813+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:26:02.982813+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:32.209 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:26:32.580 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:32.580 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 5662/6 objects degraded (94366.667%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":929,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1484673024,"bytes_avail":288425619456,"bytes_total":289910292480,"degraded_objects":5662,"degraded_total":6,"degraded_ratio":943.66666666666652,"recovering_objects_per_sec":68,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":823,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:26:32.600 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:26:32.600 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:32.918 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:32.918 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:26:32.936 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":521,"stamp":"2024-09-11T01:26:32.878007+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5662,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41978,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1449876,"kb_used_data":124964,"kb_used_omap":12,"kb_used_meta":1324851,"kb_avail":281665644,"statfs":{"total":289910292480,"available":288425619456,"internally_reserved":0,"allocated":127963136,"data_stored":126895198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1356647428},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-823,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":823,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003848"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28536,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985357+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:02.985357+0000","last_peered":"2024-09-11T01:26:02.985357+0000","last_clean":"2024-09-11T01:26:02.985357+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:02.985357+0000","last_undegraded":"2024-09-11T01:26:02.985357+0000","last_fullsized":"2024-09-11T01:26:02.985357+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20494,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984919+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:26:02.984919+0000","last_peered":"2024-09-11T01:26:02.984919+0000","last_clean":"2024-09-11T01:26:02.984919+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:26:02.984919+0000","last_undegraded":"2024-09-11T01:26:02.984919+0000","last_fullsized":"2024-09-11T01:26:02.984919+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20317,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20244,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984629+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:26:02.984629+0000","last_peered":"2024-09-11T01:26:02.984629+0000","last_clean":"2024-09-11T01:26:02.984629+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:26:02.984629+0000","last_undegraded":"2024-09-11T01:26:02.984629+0000","last_fullsized":"2024-09-11T01:26:02.984629+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20177,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984547+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:02.984547+0000","last_peered":"2024-09-11T01:26:02.984547+0000","last_clean":"2024-09-11T01:26:02.984547+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:02.984547+0000","last_undegraded":"2024-09-11T01:26:02.984547+0000","last_fullsized":"2024-09-11T01:26:02.984547+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20423,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986863+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:02.986863+0000","last_peered":"2024-09-11T01:26:02.986863+0000","last_clean":"2024-09-11T01:26:02.986863+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:02.986863+0000","last_undegraded":"2024-09-11T01:26:02.986863+0000","last_fullsized":"2024-09-11T01:26:02.986863+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":24726,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:28.304595+0000","last_change":"2024-09-11T01:26:01.184322+0000","last_active":"2024-09-11T01:26:28.304595+0000","last_peered":"2024-09-11T01:26:28.304595+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:26:28.304595+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1676,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2312,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1676}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982464+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:02.982464+0000","last_peered":"2024-09-11T01:26:02.982464+0000","last_clean":"2024-09-11T01:26:02.982464+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:02.982464+0000","last_undegraded":"2024-09-11T01:26:02.982464+0000","last_fullsized":"2024-09-11T01:26:02.982464+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986745+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:02.986745+0000","last_peered":"2024-09-11T01:26:02.986745+0000","last_clean":"2024-09-11T01:26:02.986745+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:02.986745+0000","last_undegraded":"2024-09-11T01:26:02.986745+0000","last_fullsized":"2024-09-11T01:26:02.986745+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28021,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989204+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:02.989204+0000","last_peered":"2024-09-11T01:26:02.989204+0000","last_clean":"2024-09-11T01:26:02.989204+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:02.989204+0000","last_undegraded":"2024-09-11T01:26:02.989204+0000","last_fullsized":"2024-09-11T01:26:02.989204+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982959+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:26:02.982959+0000","last_peered":"2024-09-11T01:26:02.982959+0000","last_clean":"2024-09-11T01:26:02.982959+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:26:02.982959+0000","last_undegraded":"2024-09-11T01:26:02.982959+0000","last_fullsized":"2024-09-11T01:26:02.982959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20037,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:02.982813+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:26:02.982813+0000","last_peered":"2024-09-11T01:26:02.982813+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:26:02.982813+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":141,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982616+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:02.982616+0000","last_peered":"2024-09-11T01:26:02.982616+0000","last_clean":"2024-09-11T01:26:02.982616+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:02.982616+0000","last_undegraded":"2024-09-11T01:26:02.982616+0000","last_fullsized":"2024-09-11T01:26:02.982616+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28426,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.983065+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:26:02.983065+0000","last_peered":"2024-09-11T01:26:02.983065+0000","last_clean":"2024-09-11T01:26:02.983065+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:26:02.983065+0000","last_undegraded":"2024-09-11T01:26:02.983065+0000","last_fullsized":"2024-09-11T01:26:02.983065+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989042+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:02.989042+0000","last_peered":"2024-09-11T01:26:02.989042+0000","last_clean":"2024-09-11T01:26:02.989042+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:02.989042+0000","last_undegraded":"2024-09-11T01:26:02.989042+0000","last_fullsized":"2024-09-11T01:26:02.989042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982786+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:26:02.982786+0000","last_peered":"2024-09-11T01:26:02.982786+0000","last_clean":"2024-09-11T01:26:02.982786+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:26:02.982786+0000","last_undegraded":"2024-09-11T01:26:02.982786+0000","last_fullsized":"2024-09-11T01:26:02.982786+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20427,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989519+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:02.989519+0000","last_peered":"2024-09-11T01:26:02.989519+0000","last_clean":"2024-09-11T01:26:02.989519+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:02.989519+0000","last_undegraded":"2024-09-11T01:26:02.989519+0000","last_fullsized":"2024-09-11T01:26:02.989519+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989097+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:02.989097+0000","last_peered":"2024-09-11T01:26:02.989097+0000","last_clean":"2024-09-11T01:26:02.989097+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:02.989097+0000","last_undegraded":"2024-09-11T01:26:02.989097+0000","last_fullsized":"2024-09-11T01:26:02.989097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27955,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989371+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:02.989371+0000","last_peered":"2024-09-11T01:26:02.989371+0000","last_clean":"2024-09-11T01:26:02.989371+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:02.989371+0000","last_undegraded":"2024-09-11T01:26:02.989371+0000","last_fullsized":"2024-09-11T01:26:02.989371+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985710+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:26:02.985710+0000","last_peered":"2024-09-11T01:26:02.985710+0000","last_clean":"2024-09-11T01:26:02.985710+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:26:02.985710+0000","last_undegraded":"2024-09-11T01:26:02.985710+0000","last_fullsized":"2024-09-11T01:26:02.985710+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24634,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989242+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:02.989242+0000","last_peered":"2024-09-11T01:26:02.989242+0000","last_clean":"2024-09-11T01:26:02.989242+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:02.989242+0000","last_undegraded":"2024-09-11T01:26:02.989242+0000","last_fullsized":"2024-09-11T01:26:02.989242+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989422+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:02.989422+0000","last_peered":"2024-09-11T01:26:02.989422+0000","last_clean":"2024-09-11T01:26:02.989422+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:02.989422+0000","last_undegraded":"2024-09-11T01:26:02.989422+0000","last_fullsized":"2024-09-11T01:26:02.989422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20107,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:03.078750+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:03.078750+0000","last_peered":"2024-09-11T01:26:03.078750+0000","last_clean":"2024-09-11T01:26:03.078750+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:03.078750+0000","last_undegraded":"2024-09-11T01:26:03.078750+0000","last_fullsized":"2024-09-11T01:26:03.078750+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5662,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":41977,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":125890560,"data_stored":125873165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182579,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518680,"kb_used_data":63624,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853160,"statfs":{"total":96636764160,"available":96105635840,"internally_reserved":0,"allocated":65150976,"data_stored":64795957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986352,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425628,"kb_used_data":60812,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93946212,"statfs":{"total":96636764160,"available":96200921088,"internally_reserved":0,"allocated":62271488,"data_stored":61920546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659296,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505568,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":505019,"kb_avail":93866272,"statfs":{"total":96636764160,"available":96119062528,"internally_reserved":0,"allocated":540672,"data_stored":178695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517140143},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61255680,"data_stored":61252201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:35.939 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:36.303 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:36.304 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:26:36.325 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":522,"stamp":"2024-09-11T01:26:34.878671+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42465,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1454036,"kb_used_data":124964,"kb_used_omap":12,"kb_used_meta":1329011,"kb_avail":281661484,"statfs":{"total":289910292480,"available":288421359616,"internally_reserved":0,"allocated":127963136,"data_stored":126895198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1360907268},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1310,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1310,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003839"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28536,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985357+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:02.985357+0000","last_peered":"2024-09-11T01:26:02.985357+0000","last_clean":"2024-09-11T01:26:02.985357+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:02.985357+0000","last_undegraded":"2024-09-11T01:26:02.985357+0000","last_fullsized":"2024-09-11T01:26:02.985357+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20494,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984919+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:26:02.984919+0000","last_peered":"2024-09-11T01:26:02.984919+0000","last_clean":"2024-09-11T01:26:02.984919+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:26:02.984919+0000","last_undegraded":"2024-09-11T01:26:02.984919+0000","last_fullsized":"2024-09-11T01:26:02.984919+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20317,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20244,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984629+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:26:02.984629+0000","last_peered":"2024-09-11T01:26:02.984629+0000","last_clean":"2024-09-11T01:26:02.984629+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:26:02.984629+0000","last_undegraded":"2024-09-11T01:26:02.984629+0000","last_fullsized":"2024-09-11T01:26:02.984629+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20177,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984547+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:02.984547+0000","last_peered":"2024-09-11T01:26:02.984547+0000","last_clean":"2024-09-11T01:26:02.984547+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:02.984547+0000","last_undegraded":"2024-09-11T01:26:02.984547+0000","last_fullsized":"2024-09-11T01:26:02.984547+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20423,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986863+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:02.986863+0000","last_peered":"2024-09-11T01:26:02.986863+0000","last_clean":"2024-09-11T01:26:02.986863+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:02.986863+0000","last_undegraded":"2024-09-11T01:26:02.986863+0000","last_fullsized":"2024-09-11T01:26:02.986863+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":25700,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:33.305047+0000","last_change":"2024-09-11T01:26:01.184322+0000","last_active":"2024-09-11T01:26:33.305047+0000","last_peered":"2024-09-11T01:26:33.305047+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:26:33.305047+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1189,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2799,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1189}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982464+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:02.982464+0000","last_peered":"2024-09-11T01:26:02.982464+0000","last_clean":"2024-09-11T01:26:02.982464+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:02.982464+0000","last_undegraded":"2024-09-11T01:26:02.982464+0000","last_fullsized":"2024-09-11T01:26:02.982464+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986745+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:02.986745+0000","last_peered":"2024-09-11T01:26:02.986745+0000","last_clean":"2024-09-11T01:26:02.986745+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:02.986745+0000","last_undegraded":"2024-09-11T01:26:02.986745+0000","last_fullsized":"2024-09-11T01:26:02.986745+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28021,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989204+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:02.989204+0000","last_peered":"2024-09-11T01:26:02.989204+0000","last_clean":"2024-09-11T01:26:02.989204+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:02.989204+0000","last_undegraded":"2024-09-11T01:26:02.989204+0000","last_fullsized":"2024-09-11T01:26:02.989204+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982959+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:26:02.982959+0000","last_peered":"2024-09-11T01:26:02.982959+0000","last_clean":"2024-09-11T01:26:02.982959+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:26:02.982959+0000","last_undegraded":"2024-09-11T01:26:02.982959+0000","last_fullsized":"2024-09-11T01:26:02.982959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20037,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:02.982813+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:26:02.982813+0000","last_peered":"2024-09-11T01:26:02.982813+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:26:02.982813+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":142,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982665+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:02.982665+0000","last_peered":"2024-09-11T01:26:02.982665+0000","last_clean":"2024-09-11T01:26:02.982665+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:02.982665+0000","last_undegraded":"2024-09-11T01:26:02.982665+0000","last_fullsized":"2024-09-11T01:26:02.982665+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28426,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.983065+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:26:02.983065+0000","last_peered":"2024-09-11T01:26:02.983065+0000","last_clean":"2024-09-11T01:26:02.983065+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:26:02.983065+0000","last_undegraded":"2024-09-11T01:26:02.983065+0000","last_fullsized":"2024-09-11T01:26:02.983065+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989042+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:02.989042+0000","last_peered":"2024-09-11T01:26:02.989042+0000","last_clean":"2024-09-11T01:26:02.989042+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:02.989042+0000","last_undegraded":"2024-09-11T01:26:02.989042+0000","last_fullsized":"2024-09-11T01:26:02.989042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982786+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:26:02.982786+0000","last_peered":"2024-09-11T01:26:02.982786+0000","last_clean":"2024-09-11T01:26:02.982786+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:26:02.982786+0000","last_undegraded":"2024-09-11T01:26:02.982786+0000","last_fullsized":"2024-09-11T01:26:02.982786+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20427,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989519+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:02.989519+0000","last_peered":"2024-09-11T01:26:02.989519+0000","last_clean":"2024-09-11T01:26:02.989519+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:02.989519+0000","last_undegraded":"2024-09-11T01:26:02.989519+0000","last_fullsized":"2024-09-11T01:26:02.989519+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989097+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:02.989097+0000","last_peered":"2024-09-11T01:26:02.989097+0000","last_clean":"2024-09-11T01:26:02.989097+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:02.989097+0000","last_undegraded":"2024-09-11T01:26:02.989097+0000","last_fullsized":"2024-09-11T01:26:02.989097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27956,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989418+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:02.989418+0000","last_peered":"2024-09-11T01:26:02.989418+0000","last_clean":"2024-09-11T01:26:02.989418+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:02.989418+0000","last_undegraded":"2024-09-11T01:26:02.989418+0000","last_fullsized":"2024-09-11T01:26:02.989418+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985710+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:26:02.985710+0000","last_peered":"2024-09-11T01:26:02.985710+0000","last_clean":"2024-09-11T01:26:02.985710+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:26:02.985710+0000","last_undegraded":"2024-09-11T01:26:02.985710+0000","last_fullsized":"2024-09-11T01:26:02.985710+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24634,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989242+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:02.989242+0000","last_peered":"2024-09-11T01:26:02.989242+0000","last_clean":"2024-09-11T01:26:02.989242+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:02.989242+0000","last_undegraded":"2024-09-11T01:26:02.989242+0000","last_fullsized":"2024-09-11T01:26:02.989242+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989422+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:02.989422+0000","last_peered":"2024-09-11T01:26:02.989422+0000","last_clean":"2024-09-11T01:26:02.989422+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:02.989422+0000","last_undegraded":"2024-09-11T01:26:02.989422+0000","last_fullsized":"2024-09-11T01:26:02.989422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20107,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:03.078750+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:03.078750+0000","last_peered":"2024-09-11T01:26:03.078750+0000","last_clean":"2024-09-11T01:26:03.078750+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:03.078750+0000","last_undegraded":"2024-09-11T01:26:03.078750+0000","last_fullsized":"2024-09-11T01:26:03.078750+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42464,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":125890560,"data_stored":125873165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182580,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518680,"kb_used_data":63624,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853160,"statfs":{"total":96636764160,"available":96105635840,"internally_reserved":0,"allocated":65150976,"data_stored":64795957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986352,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425628,"kb_used_data":60812,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93946212,"statfs":{"total":96636764160,"available":96200921088,"internally_reserved":0,"allocated":62271488,"data_stored":61920546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659297,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509728,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":509179,"kb_avail":93862112,"statfs":{"total":96636764160,"available":96114802688,"internally_reserved":0,"allocated":540672,"data_stored":178695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521399983},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61255680,"data_stored":61252201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:36.326 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:36.640 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:36.640 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:26:36.658 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":522,"stamp":"2024-09-11T01:26:34.878671+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42465,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1454036,"kb_used_data":124964,"kb_used_omap":12,"kb_used_meta":1329011,"kb_avail":281661484,"statfs":{"total":289910292480,"available":288421359616,"internally_reserved":0,"allocated":127963136,"data_stored":126895198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1360907268},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1310,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1310,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003839"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28536,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985357+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:02.985357+0000","last_peered":"2024-09-11T01:26:02.985357+0000","last_clean":"2024-09-11T01:26:02.985357+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:02.985357+0000","last_undegraded":"2024-09-11T01:26:02.985357+0000","last_fullsized":"2024-09-11T01:26:02.985357+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20494,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984919+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:26:02.984919+0000","last_peered":"2024-09-11T01:26:02.984919+0000","last_clean":"2024-09-11T01:26:02.984919+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:26:02.984919+0000","last_undegraded":"2024-09-11T01:26:02.984919+0000","last_fullsized":"2024-09-11T01:26:02.984919+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20317,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20244,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984629+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:26:02.984629+0000","last_peered":"2024-09-11T01:26:02.984629+0000","last_clean":"2024-09-11T01:26:02.984629+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:26:02.984629+0000","last_undegraded":"2024-09-11T01:26:02.984629+0000","last_fullsized":"2024-09-11T01:26:02.984629+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20177,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984547+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:02.984547+0000","last_peered":"2024-09-11T01:26:02.984547+0000","last_clean":"2024-09-11T01:26:02.984547+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:02.984547+0000","last_undegraded":"2024-09-11T01:26:02.984547+0000","last_fullsized":"2024-09-11T01:26:02.984547+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20423,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986863+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:02.986863+0000","last_peered":"2024-09-11T01:26:02.986863+0000","last_clean":"2024-09-11T01:26:02.986863+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:02.986863+0000","last_undegraded":"2024-09-11T01:26:02.986863+0000","last_fullsized":"2024-09-11T01:26:02.986863+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":25700,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:33.305047+0000","last_change":"2024-09-11T01:26:01.184322+0000","last_active":"2024-09-11T01:26:33.305047+0000","last_peered":"2024-09-11T01:26:33.305047+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:26:33.305047+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1189,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2799,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1189}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982464+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:02.982464+0000","last_peered":"2024-09-11T01:26:02.982464+0000","last_clean":"2024-09-11T01:26:02.982464+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:02.982464+0000","last_undegraded":"2024-09-11T01:26:02.982464+0000","last_fullsized":"2024-09-11T01:26:02.982464+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986745+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:02.986745+0000","last_peered":"2024-09-11T01:26:02.986745+0000","last_clean":"2024-09-11T01:26:02.986745+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:02.986745+0000","last_undegraded":"2024-09-11T01:26:02.986745+0000","last_fullsized":"2024-09-11T01:26:02.986745+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28021,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989204+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:02.989204+0000","last_peered":"2024-09-11T01:26:02.989204+0000","last_clean":"2024-09-11T01:26:02.989204+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:02.989204+0000","last_undegraded":"2024-09-11T01:26:02.989204+0000","last_fullsized":"2024-09-11T01:26:02.989204+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982959+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:26:02.982959+0000","last_peered":"2024-09-11T01:26:02.982959+0000","last_clean":"2024-09-11T01:26:02.982959+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:26:02.982959+0000","last_undegraded":"2024-09-11T01:26:02.982959+0000","last_fullsized":"2024-09-11T01:26:02.982959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20037,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:02.982813+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:26:02.982813+0000","last_peered":"2024-09-11T01:26:02.982813+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:26:02.982813+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":142,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982665+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:02.982665+0000","last_peered":"2024-09-11T01:26:02.982665+0000","last_clean":"2024-09-11T01:26:02.982665+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:02.982665+0000","last_undegraded":"2024-09-11T01:26:02.982665+0000","last_fullsized":"2024-09-11T01:26:02.982665+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28426,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.983065+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:26:02.983065+0000","last_peered":"2024-09-11T01:26:02.983065+0000","last_clean":"2024-09-11T01:26:02.983065+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:26:02.983065+0000","last_undegraded":"2024-09-11T01:26:02.983065+0000","last_fullsized":"2024-09-11T01:26:02.983065+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989042+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:02.989042+0000","last_peered":"2024-09-11T01:26:02.989042+0000","last_clean":"2024-09-11T01:26:02.989042+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:02.989042+0000","last_undegraded":"2024-09-11T01:26:02.989042+0000","last_fullsized":"2024-09-11T01:26:02.989042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982786+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:26:02.982786+0000","last_peered":"2024-09-11T01:26:02.982786+0000","last_clean":"2024-09-11T01:26:02.982786+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:26:02.982786+0000","last_undegraded":"2024-09-11T01:26:02.982786+0000","last_fullsized":"2024-09-11T01:26:02.982786+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20427,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989519+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:02.989519+0000","last_peered":"2024-09-11T01:26:02.989519+0000","last_clean":"2024-09-11T01:26:02.989519+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:02.989519+0000","last_undegraded":"2024-09-11T01:26:02.989519+0000","last_fullsized":"2024-09-11T01:26:02.989519+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989097+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:02.989097+0000","last_peered":"2024-09-11T01:26:02.989097+0000","last_clean":"2024-09-11T01:26:02.989097+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:02.989097+0000","last_undegraded":"2024-09-11T01:26:02.989097+0000","last_fullsized":"2024-09-11T01:26:02.989097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27956,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989418+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:02.989418+0000","last_peered":"2024-09-11T01:26:02.989418+0000","last_clean":"2024-09-11T01:26:02.989418+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:02.989418+0000","last_undegraded":"2024-09-11T01:26:02.989418+0000","last_fullsized":"2024-09-11T01:26:02.989418+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985710+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:26:02.985710+0000","last_peered":"2024-09-11T01:26:02.985710+0000","last_clean":"2024-09-11T01:26:02.985710+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:26:02.985710+0000","last_undegraded":"2024-09-11T01:26:02.985710+0000","last_fullsized":"2024-09-11T01:26:02.985710+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24634,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989242+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:02.989242+0000","last_peered":"2024-09-11T01:26:02.989242+0000","last_clean":"2024-09-11T01:26:02.989242+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:02.989242+0000","last_undegraded":"2024-09-11T01:26:02.989242+0000","last_fullsized":"2024-09-11T01:26:02.989242+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989422+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:02.989422+0000","last_peered":"2024-09-11T01:26:02.989422+0000","last_clean":"2024-09-11T01:26:02.989422+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:02.989422+0000","last_undegraded":"2024-09-11T01:26:02.989422+0000","last_fullsized":"2024-09-11T01:26:02.989422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20107,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:03.078750+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:03.078750+0000","last_peered":"2024-09-11T01:26:03.078750+0000","last_clean":"2024-09-11T01:26:03.078750+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:03.078750+0000","last_undegraded":"2024-09-11T01:26:03.078750+0000","last_fullsized":"2024-09-11T01:26:03.078750+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42464,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":125890560,"data_stored":125873165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182580,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518680,"kb_used_data":63624,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853160,"statfs":{"total":96636764160,"available":96105635840,"internally_reserved":0,"allocated":65150976,"data_stored":64795957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986352,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":425628,"kb_used_data":60812,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93946212,"statfs":{"total":96636764160,"available":96200921088,"internally_reserved":0,"allocated":62271488,"data_stored":61920546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659297,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509728,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":509179,"kb_avail":93862112,"statfs":{"total":96636764160,"available":96114802688,"internally_reserved":0,"allocated":540672,"data_stored":178695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521399983},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":61255680,"data_stored":61252201,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:36.659 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:26:36.659 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 25700, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:33.305047+0000', 'last_change': '2024-09-11T01:26:01.184322+0000', 'last_active': '2024-09-11T01:26:33.305047+0000', 'last_peered': '2024-09-11T01:26:33.305047+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:26:33.305047+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1189, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2799, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1189}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:36.659 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:26:36.659 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20037, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:02.982813+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:26:02.982813+0000', 'last_peered': '2024-09-11T01:26:02.982813+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:26:02.982813+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:36.660 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:26:37.011 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:37.011 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 5175/6 objects degraded (86250.000%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":933,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1488932864,"bytes_avail":288421359616,"bytes_total":289910292480,"degraded_objects":5175,"degraded_total":6,"degraded_ratio":862.5,"recovering_objects_per_sec":109,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1310,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:26:37.028 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:26:37.028 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:37.327 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:37.328 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:26:37.344 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":523,"stamp":"2024-09-11T01:26:36.879478+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42465,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1450436,"kb_used_data":121364,"kb_used_omap":12,"kb_used_meta":1329011,"kb_avail":281665084,"statfs":{"total":289910292480,"available":288425046016,"internally_reserved":0,"allocated":124276736,"data_stored":123208798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1360907268},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-923,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":923,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004236"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28536,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985357+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:02.985357+0000","last_peered":"2024-09-11T01:26:02.985357+0000","last_clean":"2024-09-11T01:26:02.985357+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:02.985357+0000","last_undegraded":"2024-09-11T01:26:02.985357+0000","last_fullsized":"2024-09-11T01:26:02.985357+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20494,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984919+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:26:02.984919+0000","last_peered":"2024-09-11T01:26:02.984919+0000","last_clean":"2024-09-11T01:26:02.984919+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:26:02.984919+0000","last_undegraded":"2024-09-11T01:26:02.984919+0000","last_fullsized":"2024-09-11T01:26:02.984919+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20317,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20244,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984629+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:26:02.984629+0000","last_peered":"2024-09-11T01:26:02.984629+0000","last_clean":"2024-09-11T01:26:02.984629+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:26:02.984629+0000","last_undegraded":"2024-09-11T01:26:02.984629+0000","last_fullsized":"2024-09-11T01:26:02.984629+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20177,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984547+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:02.984547+0000","last_peered":"2024-09-11T01:26:02.984547+0000","last_clean":"2024-09-11T01:26:02.984547+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:02.984547+0000","last_undegraded":"2024-09-11T01:26:02.984547+0000","last_fullsized":"2024-09-11T01:26:02.984547+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20423,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986863+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:02.986863+0000","last_peered":"2024-09-11T01:26:02.986863+0000","last_clean":"2024-09-11T01:26:02.986863+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:02.986863+0000","last_undegraded":"2024-09-11T01:26:02.986863+0000","last_fullsized":"2024-09-11T01:26:02.986863+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":25700,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:33.305047+0000","last_change":"2024-09-11T01:26:01.184322+0000","last_active":"2024-09-11T01:26:33.305047+0000","last_peered":"2024-09-11T01:26:33.305047+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:26:33.305047+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1189,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2799,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1189}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982464+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:02.982464+0000","last_peered":"2024-09-11T01:26:02.982464+0000","last_clean":"2024-09-11T01:26:02.982464+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:02.982464+0000","last_undegraded":"2024-09-11T01:26:02.982464+0000","last_fullsized":"2024-09-11T01:26:02.982464+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986745+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:02.986745+0000","last_peered":"2024-09-11T01:26:02.986745+0000","last_clean":"2024-09-11T01:26:02.986745+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:02.986745+0000","last_undegraded":"2024-09-11T01:26:02.986745+0000","last_fullsized":"2024-09-11T01:26:02.986745+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28021,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989204+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:02.989204+0000","last_peered":"2024-09-11T01:26:02.989204+0000","last_clean":"2024-09-11T01:26:02.989204+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:02.989204+0000","last_undegraded":"2024-09-11T01:26:02.989204+0000","last_fullsized":"2024-09-11T01:26:02.989204+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982959+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:26:02.982959+0000","last_peered":"2024-09-11T01:26:02.982959+0000","last_clean":"2024-09-11T01:26:02.982959+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:26:02.982959+0000","last_undegraded":"2024-09-11T01:26:02.982959+0000","last_fullsized":"2024-09-11T01:26:02.982959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20037,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:02.982813+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:26:02.982813+0000","last_peered":"2024-09-11T01:26:02.982813+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:26:02.982813+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":142,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982665+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:02.982665+0000","last_peered":"2024-09-11T01:26:02.982665+0000","last_clean":"2024-09-11T01:26:02.982665+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:02.982665+0000","last_undegraded":"2024-09-11T01:26:02.982665+0000","last_fullsized":"2024-09-11T01:26:02.982665+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28426,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.983065+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:26:02.983065+0000","last_peered":"2024-09-11T01:26:02.983065+0000","last_clean":"2024-09-11T01:26:02.983065+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:26:02.983065+0000","last_undegraded":"2024-09-11T01:26:02.983065+0000","last_fullsized":"2024-09-11T01:26:02.983065+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989042+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:02.989042+0000","last_peered":"2024-09-11T01:26:02.989042+0000","last_clean":"2024-09-11T01:26:02.989042+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:02.989042+0000","last_undegraded":"2024-09-11T01:26:02.989042+0000","last_fullsized":"2024-09-11T01:26:02.989042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982786+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:26:02.982786+0000","last_peered":"2024-09-11T01:26:02.982786+0000","last_clean":"2024-09-11T01:26:02.982786+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:26:02.982786+0000","last_undegraded":"2024-09-11T01:26:02.982786+0000","last_fullsized":"2024-09-11T01:26:02.982786+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20427,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989519+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:02.989519+0000","last_peered":"2024-09-11T01:26:02.989519+0000","last_clean":"2024-09-11T01:26:02.989519+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:02.989519+0000","last_undegraded":"2024-09-11T01:26:02.989519+0000","last_fullsized":"2024-09-11T01:26:02.989519+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989097+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:02.989097+0000","last_peered":"2024-09-11T01:26:02.989097+0000","last_clean":"2024-09-11T01:26:02.989097+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:02.989097+0000","last_undegraded":"2024-09-11T01:26:02.989097+0000","last_fullsized":"2024-09-11T01:26:02.989097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27956,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989418+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:02.989418+0000","last_peered":"2024-09-11T01:26:02.989418+0000","last_clean":"2024-09-11T01:26:02.989418+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:02.989418+0000","last_undegraded":"2024-09-11T01:26:02.989418+0000","last_fullsized":"2024-09-11T01:26:02.989418+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985710+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:26:02.985710+0000","last_peered":"2024-09-11T01:26:02.985710+0000","last_clean":"2024-09-11T01:26:02.985710+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:26:02.985710+0000","last_undegraded":"2024-09-11T01:26:02.985710+0000","last_fullsized":"2024-09-11T01:26:02.985710+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24634,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989242+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:02.989242+0000","last_peered":"2024-09-11T01:26:02.989242+0000","last_clean":"2024-09-11T01:26:02.989242+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:02.989242+0000","last_undegraded":"2024-09-11T01:26:02.989242+0000","last_fullsized":"2024-09-11T01:26:02.989242+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989422+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:02.989422+0000","last_peered":"2024-09-11T01:26:02.989422+0000","last_clean":"2024-09-11T01:26:02.989422+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:02.989422+0000","last_undegraded":"2024-09-11T01:26:02.989422+0000","last_fullsized":"2024-09-11T01:26:02.989422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20107,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:03.078750+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:03.078750+0000","last_peered":"2024-09-11T01:26:03.078750+0000","last_clean":"2024-09-11T01:26:03.078750+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:03.078750+0000","last_undegraded":"2024-09-11T01:26:03.078750+0000","last_fullsized":"2024-09-11T01:26:03.078750+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":5175,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":42464,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122286080,"data_stored":122268685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182580,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518680,"kb_used_data":63624,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853160,"statfs":{"total":96636764160,"available":96105635840,"internally_reserved":0,"allocated":65150976,"data_stored":64795957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986353,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422028,"kb_used_data":57212,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93949812,"statfs":{"total":96636764160,"available":96204607488,"internally_reserved":0,"allocated":58585088,"data_stored":58234146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659297,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509728,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":509179,"kb_avail":93862112,"statfs":{"total":96636764160,"available":96114802688,"internally_reserved":0,"allocated":540672,"data_stored":178695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521399983},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57651200,"data_stored":57647721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:40.347 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:40.702 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:40.702 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:26:40.720 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":524,"stamp":"2024-09-11T01:26:38.880071+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4604,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43036,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1437956,"kb_used_data":121364,"kb_used_omap":12,"kb_used_meta":1316531,"kb_avail":281677564,"statfs":{"total":289910292480,"available":288437825536,"internally_reserved":0,"allocated":124276736,"data_stored":123208798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1348127748},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1494,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1494,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004171"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28536,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985357+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:02.985357+0000","last_peered":"2024-09-11T01:26:02.985357+0000","last_clean":"2024-09-11T01:26:02.985357+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:02.985357+0000","last_undegraded":"2024-09-11T01:26:02.985357+0000","last_fullsized":"2024-09-11T01:26:02.985357+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20494,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984919+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:26:02.984919+0000","last_peered":"2024-09-11T01:26:02.984919+0000","last_clean":"2024-09-11T01:26:02.984919+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:26:02.984919+0000","last_undegraded":"2024-09-11T01:26:02.984919+0000","last_fullsized":"2024-09-11T01:26:02.984919+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20317,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20244,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984629+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:26:02.984629+0000","last_peered":"2024-09-11T01:26:02.984629+0000","last_clean":"2024-09-11T01:26:02.984629+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:26:02.984629+0000","last_undegraded":"2024-09-11T01:26:02.984629+0000","last_fullsized":"2024-09-11T01:26:02.984629+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20177,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984547+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:02.984547+0000","last_peered":"2024-09-11T01:26:02.984547+0000","last_clean":"2024-09-11T01:26:02.984547+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:02.984547+0000","last_undegraded":"2024-09-11T01:26:02.984547+0000","last_fullsized":"2024-09-11T01:26:02.984547+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20423,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986863+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:02.986863+0000","last_peered":"2024-09-11T01:26:02.986863+0000","last_clean":"2024-09-11T01:26:02.986863+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:02.986863+0000","last_undegraded":"2024-09-11T01:26:02.986863+0000","last_fullsized":"2024-09-11T01:26:02.986863+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":26842,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:38.306100+0000","last_change":"2024-09-11T01:26:01.184322+0000","last_active":"2024-09-11T01:26:38.306100+0000","last_peered":"2024-09-11T01:26:38.306100+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:26:38.306100+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":618,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3370,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-618}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982464+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:02.982464+0000","last_peered":"2024-09-11T01:26:02.982464+0000","last_clean":"2024-09-11T01:26:02.982464+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:02.982464+0000","last_undegraded":"2024-09-11T01:26:02.982464+0000","last_fullsized":"2024-09-11T01:26:02.982464+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986745+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:02.986745+0000","last_peered":"2024-09-11T01:26:02.986745+0000","last_clean":"2024-09-11T01:26:02.986745+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:02.986745+0000","last_undegraded":"2024-09-11T01:26:02.986745+0000","last_fullsized":"2024-09-11T01:26:02.986745+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28022,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989253+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:02.989253+0000","last_peered":"2024-09-11T01:26:02.989253+0000","last_clean":"2024-09-11T01:26:02.989253+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:02.989253+0000","last_undegraded":"2024-09-11T01:26:02.989253+0000","last_fullsized":"2024-09-11T01:26:02.989253+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982959+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:26:02.982959+0000","last_peered":"2024-09-11T01:26:02.982959+0000","last_clean":"2024-09-11T01:26:02.982959+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:26:02.982959+0000","last_undegraded":"2024-09-11T01:26:02.982959+0000","last_fullsized":"2024-09-11T01:26:02.982959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20037,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:02.982813+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:26:02.982813+0000","last_peered":"2024-09-11T01:26:02.982813+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:26:02.982813+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":142,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982665+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:02.982665+0000","last_peered":"2024-09-11T01:26:02.982665+0000","last_clean":"2024-09-11T01:26:02.982665+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:02.982665+0000","last_undegraded":"2024-09-11T01:26:02.982665+0000","last_fullsized":"2024-09-11T01:26:02.982665+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28426,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.983065+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:26:02.983065+0000","last_peered":"2024-09-11T01:26:02.983065+0000","last_clean":"2024-09-11T01:26:02.983065+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:26:02.983065+0000","last_undegraded":"2024-09-11T01:26:02.983065+0000","last_fullsized":"2024-09-11T01:26:02.983065+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989042+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:02.989042+0000","last_peered":"2024-09-11T01:26:02.989042+0000","last_clean":"2024-09-11T01:26:02.989042+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:02.989042+0000","last_undegraded":"2024-09-11T01:26:02.989042+0000","last_fullsized":"2024-09-11T01:26:02.989042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982786+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:26:02.982786+0000","last_peered":"2024-09-11T01:26:02.982786+0000","last_clean":"2024-09-11T01:26:02.982786+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:26:02.982786+0000","last_undegraded":"2024-09-11T01:26:02.982786+0000","last_fullsized":"2024-09-11T01:26:02.982786+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20427,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989519+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:02.989519+0000","last_peered":"2024-09-11T01:26:02.989519+0000","last_clean":"2024-09-11T01:26:02.989519+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:02.989519+0000","last_undegraded":"2024-09-11T01:26:02.989519+0000","last_fullsized":"2024-09-11T01:26:02.989519+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989097+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:02.989097+0000","last_peered":"2024-09-11T01:26:02.989097+0000","last_clean":"2024-09-11T01:26:02.989097+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:02.989097+0000","last_undegraded":"2024-09-11T01:26:02.989097+0000","last_fullsized":"2024-09-11T01:26:02.989097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27956,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989418+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:02.989418+0000","last_peered":"2024-09-11T01:26:02.989418+0000","last_clean":"2024-09-11T01:26:02.989418+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:02.989418+0000","last_undegraded":"2024-09-11T01:26:02.989418+0000","last_fullsized":"2024-09-11T01:26:02.989418+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985710+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:26:02.985710+0000","last_peered":"2024-09-11T01:26:02.985710+0000","last_clean":"2024-09-11T01:26:02.985710+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:26:02.985710+0000","last_undegraded":"2024-09-11T01:26:02.985710+0000","last_fullsized":"2024-09-11T01:26:02.985710+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24634,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989242+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:02.989242+0000","last_peered":"2024-09-11T01:26:02.989242+0000","last_clean":"2024-09-11T01:26:02.989242+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:02.989242+0000","last_undegraded":"2024-09-11T01:26:02.989242+0000","last_fullsized":"2024-09-11T01:26:02.989242+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989422+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:02.989422+0000","last_peered":"2024-09-11T01:26:02.989422+0000","last_clean":"2024-09-11T01:26:02.989422+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:02.989422+0000","last_undegraded":"2024-09-11T01:26:02.989422+0000","last_fullsized":"2024-09-11T01:26:02.989422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20107,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:03.078750+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:03.078750+0000","last_peered":"2024-09-11T01:26:03.078750+0000","last_clean":"2024-09-11T01:26:03.078750+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:03.078750+0000","last_undegraded":"2024-09-11T01:26:03.078750+0000","last_fullsized":"2024-09-11T01:26:03.078750+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4604,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43035,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":122286080,"data_stored":122268685,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182581,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518680,"kb_used_data":63624,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853160,"statfs":{"total":96636764160,"available":96105635840,"internally_reserved":0,"allocated":65150976,"data_stored":64795957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986353,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":422028,"kb_used_data":57212,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93949812,"statfs":{"total":96636764160,"available":96204607488,"internally_reserved":0,"allocated":58585088,"data_stored":58234146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659298,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497248,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":496699,"kb_avail":93874592,"statfs":{"total":96636764160,"available":96127582208,"internally_reserved":0,"allocated":540672,"data_stored":178695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508620463},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57651200,"data_stored":57647721,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:40.722 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:41.043 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:41.043 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:26:41.061 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":525,"stamp":"2024-09-11T01:26:40.880785+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4604,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43036,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1435104,"kb_used_data":118512,"kb_used_omap":12,"kb_used_meta":1316531,"kb_avail":281680416,"statfs":{"total":289910292480,"available":288440745984,"internally_reserved":0,"allocated":121356288,"data_stored":120288350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1348127748},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1058,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004099"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28536,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985357+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:02.985357+0000","last_peered":"2024-09-11T01:26:02.985357+0000","last_clean":"2024-09-11T01:26:02.985357+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:02.985357+0000","last_undegraded":"2024-09-11T01:26:02.985357+0000","last_fullsized":"2024-09-11T01:26:02.985357+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20494,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984919+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:26:02.984919+0000","last_peered":"2024-09-11T01:26:02.984919+0000","last_clean":"2024-09-11T01:26:02.984919+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:26:02.984919+0000","last_undegraded":"2024-09-11T01:26:02.984919+0000","last_fullsized":"2024-09-11T01:26:02.984919+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20317,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20244,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984629+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:26:02.984629+0000","last_peered":"2024-09-11T01:26:02.984629+0000","last_clean":"2024-09-11T01:26:02.984629+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:26:02.984629+0000","last_undegraded":"2024-09-11T01:26:02.984629+0000","last_fullsized":"2024-09-11T01:26:02.984629+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20177,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984547+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:02.984547+0000","last_peered":"2024-09-11T01:26:02.984547+0000","last_clean":"2024-09-11T01:26:02.984547+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:02.984547+0000","last_undegraded":"2024-09-11T01:26:02.984547+0000","last_fullsized":"2024-09-11T01:26:02.984547+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20423,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986863+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:02.986863+0000","last_peered":"2024-09-11T01:26:02.986863+0000","last_clean":"2024-09-11T01:26:02.986863+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:02.986863+0000","last_undegraded":"2024-09-11T01:26:02.986863+0000","last_fullsized":"2024-09-11T01:26:02.986863+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":26842,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:38.306100+0000","last_change":"2024-09-11T01:26:01.184322+0000","last_active":"2024-09-11T01:26:38.306100+0000","last_peered":"2024-09-11T01:26:38.306100+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:26:38.306100+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":618,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3370,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-618}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982464+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:02.982464+0000","last_peered":"2024-09-11T01:26:02.982464+0000","last_clean":"2024-09-11T01:26:02.982464+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:02.982464+0000","last_undegraded":"2024-09-11T01:26:02.982464+0000","last_fullsized":"2024-09-11T01:26:02.982464+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986745+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:02.986745+0000","last_peered":"2024-09-11T01:26:02.986745+0000","last_clean":"2024-09-11T01:26:02.986745+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:02.986745+0000","last_undegraded":"2024-09-11T01:26:02.986745+0000","last_fullsized":"2024-09-11T01:26:02.986745+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28022,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989253+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:02.989253+0000","last_peered":"2024-09-11T01:26:02.989253+0000","last_clean":"2024-09-11T01:26:02.989253+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:02.989253+0000","last_undegraded":"2024-09-11T01:26:02.989253+0000","last_fullsized":"2024-09-11T01:26:02.989253+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982959+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:26:02.982959+0000","last_peered":"2024-09-11T01:26:02.982959+0000","last_clean":"2024-09-11T01:26:02.982959+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:26:02.982959+0000","last_undegraded":"2024-09-11T01:26:02.982959+0000","last_fullsized":"2024-09-11T01:26:02.982959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20037,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:02.982813+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:26:02.982813+0000","last_peered":"2024-09-11T01:26:02.982813+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:26:02.982813+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":142,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982665+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:02.982665+0000","last_peered":"2024-09-11T01:26:02.982665+0000","last_clean":"2024-09-11T01:26:02.982665+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:02.982665+0000","last_undegraded":"2024-09-11T01:26:02.982665+0000","last_fullsized":"2024-09-11T01:26:02.982665+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28426,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.983065+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:26:02.983065+0000","last_peered":"2024-09-11T01:26:02.983065+0000","last_clean":"2024-09-11T01:26:02.983065+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:26:02.983065+0000","last_undegraded":"2024-09-11T01:26:02.983065+0000","last_fullsized":"2024-09-11T01:26:02.983065+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989042+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:02.989042+0000","last_peered":"2024-09-11T01:26:02.989042+0000","last_clean":"2024-09-11T01:26:02.989042+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:02.989042+0000","last_undegraded":"2024-09-11T01:26:02.989042+0000","last_fullsized":"2024-09-11T01:26:02.989042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982786+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:26:02.982786+0000","last_peered":"2024-09-11T01:26:02.982786+0000","last_clean":"2024-09-11T01:26:02.982786+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:26:02.982786+0000","last_undegraded":"2024-09-11T01:26:02.982786+0000","last_fullsized":"2024-09-11T01:26:02.982786+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20427,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989519+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:02.989519+0000","last_peered":"2024-09-11T01:26:02.989519+0000","last_clean":"2024-09-11T01:26:02.989519+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:02.989519+0000","last_undegraded":"2024-09-11T01:26:02.989519+0000","last_fullsized":"2024-09-11T01:26:02.989519+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989097+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:02.989097+0000","last_peered":"2024-09-11T01:26:02.989097+0000","last_clean":"2024-09-11T01:26:02.989097+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:02.989097+0000","last_undegraded":"2024-09-11T01:26:02.989097+0000","last_fullsized":"2024-09-11T01:26:02.989097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27956,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989418+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:02.989418+0000","last_peered":"2024-09-11T01:26:02.989418+0000","last_clean":"2024-09-11T01:26:02.989418+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:02.989418+0000","last_undegraded":"2024-09-11T01:26:02.989418+0000","last_fullsized":"2024-09-11T01:26:02.989418+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985710+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:26:02.985710+0000","last_peered":"2024-09-11T01:26:02.985710+0000","last_clean":"2024-09-11T01:26:02.985710+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:26:02.985710+0000","last_undegraded":"2024-09-11T01:26:02.985710+0000","last_fullsized":"2024-09-11T01:26:02.985710+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24634,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989242+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:02.989242+0000","last_peered":"2024-09-11T01:26:02.989242+0000","last_clean":"2024-09-11T01:26:02.989242+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:02.989242+0000","last_undegraded":"2024-09-11T01:26:02.989242+0000","last_fullsized":"2024-09-11T01:26:02.989242+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989422+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:02.989422+0000","last_peered":"2024-09-11T01:26:02.989422+0000","last_clean":"2024-09-11T01:26:02.989422+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:02.989422+0000","last_undegraded":"2024-09-11T01:26:02.989422+0000","last_fullsized":"2024-09-11T01:26:02.989422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20107,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:03.078750+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:03.078750+0000","last_peered":"2024-09-11T01:26:03.078750+0000","last_clean":"2024-09-11T01:26:03.078750+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:03.078750+0000","last_undegraded":"2024-09-11T01:26:03.078750+0000","last_fullsized":"2024-09-11T01:26:03.078750+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4604,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43035,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":119652352,"data_stored":119634957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182581,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518680,"kb_used_data":63624,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853160,"statfs":{"total":96636764160,"available":96105635840,"internally_reserved":0,"allocated":65150976,"data_stored":64795957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986354,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":419176,"kb_used_data":54360,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93952664,"statfs":{"total":96636764160,"available":96207527936,"internally_reserved":0,"allocated":55664640,"data_stored":55313698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659298,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497248,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":496699,"kb_avail":93874592,"statfs":{"total":96636764160,"available":96127582208,"internally_reserved":0,"allocated":540672,"data_stored":178695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508620463},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":55017472,"data_stored":55013993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:41.063 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:26:41.063 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 26842, 'reported_epoch': 63, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:38.306100+0000', 'last_change': '2024-09-11T01:26:01.184322+0000', 'last_active': '2024-09-11T01:26:38.306100+0000', 'last_peered': '2024-09-11T01:26:38.306100+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:26:38.306100+0000', 'last_undegraded': '2024-09-11T01:13:09.774578+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 42, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 618, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3370, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -618}], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:41.063 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:26:41.063 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20037, 'reported_epoch': 63, 'state': 'active+recovery_wait+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:02.982813+0000', 'last_change': '2024-09-11T01:13:10.290086+0000', 'last_active': '2024-09-11T01:26:02.982813+0000', 'last_peered': '2024-09-11T01:26:02.982813+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:26:02.982813+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3986, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3984}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:41.063 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:26:41.543 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:41.543 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 4604/6 objects degraded (76733.333%), 2 pgs degraded, 2 pgs undersized","count":4},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":938,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":63,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":2},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovery_wait+undersized+degraded+remapped","count":1},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1469546496,"bytes_avail":288440745984,"bytes_total":289910292480,"degraded_objects":4604,"degraded_total":6,"degraded_ratio":767.33333333333348,"recovering_objects_per_sec":88,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1058,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:26:41.560 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:26:41.560 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:41.864 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:41.865 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:26:41.880 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":525,"stamp":"2024-09-11T01:26:40.880785+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4604,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43036,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":46,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1435104,"kb_used_data":118512,"kb_used_omap":12,"kb_used_meta":1316531,"kb_avail":281680416,"statfs":{"total":289910292480,"available":288440745984,"internally_reserved":0,"allocated":121356288,"data_stored":120288350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1348127748},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1058,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1058,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004099"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28536,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985357+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:02.985357+0000","last_peered":"2024-09-11T01:26:02.985357+0000","last_clean":"2024-09-11T01:26:02.985357+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:02.985357+0000","last_undegraded":"2024-09-11T01:26:02.985357+0000","last_fullsized":"2024-09-11T01:26:02.985357+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20494,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984919+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:26:02.984919+0000","last_peered":"2024-09-11T01:26:02.984919+0000","last_clean":"2024-09-11T01:26:02.984919+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:26:02.984919+0000","last_undegraded":"2024-09-11T01:26:02.984919+0000","last_fullsized":"2024-09-11T01:26:02.984919+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20317,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20244,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984629+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:26:02.984629+0000","last_peered":"2024-09-11T01:26:02.984629+0000","last_clean":"2024-09-11T01:26:02.984629+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:26:02.984629+0000","last_undegraded":"2024-09-11T01:26:02.984629+0000","last_fullsized":"2024-09-11T01:26:02.984629+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20177,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984547+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:02.984547+0000","last_peered":"2024-09-11T01:26:02.984547+0000","last_clean":"2024-09-11T01:26:02.984547+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:02.984547+0000","last_undegraded":"2024-09-11T01:26:02.984547+0000","last_fullsized":"2024-09-11T01:26:02.984547+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20423,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986863+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:02.986863+0000","last_peered":"2024-09-11T01:26:02.986863+0000","last_clean":"2024-09-11T01:26:02.986863+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:02.986863+0000","last_undegraded":"2024-09-11T01:26:02.986863+0000","last_fullsized":"2024-09-11T01:26:02.986863+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":26842,"reported_epoch":63,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:38.306100+0000","last_change":"2024-09-11T01:26:01.184322+0000","last_active":"2024-09-11T01:26:38.306100+0000","last_peered":"2024-09-11T01:26:38.306100+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:26:38.306100+0000","last_undegraded":"2024-09-11T01:13:09.774578+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":42,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":618,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3370,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-618}],"blocked_by":[],"up_primary":1,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20424,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982464+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:02.982464+0000","last_peered":"2024-09-11T01:26:02.982464+0000","last_clean":"2024-09-11T01:26:02.982464+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:02.982464+0000","last_undegraded":"2024-09-11T01:26:02.982464+0000","last_fullsized":"2024-09-11T01:26:02.982464+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.986745+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:02.986745+0000","last_peered":"2024-09-11T01:26:02.986745+0000","last_clean":"2024-09-11T01:26:02.986745+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:02.986745+0000","last_undegraded":"2024-09-11T01:26:02.986745+0000","last_fullsized":"2024-09-11T01:26:02.986745+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28022,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989253+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:02.989253+0000","last_peered":"2024-09-11T01:26:02.989253+0000","last_clean":"2024-09-11T01:26:02.989253+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:02.989253+0000","last_undegraded":"2024-09-11T01:26:02.989253+0000","last_fullsized":"2024-09-11T01:26:02.989253+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982959+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:26:02.982959+0000","last_peered":"2024-09-11T01:26:02.982959+0000","last_clean":"2024-09-11T01:26:02.982959+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:26:02.982959+0000","last_undegraded":"2024-09-11T01:26:02.982959+0000","last_fullsized":"2024-09-11T01:26:02.982959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20037,"reported_epoch":63,"state":"active+recovery_wait+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:02.982813+0000","last_change":"2024-09-11T01:13:10.290086+0000","last_active":"2024-09-11T01:26:02.982813+0000","last_peered":"2024-09-11T01:26:02.982813+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:26:02.982813+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3986,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3984}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":142,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982665+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:02.982665+0000","last_peered":"2024-09-11T01:26:02.982665+0000","last_clean":"2024-09-11T01:26:02.982665+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:02.982665+0000","last_undegraded":"2024-09-11T01:26:02.982665+0000","last_fullsized":"2024-09-11T01:26:02.982665+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28426,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.983065+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:26:02.983065+0000","last_peered":"2024-09-11T01:26:02.983065+0000","last_clean":"2024-09-11T01:26:02.983065+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:26:02.983065+0000","last_undegraded":"2024-09-11T01:26:02.983065+0000","last_fullsized":"2024-09-11T01:26:02.983065+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":120,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989042+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:02.989042+0000","last_peered":"2024-09-11T01:26:02.989042+0000","last_clean":"2024-09-11T01:26:02.989042+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:02.989042+0000","last_undegraded":"2024-09-11T01:26:02.989042+0000","last_fullsized":"2024-09-11T01:26:02.989042+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982786+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:26:02.982786+0000","last_peered":"2024-09-11T01:26:02.982786+0000","last_clean":"2024-09-11T01:26:02.982786+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:26:02.982786+0000","last_undegraded":"2024-09-11T01:26:02.982786+0000","last_fullsized":"2024-09-11T01:26:02.982786+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20427,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989519+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:02.989519+0000","last_peered":"2024-09-11T01:26:02.989519+0000","last_clean":"2024-09-11T01:26:02.989519+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:02.989519+0000","last_undegraded":"2024-09-11T01:26:02.989519+0000","last_fullsized":"2024-09-11T01:26:02.989519+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989097+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:02.989097+0000","last_peered":"2024-09-11T01:26:02.989097+0000","last_clean":"2024-09-11T01:26:02.989097+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:02.989097+0000","last_undegraded":"2024-09-11T01:26:02.989097+0000","last_fullsized":"2024-09-11T01:26:02.989097+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27956,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989418+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:02.989418+0000","last_peered":"2024-09-11T01:26:02.989418+0000","last_clean":"2024-09-11T01:26:02.989418+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:02.989418+0000","last_undegraded":"2024-09-11T01:26:02.989418+0000","last_fullsized":"2024-09-11T01:26:02.989418+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985710+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:26:02.985710+0000","last_peered":"2024-09-11T01:26:02.985710+0000","last_clean":"2024-09-11T01:26:02.985710+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:26:02.985710+0000","last_undegraded":"2024-09-11T01:26:02.985710+0000","last_fullsized":"2024-09-11T01:26:02.985710+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24634,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989242+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:02.989242+0000","last_peered":"2024-09-11T01:26:02.989242+0000","last_clean":"2024-09-11T01:26:02.989242+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:02.989242+0000","last_undegraded":"2024-09-11T01:26:02.989242+0000","last_fullsized":"2024-09-11T01:26:02.989242+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989221+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:02.989221+0000","last_peered":"2024-09-11T01:26:02.989221+0000","last_clean":"2024-09-11T01:26:02.989221+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:02.989221+0000","last_undegraded":"2024-09-11T01:26:02.989221+0000","last_fullsized":"2024-09-11T01:26:02.989221+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":115,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.989422+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:02.989422+0000","last_peered":"2024-09-11T01:26:02.989422+0000","last_clean":"2024-09-11T01:26:02.989422+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:02.989422+0000","last_undegraded":"2024-09-11T01:26:02.989422+0000","last_fullsized":"2024-09-11T01:26:02.989422+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20107,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:03.078750+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:03.078750+0000","last_peered":"2024-09-11T01:26:03.078750+0000","last_clean":"2024-09-11T01:26:03.078750+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:03.078750+0000","last_undegraded":"2024-09-11T01:26:03.078750+0000","last_fullsized":"2024-09-11T01:26:03.078750+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":4604,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43035,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":119652352,"data_stored":119634957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":30,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182581,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518680,"kb_used_data":63624,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853160,"statfs":{"total":96636764160,"available":96105635840,"internally_reserved":0,"allocated":65150976,"data_stored":64795957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986354,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":419176,"kb_used_data":54360,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93952664,"statfs":{"total":96636764160,"available":96207527936,"internally_reserved":0,"allocated":55664640,"data_stored":55313698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659298,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497248,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":496699,"kb_avail":93874592,"statfs":{"total":96636764160,"available":96127582208,"internally_reserved":0,"allocated":540672,"data_stored":178695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508620463},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":55017472,"data_stored":55013993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:44.883 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:45.220 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:45.221 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:26:45.240 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":529,"stamp":"2024-09-11T01:26:44.881902+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3893,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43747,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1443440,"kb_used_data":118520,"kb_used_omap":12,"kb_used_meta":1324851,"kb_avail":281672080,"statfs":{"total":289910292480,"available":288432209920,"internally_reserved":0,"allocated":121364480,"data_stored":120292626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1356647428},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1282,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1282,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002423"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28538,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.474978+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:42.474978+0000","last_peered":"2024-09-11T01:26:42.474978+0000","last_clean":"2024-09-11T01:26:42.474978+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:42.474978+0000","last_undegraded":"2024-09-11T01:26:42.474978+0000","last_fullsized":"2024-09-11T01:26:42.474978+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20494,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984919+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:26:02.984919+0000","last_peered":"2024-09-11T01:26:02.984919+0000","last_clean":"2024-09-11T01:26:02.984919+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:26:02.984919+0000","last_undegraded":"2024-09-11T01:26:02.984919+0000","last_fullsized":"2024-09-11T01:26:02.984919+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20319,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.480804+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:42.480804+0000","last_peered":"2024-09-11T01:26:42.480804+0000","last_clean":"2024-09-11T01:26:42.480804+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:42.480804+0000","last_undegraded":"2024-09-11T01:26:42.480804+0000","last_fullsized":"2024-09-11T01:26:42.480804+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20244,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984629+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:26:02.984629+0000","last_peered":"2024-09-11T01:26:02.984629+0000","last_clean":"2024-09-11T01:26:02.984629+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:26:02.984629+0000","last_undegraded":"2024-09-11T01:26:02.984629+0000","last_fullsized":"2024-09-11T01:26:02.984629+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20179,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.473902+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:42.473902+0000","last_peered":"2024-09-11T01:26:42.473902+0000","last_clean":"2024-09-11T01:26:42.473902+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:42.473902+0000","last_undegraded":"2024-09-11T01:26:42.473902+0000","last_fullsized":"2024-09-11T01:26:42.473902+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20425,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.478860+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:42.478860+0000","last_peered":"2024-09-11T01:26:42.478860+0000","last_clean":"2024-09-11T01:26:42.478860+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:42.478860+0000","last_undegraded":"2024-09-11T01:26:42.478860+0000","last_fullsized":"2024-09-11T01:26:42.478860+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20072,"reported_epoch":64,"state":"peering","last_fresh":"2024-09-11T01:26:42.380014+0000","last_change":"2024-09-11T01:26:43.371643+0000","last_active":"2024-09-11T01:26:43.371643+0000","last_peered":"2024-09-11T01:26:42.380014+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:26:42.380014+0000","last_undegraded":"2024-09-11T01:26:42.380014+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":64,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3988,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20426,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.473830+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:42.473830+0000","last_peered":"2024-09-11T01:26:42.473830+0000","last_clean":"2024-09-11T01:26:42.473830+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:42.473830+0000","last_undegraded":"2024-09-11T01:26:42.473830+0000","last_fullsized":"2024-09-11T01:26:42.473830+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":112,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.478579+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:42.478579+0000","last_peered":"2024-09-11T01:26:42.478579+0000","last_clean":"2024-09-11T01:26:42.478579+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:42.478579+0000","last_undegraded":"2024-09-11T01:26:42.478579+0000","last_fullsized":"2024-09-11T01:26:42.478579+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28024,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.480405+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:42.480405+0000","last_peered":"2024-09-11T01:26:42.480405+0000","last_clean":"2024-09-11T01:26:42.480405+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:42.480405+0000","last_undegraded":"2024-09-11T01:26:42.480405+0000","last_fullsized":"2024-09-11T01:26:42.480405+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982959+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:26:02.982959+0000","last_peered":"2024-09-11T01:26:02.982959+0000","last_clean":"2024-09-11T01:26:02.982959+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:26:02.982959+0000","last_undegraded":"2024-09-11T01:26:02.982959+0000","last_fullsized":"2024-09-11T01:26:02.982959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20226,"reported_epoch":64,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:43.316955+0000","last_change":"2024-09-11T01:26:42.381382+0000","last_active":"2024-09-11T01:26:43.316955+0000","last_peered":"2024-09-11T01:26:43.316955+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:26:43.316955+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3893,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3893}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.474840+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:42.474840+0000","last_peered":"2024-09-11T01:26:42.474840+0000","last_clean":"2024-09-11T01:26:42.474840+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:42.474840+0000","last_undegraded":"2024-09-11T01:26:42.474840+0000","last_fullsized":"2024-09-11T01:26:42.474840+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28426,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.983065+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:26:02.983065+0000","last_peered":"2024-09-11T01:26:02.983065+0000","last_clean":"2024-09-11T01:26:02.983065+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:26:02.983065+0000","last_undegraded":"2024-09-11T01:26:02.983065+0000","last_fullsized":"2024-09-11T01:26:02.983065+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":122,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.478692+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:42.478692+0000","last_peered":"2024-09-11T01:26:42.478692+0000","last_clean":"2024-09-11T01:26:42.478692+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:42.478692+0000","last_undegraded":"2024-09-11T01:26:42.478692+0000","last_fullsized":"2024-09-11T01:26:42.478692+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982786+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:26:02.982786+0000","last_peered":"2024-09-11T01:26:02.982786+0000","last_clean":"2024-09-11T01:26:02.982786+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:26:02.982786+0000","last_undegraded":"2024-09-11T01:26:02.982786+0000","last_fullsized":"2024-09-11T01:26:02.982786+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20429,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.481528+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:42.481528+0000","last_peered":"2024-09-11T01:26:42.481528+0000","last_clean":"2024-09-11T01:26:42.481528+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:42.481528+0000","last_undegraded":"2024-09-11T01:26:42.481528+0000","last_fullsized":"2024-09-11T01:26:42.481528+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":112,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.480644+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:42.480644+0000","last_peered":"2024-09-11T01:26:42.480644+0000","last_clean":"2024-09-11T01:26:42.480644+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:42.480644+0000","last_undegraded":"2024-09-11T01:26:42.480644+0000","last_fullsized":"2024-09-11T01:26:42.480644+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27958,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.481410+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:42.481410+0000","last_peered":"2024-09-11T01:26:42.481410+0000","last_clean":"2024-09-11T01:26:42.481410+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:42.481410+0000","last_undegraded":"2024-09-11T01:26:42.481410+0000","last_fullsized":"2024-09-11T01:26:42.481410+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985710+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:26:02.985710+0000","last_peered":"2024-09-11T01:26:02.985710+0000","last_clean":"2024-09-11T01:26:02.985710+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:26:02.985710+0000","last_undegraded":"2024-09-11T01:26:02.985710+0000","last_fullsized":"2024-09-11T01:26:02.985710+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24636,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.481227+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:42.481227+0000","last_peered":"2024-09-11T01:26:42.481227+0000","last_clean":"2024-09-11T01:26:42.481227+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:42.481227+0000","last_undegraded":"2024-09-11T01:26:42.481227+0000","last_fullsized":"2024-09-11T01:26:42.481227+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":112,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.481043+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:42.481043+0000","last_peered":"2024-09-11T01:26:42.481043+0000","last_clean":"2024-09-11T01:26:42.481043+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:42.481043+0000","last_undegraded":"2024-09-11T01:26:42.481043+0000","last_fullsized":"2024-09-11T01:26:42.481043+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":117,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.481429+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:42.481429+0000","last_peered":"2024-09-11T01:26:42.481429+0000","last_clean":"2024-09-11T01:26:42.481429+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:42.481429+0000","last_undegraded":"2024-09-11T01:26:42.481429+0000","last_fullsized":"2024-09-11T01:26:42.481429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.480411+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:42.480411+0000","last_peered":"2024-09-11T01:26:42.480411+0000","last_clean":"2024-09-11T01:26:42.480411+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:42.480411+0000","last_undegraded":"2024-09-11T01:26:42.480411+0000","last_fullsized":"2024-09-11T01:26:42.480411+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3893,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43746,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":119644160,"data_stored":119633723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182582,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518696,"kb_used_data":63632,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853144,"statfs":{"total":96636764160,"available":96105619456,"internally_reserved":0,"allocated":65159168,"data_stored":64798712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986354,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":419176,"kb_used_data":54360,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93952664,"statfs":{"total":96636764160,"available":96207527936,"internally_reserved":0,"allocated":55664640,"data_stored":55313698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659299,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505568,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":505019,"kb_avail":93866272,"statfs":{"total":96636764160,"available":96119062528,"internally_reserved":0,"allocated":540672,"data_stored":180216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517140143},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":55017472,"data_stored":55013993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:45.241 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:45.615 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:45.616 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:26:45.635 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":529,"stamp":"2024-09-11T01:26:44.881902+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3893,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43747,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1443440,"kb_used_data":118520,"kb_used_omap":12,"kb_used_meta":1324851,"kb_avail":281672080,"statfs":{"total":289910292480,"available":288432209920,"internally_reserved":0,"allocated":121364480,"data_stored":120292626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1356647428},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1282,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1282,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002423"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28538,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.474978+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:42.474978+0000","last_peered":"2024-09-11T01:26:42.474978+0000","last_clean":"2024-09-11T01:26:42.474978+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:42.474978+0000","last_undegraded":"2024-09-11T01:26:42.474978+0000","last_fullsized":"2024-09-11T01:26:42.474978+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20494,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984919+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:26:02.984919+0000","last_peered":"2024-09-11T01:26:02.984919+0000","last_clean":"2024-09-11T01:26:02.984919+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:26:02.984919+0000","last_undegraded":"2024-09-11T01:26:02.984919+0000","last_fullsized":"2024-09-11T01:26:02.984919+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20319,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.480804+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:42.480804+0000","last_peered":"2024-09-11T01:26:42.480804+0000","last_clean":"2024-09-11T01:26:42.480804+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:42.480804+0000","last_undegraded":"2024-09-11T01:26:42.480804+0000","last_fullsized":"2024-09-11T01:26:42.480804+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20244,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984629+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:26:02.984629+0000","last_peered":"2024-09-11T01:26:02.984629+0000","last_clean":"2024-09-11T01:26:02.984629+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:26:02.984629+0000","last_undegraded":"2024-09-11T01:26:02.984629+0000","last_fullsized":"2024-09-11T01:26:02.984629+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20179,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.473902+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:42.473902+0000","last_peered":"2024-09-11T01:26:42.473902+0000","last_clean":"2024-09-11T01:26:42.473902+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:42.473902+0000","last_undegraded":"2024-09-11T01:26:42.473902+0000","last_fullsized":"2024-09-11T01:26:42.473902+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20425,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.478860+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:42.478860+0000","last_peered":"2024-09-11T01:26:42.478860+0000","last_clean":"2024-09-11T01:26:42.478860+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:42.478860+0000","last_undegraded":"2024-09-11T01:26:42.478860+0000","last_fullsized":"2024-09-11T01:26:42.478860+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20072,"reported_epoch":64,"state":"peering","last_fresh":"2024-09-11T01:26:42.380014+0000","last_change":"2024-09-11T01:26:43.371643+0000","last_active":"2024-09-11T01:26:43.371643+0000","last_peered":"2024-09-11T01:26:42.380014+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:26:42.380014+0000","last_undegraded":"2024-09-11T01:26:42.380014+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":64,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3988,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20426,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.473830+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:42.473830+0000","last_peered":"2024-09-11T01:26:42.473830+0000","last_clean":"2024-09-11T01:26:42.473830+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:42.473830+0000","last_undegraded":"2024-09-11T01:26:42.473830+0000","last_fullsized":"2024-09-11T01:26:42.473830+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":112,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.478579+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:42.478579+0000","last_peered":"2024-09-11T01:26:42.478579+0000","last_clean":"2024-09-11T01:26:42.478579+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:42.478579+0000","last_undegraded":"2024-09-11T01:26:42.478579+0000","last_fullsized":"2024-09-11T01:26:42.478579+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28024,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.480405+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:42.480405+0000","last_peered":"2024-09-11T01:26:42.480405+0000","last_clean":"2024-09-11T01:26:42.480405+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:42.480405+0000","last_undegraded":"2024-09-11T01:26:42.480405+0000","last_fullsized":"2024-09-11T01:26:42.480405+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982959+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:26:02.982959+0000","last_peered":"2024-09-11T01:26:02.982959+0000","last_clean":"2024-09-11T01:26:02.982959+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:26:02.982959+0000","last_undegraded":"2024-09-11T01:26:02.982959+0000","last_fullsized":"2024-09-11T01:26:02.982959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20226,"reported_epoch":64,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:43.316955+0000","last_change":"2024-09-11T01:26:42.381382+0000","last_active":"2024-09-11T01:26:43.316955+0000","last_peered":"2024-09-11T01:26:43.316955+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:26:43.316955+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3893,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3893}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.474840+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:42.474840+0000","last_peered":"2024-09-11T01:26:42.474840+0000","last_clean":"2024-09-11T01:26:42.474840+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:42.474840+0000","last_undegraded":"2024-09-11T01:26:42.474840+0000","last_fullsized":"2024-09-11T01:26:42.474840+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28426,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.983065+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:26:02.983065+0000","last_peered":"2024-09-11T01:26:02.983065+0000","last_clean":"2024-09-11T01:26:02.983065+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:26:02.983065+0000","last_undegraded":"2024-09-11T01:26:02.983065+0000","last_fullsized":"2024-09-11T01:26:02.983065+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":122,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.478692+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:42.478692+0000","last_peered":"2024-09-11T01:26:42.478692+0000","last_clean":"2024-09-11T01:26:42.478692+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:42.478692+0000","last_undegraded":"2024-09-11T01:26:42.478692+0000","last_fullsized":"2024-09-11T01:26:42.478692+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982786+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:26:02.982786+0000","last_peered":"2024-09-11T01:26:02.982786+0000","last_clean":"2024-09-11T01:26:02.982786+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:26:02.982786+0000","last_undegraded":"2024-09-11T01:26:02.982786+0000","last_fullsized":"2024-09-11T01:26:02.982786+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20429,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.481528+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:42.481528+0000","last_peered":"2024-09-11T01:26:42.481528+0000","last_clean":"2024-09-11T01:26:42.481528+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:42.481528+0000","last_undegraded":"2024-09-11T01:26:42.481528+0000","last_fullsized":"2024-09-11T01:26:42.481528+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":112,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.480644+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:42.480644+0000","last_peered":"2024-09-11T01:26:42.480644+0000","last_clean":"2024-09-11T01:26:42.480644+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:42.480644+0000","last_undegraded":"2024-09-11T01:26:42.480644+0000","last_fullsized":"2024-09-11T01:26:42.480644+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27958,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.481410+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:42.481410+0000","last_peered":"2024-09-11T01:26:42.481410+0000","last_clean":"2024-09-11T01:26:42.481410+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:42.481410+0000","last_undegraded":"2024-09-11T01:26:42.481410+0000","last_fullsized":"2024-09-11T01:26:42.481410+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985710+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:26:02.985710+0000","last_peered":"2024-09-11T01:26:02.985710+0000","last_clean":"2024-09-11T01:26:02.985710+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:26:02.985710+0000","last_undegraded":"2024-09-11T01:26:02.985710+0000","last_fullsized":"2024-09-11T01:26:02.985710+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24636,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.481227+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:42.481227+0000","last_peered":"2024-09-11T01:26:42.481227+0000","last_clean":"2024-09-11T01:26:42.481227+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:42.481227+0000","last_undegraded":"2024-09-11T01:26:42.481227+0000","last_fullsized":"2024-09-11T01:26:42.481227+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":112,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.481043+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:42.481043+0000","last_peered":"2024-09-11T01:26:42.481043+0000","last_clean":"2024-09-11T01:26:42.481043+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:42.481043+0000","last_undegraded":"2024-09-11T01:26:42.481043+0000","last_fullsized":"2024-09-11T01:26:42.481043+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":117,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.481429+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:42.481429+0000","last_peered":"2024-09-11T01:26:42.481429+0000","last_clean":"2024-09-11T01:26:42.481429+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:42.481429+0000","last_undegraded":"2024-09-11T01:26:42.481429+0000","last_fullsized":"2024-09-11T01:26:42.481429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.480411+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:42.480411+0000","last_peered":"2024-09-11T01:26:42.480411+0000","last_clean":"2024-09-11T01:26:42.480411+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:42.480411+0000","last_undegraded":"2024-09-11T01:26:42.480411+0000","last_fullsized":"2024-09-11T01:26:42.480411+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3893,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43746,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":119644160,"data_stored":119633723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182582,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518696,"kb_used_data":63632,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853144,"statfs":{"total":96636764160,"available":96105619456,"internally_reserved":0,"allocated":65159168,"data_stored":64798712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986354,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":419176,"kb_used_data":54360,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93952664,"statfs":{"total":96636764160,"available":96207527936,"internally_reserved":0,"allocated":55664640,"data_stored":55313698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659299,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505568,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":505019,"kb_avail":93866272,"statfs":{"total":96636764160,"available":96119062528,"internally_reserved":0,"allocated":540672,"data_stored":180216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517140143},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":55017472,"data_stored":55013993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:45.636 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.9 is not active+clean 2024-09-11T01:26:45.637 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.9', 'version': "49'19930", 'reported_seq': 20072, 'reported_epoch': 64, 'state': 'peering', 'last_fresh': '2024-09-11T01:26:42.380014+0000', 'last_change': '2024-09-11T01:26:43.371643+0000', 'last_active': '2024-09-11T01:26:43.371643+0000', 'last_peered': '2024-09-11T01:26:42.380014+0000', 'last_clean': '2024-09-11T01:13:07.631917+0000', 'last_became_active': '2024-09-11T01:13:10.268758+0000', 'last_became_peered': '2024-09-11T01:13:10.268758+0000', 'last_unstale': '2024-09-11T01:26:42.380014+0000', 'last_undegraded': '2024-09-11T01:26:42.380014+0000', 'last_fullsized': '2024-09-11T01:13:09.656066+0000', 'mapping_epoch': 64, 'log_start': "45'9900", 'ondisk_log_start': "45'9900", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3190", 'last_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:54.410173+0000', 'objects_scrubbed': 3137, 'log_size': 10030, 'log_dups_size': 0, 'ondisk_log_size': 10030, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1319, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16787, 'num_write_kb': 27300, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3988, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-11T01:26:45.637 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:26:45.637 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 20226, 'reported_epoch': 64, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:43.316955+0000', 'last_change': '2024-09-11T01:26:42.381382+0000', 'last_active': '2024-09-11T01:26:43.316955+0000', 'last_peered': '2024-09-11T01:26:43.316955+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:26:43.316955+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3893, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 93, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3893}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:45.637 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:26:46.052 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:46.052 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 3893/6 objects degraded (64883.333%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":942,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":22},{"state_name":"active+recovering+undersized+degraded+remapped","count":1},{"state_name":"peering","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1478082560,"bytes_avail":288432209920,"bytes_total":289910292480,"inactive_pgs_ratio":0.041666667908430099,"degraded_objects":3893,"degraded_total":6,"degraded_ratio":648.83333333333337,"recovering_objects_per_sec":160,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1282,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:26:46.077 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:26:46.077 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:46.436 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:46.436 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:26:46.460 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":529,"stamp":"2024-09-11T01:26:44.881902+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3893,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43747,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1443440,"kb_used_data":118520,"kb_used_omap":12,"kb_used_meta":1324851,"kb_avail":281672080,"statfs":{"total":289910292480,"available":288432209920,"internally_reserved":0,"allocated":121364480,"data_stored":120292626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1356647428},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1282,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1282,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002423"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28538,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.474978+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:42.474978+0000","last_peered":"2024-09-11T01:26:42.474978+0000","last_clean":"2024-09-11T01:26:42.474978+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:42.474978+0000","last_undegraded":"2024-09-11T01:26:42.474978+0000","last_fullsized":"2024-09-11T01:26:42.474978+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20494,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984919+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:26:02.984919+0000","last_peered":"2024-09-11T01:26:02.984919+0000","last_clean":"2024-09-11T01:26:02.984919+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:26:02.984919+0000","last_undegraded":"2024-09-11T01:26:02.984919+0000","last_fullsized":"2024-09-11T01:26:02.984919+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20319,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.480804+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:42.480804+0000","last_peered":"2024-09-11T01:26:42.480804+0000","last_clean":"2024-09-11T01:26:42.480804+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:42.480804+0000","last_undegraded":"2024-09-11T01:26:42.480804+0000","last_fullsized":"2024-09-11T01:26:42.480804+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20244,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.984629+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:26:02.984629+0000","last_peered":"2024-09-11T01:26:02.984629+0000","last_clean":"2024-09-11T01:26:02.984629+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:26:02.984629+0000","last_undegraded":"2024-09-11T01:26:02.984629+0000","last_fullsized":"2024-09-11T01:26:02.984629+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20179,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.473902+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:42.473902+0000","last_peered":"2024-09-11T01:26:42.473902+0000","last_clean":"2024-09-11T01:26:42.473902+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:42.473902+0000","last_undegraded":"2024-09-11T01:26:42.473902+0000","last_fullsized":"2024-09-11T01:26:42.473902+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20425,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.478860+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:42.478860+0000","last_peered":"2024-09-11T01:26:42.478860+0000","last_clean":"2024-09-11T01:26:42.478860+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:42.478860+0000","last_undegraded":"2024-09-11T01:26:42.478860+0000","last_fullsized":"2024-09-11T01:26:42.478860+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20072,"reported_epoch":64,"state":"peering","last_fresh":"2024-09-11T01:26:42.380014+0000","last_change":"2024-09-11T01:26:43.371643+0000","last_active":"2024-09-11T01:26:43.371643+0000","last_peered":"2024-09-11T01:26:42.380014+0000","last_clean":"2024-09-11T01:13:07.631917+0000","last_became_active":"2024-09-11T01:13:10.268758+0000","last_became_peered":"2024-09-11T01:13:10.268758+0000","last_unstale":"2024-09-11T01:26:42.380014+0000","last_undegraded":"2024-09-11T01:26:42.380014+0000","last_fullsized":"2024-09-11T01:13:09.656066+0000","mapping_epoch":64,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3988,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20426,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.473830+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:42.473830+0000","last_peered":"2024-09-11T01:26:42.473830+0000","last_clean":"2024-09-11T01:26:42.473830+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:42.473830+0000","last_undegraded":"2024-09-11T01:26:42.473830+0000","last_fullsized":"2024-09-11T01:26:42.473830+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":112,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.478579+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:42.478579+0000","last_peered":"2024-09-11T01:26:42.478579+0000","last_clean":"2024-09-11T01:26:42.478579+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:42.478579+0000","last_undegraded":"2024-09-11T01:26:42.478579+0000","last_fullsized":"2024-09-11T01:26:42.478579+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28024,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.480405+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:42.480405+0000","last_peered":"2024-09-11T01:26:42.480405+0000","last_clean":"2024-09-11T01:26:42.480405+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:42.480405+0000","last_undegraded":"2024-09-11T01:26:42.480405+0000","last_fullsized":"2024-09-11T01:26:42.480405+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982959+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:26:02.982959+0000","last_peered":"2024-09-11T01:26:02.982959+0000","last_clean":"2024-09-11T01:26:02.982959+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:26:02.982959+0000","last_undegraded":"2024-09-11T01:26:02.982959+0000","last_fullsized":"2024-09-11T01:26:02.982959+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":20226,"reported_epoch":64,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:43.316955+0000","last_change":"2024-09-11T01:26:42.381382+0000","last_active":"2024-09-11T01:26:43.316955+0000","last_peered":"2024-09-11T01:26:43.316955+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:26:43.316955+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3893,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":93,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3893}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":144,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.474840+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:42.474840+0000","last_peered":"2024-09-11T01:26:42.474840+0000","last_clean":"2024-09-11T01:26:42.474840+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:42.474840+0000","last_undegraded":"2024-09-11T01:26:42.474840+0000","last_fullsized":"2024-09-11T01:26:42.474840+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28426,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.983065+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:26:02.983065+0000","last_peered":"2024-09-11T01:26:02.983065+0000","last_clean":"2024-09-11T01:26:02.983065+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:26:02.983065+0000","last_undegraded":"2024-09-11T01:26:02.983065+0000","last_fullsized":"2024-09-11T01:26:02.983065+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":122,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.478692+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:42.478692+0000","last_peered":"2024-09-11T01:26:42.478692+0000","last_clean":"2024-09-11T01:26:42.478692+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:42.478692+0000","last_undegraded":"2024-09-11T01:26:42.478692+0000","last_fullsized":"2024-09-11T01:26:42.478692+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27818,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.982786+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:26:02.982786+0000","last_peered":"2024-09-11T01:26:02.982786+0000","last_clean":"2024-09-11T01:26:02.982786+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:26:02.982786+0000","last_undegraded":"2024-09-11T01:26:02.982786+0000","last_fullsized":"2024-09-11T01:26:02.982786+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20429,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.481528+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:42.481528+0000","last_peered":"2024-09-11T01:26:42.481528+0000","last_clean":"2024-09-11T01:26:42.481528+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:42.481528+0000","last_undegraded":"2024-09-11T01:26:42.481528+0000","last_fullsized":"2024-09-11T01:26:42.481528+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":112,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.480644+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:42.480644+0000","last_peered":"2024-09-11T01:26:42.480644+0000","last_clean":"2024-09-11T01:26:42.480644+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:42.480644+0000","last_undegraded":"2024-09-11T01:26:42.480644+0000","last_fullsized":"2024-09-11T01:26:42.480644+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27958,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.481410+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:42.481410+0000","last_peered":"2024-09-11T01:26:42.481410+0000","last_clean":"2024-09-11T01:26:42.481410+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:42.481410+0000","last_undegraded":"2024-09-11T01:26:42.481410+0000","last_fullsized":"2024-09-11T01:26:42.481410+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":110,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-09-11T01:26:02.985710+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:26:02.985710+0000","last_peered":"2024-09-11T01:26:02.985710+0000","last_clean":"2024-09-11T01:26:02.985710+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:26:02.985710+0000","last_undegraded":"2024-09-11T01:26:02.985710+0000","last_fullsized":"2024-09-11T01:26:02.985710+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24636,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.481227+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:42.481227+0000","last_peered":"2024-09-11T01:26:42.481227+0000","last_clean":"2024-09-11T01:26:42.481227+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:42.481227+0000","last_undegraded":"2024-09-11T01:26:42.481227+0000","last_fullsized":"2024-09-11T01:26:42.481227+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":112,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.481043+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:42.481043+0000","last_peered":"2024-09-11T01:26:42.481043+0000","last_clean":"2024-09-11T01:26:42.481043+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:42.481043+0000","last_undegraded":"2024-09-11T01:26:42.481043+0000","last_fullsized":"2024-09-11T01:26:42.481043+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":117,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.481429+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:42.481429+0000","last_peered":"2024-09-11T01:26:42.481429+0000","last_clean":"2024-09-11T01:26:42.481429+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:42.481429+0000","last_undegraded":"2024-09-11T01:26:42.481429+0000","last_fullsized":"2024-09-11T01:26:42.481429+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20109,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-09-11T01:26:42.480411+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:42.480411+0000","last_peered":"2024-09-11T01:26:42.480411+0000","last_clean":"2024-09-11T01:26:42.480411+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:42.480411+0000","last_undegraded":"2024-09-11T01:26:42.480411+0000","last_fullsized":"2024-09-11T01:26:42.480411+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3893,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":43746,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":119644160,"data_stored":119633723,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182582,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518696,"kb_used_data":63632,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853144,"statfs":{"total":96636764160,"available":96105619456,"internally_reserved":0,"allocated":65159168,"data_stored":64798712,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1],"upper_bound":2},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986354,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":419176,"kb_used_data":54360,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93952664,"statfs":{"total":96636764160,"available":96207527936,"internally_reserved":0,"allocated":55664640,"data_stored":55313698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659299,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505568,"kb_used_data":528,"kb_used_omap":4,"kb_used_meta":505019,"kb_avail":93866272,"statfs":{"total":96636764160,"available":96119062528,"internally_reserved":0,"allocated":540672,"data_stored":180216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":517140143},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":55017472,"data_stored":55013993,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:49.463 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:49.820 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:49.820 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:26:49.841 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":531,"stamp":"2024-09-11T01:26:48.882880+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3159,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44481,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1445708,"kb_used_data":116636,"kb_used_omap":12,"kb_used_meta":1329011,"kb_avail":281669812,"statfs":{"total":289910292480,"available":288429887488,"internally_reserved":0,"allocated":119435264,"data_stored":118341599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1360907268},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1445,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1445,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002094"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28540,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.496965+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:43.496965+0000","last_peered":"2024-09-11T01:26:43.496965+0000","last_clean":"2024-09-11T01:26:43.496965+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:43.496965+0000","last_undegraded":"2024-09-11T01:26:43.496965+0000","last_fullsized":"2024-09-11T01:26:43.496965+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20498,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837072+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:26:43.837072+0000","last_peered":"2024-09-11T01:26:43.837072+0000","last_clean":"2024-09-11T01:26:43.837072+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:26:43.837072+0000","last_undegraded":"2024-09-11T01:26:43.837072+0000","last_fullsized":"2024-09-11T01:26:43.837072+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20321,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493170+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:43.493170+0000","last_peered":"2024-09-11T01:26:43.493170+0000","last_clean":"2024-09-11T01:26:43.493170+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:43.493170+0000","last_undegraded":"2024-09-11T01:26:43.493170+0000","last_fullsized":"2024-09-11T01:26:43.493170+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20248,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837042+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:26:43.837042+0000","last_peered":"2024-09-11T01:26:43.837042+0000","last_clean":"2024-09-11T01:26:43.837042+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:26:43.837042+0000","last_undegraded":"2024-09-11T01:26:43.837042+0000","last_fullsized":"2024-09-11T01:26:43.837042+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20181,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.495408+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:43.495408+0000","last_peered":"2024-09-11T01:26:43.495408+0000","last_clean":"2024-09-11T01:26:43.495408+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:43.495408+0000","last_undegraded":"2024-09-11T01:26:43.495408+0000","last_fullsized":"2024-09-11T01:26:43.495408+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20427,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492918+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:43.492918+0000","last_peered":"2024-09-11T01:26:43.492918+0000","last_clean":"2024-09-11T01:26:43.492918+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:43.492918+0000","last_undegraded":"2024-09-11T01:26:43.492918+0000","last_fullsized":"2024-09-11T01:26:43.492918+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20081,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.604490+0000","last_change":"2024-09-11T01:26:43.604424+0000","last_active":"2024-09-11T01:26:43.604490+0000","last_peered":"2024-09-11T01:26:43.604490+0000","last_clean":"2024-09-11T01:26:43.604490+0000","last_became_active":"2024-09-11T01:26:43.498430+0000","last_became_peered":"2024-09-11T01:26:43.498430+0000","last_unstale":"2024-09-11T01:26:43.604490+0000","last_undegraded":"2024-09-11T01:26:43.604490+0000","last_fullsized":"2024-09-11T01:26:43.604490+0000","mapping_epoch":64,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:13.180371+0000","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3988,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20428,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.495316+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:43.495316+0000","last_peered":"2024-09-11T01:26:43.495316+0000","last_clean":"2024-09-11T01:26:43.495316+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:43.495316+0000","last_undegraded":"2024-09-11T01:26:43.495316+0000","last_fullsized":"2024-09-11T01:26:43.495316+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492641+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:43.492641+0000","last_peered":"2024-09-11T01:26:43.492641+0000","last_clean":"2024-09-11T01:26:43.492641+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:43.492641+0000","last_undegraded":"2024-09-11T01:26:43.492641+0000","last_fullsized":"2024-09-11T01:26:43.492641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28026,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493375+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:43.493375+0000","last_peered":"2024-09-11T01:26:43.493375+0000","last_clean":"2024-09-11T01:26:43.493375+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:43.493375+0000","last_undegraded":"2024-09-11T01:26:43.493375+0000","last_fullsized":"2024-09-11T01:26:43.493375+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836907+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:26:43.836907+0000","last_peered":"2024-09-11T01:26:43.836907+0000","last_clean":"2024-09-11T01:26:43.836907+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:26:43.836907+0000","last_undegraded":"2024-09-11T01:26:43.836907+0000","last_fullsized":"2024-09-11T01:26:43.836907+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":21696,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:48.323344+0000","last_change":"2024-09-11T01:26:42.381382+0000","last_active":"2024-09-11T01:26:48.323344+0000","last_peered":"2024-09-11T01:26:48.323344+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:26:48.323344+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3159,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":827,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3159}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":146,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.497081+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:43.497081+0000","last_peered":"2024-09-11T01:26:43.497081+0000","last_clean":"2024-09-11T01:26:43.497081+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:43.497081+0000","last_undegraded":"2024-09-11T01:26:43.497081+0000","last_fullsized":"2024-09-11T01:26:43.497081+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28431,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836927+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:26:43.836927+0000","last_peered":"2024-09-11T01:26:43.836927+0000","last_clean":"2024-09-11T01:26:43.836927+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:26:43.836927+0000","last_undegraded":"2024-09-11T01:26:43.836927+0000","last_fullsized":"2024-09-11T01:26:43.836927+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492885+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:43.492885+0000","last_peered":"2024-09-11T01:26:43.492885+0000","last_clean":"2024-09-11T01:26:43.492885+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:43.492885+0000","last_undegraded":"2024-09-11T01:26:43.492885+0000","last_fullsized":"2024-09-11T01:26:43.492885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27822,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836526+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:26:43.836526+0000","last_peered":"2024-09-11T01:26:43.836526+0000","last_clean":"2024-09-11T01:26:43.836526+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:26:43.836526+0000","last_undegraded":"2024-09-11T01:26:43.836526+0000","last_fullsized":"2024-09-11T01:26:43.836526+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20431,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492883+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:43.492883+0000","last_peered":"2024-09-11T01:26:43.492883+0000","last_clean":"2024-09-11T01:26:43.492883+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:43.492883+0000","last_undegraded":"2024-09-11T01:26:43.492883+0000","last_fullsized":"2024-09-11T01:26:43.492883+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493122+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:43.493122+0000","last_peered":"2024-09-11T01:26:43.493122+0000","last_clean":"2024-09-11T01:26:43.493122+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:43.493122+0000","last_undegraded":"2024-09-11T01:26:43.493122+0000","last_fullsized":"2024-09-11T01:26:43.493122+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27960,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493218+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:43.493218+0000","last_peered":"2024-09-11T01:26:43.493218+0000","last_clean":"2024-09-11T01:26:43.493218+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:43.493218+0000","last_undegraded":"2024-09-11T01:26:43.493218+0000","last_fullsized":"2024-09-11T01:26:43.493218+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837106+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:26:43.837106+0000","last_peered":"2024-09-11T01:26:43.837106+0000","last_clean":"2024-09-11T01:26:43.837106+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:26:43.837106+0000","last_undegraded":"2024-09-11T01:26:43.837106+0000","last_fullsized":"2024-09-11T01:26:43.837106+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24639,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493408+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:43.493408+0000","last_peered":"2024-09-11T01:26:43.493408+0000","last_clean":"2024-09-11T01:26:43.493408+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:43.493408+0000","last_undegraded":"2024-09-11T01:26:43.493408+0000","last_fullsized":"2024-09-11T01:26:43.493408+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492970+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:43.492970+0000","last_peered":"2024-09-11T01:26:43.492970+0000","last_clean":"2024-09-11T01:26:43.492970+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:43.492970+0000","last_undegraded":"2024-09-11T01:26:43.492970+0000","last_fullsized":"2024-09-11T01:26:43.492970+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492818+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:43.492818+0000","last_peered":"2024-09-11T01:26:43.492818+0000","last_clean":"2024-09-11T01:26:43.492818+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:43.492818+0000","last_undegraded":"2024-09-11T01:26:43.492818+0000","last_fullsized":"2024-09-11T01:26:43.492818+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20111,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492860+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:43.492860+0000","last_peered":"2024-09-11T01:26:43.492860+0000","last_clean":"2024-09-11T01:26:43.492860+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:43.492860+0000","last_undegraded":"2024-09-11T01:26:43.492860+0000","last_fullsized":"2024-09-11T01:26:43.492860+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3159,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44480,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":117559296,"data_stored":117548859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182583,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518696,"kb_used_data":63640,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853144,"statfs":{"total":96636764160,"available":96105619456,"internally_reserved":0,"allocated":65167360,"data_stored":64801446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986355,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":417276,"kb_used_data":52460,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93954564,"statfs":{"total":96636764160,"available":96209473536,"internally_reserved":0,"allocated":53719040,"data_stored":53357203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659300,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509736,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":509179,"kb_avail":93862104,"statfs":{"total":96636764160,"available":96114794496,"internally_reserved":0,"allocated":548864,"data_stored":182950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521399983},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52932608,"data_stored":52929129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:49.842 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:50.231 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:50.232 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:26:50.251 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":531,"stamp":"2024-09-11T01:26:48.882880+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3159,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44481,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":58,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1445708,"kb_used_data":116636,"kb_used_omap":12,"kb_used_meta":1329011,"kb_avail":281669812,"statfs":{"total":289910292480,"available":288429887488,"internally_reserved":0,"allocated":119435264,"data_stored":118341599,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1360907268},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1445,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1445,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002094"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28540,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.496965+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:43.496965+0000","last_peered":"2024-09-11T01:26:43.496965+0000","last_clean":"2024-09-11T01:26:43.496965+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:43.496965+0000","last_undegraded":"2024-09-11T01:26:43.496965+0000","last_fullsized":"2024-09-11T01:26:43.496965+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20498,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837072+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:26:43.837072+0000","last_peered":"2024-09-11T01:26:43.837072+0000","last_clean":"2024-09-11T01:26:43.837072+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:26:43.837072+0000","last_undegraded":"2024-09-11T01:26:43.837072+0000","last_fullsized":"2024-09-11T01:26:43.837072+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20321,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493170+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:43.493170+0000","last_peered":"2024-09-11T01:26:43.493170+0000","last_clean":"2024-09-11T01:26:43.493170+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:43.493170+0000","last_undegraded":"2024-09-11T01:26:43.493170+0000","last_fullsized":"2024-09-11T01:26:43.493170+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20248,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837042+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:26:43.837042+0000","last_peered":"2024-09-11T01:26:43.837042+0000","last_clean":"2024-09-11T01:26:43.837042+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:26:43.837042+0000","last_undegraded":"2024-09-11T01:26:43.837042+0000","last_fullsized":"2024-09-11T01:26:43.837042+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20181,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.495408+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:43.495408+0000","last_peered":"2024-09-11T01:26:43.495408+0000","last_clean":"2024-09-11T01:26:43.495408+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:43.495408+0000","last_undegraded":"2024-09-11T01:26:43.495408+0000","last_fullsized":"2024-09-11T01:26:43.495408+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20427,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492918+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:43.492918+0000","last_peered":"2024-09-11T01:26:43.492918+0000","last_clean":"2024-09-11T01:26:43.492918+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:43.492918+0000","last_undegraded":"2024-09-11T01:26:43.492918+0000","last_fullsized":"2024-09-11T01:26:43.492918+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20081,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.604490+0000","last_change":"2024-09-11T01:26:43.604424+0000","last_active":"2024-09-11T01:26:43.604490+0000","last_peered":"2024-09-11T01:26:43.604490+0000","last_clean":"2024-09-11T01:26:43.604490+0000","last_became_active":"2024-09-11T01:26:43.498430+0000","last_became_peered":"2024-09-11T01:26:43.498430+0000","last_unstale":"2024-09-11T01:26:43.604490+0000","last_undegraded":"2024-09-11T01:26:43.604490+0000","last_fullsized":"2024-09-11T01:26:43.604490+0000","mapping_epoch":64,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:13.180371+0000","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3988,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20428,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.495316+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:43.495316+0000","last_peered":"2024-09-11T01:26:43.495316+0000","last_clean":"2024-09-11T01:26:43.495316+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:43.495316+0000","last_undegraded":"2024-09-11T01:26:43.495316+0000","last_fullsized":"2024-09-11T01:26:43.495316+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492641+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:43.492641+0000","last_peered":"2024-09-11T01:26:43.492641+0000","last_clean":"2024-09-11T01:26:43.492641+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:43.492641+0000","last_undegraded":"2024-09-11T01:26:43.492641+0000","last_fullsized":"2024-09-11T01:26:43.492641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28026,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493375+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:43.493375+0000","last_peered":"2024-09-11T01:26:43.493375+0000","last_clean":"2024-09-11T01:26:43.493375+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:43.493375+0000","last_undegraded":"2024-09-11T01:26:43.493375+0000","last_fullsized":"2024-09-11T01:26:43.493375+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836907+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:26:43.836907+0000","last_peered":"2024-09-11T01:26:43.836907+0000","last_clean":"2024-09-11T01:26:43.836907+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:26:43.836907+0000","last_undegraded":"2024-09-11T01:26:43.836907+0000","last_fullsized":"2024-09-11T01:26:43.836907+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":21696,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:48.323344+0000","last_change":"2024-09-11T01:26:42.381382+0000","last_active":"2024-09-11T01:26:48.323344+0000","last_peered":"2024-09-11T01:26:48.323344+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:26:48.323344+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3159,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":827,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3159}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":146,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.497081+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:43.497081+0000","last_peered":"2024-09-11T01:26:43.497081+0000","last_clean":"2024-09-11T01:26:43.497081+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:43.497081+0000","last_undegraded":"2024-09-11T01:26:43.497081+0000","last_fullsized":"2024-09-11T01:26:43.497081+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28431,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836927+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:26:43.836927+0000","last_peered":"2024-09-11T01:26:43.836927+0000","last_clean":"2024-09-11T01:26:43.836927+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:26:43.836927+0000","last_undegraded":"2024-09-11T01:26:43.836927+0000","last_fullsized":"2024-09-11T01:26:43.836927+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492885+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:43.492885+0000","last_peered":"2024-09-11T01:26:43.492885+0000","last_clean":"2024-09-11T01:26:43.492885+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:43.492885+0000","last_undegraded":"2024-09-11T01:26:43.492885+0000","last_fullsized":"2024-09-11T01:26:43.492885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27822,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836526+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:26:43.836526+0000","last_peered":"2024-09-11T01:26:43.836526+0000","last_clean":"2024-09-11T01:26:43.836526+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:26:43.836526+0000","last_undegraded":"2024-09-11T01:26:43.836526+0000","last_fullsized":"2024-09-11T01:26:43.836526+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20431,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492883+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:43.492883+0000","last_peered":"2024-09-11T01:26:43.492883+0000","last_clean":"2024-09-11T01:26:43.492883+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:43.492883+0000","last_undegraded":"2024-09-11T01:26:43.492883+0000","last_fullsized":"2024-09-11T01:26:43.492883+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493122+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:43.493122+0000","last_peered":"2024-09-11T01:26:43.493122+0000","last_clean":"2024-09-11T01:26:43.493122+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:43.493122+0000","last_undegraded":"2024-09-11T01:26:43.493122+0000","last_fullsized":"2024-09-11T01:26:43.493122+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27960,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493218+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:43.493218+0000","last_peered":"2024-09-11T01:26:43.493218+0000","last_clean":"2024-09-11T01:26:43.493218+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:43.493218+0000","last_undegraded":"2024-09-11T01:26:43.493218+0000","last_fullsized":"2024-09-11T01:26:43.493218+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837106+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:26:43.837106+0000","last_peered":"2024-09-11T01:26:43.837106+0000","last_clean":"2024-09-11T01:26:43.837106+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:26:43.837106+0000","last_undegraded":"2024-09-11T01:26:43.837106+0000","last_fullsized":"2024-09-11T01:26:43.837106+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24639,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493408+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:43.493408+0000","last_peered":"2024-09-11T01:26:43.493408+0000","last_clean":"2024-09-11T01:26:43.493408+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:43.493408+0000","last_undegraded":"2024-09-11T01:26:43.493408+0000","last_fullsized":"2024-09-11T01:26:43.493408+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492970+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:43.492970+0000","last_peered":"2024-09-11T01:26:43.492970+0000","last_clean":"2024-09-11T01:26:43.492970+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:43.492970+0000","last_undegraded":"2024-09-11T01:26:43.492970+0000","last_fullsized":"2024-09-11T01:26:43.492970+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492818+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:43.492818+0000","last_peered":"2024-09-11T01:26:43.492818+0000","last_clean":"2024-09-11T01:26:43.492818+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:43.492818+0000","last_undegraded":"2024-09-11T01:26:43.492818+0000","last_fullsized":"2024-09-11T01:26:43.492818+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20111,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492860+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:43.492860+0000","last_peered":"2024-09-11T01:26:43.492860+0000","last_clean":"2024-09-11T01:26:43.492860+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:43.492860+0000","last_undegraded":"2024-09-11T01:26:43.492860+0000","last_fullsized":"2024-09-11T01:26:43.492860+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3159,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44480,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":117559296,"data_stored":117548859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182583,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518696,"kb_used_data":63640,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853144,"statfs":{"total":96636764160,"available":96105619456,"internally_reserved":0,"allocated":65167360,"data_stored":64801446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986355,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":417276,"kb_used_data":52460,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93954564,"statfs":{"total":96636764160,"available":96209473536,"internally_reserved":0,"allocated":53719040,"data_stored":53357203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659300,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509736,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":509179,"kb_avail":93862104,"statfs":{"total":96636764160,"available":96114794496,"internally_reserved":0,"allocated":548864,"data_stored":182950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521399983},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":52932608,"data_stored":52929129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:50.253 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:26:50.253 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 21696, 'reported_epoch': 65, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:48.323344+0000', 'last_change': '2024-09-11T01:26:42.381382+0000', 'last_active': '2024-09-11T01:26:48.323344+0000', 'last_peered': '2024-09-11T01:26:48.323344+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:26:48.323344+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3159, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 827, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3159}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:50.253 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:26:50.712 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:50.712 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 3159/6 objects degraded (52650.000%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":947,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1480404992,"bytes_avail":288429887488,"bytes_total":289910292480,"degraded_objects":3159,"degraded_total":6,"degraded_ratio":526.5,"recovering_objects_per_sec":180,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1445,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:26:50.733 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:26:50.734 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:51.076 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:51.077 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:26:51.096 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":532,"stamp":"2024-09-11T01:26:50.883446+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3159,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44481,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1443308,"kb_used_data":114236,"kb_used_omap":12,"kb_used_meta":1329011,"kb_avail":281672212,"statfs":{"total":289910292480,"available":288432345088,"internally_reserved":0,"allocated":116977664,"data_stored":115883999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1360907268},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1445,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1445,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.412334"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28540,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.496965+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:43.496965+0000","last_peered":"2024-09-11T01:26:43.496965+0000","last_clean":"2024-09-11T01:26:43.496965+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:43.496965+0000","last_undegraded":"2024-09-11T01:26:43.496965+0000","last_fullsized":"2024-09-11T01:26:43.496965+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20498,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837072+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:26:43.837072+0000","last_peered":"2024-09-11T01:26:43.837072+0000","last_clean":"2024-09-11T01:26:43.837072+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:26:43.837072+0000","last_undegraded":"2024-09-11T01:26:43.837072+0000","last_fullsized":"2024-09-11T01:26:43.837072+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20321,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493170+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:43.493170+0000","last_peered":"2024-09-11T01:26:43.493170+0000","last_clean":"2024-09-11T01:26:43.493170+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:43.493170+0000","last_undegraded":"2024-09-11T01:26:43.493170+0000","last_fullsized":"2024-09-11T01:26:43.493170+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20248,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837042+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:26:43.837042+0000","last_peered":"2024-09-11T01:26:43.837042+0000","last_clean":"2024-09-11T01:26:43.837042+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:26:43.837042+0000","last_undegraded":"2024-09-11T01:26:43.837042+0000","last_fullsized":"2024-09-11T01:26:43.837042+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20181,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.495408+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:43.495408+0000","last_peered":"2024-09-11T01:26:43.495408+0000","last_clean":"2024-09-11T01:26:43.495408+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:43.495408+0000","last_undegraded":"2024-09-11T01:26:43.495408+0000","last_fullsized":"2024-09-11T01:26:43.495408+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20427,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492918+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:43.492918+0000","last_peered":"2024-09-11T01:26:43.492918+0000","last_clean":"2024-09-11T01:26:43.492918+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:43.492918+0000","last_undegraded":"2024-09-11T01:26:43.492918+0000","last_fullsized":"2024-09-11T01:26:43.492918+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20081,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.604490+0000","last_change":"2024-09-11T01:26:43.604424+0000","last_active":"2024-09-11T01:26:43.604490+0000","last_peered":"2024-09-11T01:26:43.604490+0000","last_clean":"2024-09-11T01:26:43.604490+0000","last_became_active":"2024-09-11T01:26:43.498430+0000","last_became_peered":"2024-09-11T01:26:43.498430+0000","last_unstale":"2024-09-11T01:26:43.604490+0000","last_undegraded":"2024-09-11T01:26:43.604490+0000","last_fullsized":"2024-09-11T01:26:43.604490+0000","mapping_epoch":64,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:13.180371+0000","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3988,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20428,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.495316+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:43.495316+0000","last_peered":"2024-09-11T01:26:43.495316+0000","last_clean":"2024-09-11T01:26:43.495316+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:43.495316+0000","last_undegraded":"2024-09-11T01:26:43.495316+0000","last_fullsized":"2024-09-11T01:26:43.495316+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492641+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:43.492641+0000","last_peered":"2024-09-11T01:26:43.492641+0000","last_clean":"2024-09-11T01:26:43.492641+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:43.492641+0000","last_undegraded":"2024-09-11T01:26:43.492641+0000","last_fullsized":"2024-09-11T01:26:43.492641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28026,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493375+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:43.493375+0000","last_peered":"2024-09-11T01:26:43.493375+0000","last_clean":"2024-09-11T01:26:43.493375+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:43.493375+0000","last_undegraded":"2024-09-11T01:26:43.493375+0000","last_fullsized":"2024-09-11T01:26:43.493375+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836907+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:26:43.836907+0000","last_peered":"2024-09-11T01:26:43.836907+0000","last_clean":"2024-09-11T01:26:43.836907+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:26:43.836907+0000","last_undegraded":"2024-09-11T01:26:43.836907+0000","last_fullsized":"2024-09-11T01:26:43.836907+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":21696,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:48.323344+0000","last_change":"2024-09-11T01:26:42.381382+0000","last_active":"2024-09-11T01:26:48.323344+0000","last_peered":"2024-09-11T01:26:48.323344+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:26:48.323344+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3159,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":827,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3159}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":146,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.497081+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:43.497081+0000","last_peered":"2024-09-11T01:26:43.497081+0000","last_clean":"2024-09-11T01:26:43.497081+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:43.497081+0000","last_undegraded":"2024-09-11T01:26:43.497081+0000","last_fullsized":"2024-09-11T01:26:43.497081+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28431,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836927+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:26:43.836927+0000","last_peered":"2024-09-11T01:26:43.836927+0000","last_clean":"2024-09-11T01:26:43.836927+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:26:43.836927+0000","last_undegraded":"2024-09-11T01:26:43.836927+0000","last_fullsized":"2024-09-11T01:26:43.836927+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492885+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:43.492885+0000","last_peered":"2024-09-11T01:26:43.492885+0000","last_clean":"2024-09-11T01:26:43.492885+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:43.492885+0000","last_undegraded":"2024-09-11T01:26:43.492885+0000","last_fullsized":"2024-09-11T01:26:43.492885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27822,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836526+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:26:43.836526+0000","last_peered":"2024-09-11T01:26:43.836526+0000","last_clean":"2024-09-11T01:26:43.836526+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:26:43.836526+0000","last_undegraded":"2024-09-11T01:26:43.836526+0000","last_fullsized":"2024-09-11T01:26:43.836526+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20431,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492883+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:43.492883+0000","last_peered":"2024-09-11T01:26:43.492883+0000","last_clean":"2024-09-11T01:26:43.492883+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:43.492883+0000","last_undegraded":"2024-09-11T01:26:43.492883+0000","last_fullsized":"2024-09-11T01:26:43.492883+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493122+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:43.493122+0000","last_peered":"2024-09-11T01:26:43.493122+0000","last_clean":"2024-09-11T01:26:43.493122+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:43.493122+0000","last_undegraded":"2024-09-11T01:26:43.493122+0000","last_fullsized":"2024-09-11T01:26:43.493122+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27960,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493218+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:43.493218+0000","last_peered":"2024-09-11T01:26:43.493218+0000","last_clean":"2024-09-11T01:26:43.493218+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:43.493218+0000","last_undegraded":"2024-09-11T01:26:43.493218+0000","last_fullsized":"2024-09-11T01:26:43.493218+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837106+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:26:43.837106+0000","last_peered":"2024-09-11T01:26:43.837106+0000","last_clean":"2024-09-11T01:26:43.837106+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:26:43.837106+0000","last_undegraded":"2024-09-11T01:26:43.837106+0000","last_fullsized":"2024-09-11T01:26:43.837106+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24639,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493408+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:43.493408+0000","last_peered":"2024-09-11T01:26:43.493408+0000","last_clean":"2024-09-11T01:26:43.493408+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:43.493408+0000","last_undegraded":"2024-09-11T01:26:43.493408+0000","last_fullsized":"2024-09-11T01:26:43.493408+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492970+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:43.492970+0000","last_peered":"2024-09-11T01:26:43.492970+0000","last_clean":"2024-09-11T01:26:43.492970+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:43.492970+0000","last_undegraded":"2024-09-11T01:26:43.492970+0000","last_fullsized":"2024-09-11T01:26:43.492970+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492818+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:43.492818+0000","last_peered":"2024-09-11T01:26:43.492818+0000","last_clean":"2024-09-11T01:26:43.492818+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:43.492818+0000","last_undegraded":"2024-09-11T01:26:43.492818+0000","last_fullsized":"2024-09-11T01:26:43.492818+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20111,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492860+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:43.492860+0000","last_peered":"2024-09-11T01:26:43.492860+0000","last_clean":"2024-09-11T01:26:43.492860+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:43.492860+0000","last_undegraded":"2024-09-11T01:26:43.492860+0000","last_fullsized":"2024-09-11T01:26:43.492860+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3159,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44480,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":115101696,"data_stored":115091259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182583,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518696,"kb_used_data":63640,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853144,"statfs":{"total":96636764160,"available":96105619456,"internally_reserved":0,"allocated":65167360,"data_stored":64801446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986356,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":414876,"kb_used_data":50060,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93956964,"statfs":{"total":96636764160,"available":96211931136,"internally_reserved":0,"allocated":51261440,"data_stored":50899603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659300,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509736,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":509179,"kb_avail":93862104,"statfs":{"total":96636764160,"available":96114794496,"internally_reserved":0,"allocated":548864,"data_stored":182950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521399983},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":50475008,"data_stored":50471529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:54.099 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:54.452 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:54.453 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:26:54.472 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":533,"stamp":"2024-09-11T01:26:52.883805+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3159,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44481,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1443308,"kb_used_data":114236,"kb_used_omap":12,"kb_used_meta":1329011,"kb_avail":281672212,"statfs":{"total":289910292480,"available":288432345088,"internally_reserved":0,"allocated":116977664,"data_stored":115883999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1360907268},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1445,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1445,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002645"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28540,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.496965+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:43.496965+0000","last_peered":"2024-09-11T01:26:43.496965+0000","last_clean":"2024-09-11T01:26:43.496965+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:43.496965+0000","last_undegraded":"2024-09-11T01:26:43.496965+0000","last_fullsized":"2024-09-11T01:26:43.496965+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20498,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837072+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:26:43.837072+0000","last_peered":"2024-09-11T01:26:43.837072+0000","last_clean":"2024-09-11T01:26:43.837072+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:26:43.837072+0000","last_undegraded":"2024-09-11T01:26:43.837072+0000","last_fullsized":"2024-09-11T01:26:43.837072+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20321,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493170+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:43.493170+0000","last_peered":"2024-09-11T01:26:43.493170+0000","last_clean":"2024-09-11T01:26:43.493170+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:43.493170+0000","last_undegraded":"2024-09-11T01:26:43.493170+0000","last_fullsized":"2024-09-11T01:26:43.493170+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20248,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837042+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:26:43.837042+0000","last_peered":"2024-09-11T01:26:43.837042+0000","last_clean":"2024-09-11T01:26:43.837042+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:26:43.837042+0000","last_undegraded":"2024-09-11T01:26:43.837042+0000","last_fullsized":"2024-09-11T01:26:43.837042+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20181,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.495408+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:43.495408+0000","last_peered":"2024-09-11T01:26:43.495408+0000","last_clean":"2024-09-11T01:26:43.495408+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:43.495408+0000","last_undegraded":"2024-09-11T01:26:43.495408+0000","last_fullsized":"2024-09-11T01:26:43.495408+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20427,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492918+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:43.492918+0000","last_peered":"2024-09-11T01:26:43.492918+0000","last_clean":"2024-09-11T01:26:43.492918+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:43.492918+0000","last_undegraded":"2024-09-11T01:26:43.492918+0000","last_fullsized":"2024-09-11T01:26:43.492918+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20081,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.604490+0000","last_change":"2024-09-11T01:26:43.604424+0000","last_active":"2024-09-11T01:26:43.604490+0000","last_peered":"2024-09-11T01:26:43.604490+0000","last_clean":"2024-09-11T01:26:43.604490+0000","last_became_active":"2024-09-11T01:26:43.498430+0000","last_became_peered":"2024-09-11T01:26:43.498430+0000","last_unstale":"2024-09-11T01:26:43.604490+0000","last_undegraded":"2024-09-11T01:26:43.604490+0000","last_fullsized":"2024-09-11T01:26:43.604490+0000","mapping_epoch":64,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:13.180371+0000","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3988,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20428,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.495316+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:43.495316+0000","last_peered":"2024-09-11T01:26:43.495316+0000","last_clean":"2024-09-11T01:26:43.495316+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:43.495316+0000","last_undegraded":"2024-09-11T01:26:43.495316+0000","last_fullsized":"2024-09-11T01:26:43.495316+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492641+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:43.492641+0000","last_peered":"2024-09-11T01:26:43.492641+0000","last_clean":"2024-09-11T01:26:43.492641+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:43.492641+0000","last_undegraded":"2024-09-11T01:26:43.492641+0000","last_fullsized":"2024-09-11T01:26:43.492641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28026,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493375+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:43.493375+0000","last_peered":"2024-09-11T01:26:43.493375+0000","last_clean":"2024-09-11T01:26:43.493375+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:43.493375+0000","last_undegraded":"2024-09-11T01:26:43.493375+0000","last_fullsized":"2024-09-11T01:26:43.493375+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836907+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:26:43.836907+0000","last_peered":"2024-09-11T01:26:43.836907+0000","last_clean":"2024-09-11T01:26:43.836907+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:26:43.836907+0000","last_undegraded":"2024-09-11T01:26:43.836907+0000","last_fullsized":"2024-09-11T01:26:43.836907+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":21696,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:48.323344+0000","last_change":"2024-09-11T01:26:42.381382+0000","last_active":"2024-09-11T01:26:48.323344+0000","last_peered":"2024-09-11T01:26:48.323344+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:26:48.323344+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3159,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":827,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3159}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":146,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.497081+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:43.497081+0000","last_peered":"2024-09-11T01:26:43.497081+0000","last_clean":"2024-09-11T01:26:43.497081+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:43.497081+0000","last_undegraded":"2024-09-11T01:26:43.497081+0000","last_fullsized":"2024-09-11T01:26:43.497081+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28431,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836927+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:26:43.836927+0000","last_peered":"2024-09-11T01:26:43.836927+0000","last_clean":"2024-09-11T01:26:43.836927+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:26:43.836927+0000","last_undegraded":"2024-09-11T01:26:43.836927+0000","last_fullsized":"2024-09-11T01:26:43.836927+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492885+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:43.492885+0000","last_peered":"2024-09-11T01:26:43.492885+0000","last_clean":"2024-09-11T01:26:43.492885+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:43.492885+0000","last_undegraded":"2024-09-11T01:26:43.492885+0000","last_fullsized":"2024-09-11T01:26:43.492885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27822,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836526+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:26:43.836526+0000","last_peered":"2024-09-11T01:26:43.836526+0000","last_clean":"2024-09-11T01:26:43.836526+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:26:43.836526+0000","last_undegraded":"2024-09-11T01:26:43.836526+0000","last_fullsized":"2024-09-11T01:26:43.836526+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20431,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492883+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:43.492883+0000","last_peered":"2024-09-11T01:26:43.492883+0000","last_clean":"2024-09-11T01:26:43.492883+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:43.492883+0000","last_undegraded":"2024-09-11T01:26:43.492883+0000","last_fullsized":"2024-09-11T01:26:43.492883+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493122+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:43.493122+0000","last_peered":"2024-09-11T01:26:43.493122+0000","last_clean":"2024-09-11T01:26:43.493122+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:43.493122+0000","last_undegraded":"2024-09-11T01:26:43.493122+0000","last_fullsized":"2024-09-11T01:26:43.493122+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27960,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493218+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:43.493218+0000","last_peered":"2024-09-11T01:26:43.493218+0000","last_clean":"2024-09-11T01:26:43.493218+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:43.493218+0000","last_undegraded":"2024-09-11T01:26:43.493218+0000","last_fullsized":"2024-09-11T01:26:43.493218+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837106+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:26:43.837106+0000","last_peered":"2024-09-11T01:26:43.837106+0000","last_clean":"2024-09-11T01:26:43.837106+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:26:43.837106+0000","last_undegraded":"2024-09-11T01:26:43.837106+0000","last_fullsized":"2024-09-11T01:26:43.837106+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24639,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493408+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:43.493408+0000","last_peered":"2024-09-11T01:26:43.493408+0000","last_clean":"2024-09-11T01:26:43.493408+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:43.493408+0000","last_undegraded":"2024-09-11T01:26:43.493408+0000","last_fullsized":"2024-09-11T01:26:43.493408+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492970+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:43.492970+0000","last_peered":"2024-09-11T01:26:43.492970+0000","last_clean":"2024-09-11T01:26:43.492970+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:43.492970+0000","last_undegraded":"2024-09-11T01:26:43.492970+0000","last_fullsized":"2024-09-11T01:26:43.492970+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492818+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:43.492818+0000","last_peered":"2024-09-11T01:26:43.492818+0000","last_clean":"2024-09-11T01:26:43.492818+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:43.492818+0000","last_undegraded":"2024-09-11T01:26:43.492818+0000","last_fullsized":"2024-09-11T01:26:43.492818+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20111,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492860+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:43.492860+0000","last_peered":"2024-09-11T01:26:43.492860+0000","last_clean":"2024-09-11T01:26:43.492860+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:43.492860+0000","last_undegraded":"2024-09-11T01:26:43.492860+0000","last_fullsized":"2024-09-11T01:26:43.492860+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3159,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44480,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":115101696,"data_stored":115091259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182583,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518696,"kb_used_data":63640,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853144,"statfs":{"total":96636764160,"available":96105619456,"internally_reserved":0,"allocated":65167360,"data_stored":64801446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986356,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":414876,"kb_used_data":50060,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93956964,"statfs":{"total":96636764160,"available":96211931136,"internally_reserved":0,"allocated":51261440,"data_stored":50899603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659300,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509736,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":509179,"kb_avail":93862104,"statfs":{"total":96636764160,"available":96114794496,"internally_reserved":0,"allocated":548864,"data_stored":182950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521399983},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":50475008,"data_stored":50471529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:54.474 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:54.807 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:54.808 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:26:54.827 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":533,"stamp":"2024-09-11T01:26:52.883805+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3159,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44481,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1443308,"kb_used_data":114236,"kb_used_omap":12,"kb_used_meta":1329011,"kb_avail":281672212,"statfs":{"total":289910292480,"available":288432345088,"internally_reserved":0,"allocated":116977664,"data_stored":115883999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1360907268},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1445,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1445,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002645"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28540,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.496965+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:43.496965+0000","last_peered":"2024-09-11T01:26:43.496965+0000","last_clean":"2024-09-11T01:26:43.496965+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:43.496965+0000","last_undegraded":"2024-09-11T01:26:43.496965+0000","last_fullsized":"2024-09-11T01:26:43.496965+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20498,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837072+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:26:43.837072+0000","last_peered":"2024-09-11T01:26:43.837072+0000","last_clean":"2024-09-11T01:26:43.837072+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:26:43.837072+0000","last_undegraded":"2024-09-11T01:26:43.837072+0000","last_fullsized":"2024-09-11T01:26:43.837072+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20321,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493170+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:43.493170+0000","last_peered":"2024-09-11T01:26:43.493170+0000","last_clean":"2024-09-11T01:26:43.493170+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:43.493170+0000","last_undegraded":"2024-09-11T01:26:43.493170+0000","last_fullsized":"2024-09-11T01:26:43.493170+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20248,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837042+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:26:43.837042+0000","last_peered":"2024-09-11T01:26:43.837042+0000","last_clean":"2024-09-11T01:26:43.837042+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:26:43.837042+0000","last_undegraded":"2024-09-11T01:26:43.837042+0000","last_fullsized":"2024-09-11T01:26:43.837042+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20181,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.495408+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:43.495408+0000","last_peered":"2024-09-11T01:26:43.495408+0000","last_clean":"2024-09-11T01:26:43.495408+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:43.495408+0000","last_undegraded":"2024-09-11T01:26:43.495408+0000","last_fullsized":"2024-09-11T01:26:43.495408+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20427,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492918+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:43.492918+0000","last_peered":"2024-09-11T01:26:43.492918+0000","last_clean":"2024-09-11T01:26:43.492918+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:43.492918+0000","last_undegraded":"2024-09-11T01:26:43.492918+0000","last_fullsized":"2024-09-11T01:26:43.492918+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20081,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.604490+0000","last_change":"2024-09-11T01:26:43.604424+0000","last_active":"2024-09-11T01:26:43.604490+0000","last_peered":"2024-09-11T01:26:43.604490+0000","last_clean":"2024-09-11T01:26:43.604490+0000","last_became_active":"2024-09-11T01:26:43.498430+0000","last_became_peered":"2024-09-11T01:26:43.498430+0000","last_unstale":"2024-09-11T01:26:43.604490+0000","last_undegraded":"2024-09-11T01:26:43.604490+0000","last_fullsized":"2024-09-11T01:26:43.604490+0000","mapping_epoch":64,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:13.180371+0000","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3988,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20428,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.495316+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:43.495316+0000","last_peered":"2024-09-11T01:26:43.495316+0000","last_clean":"2024-09-11T01:26:43.495316+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:43.495316+0000","last_undegraded":"2024-09-11T01:26:43.495316+0000","last_fullsized":"2024-09-11T01:26:43.495316+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492641+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:43.492641+0000","last_peered":"2024-09-11T01:26:43.492641+0000","last_clean":"2024-09-11T01:26:43.492641+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:43.492641+0000","last_undegraded":"2024-09-11T01:26:43.492641+0000","last_fullsized":"2024-09-11T01:26:43.492641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28026,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493375+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:43.493375+0000","last_peered":"2024-09-11T01:26:43.493375+0000","last_clean":"2024-09-11T01:26:43.493375+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:43.493375+0000","last_undegraded":"2024-09-11T01:26:43.493375+0000","last_fullsized":"2024-09-11T01:26:43.493375+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836907+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:26:43.836907+0000","last_peered":"2024-09-11T01:26:43.836907+0000","last_clean":"2024-09-11T01:26:43.836907+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:26:43.836907+0000","last_undegraded":"2024-09-11T01:26:43.836907+0000","last_fullsized":"2024-09-11T01:26:43.836907+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":21696,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:48.323344+0000","last_change":"2024-09-11T01:26:42.381382+0000","last_active":"2024-09-11T01:26:48.323344+0000","last_peered":"2024-09-11T01:26:48.323344+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:26:48.323344+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3159,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":827,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-3159}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":146,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.497081+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:43.497081+0000","last_peered":"2024-09-11T01:26:43.497081+0000","last_clean":"2024-09-11T01:26:43.497081+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:43.497081+0000","last_undegraded":"2024-09-11T01:26:43.497081+0000","last_fullsized":"2024-09-11T01:26:43.497081+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28431,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836927+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:26:43.836927+0000","last_peered":"2024-09-11T01:26:43.836927+0000","last_clean":"2024-09-11T01:26:43.836927+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:26:43.836927+0000","last_undegraded":"2024-09-11T01:26:43.836927+0000","last_fullsized":"2024-09-11T01:26:43.836927+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492885+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:43.492885+0000","last_peered":"2024-09-11T01:26:43.492885+0000","last_clean":"2024-09-11T01:26:43.492885+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:43.492885+0000","last_undegraded":"2024-09-11T01:26:43.492885+0000","last_fullsized":"2024-09-11T01:26:43.492885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27822,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836526+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:26:43.836526+0000","last_peered":"2024-09-11T01:26:43.836526+0000","last_clean":"2024-09-11T01:26:43.836526+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:26:43.836526+0000","last_undegraded":"2024-09-11T01:26:43.836526+0000","last_fullsized":"2024-09-11T01:26:43.836526+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20431,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492883+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:43.492883+0000","last_peered":"2024-09-11T01:26:43.492883+0000","last_clean":"2024-09-11T01:26:43.492883+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:43.492883+0000","last_undegraded":"2024-09-11T01:26:43.492883+0000","last_fullsized":"2024-09-11T01:26:43.492883+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493122+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:43.493122+0000","last_peered":"2024-09-11T01:26:43.493122+0000","last_clean":"2024-09-11T01:26:43.493122+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:43.493122+0000","last_undegraded":"2024-09-11T01:26:43.493122+0000","last_fullsized":"2024-09-11T01:26:43.493122+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27960,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493218+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:43.493218+0000","last_peered":"2024-09-11T01:26:43.493218+0000","last_clean":"2024-09-11T01:26:43.493218+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:43.493218+0000","last_undegraded":"2024-09-11T01:26:43.493218+0000","last_fullsized":"2024-09-11T01:26:43.493218+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837106+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:26:43.837106+0000","last_peered":"2024-09-11T01:26:43.837106+0000","last_clean":"2024-09-11T01:26:43.837106+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:26:43.837106+0000","last_undegraded":"2024-09-11T01:26:43.837106+0000","last_fullsized":"2024-09-11T01:26:43.837106+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24639,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493408+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:43.493408+0000","last_peered":"2024-09-11T01:26:43.493408+0000","last_clean":"2024-09-11T01:26:43.493408+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:43.493408+0000","last_undegraded":"2024-09-11T01:26:43.493408+0000","last_fullsized":"2024-09-11T01:26:43.493408+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492970+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:43.492970+0000","last_peered":"2024-09-11T01:26:43.492970+0000","last_clean":"2024-09-11T01:26:43.492970+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:43.492970+0000","last_undegraded":"2024-09-11T01:26:43.492970+0000","last_fullsized":"2024-09-11T01:26:43.492970+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492818+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:43.492818+0000","last_peered":"2024-09-11T01:26:43.492818+0000","last_clean":"2024-09-11T01:26:43.492818+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:43.492818+0000","last_undegraded":"2024-09-11T01:26:43.492818+0000","last_fullsized":"2024-09-11T01:26:43.492818+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20111,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492860+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:43.492860+0000","last_peered":"2024-09-11T01:26:43.492860+0000","last_clean":"2024-09-11T01:26:43.492860+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:43.492860+0000","last_undegraded":"2024-09-11T01:26:43.492860+0000","last_fullsized":"2024-09-11T01:26:43.492860+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":3159,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":44480,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":115101696,"data_stored":115091259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182583,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518696,"kb_used_data":63640,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853144,"statfs":{"total":96636764160,"available":96105619456,"internally_reserved":0,"allocated":65167360,"data_stored":64801446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986356,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":414876,"kb_used_data":50060,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93956964,"statfs":{"total":96636764160,"available":96211931136,"internally_reserved":0,"allocated":51261440,"data_stored":50899603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659300,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509736,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":509179,"kb_avail":93862104,"statfs":{"total":96636764160,"available":96114794496,"internally_reserved":0,"allocated":548864,"data_stored":182950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521399983},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":50475008,"data_stored":50471529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:54.829 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:26:54.829 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 21696, 'reported_epoch': 65, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:48.323344+0000', 'last_change': '2024-09-11T01:26:42.381382+0000', 'last_active': '2024-09-11T01:26:48.323344+0000', 'last_peered': '2024-09-11T01:26:48.323344+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:26:48.323344+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 3159, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 827, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -3159}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:54.829 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:26:55.282 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:55.282 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2268/6 objects degraded (37800.000%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":951,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1469427712,"bytes_avail":288440864768,"bytes_total":289910292480,"degraded_objects":2268,"degraded_total":6,"degraded_ratio":378,"recovering_objects_per_sec":142,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1625,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:26:55.306 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:26:55.306 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:55.643 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:55.643 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:26:55.663 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":534,"stamp":"2024-09-11T01:26:54.884178+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45372,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1434988,"kb_used_data":114236,"kb_used_omap":12,"kb_used_meta":1320691,"kb_avail":281680532,"statfs":{"total":289910292480,"available":288440864768,"internally_reserved":0,"allocated":116977664,"data_stored":115883999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1352387588},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1625,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1625,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.391007"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28540,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.496965+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:43.496965+0000","last_peered":"2024-09-11T01:26:43.496965+0000","last_clean":"2024-09-11T01:26:43.496965+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:43.496965+0000","last_undegraded":"2024-09-11T01:26:43.496965+0000","last_fullsized":"2024-09-11T01:26:43.496965+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20498,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837072+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:26:43.837072+0000","last_peered":"2024-09-11T01:26:43.837072+0000","last_clean":"2024-09-11T01:26:43.837072+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:26:43.837072+0000","last_undegraded":"2024-09-11T01:26:43.837072+0000","last_fullsized":"2024-09-11T01:26:43.837072+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20321,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493170+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:43.493170+0000","last_peered":"2024-09-11T01:26:43.493170+0000","last_clean":"2024-09-11T01:26:43.493170+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:43.493170+0000","last_undegraded":"2024-09-11T01:26:43.493170+0000","last_fullsized":"2024-09-11T01:26:43.493170+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20248,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837042+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:26:43.837042+0000","last_peered":"2024-09-11T01:26:43.837042+0000","last_clean":"2024-09-11T01:26:43.837042+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:26:43.837042+0000","last_undegraded":"2024-09-11T01:26:43.837042+0000","last_fullsized":"2024-09-11T01:26:43.837042+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20181,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.495408+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:43.495408+0000","last_peered":"2024-09-11T01:26:43.495408+0000","last_clean":"2024-09-11T01:26:43.495408+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:43.495408+0000","last_undegraded":"2024-09-11T01:26:43.495408+0000","last_fullsized":"2024-09-11T01:26:43.495408+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20427,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492918+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:43.492918+0000","last_peered":"2024-09-11T01:26:43.492918+0000","last_clean":"2024-09-11T01:26:43.492918+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:43.492918+0000","last_undegraded":"2024-09-11T01:26:43.492918+0000","last_fullsized":"2024-09-11T01:26:43.492918+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20081,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.604490+0000","last_change":"2024-09-11T01:26:43.604424+0000","last_active":"2024-09-11T01:26:43.604490+0000","last_peered":"2024-09-11T01:26:43.604490+0000","last_clean":"2024-09-11T01:26:43.604490+0000","last_became_active":"2024-09-11T01:26:43.498430+0000","last_became_peered":"2024-09-11T01:26:43.498430+0000","last_unstale":"2024-09-11T01:26:43.604490+0000","last_undegraded":"2024-09-11T01:26:43.604490+0000","last_fullsized":"2024-09-11T01:26:43.604490+0000","mapping_epoch":64,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:13.180371+0000","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3988,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20428,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.495316+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:43.495316+0000","last_peered":"2024-09-11T01:26:43.495316+0000","last_clean":"2024-09-11T01:26:43.495316+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:43.495316+0000","last_undegraded":"2024-09-11T01:26:43.495316+0000","last_fullsized":"2024-09-11T01:26:43.495316+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492641+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:43.492641+0000","last_peered":"2024-09-11T01:26:43.492641+0000","last_clean":"2024-09-11T01:26:43.492641+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:43.492641+0000","last_undegraded":"2024-09-11T01:26:43.492641+0000","last_fullsized":"2024-09-11T01:26:43.492641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28026,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493375+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:43.493375+0000","last_peered":"2024-09-11T01:26:43.493375+0000","last_clean":"2024-09-11T01:26:43.493375+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:43.493375+0000","last_undegraded":"2024-09-11T01:26:43.493375+0000","last_fullsized":"2024-09-11T01:26:43.493375+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836907+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:26:43.836907+0000","last_peered":"2024-09-11T01:26:43.836907+0000","last_clean":"2024-09-11T01:26:43.836907+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:26:43.836907+0000","last_undegraded":"2024-09-11T01:26:43.836907+0000","last_fullsized":"2024-09-11T01:26:43.836907+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":23478,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:53.682555+0000","last_change":"2024-09-11T01:26:42.381382+0000","last_active":"2024-09-11T01:26:53.682555+0000","last_peered":"2024-09-11T01:26:53.682555+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:26:53.682555+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1718,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-2268}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":146,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.497081+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:43.497081+0000","last_peered":"2024-09-11T01:26:43.497081+0000","last_clean":"2024-09-11T01:26:43.497081+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:43.497081+0000","last_undegraded":"2024-09-11T01:26:43.497081+0000","last_fullsized":"2024-09-11T01:26:43.497081+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28431,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836927+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:26:43.836927+0000","last_peered":"2024-09-11T01:26:43.836927+0000","last_clean":"2024-09-11T01:26:43.836927+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:26:43.836927+0000","last_undegraded":"2024-09-11T01:26:43.836927+0000","last_fullsized":"2024-09-11T01:26:43.836927+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492885+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:43.492885+0000","last_peered":"2024-09-11T01:26:43.492885+0000","last_clean":"2024-09-11T01:26:43.492885+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:43.492885+0000","last_undegraded":"2024-09-11T01:26:43.492885+0000","last_fullsized":"2024-09-11T01:26:43.492885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27822,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836526+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:26:43.836526+0000","last_peered":"2024-09-11T01:26:43.836526+0000","last_clean":"2024-09-11T01:26:43.836526+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:26:43.836526+0000","last_undegraded":"2024-09-11T01:26:43.836526+0000","last_fullsized":"2024-09-11T01:26:43.836526+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20431,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492883+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:43.492883+0000","last_peered":"2024-09-11T01:26:43.492883+0000","last_clean":"2024-09-11T01:26:43.492883+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:43.492883+0000","last_undegraded":"2024-09-11T01:26:43.492883+0000","last_fullsized":"2024-09-11T01:26:43.492883+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493122+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:43.493122+0000","last_peered":"2024-09-11T01:26:43.493122+0000","last_clean":"2024-09-11T01:26:43.493122+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:43.493122+0000","last_undegraded":"2024-09-11T01:26:43.493122+0000","last_fullsized":"2024-09-11T01:26:43.493122+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27960,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493218+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:43.493218+0000","last_peered":"2024-09-11T01:26:43.493218+0000","last_clean":"2024-09-11T01:26:43.493218+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:43.493218+0000","last_undegraded":"2024-09-11T01:26:43.493218+0000","last_fullsized":"2024-09-11T01:26:43.493218+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837106+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:26:43.837106+0000","last_peered":"2024-09-11T01:26:43.837106+0000","last_clean":"2024-09-11T01:26:43.837106+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:26:43.837106+0000","last_undegraded":"2024-09-11T01:26:43.837106+0000","last_fullsized":"2024-09-11T01:26:43.837106+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24639,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493408+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:43.493408+0000","last_peered":"2024-09-11T01:26:43.493408+0000","last_clean":"2024-09-11T01:26:43.493408+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:43.493408+0000","last_undegraded":"2024-09-11T01:26:43.493408+0000","last_fullsized":"2024-09-11T01:26:43.493408+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492970+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:43.492970+0000","last_peered":"2024-09-11T01:26:43.492970+0000","last_clean":"2024-09-11T01:26:43.492970+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:43.492970+0000","last_undegraded":"2024-09-11T01:26:43.492970+0000","last_fullsized":"2024-09-11T01:26:43.492970+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492818+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:43.492818+0000","last_peered":"2024-09-11T01:26:43.492818+0000","last_clean":"2024-09-11T01:26:43.492818+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:43.492818+0000","last_undegraded":"2024-09-11T01:26:43.492818+0000","last_fullsized":"2024-09-11T01:26:43.492818+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20111,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492860+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:43.492860+0000","last_peered":"2024-09-11T01:26:43.492860+0000","last_clean":"2024-09-11T01:26:43.492860+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:43.492860+0000","last_undegraded":"2024-09-11T01:26:43.492860+0000","last_fullsized":"2024-09-11T01:26:43.492860+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2268,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":45371,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":115101696,"data_stored":115091259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182585,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518696,"kb_used_data":63640,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853144,"statfs":{"total":96636764160,"available":96105619456,"internally_reserved":0,"allocated":65167360,"data_stored":64801446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986356,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":414876,"kb_used_data":50060,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93956964,"statfs":{"total":96636764160,"available":96211931136,"internally_reserved":0,"allocated":51261440,"data_stored":50899603,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659301,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501416,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":500859,"kb_avail":93870424,"statfs":{"total":96636764160,"available":96123314176,"internally_reserved":0,"allocated":548864,"data_stored":182950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512880303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":50475008,"data_stored":50471529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:58.665 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:59.003 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:59.004 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:26:59.024 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":536,"stamp":"2024-09-11T01:26:58.885367+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46448,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1440908,"kb_used_data":111836,"kb_used_omap":12,"kb_used_meta":1329011,"kb_avail":281674612,"statfs":{"total":289910292480,"available":288434802688,"internally_reserved":0,"allocated":114520064,"data_stored":113426399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1360907268},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2701,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2701,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002947"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28540,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.496965+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:43.496965+0000","last_peered":"2024-09-11T01:26:43.496965+0000","last_clean":"2024-09-11T01:26:43.496965+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:43.496965+0000","last_undegraded":"2024-09-11T01:26:43.496965+0000","last_fullsized":"2024-09-11T01:26:43.496965+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20498,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837072+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:26:43.837072+0000","last_peered":"2024-09-11T01:26:43.837072+0000","last_clean":"2024-09-11T01:26:43.837072+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:26:43.837072+0000","last_undegraded":"2024-09-11T01:26:43.837072+0000","last_fullsized":"2024-09-11T01:26:43.837072+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20321,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493170+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:43.493170+0000","last_peered":"2024-09-11T01:26:43.493170+0000","last_clean":"2024-09-11T01:26:43.493170+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:43.493170+0000","last_undegraded":"2024-09-11T01:26:43.493170+0000","last_fullsized":"2024-09-11T01:26:43.493170+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20248,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837042+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:26:43.837042+0000","last_peered":"2024-09-11T01:26:43.837042+0000","last_clean":"2024-09-11T01:26:43.837042+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:26:43.837042+0000","last_undegraded":"2024-09-11T01:26:43.837042+0000","last_fullsized":"2024-09-11T01:26:43.837042+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20181,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.495408+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:43.495408+0000","last_peered":"2024-09-11T01:26:43.495408+0000","last_clean":"2024-09-11T01:26:43.495408+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:43.495408+0000","last_undegraded":"2024-09-11T01:26:43.495408+0000","last_fullsized":"2024-09-11T01:26:43.495408+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20427,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492918+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:43.492918+0000","last_peered":"2024-09-11T01:26:43.492918+0000","last_clean":"2024-09-11T01:26:43.492918+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:43.492918+0000","last_undegraded":"2024-09-11T01:26:43.492918+0000","last_fullsized":"2024-09-11T01:26:43.492918+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20081,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.604490+0000","last_change":"2024-09-11T01:26:43.604424+0000","last_active":"2024-09-11T01:26:43.604490+0000","last_peered":"2024-09-11T01:26:43.604490+0000","last_clean":"2024-09-11T01:26:43.604490+0000","last_became_active":"2024-09-11T01:26:43.498430+0000","last_became_peered":"2024-09-11T01:26:43.498430+0000","last_unstale":"2024-09-11T01:26:43.604490+0000","last_undegraded":"2024-09-11T01:26:43.604490+0000","last_fullsized":"2024-09-11T01:26:43.604490+0000","mapping_epoch":64,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:13.180371+0000","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3988,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20428,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.495316+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:43.495316+0000","last_peered":"2024-09-11T01:26:43.495316+0000","last_clean":"2024-09-11T01:26:43.495316+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:43.495316+0000","last_undegraded":"2024-09-11T01:26:43.495316+0000","last_fullsized":"2024-09-11T01:26:43.495316+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492641+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:43.492641+0000","last_peered":"2024-09-11T01:26:43.492641+0000","last_clean":"2024-09-11T01:26:43.492641+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:43.492641+0000","last_undegraded":"2024-09-11T01:26:43.492641+0000","last_fullsized":"2024-09-11T01:26:43.492641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28026,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493375+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:43.493375+0000","last_peered":"2024-09-11T01:26:43.493375+0000","last_clean":"2024-09-11T01:26:43.493375+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:43.493375+0000","last_undegraded":"2024-09-11T01:26:43.493375+0000","last_fullsized":"2024-09-11T01:26:43.493375+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836907+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:26:43.836907+0000","last_peered":"2024-09-11T01:26:43.836907+0000","last_clean":"2024-09-11T01:26:43.836907+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:26:43.836907+0000","last_undegraded":"2024-09-11T01:26:43.836907+0000","last_fullsized":"2024-09-11T01:26:43.836907+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":25630,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:58.689888+0000","last_change":"2024-09-11T01:26:42.381382+0000","last_active":"2024-09-11T01:26:58.689888+0000","last_peered":"2024-09-11T01:26:58.689888+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:26:58.689888+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2794,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1192}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":146,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.497081+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:43.497081+0000","last_peered":"2024-09-11T01:26:43.497081+0000","last_clean":"2024-09-11T01:26:43.497081+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:43.497081+0000","last_undegraded":"2024-09-11T01:26:43.497081+0000","last_fullsized":"2024-09-11T01:26:43.497081+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28431,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836927+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:26:43.836927+0000","last_peered":"2024-09-11T01:26:43.836927+0000","last_clean":"2024-09-11T01:26:43.836927+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:26:43.836927+0000","last_undegraded":"2024-09-11T01:26:43.836927+0000","last_fullsized":"2024-09-11T01:26:43.836927+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492885+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:43.492885+0000","last_peered":"2024-09-11T01:26:43.492885+0000","last_clean":"2024-09-11T01:26:43.492885+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:43.492885+0000","last_undegraded":"2024-09-11T01:26:43.492885+0000","last_fullsized":"2024-09-11T01:26:43.492885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27822,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836526+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:26:43.836526+0000","last_peered":"2024-09-11T01:26:43.836526+0000","last_clean":"2024-09-11T01:26:43.836526+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:26:43.836526+0000","last_undegraded":"2024-09-11T01:26:43.836526+0000","last_fullsized":"2024-09-11T01:26:43.836526+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20431,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492883+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:43.492883+0000","last_peered":"2024-09-11T01:26:43.492883+0000","last_clean":"2024-09-11T01:26:43.492883+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:43.492883+0000","last_undegraded":"2024-09-11T01:26:43.492883+0000","last_fullsized":"2024-09-11T01:26:43.492883+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493122+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:43.493122+0000","last_peered":"2024-09-11T01:26:43.493122+0000","last_clean":"2024-09-11T01:26:43.493122+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:43.493122+0000","last_undegraded":"2024-09-11T01:26:43.493122+0000","last_fullsized":"2024-09-11T01:26:43.493122+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27960,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493218+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:43.493218+0000","last_peered":"2024-09-11T01:26:43.493218+0000","last_clean":"2024-09-11T01:26:43.493218+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:43.493218+0000","last_undegraded":"2024-09-11T01:26:43.493218+0000","last_fullsized":"2024-09-11T01:26:43.493218+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837106+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:26:43.837106+0000","last_peered":"2024-09-11T01:26:43.837106+0000","last_clean":"2024-09-11T01:26:43.837106+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:26:43.837106+0000","last_undegraded":"2024-09-11T01:26:43.837106+0000","last_fullsized":"2024-09-11T01:26:43.837106+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24639,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493408+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:43.493408+0000","last_peered":"2024-09-11T01:26:43.493408+0000","last_clean":"2024-09-11T01:26:43.493408+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:43.493408+0000","last_undegraded":"2024-09-11T01:26:43.493408+0000","last_fullsized":"2024-09-11T01:26:43.493408+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492970+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:43.492970+0000","last_peered":"2024-09-11T01:26:43.492970+0000","last_clean":"2024-09-11T01:26:43.492970+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:43.492970+0000","last_undegraded":"2024-09-11T01:26:43.492970+0000","last_fullsized":"2024-09-11T01:26:43.492970+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492818+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:43.492818+0000","last_peered":"2024-09-11T01:26:43.492818+0000","last_clean":"2024-09-11T01:26:43.492818+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:43.492818+0000","last_undegraded":"2024-09-11T01:26:43.492818+0000","last_fullsized":"2024-09-11T01:26:43.492818+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20111,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492860+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:43.492860+0000","last_peered":"2024-09-11T01:26:43.492860+0000","last_clean":"2024-09-11T01:26:43.492860+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:43.492860+0000","last_undegraded":"2024-09-11T01:26:43.492860+0000","last_fullsized":"2024-09-11T01:26:43.492860+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46447,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":112644096,"data_stored":112633659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182586,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518696,"kb_used_data":63640,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853144,"statfs":{"total":96636764160,"available":96105619456,"internally_reserved":0,"allocated":65167360,"data_stored":64801446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986357,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":412476,"kb_used_data":47660,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93959364,"statfs":{"total":96636764160,"available":96214388736,"internally_reserved":0,"allocated":48803840,"data_stored":48442003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659302,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509736,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":509179,"kb_avail":93862104,"statfs":{"total":96636764160,"available":96114794496,"internally_reserved":0,"allocated":548864,"data_stored":182950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521399983},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48017408,"data_stored":48013929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:59.025 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:26:59.353 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:59.353 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:26:59.373 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":536,"stamp":"2024-09-11T01:26:58.885367+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46448,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1440908,"kb_used_data":111836,"kb_used_omap":12,"kb_used_meta":1329011,"kb_avail":281674612,"statfs":{"total":289910292480,"available":288434802688,"internally_reserved":0,"allocated":114520064,"data_stored":113426399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1360907268},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2701,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2701,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002947"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28540,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.496965+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:43.496965+0000","last_peered":"2024-09-11T01:26:43.496965+0000","last_clean":"2024-09-11T01:26:43.496965+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:43.496965+0000","last_undegraded":"2024-09-11T01:26:43.496965+0000","last_fullsized":"2024-09-11T01:26:43.496965+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20498,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837072+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:26:43.837072+0000","last_peered":"2024-09-11T01:26:43.837072+0000","last_clean":"2024-09-11T01:26:43.837072+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:26:43.837072+0000","last_undegraded":"2024-09-11T01:26:43.837072+0000","last_fullsized":"2024-09-11T01:26:43.837072+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20321,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493170+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:43.493170+0000","last_peered":"2024-09-11T01:26:43.493170+0000","last_clean":"2024-09-11T01:26:43.493170+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:43.493170+0000","last_undegraded":"2024-09-11T01:26:43.493170+0000","last_fullsized":"2024-09-11T01:26:43.493170+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20248,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837042+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:26:43.837042+0000","last_peered":"2024-09-11T01:26:43.837042+0000","last_clean":"2024-09-11T01:26:43.837042+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:26:43.837042+0000","last_undegraded":"2024-09-11T01:26:43.837042+0000","last_fullsized":"2024-09-11T01:26:43.837042+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20181,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.495408+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:43.495408+0000","last_peered":"2024-09-11T01:26:43.495408+0000","last_clean":"2024-09-11T01:26:43.495408+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:43.495408+0000","last_undegraded":"2024-09-11T01:26:43.495408+0000","last_fullsized":"2024-09-11T01:26:43.495408+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20427,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492918+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:43.492918+0000","last_peered":"2024-09-11T01:26:43.492918+0000","last_clean":"2024-09-11T01:26:43.492918+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:43.492918+0000","last_undegraded":"2024-09-11T01:26:43.492918+0000","last_fullsized":"2024-09-11T01:26:43.492918+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20081,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.604490+0000","last_change":"2024-09-11T01:26:43.604424+0000","last_active":"2024-09-11T01:26:43.604490+0000","last_peered":"2024-09-11T01:26:43.604490+0000","last_clean":"2024-09-11T01:26:43.604490+0000","last_became_active":"2024-09-11T01:26:43.498430+0000","last_became_peered":"2024-09-11T01:26:43.498430+0000","last_unstale":"2024-09-11T01:26:43.604490+0000","last_undegraded":"2024-09-11T01:26:43.604490+0000","last_fullsized":"2024-09-11T01:26:43.604490+0000","mapping_epoch":64,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:13.180371+0000","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3988,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20428,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.495316+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:43.495316+0000","last_peered":"2024-09-11T01:26:43.495316+0000","last_clean":"2024-09-11T01:26:43.495316+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:43.495316+0000","last_undegraded":"2024-09-11T01:26:43.495316+0000","last_fullsized":"2024-09-11T01:26:43.495316+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492641+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:43.492641+0000","last_peered":"2024-09-11T01:26:43.492641+0000","last_clean":"2024-09-11T01:26:43.492641+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:43.492641+0000","last_undegraded":"2024-09-11T01:26:43.492641+0000","last_fullsized":"2024-09-11T01:26:43.492641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28026,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493375+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:43.493375+0000","last_peered":"2024-09-11T01:26:43.493375+0000","last_clean":"2024-09-11T01:26:43.493375+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:43.493375+0000","last_undegraded":"2024-09-11T01:26:43.493375+0000","last_fullsized":"2024-09-11T01:26:43.493375+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836907+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:26:43.836907+0000","last_peered":"2024-09-11T01:26:43.836907+0000","last_clean":"2024-09-11T01:26:43.836907+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:26:43.836907+0000","last_undegraded":"2024-09-11T01:26:43.836907+0000","last_fullsized":"2024-09-11T01:26:43.836907+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":25630,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:58.689888+0000","last_change":"2024-09-11T01:26:42.381382+0000","last_active":"2024-09-11T01:26:58.689888+0000","last_peered":"2024-09-11T01:26:58.689888+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:26:58.689888+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2794,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1192}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":146,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.497081+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:43.497081+0000","last_peered":"2024-09-11T01:26:43.497081+0000","last_clean":"2024-09-11T01:26:43.497081+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:43.497081+0000","last_undegraded":"2024-09-11T01:26:43.497081+0000","last_fullsized":"2024-09-11T01:26:43.497081+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28431,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836927+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:26:43.836927+0000","last_peered":"2024-09-11T01:26:43.836927+0000","last_clean":"2024-09-11T01:26:43.836927+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:26:43.836927+0000","last_undegraded":"2024-09-11T01:26:43.836927+0000","last_fullsized":"2024-09-11T01:26:43.836927+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492885+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:43.492885+0000","last_peered":"2024-09-11T01:26:43.492885+0000","last_clean":"2024-09-11T01:26:43.492885+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:43.492885+0000","last_undegraded":"2024-09-11T01:26:43.492885+0000","last_fullsized":"2024-09-11T01:26:43.492885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27822,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836526+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:26:43.836526+0000","last_peered":"2024-09-11T01:26:43.836526+0000","last_clean":"2024-09-11T01:26:43.836526+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:26:43.836526+0000","last_undegraded":"2024-09-11T01:26:43.836526+0000","last_fullsized":"2024-09-11T01:26:43.836526+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20431,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492883+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:43.492883+0000","last_peered":"2024-09-11T01:26:43.492883+0000","last_clean":"2024-09-11T01:26:43.492883+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:43.492883+0000","last_undegraded":"2024-09-11T01:26:43.492883+0000","last_fullsized":"2024-09-11T01:26:43.492883+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493122+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:43.493122+0000","last_peered":"2024-09-11T01:26:43.493122+0000","last_clean":"2024-09-11T01:26:43.493122+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:43.493122+0000","last_undegraded":"2024-09-11T01:26:43.493122+0000","last_fullsized":"2024-09-11T01:26:43.493122+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27960,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493218+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:43.493218+0000","last_peered":"2024-09-11T01:26:43.493218+0000","last_clean":"2024-09-11T01:26:43.493218+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:43.493218+0000","last_undegraded":"2024-09-11T01:26:43.493218+0000","last_fullsized":"2024-09-11T01:26:43.493218+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837106+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:26:43.837106+0000","last_peered":"2024-09-11T01:26:43.837106+0000","last_clean":"2024-09-11T01:26:43.837106+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:26:43.837106+0000","last_undegraded":"2024-09-11T01:26:43.837106+0000","last_fullsized":"2024-09-11T01:26:43.837106+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24639,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493408+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:43.493408+0000","last_peered":"2024-09-11T01:26:43.493408+0000","last_clean":"2024-09-11T01:26:43.493408+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:43.493408+0000","last_undegraded":"2024-09-11T01:26:43.493408+0000","last_fullsized":"2024-09-11T01:26:43.493408+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492970+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:43.492970+0000","last_peered":"2024-09-11T01:26:43.492970+0000","last_clean":"2024-09-11T01:26:43.492970+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:43.492970+0000","last_undegraded":"2024-09-11T01:26:43.492970+0000","last_fullsized":"2024-09-11T01:26:43.492970+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492818+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:43.492818+0000","last_peered":"2024-09-11T01:26:43.492818+0000","last_clean":"2024-09-11T01:26:43.492818+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:43.492818+0000","last_undegraded":"2024-09-11T01:26:43.492818+0000","last_fullsized":"2024-09-11T01:26:43.492818+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20111,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492860+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:43.492860+0000","last_peered":"2024-09-11T01:26:43.492860+0000","last_clean":"2024-09-11T01:26:43.492860+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:43.492860+0000","last_undegraded":"2024-09-11T01:26:43.492860+0000","last_fullsized":"2024-09-11T01:26:43.492860+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46447,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":112644096,"data_stored":112633659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182586,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518696,"kb_used_data":63640,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853144,"statfs":{"total":96636764160,"available":96105619456,"internally_reserved":0,"allocated":65167360,"data_stored":64801446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986357,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":412476,"kb_used_data":47660,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93959364,"statfs":{"total":96636764160,"available":96214388736,"internally_reserved":0,"allocated":48803840,"data_stored":48442003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659302,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509736,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":509179,"kb_avail":93862104,"statfs":{"total":96636764160,"available":96114794496,"internally_reserved":0,"allocated":548864,"data_stored":182950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521399983},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48017408,"data_stored":48013929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:26:59.374 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:26:59.374 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 25630, 'reported_epoch': 65, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:58.689888+0000', 'last_change': '2024-09-11T01:26:42.381382+0000', 'last_active': '2024-09-11T01:26:58.689888+0000', 'last_peered': '2024-09-11T01:26:58.689888+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:26:58.689888+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1192, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2794, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1192}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:26:59.375 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:26:59.800 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:26:59.801 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 1192/6 objects degraded (19866.667%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":956,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1475489792,"bytes_avail":288434802688,"bytes_total":289910292480,"degraded_objects":1192,"degraded_total":6,"degraded_ratio":198.66666666666663,"recovering_objects_per_sec":225,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":2701,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:26:59.819 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:26:59.819 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:00.139 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:27:00.140 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:27:00.157 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":536,"stamp":"2024-09-11T01:26:58.885367+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46448,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1440908,"kb_used_data":111836,"kb_used_omap":12,"kb_used_meta":1329011,"kb_avail":281674612,"statfs":{"total":289910292480,"available":288434802688,"internally_reserved":0,"allocated":114520064,"data_stored":113426399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1360907268},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2701,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2701,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002947"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28540,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.496965+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:43.496965+0000","last_peered":"2024-09-11T01:26:43.496965+0000","last_clean":"2024-09-11T01:26:43.496965+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:43.496965+0000","last_undegraded":"2024-09-11T01:26:43.496965+0000","last_fullsized":"2024-09-11T01:26:43.496965+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20498,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837072+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:26:43.837072+0000","last_peered":"2024-09-11T01:26:43.837072+0000","last_clean":"2024-09-11T01:26:43.837072+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:26:43.837072+0000","last_undegraded":"2024-09-11T01:26:43.837072+0000","last_fullsized":"2024-09-11T01:26:43.837072+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20321,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493170+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:43.493170+0000","last_peered":"2024-09-11T01:26:43.493170+0000","last_clean":"2024-09-11T01:26:43.493170+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:43.493170+0000","last_undegraded":"2024-09-11T01:26:43.493170+0000","last_fullsized":"2024-09-11T01:26:43.493170+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20248,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837042+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:26:43.837042+0000","last_peered":"2024-09-11T01:26:43.837042+0000","last_clean":"2024-09-11T01:26:43.837042+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:26:43.837042+0000","last_undegraded":"2024-09-11T01:26:43.837042+0000","last_fullsized":"2024-09-11T01:26:43.837042+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20181,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.495408+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:43.495408+0000","last_peered":"2024-09-11T01:26:43.495408+0000","last_clean":"2024-09-11T01:26:43.495408+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:43.495408+0000","last_undegraded":"2024-09-11T01:26:43.495408+0000","last_fullsized":"2024-09-11T01:26:43.495408+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20427,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492918+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:43.492918+0000","last_peered":"2024-09-11T01:26:43.492918+0000","last_clean":"2024-09-11T01:26:43.492918+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:43.492918+0000","last_undegraded":"2024-09-11T01:26:43.492918+0000","last_fullsized":"2024-09-11T01:26:43.492918+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20081,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.604490+0000","last_change":"2024-09-11T01:26:43.604424+0000","last_active":"2024-09-11T01:26:43.604490+0000","last_peered":"2024-09-11T01:26:43.604490+0000","last_clean":"2024-09-11T01:26:43.604490+0000","last_became_active":"2024-09-11T01:26:43.498430+0000","last_became_peered":"2024-09-11T01:26:43.498430+0000","last_unstale":"2024-09-11T01:26:43.604490+0000","last_undegraded":"2024-09-11T01:26:43.604490+0000","last_fullsized":"2024-09-11T01:26:43.604490+0000","mapping_epoch":64,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:13.180371+0000","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3988,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20428,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.495316+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:43.495316+0000","last_peered":"2024-09-11T01:26:43.495316+0000","last_clean":"2024-09-11T01:26:43.495316+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:43.495316+0000","last_undegraded":"2024-09-11T01:26:43.495316+0000","last_fullsized":"2024-09-11T01:26:43.495316+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492641+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:43.492641+0000","last_peered":"2024-09-11T01:26:43.492641+0000","last_clean":"2024-09-11T01:26:43.492641+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:43.492641+0000","last_undegraded":"2024-09-11T01:26:43.492641+0000","last_fullsized":"2024-09-11T01:26:43.492641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28026,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493375+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:43.493375+0000","last_peered":"2024-09-11T01:26:43.493375+0000","last_clean":"2024-09-11T01:26:43.493375+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:43.493375+0000","last_undegraded":"2024-09-11T01:26:43.493375+0000","last_fullsized":"2024-09-11T01:26:43.493375+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836907+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:26:43.836907+0000","last_peered":"2024-09-11T01:26:43.836907+0000","last_clean":"2024-09-11T01:26:43.836907+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:26:43.836907+0000","last_undegraded":"2024-09-11T01:26:43.836907+0000","last_fullsized":"2024-09-11T01:26:43.836907+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":25630,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:58.689888+0000","last_change":"2024-09-11T01:26:42.381382+0000","last_active":"2024-09-11T01:26:58.689888+0000","last_peered":"2024-09-11T01:26:58.689888+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:26:58.689888+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2794,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1192}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":146,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.497081+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:43.497081+0000","last_peered":"2024-09-11T01:26:43.497081+0000","last_clean":"2024-09-11T01:26:43.497081+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:43.497081+0000","last_undegraded":"2024-09-11T01:26:43.497081+0000","last_fullsized":"2024-09-11T01:26:43.497081+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28431,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836927+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:26:43.836927+0000","last_peered":"2024-09-11T01:26:43.836927+0000","last_clean":"2024-09-11T01:26:43.836927+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:26:43.836927+0000","last_undegraded":"2024-09-11T01:26:43.836927+0000","last_fullsized":"2024-09-11T01:26:43.836927+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492885+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:43.492885+0000","last_peered":"2024-09-11T01:26:43.492885+0000","last_clean":"2024-09-11T01:26:43.492885+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:43.492885+0000","last_undegraded":"2024-09-11T01:26:43.492885+0000","last_fullsized":"2024-09-11T01:26:43.492885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27822,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836526+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:26:43.836526+0000","last_peered":"2024-09-11T01:26:43.836526+0000","last_clean":"2024-09-11T01:26:43.836526+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:26:43.836526+0000","last_undegraded":"2024-09-11T01:26:43.836526+0000","last_fullsized":"2024-09-11T01:26:43.836526+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20431,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492883+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:43.492883+0000","last_peered":"2024-09-11T01:26:43.492883+0000","last_clean":"2024-09-11T01:26:43.492883+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:43.492883+0000","last_undegraded":"2024-09-11T01:26:43.492883+0000","last_fullsized":"2024-09-11T01:26:43.492883+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493122+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:43.493122+0000","last_peered":"2024-09-11T01:26:43.493122+0000","last_clean":"2024-09-11T01:26:43.493122+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:43.493122+0000","last_undegraded":"2024-09-11T01:26:43.493122+0000","last_fullsized":"2024-09-11T01:26:43.493122+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27960,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493218+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:43.493218+0000","last_peered":"2024-09-11T01:26:43.493218+0000","last_clean":"2024-09-11T01:26:43.493218+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:43.493218+0000","last_undegraded":"2024-09-11T01:26:43.493218+0000","last_fullsized":"2024-09-11T01:26:43.493218+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837106+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:26:43.837106+0000","last_peered":"2024-09-11T01:26:43.837106+0000","last_clean":"2024-09-11T01:26:43.837106+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:26:43.837106+0000","last_undegraded":"2024-09-11T01:26:43.837106+0000","last_fullsized":"2024-09-11T01:26:43.837106+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24639,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493408+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:43.493408+0000","last_peered":"2024-09-11T01:26:43.493408+0000","last_clean":"2024-09-11T01:26:43.493408+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:43.493408+0000","last_undegraded":"2024-09-11T01:26:43.493408+0000","last_fullsized":"2024-09-11T01:26:43.493408+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492970+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:43.492970+0000","last_peered":"2024-09-11T01:26:43.492970+0000","last_clean":"2024-09-11T01:26:43.492970+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:43.492970+0000","last_undegraded":"2024-09-11T01:26:43.492970+0000","last_fullsized":"2024-09-11T01:26:43.492970+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492818+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:43.492818+0000","last_peered":"2024-09-11T01:26:43.492818+0000","last_clean":"2024-09-11T01:26:43.492818+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:43.492818+0000","last_undegraded":"2024-09-11T01:26:43.492818+0000","last_fullsized":"2024-09-11T01:26:43.492818+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20111,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492860+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:43.492860+0000","last_peered":"2024-09-11T01:26:43.492860+0000","last_clean":"2024-09-11T01:26:43.492860+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:43.492860+0000","last_undegraded":"2024-09-11T01:26:43.492860+0000","last_fullsized":"2024-09-11T01:26:43.492860+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46447,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":112644096,"data_stored":112633659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182586,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518696,"kb_used_data":63640,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853144,"statfs":{"total":96636764160,"available":96105619456,"internally_reserved":0,"allocated":65167360,"data_stored":64801446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986357,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":412476,"kb_used_data":47660,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93959364,"statfs":{"total":96636764160,"available":96214388736,"internally_reserved":0,"allocated":48803840,"data_stored":48442003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659302,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509736,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":509179,"kb_avail":93862104,"statfs":{"total":96636764160,"available":96114794496,"internally_reserved":0,"allocated":548864,"data_stored":182950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521399983},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":48017408,"data_stored":48013929,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:03.159 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:03.485 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:27:03.486 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:27:03.504 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":538,"stamp":"2024-09-11T01:27:02.886732+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46448,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1438508,"kb_used_data":109436,"kb_used_omap":12,"kb_used_meta":1329011,"kb_avail":281677012,"statfs":{"total":289910292480,"available":288437260288,"internally_reserved":0,"allocated":112062464,"data_stored":110968799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1360907268},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1967,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1967,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003286"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28540,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.496965+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:43.496965+0000","last_peered":"2024-09-11T01:26:43.496965+0000","last_clean":"2024-09-11T01:26:43.496965+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:43.496965+0000","last_undegraded":"2024-09-11T01:26:43.496965+0000","last_fullsized":"2024-09-11T01:26:43.496965+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20498,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837072+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:26:43.837072+0000","last_peered":"2024-09-11T01:26:43.837072+0000","last_clean":"2024-09-11T01:26:43.837072+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:26:43.837072+0000","last_undegraded":"2024-09-11T01:26:43.837072+0000","last_fullsized":"2024-09-11T01:26:43.837072+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20321,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493170+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:43.493170+0000","last_peered":"2024-09-11T01:26:43.493170+0000","last_clean":"2024-09-11T01:26:43.493170+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:43.493170+0000","last_undegraded":"2024-09-11T01:26:43.493170+0000","last_fullsized":"2024-09-11T01:26:43.493170+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20248,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837042+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:26:43.837042+0000","last_peered":"2024-09-11T01:26:43.837042+0000","last_clean":"2024-09-11T01:26:43.837042+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:26:43.837042+0000","last_undegraded":"2024-09-11T01:26:43.837042+0000","last_fullsized":"2024-09-11T01:26:43.837042+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20181,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.495408+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:43.495408+0000","last_peered":"2024-09-11T01:26:43.495408+0000","last_clean":"2024-09-11T01:26:43.495408+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:43.495408+0000","last_undegraded":"2024-09-11T01:26:43.495408+0000","last_fullsized":"2024-09-11T01:26:43.495408+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20427,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492918+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:43.492918+0000","last_peered":"2024-09-11T01:26:43.492918+0000","last_clean":"2024-09-11T01:26:43.492918+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:43.492918+0000","last_undegraded":"2024-09-11T01:26:43.492918+0000","last_fullsized":"2024-09-11T01:26:43.492918+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20081,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.604490+0000","last_change":"2024-09-11T01:26:43.604424+0000","last_active":"2024-09-11T01:26:43.604490+0000","last_peered":"2024-09-11T01:26:43.604490+0000","last_clean":"2024-09-11T01:26:43.604490+0000","last_became_active":"2024-09-11T01:26:43.498430+0000","last_became_peered":"2024-09-11T01:26:43.498430+0000","last_unstale":"2024-09-11T01:26:43.604490+0000","last_undegraded":"2024-09-11T01:26:43.604490+0000","last_fullsized":"2024-09-11T01:26:43.604490+0000","mapping_epoch":64,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:13.180371+0000","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3988,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20428,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.495316+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:43.495316+0000","last_peered":"2024-09-11T01:26:43.495316+0000","last_clean":"2024-09-11T01:26:43.495316+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:43.495316+0000","last_undegraded":"2024-09-11T01:26:43.495316+0000","last_fullsized":"2024-09-11T01:26:43.495316+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492641+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:43.492641+0000","last_peered":"2024-09-11T01:26:43.492641+0000","last_clean":"2024-09-11T01:26:43.492641+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:43.492641+0000","last_undegraded":"2024-09-11T01:26:43.492641+0000","last_fullsized":"2024-09-11T01:26:43.492641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28026,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493375+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:43.493375+0000","last_peered":"2024-09-11T01:26:43.493375+0000","last_clean":"2024-09-11T01:26:43.493375+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:43.493375+0000","last_undegraded":"2024-09-11T01:26:43.493375+0000","last_fullsized":"2024-09-11T01:26:43.493375+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836907+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:26:43.836907+0000","last_peered":"2024-09-11T01:26:43.836907+0000","last_clean":"2024-09-11T01:26:43.836907+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:26:43.836907+0000","last_undegraded":"2024-09-11T01:26:43.836907+0000","last_fullsized":"2024-09-11T01:26:43.836907+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":25630,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:58.689888+0000","last_change":"2024-09-11T01:26:42.381382+0000","last_active":"2024-09-11T01:26:58.689888+0000","last_peered":"2024-09-11T01:26:58.689888+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:26:58.689888+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2794,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1192}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":146,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.497081+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:43.497081+0000","last_peered":"2024-09-11T01:26:43.497081+0000","last_clean":"2024-09-11T01:26:43.497081+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:43.497081+0000","last_undegraded":"2024-09-11T01:26:43.497081+0000","last_fullsized":"2024-09-11T01:26:43.497081+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28431,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836927+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:26:43.836927+0000","last_peered":"2024-09-11T01:26:43.836927+0000","last_clean":"2024-09-11T01:26:43.836927+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:26:43.836927+0000","last_undegraded":"2024-09-11T01:26:43.836927+0000","last_fullsized":"2024-09-11T01:26:43.836927+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492885+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:43.492885+0000","last_peered":"2024-09-11T01:26:43.492885+0000","last_clean":"2024-09-11T01:26:43.492885+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:43.492885+0000","last_undegraded":"2024-09-11T01:26:43.492885+0000","last_fullsized":"2024-09-11T01:26:43.492885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27822,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836526+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:26:43.836526+0000","last_peered":"2024-09-11T01:26:43.836526+0000","last_clean":"2024-09-11T01:26:43.836526+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:26:43.836526+0000","last_undegraded":"2024-09-11T01:26:43.836526+0000","last_fullsized":"2024-09-11T01:26:43.836526+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20431,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492883+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:43.492883+0000","last_peered":"2024-09-11T01:26:43.492883+0000","last_clean":"2024-09-11T01:26:43.492883+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:43.492883+0000","last_undegraded":"2024-09-11T01:26:43.492883+0000","last_fullsized":"2024-09-11T01:26:43.492883+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493122+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:43.493122+0000","last_peered":"2024-09-11T01:26:43.493122+0000","last_clean":"2024-09-11T01:26:43.493122+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:43.493122+0000","last_undegraded":"2024-09-11T01:26:43.493122+0000","last_fullsized":"2024-09-11T01:26:43.493122+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27960,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493218+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:43.493218+0000","last_peered":"2024-09-11T01:26:43.493218+0000","last_clean":"2024-09-11T01:26:43.493218+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:43.493218+0000","last_undegraded":"2024-09-11T01:26:43.493218+0000","last_fullsized":"2024-09-11T01:26:43.493218+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837106+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:26:43.837106+0000","last_peered":"2024-09-11T01:26:43.837106+0000","last_clean":"2024-09-11T01:26:43.837106+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:26:43.837106+0000","last_undegraded":"2024-09-11T01:26:43.837106+0000","last_fullsized":"2024-09-11T01:26:43.837106+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24639,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493408+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:43.493408+0000","last_peered":"2024-09-11T01:26:43.493408+0000","last_clean":"2024-09-11T01:26:43.493408+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:43.493408+0000","last_undegraded":"2024-09-11T01:26:43.493408+0000","last_fullsized":"2024-09-11T01:26:43.493408+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492970+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:43.492970+0000","last_peered":"2024-09-11T01:26:43.492970+0000","last_clean":"2024-09-11T01:26:43.492970+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:43.492970+0000","last_undegraded":"2024-09-11T01:26:43.492970+0000","last_fullsized":"2024-09-11T01:26:43.492970+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492818+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:43.492818+0000","last_peered":"2024-09-11T01:26:43.492818+0000","last_clean":"2024-09-11T01:26:43.492818+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:43.492818+0000","last_undegraded":"2024-09-11T01:26:43.492818+0000","last_fullsized":"2024-09-11T01:26:43.492818+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20111,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492860+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:43.492860+0000","last_peered":"2024-09-11T01:26:43.492860+0000","last_clean":"2024-09-11T01:26:43.492860+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:43.492860+0000","last_undegraded":"2024-09-11T01:26:43.492860+0000","last_fullsized":"2024-09-11T01:26:43.492860+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46447,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110186496,"data_stored":110176059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182586,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518696,"kb_used_data":63640,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853144,"statfs":{"total":96636764160,"available":96105619456,"internally_reserved":0,"allocated":65167360,"data_stored":64801446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986358,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":410076,"kb_used_data":45260,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93961764,"statfs":{"total":96636764160,"available":96216846336,"internally_reserved":0,"allocated":46346240,"data_stored":45984403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659302,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509736,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":509179,"kb_avail":93862104,"statfs":{"total":96636764160,"available":96114794496,"internally_reserved":0,"allocated":548864,"data_stored":182950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521399983},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45559808,"data_stored":45556329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:03.505 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:03.830 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:27:03.831 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:27:03.849 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":538,"stamp":"2024-09-11T01:27:02.886732+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46448,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1438508,"kb_used_data":109436,"kb_used_omap":12,"kb_used_meta":1329011,"kb_avail":281677012,"statfs":{"total":289910292480,"available":288437260288,"internally_reserved":0,"allocated":112062464,"data_stored":110968799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1360907268},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1967,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1967,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003286"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28540,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.496965+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:43.496965+0000","last_peered":"2024-09-11T01:26:43.496965+0000","last_clean":"2024-09-11T01:26:43.496965+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:43.496965+0000","last_undegraded":"2024-09-11T01:26:43.496965+0000","last_fullsized":"2024-09-11T01:26:43.496965+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20498,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837072+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:26:43.837072+0000","last_peered":"2024-09-11T01:26:43.837072+0000","last_clean":"2024-09-11T01:26:43.837072+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:26:43.837072+0000","last_undegraded":"2024-09-11T01:26:43.837072+0000","last_fullsized":"2024-09-11T01:26:43.837072+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20321,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493170+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:43.493170+0000","last_peered":"2024-09-11T01:26:43.493170+0000","last_clean":"2024-09-11T01:26:43.493170+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:43.493170+0000","last_undegraded":"2024-09-11T01:26:43.493170+0000","last_fullsized":"2024-09-11T01:26:43.493170+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20248,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837042+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:26:43.837042+0000","last_peered":"2024-09-11T01:26:43.837042+0000","last_clean":"2024-09-11T01:26:43.837042+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:26:43.837042+0000","last_undegraded":"2024-09-11T01:26:43.837042+0000","last_fullsized":"2024-09-11T01:26:43.837042+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20181,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.495408+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:43.495408+0000","last_peered":"2024-09-11T01:26:43.495408+0000","last_clean":"2024-09-11T01:26:43.495408+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:43.495408+0000","last_undegraded":"2024-09-11T01:26:43.495408+0000","last_fullsized":"2024-09-11T01:26:43.495408+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20427,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492918+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:43.492918+0000","last_peered":"2024-09-11T01:26:43.492918+0000","last_clean":"2024-09-11T01:26:43.492918+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:43.492918+0000","last_undegraded":"2024-09-11T01:26:43.492918+0000","last_fullsized":"2024-09-11T01:26:43.492918+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20081,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.604490+0000","last_change":"2024-09-11T01:26:43.604424+0000","last_active":"2024-09-11T01:26:43.604490+0000","last_peered":"2024-09-11T01:26:43.604490+0000","last_clean":"2024-09-11T01:26:43.604490+0000","last_became_active":"2024-09-11T01:26:43.498430+0000","last_became_peered":"2024-09-11T01:26:43.498430+0000","last_unstale":"2024-09-11T01:26:43.604490+0000","last_undegraded":"2024-09-11T01:26:43.604490+0000","last_fullsized":"2024-09-11T01:26:43.604490+0000","mapping_epoch":64,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:13.180371+0000","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3988,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20428,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.495316+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:43.495316+0000","last_peered":"2024-09-11T01:26:43.495316+0000","last_clean":"2024-09-11T01:26:43.495316+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:43.495316+0000","last_undegraded":"2024-09-11T01:26:43.495316+0000","last_fullsized":"2024-09-11T01:26:43.495316+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492641+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:43.492641+0000","last_peered":"2024-09-11T01:26:43.492641+0000","last_clean":"2024-09-11T01:26:43.492641+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:43.492641+0000","last_undegraded":"2024-09-11T01:26:43.492641+0000","last_fullsized":"2024-09-11T01:26:43.492641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28026,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493375+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:43.493375+0000","last_peered":"2024-09-11T01:26:43.493375+0000","last_clean":"2024-09-11T01:26:43.493375+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:43.493375+0000","last_undegraded":"2024-09-11T01:26:43.493375+0000","last_fullsized":"2024-09-11T01:26:43.493375+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836907+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:26:43.836907+0000","last_peered":"2024-09-11T01:26:43.836907+0000","last_clean":"2024-09-11T01:26:43.836907+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:26:43.836907+0000","last_undegraded":"2024-09-11T01:26:43.836907+0000","last_fullsized":"2024-09-11T01:26:43.836907+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":25630,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:58.689888+0000","last_change":"2024-09-11T01:26:42.381382+0000","last_active":"2024-09-11T01:26:58.689888+0000","last_peered":"2024-09-11T01:26:58.689888+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:26:58.689888+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2794,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1192}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":146,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.497081+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:43.497081+0000","last_peered":"2024-09-11T01:26:43.497081+0000","last_clean":"2024-09-11T01:26:43.497081+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:43.497081+0000","last_undegraded":"2024-09-11T01:26:43.497081+0000","last_fullsized":"2024-09-11T01:26:43.497081+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28431,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836927+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:26:43.836927+0000","last_peered":"2024-09-11T01:26:43.836927+0000","last_clean":"2024-09-11T01:26:43.836927+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:26:43.836927+0000","last_undegraded":"2024-09-11T01:26:43.836927+0000","last_fullsized":"2024-09-11T01:26:43.836927+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492885+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:43.492885+0000","last_peered":"2024-09-11T01:26:43.492885+0000","last_clean":"2024-09-11T01:26:43.492885+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:43.492885+0000","last_undegraded":"2024-09-11T01:26:43.492885+0000","last_fullsized":"2024-09-11T01:26:43.492885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27822,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836526+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:26:43.836526+0000","last_peered":"2024-09-11T01:26:43.836526+0000","last_clean":"2024-09-11T01:26:43.836526+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:26:43.836526+0000","last_undegraded":"2024-09-11T01:26:43.836526+0000","last_fullsized":"2024-09-11T01:26:43.836526+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20431,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492883+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:43.492883+0000","last_peered":"2024-09-11T01:26:43.492883+0000","last_clean":"2024-09-11T01:26:43.492883+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:43.492883+0000","last_undegraded":"2024-09-11T01:26:43.492883+0000","last_fullsized":"2024-09-11T01:26:43.492883+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493122+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:43.493122+0000","last_peered":"2024-09-11T01:26:43.493122+0000","last_clean":"2024-09-11T01:26:43.493122+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:43.493122+0000","last_undegraded":"2024-09-11T01:26:43.493122+0000","last_fullsized":"2024-09-11T01:26:43.493122+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27960,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493218+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:43.493218+0000","last_peered":"2024-09-11T01:26:43.493218+0000","last_clean":"2024-09-11T01:26:43.493218+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:43.493218+0000","last_undegraded":"2024-09-11T01:26:43.493218+0000","last_fullsized":"2024-09-11T01:26:43.493218+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837106+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:26:43.837106+0000","last_peered":"2024-09-11T01:26:43.837106+0000","last_clean":"2024-09-11T01:26:43.837106+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:26:43.837106+0000","last_undegraded":"2024-09-11T01:26:43.837106+0000","last_fullsized":"2024-09-11T01:26:43.837106+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24639,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493408+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:43.493408+0000","last_peered":"2024-09-11T01:26:43.493408+0000","last_clean":"2024-09-11T01:26:43.493408+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:43.493408+0000","last_undegraded":"2024-09-11T01:26:43.493408+0000","last_fullsized":"2024-09-11T01:26:43.493408+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492970+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:43.492970+0000","last_peered":"2024-09-11T01:26:43.492970+0000","last_clean":"2024-09-11T01:26:43.492970+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:43.492970+0000","last_undegraded":"2024-09-11T01:26:43.492970+0000","last_fullsized":"2024-09-11T01:26:43.492970+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492818+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:43.492818+0000","last_peered":"2024-09-11T01:26:43.492818+0000","last_clean":"2024-09-11T01:26:43.492818+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:43.492818+0000","last_undegraded":"2024-09-11T01:26:43.492818+0000","last_fullsized":"2024-09-11T01:26:43.492818+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20111,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492860+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:43.492860+0000","last_peered":"2024-09-11T01:26:43.492860+0000","last_clean":"2024-09-11T01:26:43.492860+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:43.492860+0000","last_undegraded":"2024-09-11T01:26:43.492860+0000","last_fullsized":"2024-09-11T01:26:43.492860+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46447,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110186496,"data_stored":110176059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182586,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518696,"kb_used_data":63640,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853144,"statfs":{"total":96636764160,"available":96105619456,"internally_reserved":0,"allocated":65167360,"data_stored":64801446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986358,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":410076,"kb_used_data":45260,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93961764,"statfs":{"total":96636764160,"available":96216846336,"internally_reserved":0,"allocated":46346240,"data_stored":45984403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659302,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509736,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":509179,"kb_avail":93862104,"statfs":{"total":96636764160,"available":96114794496,"internally_reserved":0,"allocated":548864,"data_stored":182950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521399983},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45559808,"data_stored":45556329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:03.851 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:27:03.851 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 25630, 'reported_epoch': 65, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:26:58.689888+0000', 'last_change': '2024-09-11T01:26:42.381382+0000', 'last_active': '2024-09-11T01:26:58.689888+0000', 'last_peered': '2024-09-11T01:26:58.689888+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:26:58.689888+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1192, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 2794, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -1192}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:27:03.851 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:27:04.219 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:27:04.219 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 1192/6 objects degraded (19866.667%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":960,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":65,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":1},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1473032192,"bytes_avail":288437260288,"bytes_total":289910292480,"degraded_objects":1192,"degraded_total":6,"degraded_ratio":198.66666666666663,"recovering_objects_per_sec":163,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1967,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:27:04.237 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:27:04.237 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:04.545 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:27:04.546 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:27:04.563 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":538,"stamp":"2024-09-11T01:27:02.886732+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46448,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1438508,"kb_used_data":109436,"kb_used_omap":12,"kb_used_meta":1329011,"kb_avail":281677012,"statfs":{"total":289910292480,"available":288437260288,"internally_reserved":0,"allocated":112062464,"data_stored":110968799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1360907268},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1967,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1967,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003286"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28540,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.496965+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:43.496965+0000","last_peered":"2024-09-11T01:26:43.496965+0000","last_clean":"2024-09-11T01:26:43.496965+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:43.496965+0000","last_undegraded":"2024-09-11T01:26:43.496965+0000","last_fullsized":"2024-09-11T01:26:43.496965+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20498,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837072+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:26:43.837072+0000","last_peered":"2024-09-11T01:26:43.837072+0000","last_clean":"2024-09-11T01:26:43.837072+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:26:43.837072+0000","last_undegraded":"2024-09-11T01:26:43.837072+0000","last_fullsized":"2024-09-11T01:26:43.837072+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20321,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493170+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:43.493170+0000","last_peered":"2024-09-11T01:26:43.493170+0000","last_clean":"2024-09-11T01:26:43.493170+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:43.493170+0000","last_undegraded":"2024-09-11T01:26:43.493170+0000","last_fullsized":"2024-09-11T01:26:43.493170+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20248,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837042+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:26:43.837042+0000","last_peered":"2024-09-11T01:26:43.837042+0000","last_clean":"2024-09-11T01:26:43.837042+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:26:43.837042+0000","last_undegraded":"2024-09-11T01:26:43.837042+0000","last_fullsized":"2024-09-11T01:26:43.837042+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20181,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.495408+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:43.495408+0000","last_peered":"2024-09-11T01:26:43.495408+0000","last_clean":"2024-09-11T01:26:43.495408+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:43.495408+0000","last_undegraded":"2024-09-11T01:26:43.495408+0000","last_fullsized":"2024-09-11T01:26:43.495408+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20427,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492918+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:43.492918+0000","last_peered":"2024-09-11T01:26:43.492918+0000","last_clean":"2024-09-11T01:26:43.492918+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:43.492918+0000","last_undegraded":"2024-09-11T01:26:43.492918+0000","last_fullsized":"2024-09-11T01:26:43.492918+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20081,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.604490+0000","last_change":"2024-09-11T01:26:43.604424+0000","last_active":"2024-09-11T01:26:43.604490+0000","last_peered":"2024-09-11T01:26:43.604490+0000","last_clean":"2024-09-11T01:26:43.604490+0000","last_became_active":"2024-09-11T01:26:43.498430+0000","last_became_peered":"2024-09-11T01:26:43.498430+0000","last_unstale":"2024-09-11T01:26:43.604490+0000","last_undegraded":"2024-09-11T01:26:43.604490+0000","last_fullsized":"2024-09-11T01:26:43.604490+0000","mapping_epoch":64,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:13.180371+0000","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3988,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20428,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.495316+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:43.495316+0000","last_peered":"2024-09-11T01:26:43.495316+0000","last_clean":"2024-09-11T01:26:43.495316+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:43.495316+0000","last_undegraded":"2024-09-11T01:26:43.495316+0000","last_fullsized":"2024-09-11T01:26:43.495316+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492641+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:43.492641+0000","last_peered":"2024-09-11T01:26:43.492641+0000","last_clean":"2024-09-11T01:26:43.492641+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:43.492641+0000","last_undegraded":"2024-09-11T01:26:43.492641+0000","last_fullsized":"2024-09-11T01:26:43.492641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28026,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493375+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:43.493375+0000","last_peered":"2024-09-11T01:26:43.493375+0000","last_clean":"2024-09-11T01:26:43.493375+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:43.493375+0000","last_undegraded":"2024-09-11T01:26:43.493375+0000","last_fullsized":"2024-09-11T01:26:43.493375+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836907+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:26:43.836907+0000","last_peered":"2024-09-11T01:26:43.836907+0000","last_clean":"2024-09-11T01:26:43.836907+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:26:43.836907+0000","last_undegraded":"2024-09-11T01:26:43.836907+0000","last_fullsized":"2024-09-11T01:26:43.836907+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":25630,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:26:58.689888+0000","last_change":"2024-09-11T01:26:42.381382+0000","last_active":"2024-09-11T01:26:58.689888+0000","last_peered":"2024-09-11T01:26:58.689888+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:26:58.689888+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2794,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-1192}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":146,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.497081+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:43.497081+0000","last_peered":"2024-09-11T01:26:43.497081+0000","last_clean":"2024-09-11T01:26:43.497081+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:43.497081+0000","last_undegraded":"2024-09-11T01:26:43.497081+0000","last_fullsized":"2024-09-11T01:26:43.497081+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28431,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836927+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:26:43.836927+0000","last_peered":"2024-09-11T01:26:43.836927+0000","last_clean":"2024-09-11T01:26:43.836927+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:26:43.836927+0000","last_undegraded":"2024-09-11T01:26:43.836927+0000","last_fullsized":"2024-09-11T01:26:43.836927+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492885+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:43.492885+0000","last_peered":"2024-09-11T01:26:43.492885+0000","last_clean":"2024-09-11T01:26:43.492885+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:43.492885+0000","last_undegraded":"2024-09-11T01:26:43.492885+0000","last_fullsized":"2024-09-11T01:26:43.492885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27822,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836526+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:26:43.836526+0000","last_peered":"2024-09-11T01:26:43.836526+0000","last_clean":"2024-09-11T01:26:43.836526+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:26:43.836526+0000","last_undegraded":"2024-09-11T01:26:43.836526+0000","last_fullsized":"2024-09-11T01:26:43.836526+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20431,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492883+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:43.492883+0000","last_peered":"2024-09-11T01:26:43.492883+0000","last_clean":"2024-09-11T01:26:43.492883+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:43.492883+0000","last_undegraded":"2024-09-11T01:26:43.492883+0000","last_fullsized":"2024-09-11T01:26:43.492883+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493122+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:43.493122+0000","last_peered":"2024-09-11T01:26:43.493122+0000","last_clean":"2024-09-11T01:26:43.493122+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:43.493122+0000","last_undegraded":"2024-09-11T01:26:43.493122+0000","last_fullsized":"2024-09-11T01:26:43.493122+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27960,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493218+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:43.493218+0000","last_peered":"2024-09-11T01:26:43.493218+0000","last_clean":"2024-09-11T01:26:43.493218+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:43.493218+0000","last_undegraded":"2024-09-11T01:26:43.493218+0000","last_fullsized":"2024-09-11T01:26:43.493218+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837106+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:26:43.837106+0000","last_peered":"2024-09-11T01:26:43.837106+0000","last_clean":"2024-09-11T01:26:43.837106+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:26:43.837106+0000","last_undegraded":"2024-09-11T01:26:43.837106+0000","last_fullsized":"2024-09-11T01:26:43.837106+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24639,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493408+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:43.493408+0000","last_peered":"2024-09-11T01:26:43.493408+0000","last_clean":"2024-09-11T01:26:43.493408+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:43.493408+0000","last_undegraded":"2024-09-11T01:26:43.493408+0000","last_fullsized":"2024-09-11T01:26:43.493408+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492970+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:43.492970+0000","last_peered":"2024-09-11T01:26:43.492970+0000","last_clean":"2024-09-11T01:26:43.492970+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:43.492970+0000","last_undegraded":"2024-09-11T01:26:43.492970+0000","last_fullsized":"2024-09-11T01:26:43.492970+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492818+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:43.492818+0000","last_peered":"2024-09-11T01:26:43.492818+0000","last_clean":"2024-09-11T01:26:43.492818+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:43.492818+0000","last_undegraded":"2024-09-11T01:26:43.492818+0000","last_fullsized":"2024-09-11T01:26:43.492818+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20111,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492860+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:43.492860+0000","last_peered":"2024-09-11T01:26:43.492860+0000","last_clean":"2024-09-11T01:26:43.492860+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:43.492860+0000","last_undegraded":"2024-09-11T01:26:43.492860+0000","last_fullsized":"2024-09-11T01:26:43.492860+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":46447,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":110186496,"data_stored":110176059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182586,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518696,"kb_used_data":63640,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853144,"statfs":{"total":96636764160,"available":96105619456,"internally_reserved":0,"allocated":65167360,"data_stored":64801446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986358,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":410076,"kb_used_data":45260,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93961764,"statfs":{"total":96636764160,"available":96216846336,"internally_reserved":0,"allocated":46346240,"data_stored":45984403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659302,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":509736,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":509179,"kb_avail":93862104,"statfs":{"total":96636764160,"available":96114794496,"internally_reserved":0,"allocated":548864,"data_stored":182950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":521399983},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":45559808,"data_stored":45556329,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:07.564 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:07.871 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:27:07.872 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:27:07.889 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":542,"stamp":"2024-09-11T01:27:07.519050+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":440,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47200,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1423628,"kb_used_data":107036,"kb_used_omap":12,"kb_used_meta":1316531,"kb_avail":281691892,"statfs":{"total":289910292480,"available":288452497408,"internally_reserved":0,"allocated":109604864,"data_stored":108511199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1348127748},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-752,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":752,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.633683"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28540,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.496965+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:43.496965+0000","last_peered":"2024-09-11T01:26:43.496965+0000","last_clean":"2024-09-11T01:26:43.496965+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:43.496965+0000","last_undegraded":"2024-09-11T01:26:43.496965+0000","last_fullsized":"2024-09-11T01:26:43.496965+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20498,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837072+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:26:43.837072+0000","last_peered":"2024-09-11T01:26:43.837072+0000","last_clean":"2024-09-11T01:26:43.837072+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:26:43.837072+0000","last_undegraded":"2024-09-11T01:26:43.837072+0000","last_fullsized":"2024-09-11T01:26:43.837072+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20321,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493170+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:43.493170+0000","last_peered":"2024-09-11T01:26:43.493170+0000","last_clean":"2024-09-11T01:26:43.493170+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:43.493170+0000","last_undegraded":"2024-09-11T01:26:43.493170+0000","last_fullsized":"2024-09-11T01:26:43.493170+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20248,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837042+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:26:43.837042+0000","last_peered":"2024-09-11T01:26:43.837042+0000","last_clean":"2024-09-11T01:26:43.837042+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:26:43.837042+0000","last_undegraded":"2024-09-11T01:26:43.837042+0000","last_fullsized":"2024-09-11T01:26:43.837042+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20181,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.495408+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:43.495408+0000","last_peered":"2024-09-11T01:26:43.495408+0000","last_clean":"2024-09-11T01:26:43.495408+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:43.495408+0000","last_undegraded":"2024-09-11T01:26:43.495408+0000","last_fullsized":"2024-09-11T01:26:43.495408+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20427,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492918+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:43.492918+0000","last_peered":"2024-09-11T01:26:43.492918+0000","last_clean":"2024-09-11T01:26:43.492918+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:43.492918+0000","last_undegraded":"2024-09-11T01:26:43.492918+0000","last_fullsized":"2024-09-11T01:26:43.492918+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20081,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.604490+0000","last_change":"2024-09-11T01:26:43.604424+0000","last_active":"2024-09-11T01:26:43.604490+0000","last_peered":"2024-09-11T01:26:43.604490+0000","last_clean":"2024-09-11T01:26:43.604490+0000","last_became_active":"2024-09-11T01:26:43.498430+0000","last_became_peered":"2024-09-11T01:26:43.498430+0000","last_unstale":"2024-09-11T01:26:43.604490+0000","last_undegraded":"2024-09-11T01:26:43.604490+0000","last_fullsized":"2024-09-11T01:26:43.604490+0000","mapping_epoch":64,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:13.180371+0000","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3988,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20428,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.495316+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:43.495316+0000","last_peered":"2024-09-11T01:26:43.495316+0000","last_clean":"2024-09-11T01:26:43.495316+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:43.495316+0000","last_undegraded":"2024-09-11T01:26:43.495316+0000","last_fullsized":"2024-09-11T01:26:43.495316+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492641+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:43.492641+0000","last_peered":"2024-09-11T01:26:43.492641+0000","last_clean":"2024-09-11T01:26:43.492641+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:43.492641+0000","last_undegraded":"2024-09-11T01:26:43.492641+0000","last_fullsized":"2024-09-11T01:26:43.492641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28026,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493375+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:43.493375+0000","last_peered":"2024-09-11T01:26:43.493375+0000","last_clean":"2024-09-11T01:26:43.493375+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:43.493375+0000","last_undegraded":"2024-09-11T01:26:43.493375+0000","last_fullsized":"2024-09-11T01:26:43.493375+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836907+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:26:43.836907+0000","last_peered":"2024-09-11T01:26:43.836907+0000","last_clean":"2024-09-11T01:26:43.836907+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:26:43.836907+0000","last_undegraded":"2024-09-11T01:26:43.836907+0000","last_fullsized":"2024-09-11T01:26:43.836907+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":27134,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:03.687633+0000","last_change":"2024-09-11T01:26:42.381382+0000","last_active":"2024-09-11T01:27:03.687633+0000","last_peered":"2024-09-11T01:27:03.687633+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:27:03.687633+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":440,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-440}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":146,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.497081+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:43.497081+0000","last_peered":"2024-09-11T01:26:43.497081+0000","last_clean":"2024-09-11T01:26:43.497081+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:43.497081+0000","last_undegraded":"2024-09-11T01:26:43.497081+0000","last_fullsized":"2024-09-11T01:26:43.497081+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28431,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836927+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:26:43.836927+0000","last_peered":"2024-09-11T01:26:43.836927+0000","last_clean":"2024-09-11T01:26:43.836927+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:26:43.836927+0000","last_undegraded":"2024-09-11T01:26:43.836927+0000","last_fullsized":"2024-09-11T01:26:43.836927+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492885+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:43.492885+0000","last_peered":"2024-09-11T01:26:43.492885+0000","last_clean":"2024-09-11T01:26:43.492885+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:43.492885+0000","last_undegraded":"2024-09-11T01:26:43.492885+0000","last_fullsized":"2024-09-11T01:26:43.492885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27822,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836526+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:26:43.836526+0000","last_peered":"2024-09-11T01:26:43.836526+0000","last_clean":"2024-09-11T01:26:43.836526+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:26:43.836526+0000","last_undegraded":"2024-09-11T01:26:43.836526+0000","last_fullsized":"2024-09-11T01:26:43.836526+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20431,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492883+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:43.492883+0000","last_peered":"2024-09-11T01:26:43.492883+0000","last_clean":"2024-09-11T01:26:43.492883+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:43.492883+0000","last_undegraded":"2024-09-11T01:26:43.492883+0000","last_fullsized":"2024-09-11T01:26:43.492883+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493122+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:43.493122+0000","last_peered":"2024-09-11T01:26:43.493122+0000","last_clean":"2024-09-11T01:26:43.493122+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:43.493122+0000","last_undegraded":"2024-09-11T01:26:43.493122+0000","last_fullsized":"2024-09-11T01:26:43.493122+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27960,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493218+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:43.493218+0000","last_peered":"2024-09-11T01:26:43.493218+0000","last_clean":"2024-09-11T01:26:43.493218+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:43.493218+0000","last_undegraded":"2024-09-11T01:26:43.493218+0000","last_fullsized":"2024-09-11T01:26:43.493218+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837106+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:26:43.837106+0000","last_peered":"2024-09-11T01:26:43.837106+0000","last_clean":"2024-09-11T01:26:43.837106+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:26:43.837106+0000","last_undegraded":"2024-09-11T01:26:43.837106+0000","last_fullsized":"2024-09-11T01:26:43.837106+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24639,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493408+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:43.493408+0000","last_peered":"2024-09-11T01:26:43.493408+0000","last_clean":"2024-09-11T01:26:43.493408+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:43.493408+0000","last_undegraded":"2024-09-11T01:26:43.493408+0000","last_fullsized":"2024-09-11T01:26:43.493408+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492970+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:43.492970+0000","last_peered":"2024-09-11T01:26:43.492970+0000","last_clean":"2024-09-11T01:26:43.492970+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:43.492970+0000","last_undegraded":"2024-09-11T01:26:43.492970+0000","last_fullsized":"2024-09-11T01:26:43.492970+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492818+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:43.492818+0000","last_peered":"2024-09-11T01:26:43.492818+0000","last_clean":"2024-09-11T01:26:43.492818+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:43.492818+0000","last_undegraded":"2024-09-11T01:26:43.492818+0000","last_fullsized":"2024-09-11T01:26:43.492818+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20111,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492860+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:43.492860+0000","last_peered":"2024-09-11T01:26:43.492860+0000","last_clean":"2024-09-11T01:26:43.492860+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:43.492860+0000","last_undegraded":"2024-09-11T01:26:43.492860+0000","last_fullsized":"2024-09-11T01:26:43.492860+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":440,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47199,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":107728896,"data_stored":107718459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182587,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518696,"kb_used_data":63640,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853144,"statfs":{"total":96636764160,"available":96105619456,"internally_reserved":0,"allocated":65167360,"data_stored":64801446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986359,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":407676,"kb_used_data":42860,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93964164,"statfs":{"total":96636764160,"available":96219303936,"internally_reserved":0,"allocated":43888640,"data_stored":43526803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659303,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497256,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":496699,"kb_avail":93874584,"statfs":{"total":96636764160,"available":96127574016,"internally_reserved":0,"allocated":548864,"data_stored":182950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508620463},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":43102208,"data_stored":43098729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:07.891 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:08.190 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:27:08.191 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:27:08.209 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":542,"stamp":"2024-09-11T01:27:07.519050+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":440,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47200,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1423628,"kb_used_data":107036,"kb_used_omap":12,"kb_used_meta":1316531,"kb_avail":281691892,"statfs":{"total":289910292480,"available":288452497408,"internally_reserved":0,"allocated":109604864,"data_stored":108511199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1348127748},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-752,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":752,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.633683"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28540,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.496965+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:43.496965+0000","last_peered":"2024-09-11T01:26:43.496965+0000","last_clean":"2024-09-11T01:26:43.496965+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:43.496965+0000","last_undegraded":"2024-09-11T01:26:43.496965+0000","last_fullsized":"2024-09-11T01:26:43.496965+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20498,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837072+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:26:43.837072+0000","last_peered":"2024-09-11T01:26:43.837072+0000","last_clean":"2024-09-11T01:26:43.837072+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:26:43.837072+0000","last_undegraded":"2024-09-11T01:26:43.837072+0000","last_fullsized":"2024-09-11T01:26:43.837072+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20321,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493170+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:43.493170+0000","last_peered":"2024-09-11T01:26:43.493170+0000","last_clean":"2024-09-11T01:26:43.493170+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:43.493170+0000","last_undegraded":"2024-09-11T01:26:43.493170+0000","last_fullsized":"2024-09-11T01:26:43.493170+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20248,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837042+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:26:43.837042+0000","last_peered":"2024-09-11T01:26:43.837042+0000","last_clean":"2024-09-11T01:26:43.837042+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:26:43.837042+0000","last_undegraded":"2024-09-11T01:26:43.837042+0000","last_fullsized":"2024-09-11T01:26:43.837042+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20181,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.495408+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:43.495408+0000","last_peered":"2024-09-11T01:26:43.495408+0000","last_clean":"2024-09-11T01:26:43.495408+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:43.495408+0000","last_undegraded":"2024-09-11T01:26:43.495408+0000","last_fullsized":"2024-09-11T01:26:43.495408+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20427,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492918+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:43.492918+0000","last_peered":"2024-09-11T01:26:43.492918+0000","last_clean":"2024-09-11T01:26:43.492918+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:43.492918+0000","last_undegraded":"2024-09-11T01:26:43.492918+0000","last_fullsized":"2024-09-11T01:26:43.492918+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20081,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.604490+0000","last_change":"2024-09-11T01:26:43.604424+0000","last_active":"2024-09-11T01:26:43.604490+0000","last_peered":"2024-09-11T01:26:43.604490+0000","last_clean":"2024-09-11T01:26:43.604490+0000","last_became_active":"2024-09-11T01:26:43.498430+0000","last_became_peered":"2024-09-11T01:26:43.498430+0000","last_unstale":"2024-09-11T01:26:43.604490+0000","last_undegraded":"2024-09-11T01:26:43.604490+0000","last_fullsized":"2024-09-11T01:26:43.604490+0000","mapping_epoch":64,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:13.180371+0000","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3988,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20428,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.495316+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:43.495316+0000","last_peered":"2024-09-11T01:26:43.495316+0000","last_clean":"2024-09-11T01:26:43.495316+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:43.495316+0000","last_undegraded":"2024-09-11T01:26:43.495316+0000","last_fullsized":"2024-09-11T01:26:43.495316+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492641+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:43.492641+0000","last_peered":"2024-09-11T01:26:43.492641+0000","last_clean":"2024-09-11T01:26:43.492641+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:43.492641+0000","last_undegraded":"2024-09-11T01:26:43.492641+0000","last_fullsized":"2024-09-11T01:26:43.492641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28026,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493375+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:43.493375+0000","last_peered":"2024-09-11T01:26:43.493375+0000","last_clean":"2024-09-11T01:26:43.493375+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:43.493375+0000","last_undegraded":"2024-09-11T01:26:43.493375+0000","last_fullsized":"2024-09-11T01:26:43.493375+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836907+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:26:43.836907+0000","last_peered":"2024-09-11T01:26:43.836907+0000","last_clean":"2024-09-11T01:26:43.836907+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:26:43.836907+0000","last_undegraded":"2024-09-11T01:26:43.836907+0000","last_fullsized":"2024-09-11T01:26:43.836907+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":27134,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:03.687633+0000","last_change":"2024-09-11T01:26:42.381382+0000","last_active":"2024-09-11T01:27:03.687633+0000","last_peered":"2024-09-11T01:27:03.687633+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:27:03.687633+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":440,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-440}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":146,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.497081+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:43.497081+0000","last_peered":"2024-09-11T01:26:43.497081+0000","last_clean":"2024-09-11T01:26:43.497081+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:43.497081+0000","last_undegraded":"2024-09-11T01:26:43.497081+0000","last_fullsized":"2024-09-11T01:26:43.497081+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28431,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836927+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:26:43.836927+0000","last_peered":"2024-09-11T01:26:43.836927+0000","last_clean":"2024-09-11T01:26:43.836927+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:26:43.836927+0000","last_undegraded":"2024-09-11T01:26:43.836927+0000","last_fullsized":"2024-09-11T01:26:43.836927+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492885+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:43.492885+0000","last_peered":"2024-09-11T01:26:43.492885+0000","last_clean":"2024-09-11T01:26:43.492885+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:43.492885+0000","last_undegraded":"2024-09-11T01:26:43.492885+0000","last_fullsized":"2024-09-11T01:26:43.492885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27822,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836526+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:26:43.836526+0000","last_peered":"2024-09-11T01:26:43.836526+0000","last_clean":"2024-09-11T01:26:43.836526+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:26:43.836526+0000","last_undegraded":"2024-09-11T01:26:43.836526+0000","last_fullsized":"2024-09-11T01:26:43.836526+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20431,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492883+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:43.492883+0000","last_peered":"2024-09-11T01:26:43.492883+0000","last_clean":"2024-09-11T01:26:43.492883+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:43.492883+0000","last_undegraded":"2024-09-11T01:26:43.492883+0000","last_fullsized":"2024-09-11T01:26:43.492883+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493122+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:43.493122+0000","last_peered":"2024-09-11T01:26:43.493122+0000","last_clean":"2024-09-11T01:26:43.493122+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:43.493122+0000","last_undegraded":"2024-09-11T01:26:43.493122+0000","last_fullsized":"2024-09-11T01:26:43.493122+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27960,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493218+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:43.493218+0000","last_peered":"2024-09-11T01:26:43.493218+0000","last_clean":"2024-09-11T01:26:43.493218+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:43.493218+0000","last_undegraded":"2024-09-11T01:26:43.493218+0000","last_fullsized":"2024-09-11T01:26:43.493218+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837106+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:26:43.837106+0000","last_peered":"2024-09-11T01:26:43.837106+0000","last_clean":"2024-09-11T01:26:43.837106+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:26:43.837106+0000","last_undegraded":"2024-09-11T01:26:43.837106+0000","last_fullsized":"2024-09-11T01:26:43.837106+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24639,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493408+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:43.493408+0000","last_peered":"2024-09-11T01:26:43.493408+0000","last_clean":"2024-09-11T01:26:43.493408+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:43.493408+0000","last_undegraded":"2024-09-11T01:26:43.493408+0000","last_fullsized":"2024-09-11T01:26:43.493408+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492970+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:43.492970+0000","last_peered":"2024-09-11T01:26:43.492970+0000","last_clean":"2024-09-11T01:26:43.492970+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:43.492970+0000","last_undegraded":"2024-09-11T01:26:43.492970+0000","last_fullsized":"2024-09-11T01:26:43.492970+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492818+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:43.492818+0000","last_peered":"2024-09-11T01:26:43.492818+0000","last_clean":"2024-09-11T01:26:43.492818+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:43.492818+0000","last_undegraded":"2024-09-11T01:26:43.492818+0000","last_fullsized":"2024-09-11T01:26:43.492818+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20111,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492860+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:43.492860+0000","last_peered":"2024-09-11T01:26:43.492860+0000","last_clean":"2024-09-11T01:26:43.492860+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:43.492860+0000","last_undegraded":"2024-09-11T01:26:43.492860+0000","last_fullsized":"2024-09-11T01:26:43.492860+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":440,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47199,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":107728896,"data_stored":107718459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182587,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518696,"kb_used_data":63640,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853144,"statfs":{"total":96636764160,"available":96105619456,"internally_reserved":0,"allocated":65167360,"data_stored":64801446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986359,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":407676,"kb_used_data":42860,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93964164,"statfs":{"total":96636764160,"available":96219303936,"internally_reserved":0,"allocated":43888640,"data_stored":43526803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659303,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497256,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":496699,"kb_avail":93874584,"statfs":{"total":96636764160,"available":96127574016,"internally_reserved":0,"allocated":548864,"data_stored":182950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508620463},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":43102208,"data_stored":43098729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:08.210 INFO:tasks.rep_lost_unfound_delete.ceph_manager:PG 2.2 is not active+clean 2024-09-11T01:27:08.210 INFO:tasks.rep_lost_unfound_delete.ceph_manager:{'pgid': '2.2', 'version': "49'19868", 'reported_seq': 27134, 'reported_epoch': 65, 'state': 'active+recovering+undersized+degraded+remapped', 'last_fresh': '2024-09-11T01:27:03.687633+0000', 'last_change': '2024-09-11T01:26:42.381382+0000', 'last_active': '2024-09-11T01:27:03.687633+0000', 'last_peered': '2024-09-11T01:27:03.687633+0000', 'last_clean': '2024-09-11T01:13:07.608035+0000', 'last_became_active': '2024-09-11T01:13:10.266110+0000', 'last_became_peered': '2024-09-11T01:13:10.266110+0000', 'last_unstale': '2024-09-11T01:27:03.687633+0000', 'last_undegraded': '2024-09-11T01:13:09.834189+0000', 'last_fullsized': '2024-09-11T01:13:09.648386+0000', 'mapping_epoch': 42, 'log_start': "45'9800", 'ondisk_log_start': "45'9800", 'created': 16, 'last_epoch_clean': 39, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "36'3497", 'last_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-11T01:11:30.697577+0000', 'last_clean_scrub_stamp': '2024-09-11T01:12:59.058452+0000', 'objects_scrubbed': 3405, 'log_size': 10068, 'log_dups_size': 0, 'ondisk_log_size': 10068, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 2, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 1919, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 440, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 16455, 'num_write_kb': 26100, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 3546, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0], 'avail_no_missing': ['0', '2'], 'object_location_counts': [{'shards': '0', 'objects': -440}], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-11T01:27:08.211 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-11T01:27:08.567 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:27:08.568 INFO:teuthology.orchestra.run.smithi122.stdout:{"fsid":"6d328773-3914-4412-91b7-393379984de3","health":{"status":"HEALTH_WARN","checks":{"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 440/6 objects degraded (7333.333%), 1 pg degraded, 1 pg undersized","count":2},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":965,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":67,"num_osds":3,"num_up_osds":3,"osd_up_since":1726017187,"num_in_osds":3,"osd_in_since":1726017179,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":23},{"state_name":"active+recovering+undersized+degraded+remapped","count":1}],"num_pgs":24,"num_pools":2,"num_objects":3,"data_bytes":636,"bytes_used":1457795072,"bytes_avail":288452497408,"bytes_total":289910292480,"degraded_objects":440,"degraded_total":6,"degraded_ratio":73.333333333333329,"recovering_objects_per_sec":182,"recovering_bytes_per_sec":0,"recovering_keys_per_sec":0,"num_objects_recovered":1828,"num_bytes_recovered":0,"num_keys_recovered":0},"fsmap":{"epoch":1,"btime":"2024-09-11T01:11:03:507954+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":6,"modified":"2024-09-11T01:14:08.574412+0000","services":{}},"progress_events":{}} 2024-09-11T01:27:08.587 INFO:tasks.rep_lost_unfound_delete.ceph_manager:making progress, resetting timeout 2024-09-11T01:27:08.587 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:08.871 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:27:08.872 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:27:08.888 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":542,"stamp":"2024-09-11T01:27:07.519050+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":440,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47200,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":47,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1423628,"kb_used_data":107036,"kb_used_omap":12,"kb_used_meta":1316531,"kb_avail":281691892,"statfs":{"total":289910292480,"available":288452497408,"internally_reserved":0,"allocated":109604864,"data_stored":108511199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1348127748},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-752,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":752,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.633683"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28540,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.496965+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:26:43.496965+0000","last_peered":"2024-09-11T01:26:43.496965+0000","last_clean":"2024-09-11T01:26:43.496965+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:26:43.496965+0000","last_undegraded":"2024-09-11T01:26:43.496965+0000","last_fullsized":"2024-09-11T01:26:43.496965+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20498,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837072+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:26:43.837072+0000","last_peered":"2024-09-11T01:26:43.837072+0000","last_clean":"2024-09-11T01:26:43.837072+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:26:43.837072+0000","last_undegraded":"2024-09-11T01:26:43.837072+0000","last_fullsized":"2024-09-11T01:26:43.837072+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20321,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493170+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:26:43.493170+0000","last_peered":"2024-09-11T01:26:43.493170+0000","last_clean":"2024-09-11T01:26:43.493170+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:26:43.493170+0000","last_undegraded":"2024-09-11T01:26:43.493170+0000","last_fullsized":"2024-09-11T01:26:43.493170+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20248,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837042+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:26:43.837042+0000","last_peered":"2024-09-11T01:26:43.837042+0000","last_clean":"2024-09-11T01:26:43.837042+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:26:43.837042+0000","last_undegraded":"2024-09-11T01:26:43.837042+0000","last_fullsized":"2024-09-11T01:26:43.837042+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20181,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.495408+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:26:43.495408+0000","last_peered":"2024-09-11T01:26:43.495408+0000","last_clean":"2024-09-11T01:26:43.495408+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:26:43.495408+0000","last_undegraded":"2024-09-11T01:26:43.495408+0000","last_fullsized":"2024-09-11T01:26:43.495408+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20427,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492918+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:26:43.492918+0000","last_peered":"2024-09-11T01:26:43.492918+0000","last_clean":"2024-09-11T01:26:43.492918+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:26:43.492918+0000","last_undegraded":"2024-09-11T01:26:43.492918+0000","last_fullsized":"2024-09-11T01:26:43.492918+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20081,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.604490+0000","last_change":"2024-09-11T01:26:43.604424+0000","last_active":"2024-09-11T01:26:43.604490+0000","last_peered":"2024-09-11T01:26:43.604490+0000","last_clean":"2024-09-11T01:26:43.604490+0000","last_became_active":"2024-09-11T01:26:43.498430+0000","last_became_peered":"2024-09-11T01:26:43.498430+0000","last_unstale":"2024-09-11T01:26:43.604490+0000","last_undegraded":"2024-09-11T01:26:43.604490+0000","last_fullsized":"2024-09-11T01:26:43.604490+0000","mapping_epoch":64,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:13.180371+0000","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3988,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20428,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.495316+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:26:43.495316+0000","last_peered":"2024-09-11T01:26:43.495316+0000","last_clean":"2024-09-11T01:26:43.495316+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:26:43.495316+0000","last_undegraded":"2024-09-11T01:26:43.495316+0000","last_fullsized":"2024-09-11T01:26:43.495316+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492641+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:26:43.492641+0000","last_peered":"2024-09-11T01:26:43.492641+0000","last_clean":"2024-09-11T01:26:43.492641+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:26:43.492641+0000","last_undegraded":"2024-09-11T01:26:43.492641+0000","last_fullsized":"2024-09-11T01:26:43.492641+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28026,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493375+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:26:43.493375+0000","last_peered":"2024-09-11T01:26:43.493375+0000","last_clean":"2024-09-11T01:26:43.493375+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:26:43.493375+0000","last_undegraded":"2024-09-11T01:26:43.493375+0000","last_fullsized":"2024-09-11T01:26:43.493375+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836907+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:26:43.836907+0000","last_peered":"2024-09-11T01:26:43.836907+0000","last_clean":"2024-09-11T01:26:43.836907+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:26:43.836907+0000","last_undegraded":"2024-09-11T01:26:43.836907+0000","last_fullsized":"2024-09-11T01:26:43.836907+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":27134,"reported_epoch":65,"state":"active+recovering+undersized+degraded+remapped","last_fresh":"2024-09-11T01:27:03.687633+0000","last_change":"2024-09-11T01:26:42.381382+0000","last_active":"2024-09-11T01:27:03.687633+0000","last_peered":"2024-09-11T01:27:03.687633+0000","last_clean":"2024-09-11T01:13:07.608035+0000","last_became_active":"2024-09-11T01:13:10.266110+0000","last_became_peered":"2024-09-11T01:13:10.266110+0000","last_unstale":"2024-09-11T01:27:03.687633+0000","last_undegraded":"2024-09-11T01:13:09.834189+0000","last_fullsized":"2024-09-11T01:13:09.648386+0000","mapping_epoch":42,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"no scrub is scheduled","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":440,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3546,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0],"avail_no_missing":["0","2"],"object_location_counts":[{"shards":"0","objects":-440}],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":146,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.497081+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:26:43.497081+0000","last_peered":"2024-09-11T01:26:43.497081+0000","last_clean":"2024-09-11T01:26:43.497081+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:26:43.497081+0000","last_undegraded":"2024-09-11T01:26:43.497081+0000","last_fullsized":"2024-09-11T01:26:43.497081+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28431,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836927+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:26:43.836927+0000","last_peered":"2024-09-11T01:26:43.836927+0000","last_clean":"2024-09-11T01:26:43.836927+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:26:43.836927+0000","last_undegraded":"2024-09-11T01:26:43.836927+0000","last_fullsized":"2024-09-11T01:26:43.836927+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":124,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492885+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:26:43.492885+0000","last_peered":"2024-09-11T01:26:43.492885+0000","last_clean":"2024-09-11T01:26:43.492885+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:26:43.492885+0000","last_undegraded":"2024-09-11T01:26:43.492885+0000","last_fullsized":"2024-09-11T01:26:43.492885+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27822,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.836526+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:26:43.836526+0000","last_peered":"2024-09-11T01:26:43.836526+0000","last_clean":"2024-09-11T01:26:43.836526+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:26:43.836526+0000","last_undegraded":"2024-09-11T01:26:43.836526+0000","last_fullsized":"2024-09-11T01:26:43.836526+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20431,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492883+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:26:43.492883+0000","last_peered":"2024-09-11T01:26:43.492883+0000","last_clean":"2024-09-11T01:26:43.492883+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:26:43.492883+0000","last_undegraded":"2024-09-11T01:26:43.492883+0000","last_fullsized":"2024-09-11T01:26:43.492883+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493122+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:26:43.493122+0000","last_peered":"2024-09-11T01:26:43.493122+0000","last_clean":"2024-09-11T01:26:43.493122+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:26:43.493122+0000","last_undegraded":"2024-09-11T01:26:43.493122+0000","last_fullsized":"2024-09-11T01:26:43.493122+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27960,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493218+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:26:43.493218+0000","last_peered":"2024-09-11T01:26:43.493218+0000","last_clean":"2024-09-11T01:26:43.493218+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:26:43.493218+0000","last_undegraded":"2024-09-11T01:26:43.493218+0000","last_fullsized":"2024-09-11T01:26:43.493218+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":115,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.837106+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:26:43.837106+0000","last_peered":"2024-09-11T01:26:43.837106+0000","last_clean":"2024-09-11T01:26:43.837106+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:26:43.837106+0000","last_undegraded":"2024-09-11T01:26:43.837106+0000","last_fullsized":"2024-09-11T01:26:43.837106+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24639,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.493408+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:26:43.493408+0000","last_peered":"2024-09-11T01:26:43.493408+0000","last_clean":"2024-09-11T01:26:43.493408+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:26:43.493408+0000","last_undegraded":"2024-09-11T01:26:43.493408+0000","last_fullsized":"2024-09-11T01:26:43.493408+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":114,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492970+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:26:43.492970+0000","last_peered":"2024-09-11T01:26:43.492970+0000","last_clean":"2024-09-11T01:26:43.492970+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:26:43.492970+0000","last_undegraded":"2024-09-11T01:26:43.492970+0000","last_fullsized":"2024-09-11T01:26:43.492970+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492818+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:26:43.492818+0000","last_peered":"2024-09-11T01:26:43.492818+0000","last_clean":"2024-09-11T01:26:43.492818+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:26:43.492818+0000","last_undegraded":"2024-09-11T01:26:43.492818+0000","last_fullsized":"2024-09-11T01:26:43.492818+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20111,"reported_epoch":65,"state":"active+clean","last_fresh":"2024-09-11T01:26:43.492860+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:26:43.492860+0000","last_peered":"2024-09-11T01:26:43.492860+0000","last_clean":"2024-09-11T01:26:43.492860+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:26:43.492860+0000","last_undegraded":"2024-09-11T01:26:43.492860+0000","last_fullsized":"2024-09-11T01:26:43.492860+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":440,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47199,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":107728896,"data_stored":107718459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":31,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182587,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":518696,"kb_used_data":63640,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93853144,"statfs":{"total":96636764160,"available":96105619456,"internally_reserved":0,"allocated":65167360,"data_stored":64801446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986359,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":407676,"kb_used_data":42860,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93964164,"statfs":{"total":96636764160,"available":96219303936,"internally_reserved":0,"allocated":43888640,"data_stored":43526803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659303,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":497256,"kb_used_data":536,"kb_used_omap":4,"kb_used_meta":496699,"kb_avail":93874584,"statfs":{"total":96636764160,"available":96127574016,"internally_reserved":0,"allocated":548864,"data_stored":182950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":508620463},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":64622592,"data_stored":64619113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":43102208,"data_stored":43098729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:11.890 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:12.200 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:27:12.201 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:27:12.218 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":544,"stamp":"2024-09-11T01:27:10.888793+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47640,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1425336,"kb_used_data":104584,"kb_used_omap":12,"kb_used_meta":1320691,"kb_avail":281690184,"statfs":{"total":289910292480,"available":288450748416,"internally_reserved":0,"allocated":107094016,"data_stored":105967516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1352387588},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1192,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002060"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28544,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.524154+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:27:07.524154+0000","last_peered":"2024-09-11T01:27:07.524154+0000","last_clean":"2024-09-11T01:27:07.524154+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:27:07.524154+0000","last_undegraded":"2024-09-11T01:27:07.524154+0000","last_fullsized":"2024-09-11T01:27:07.524154+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20502,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.523059+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:27:07.523059+0000","last_peered":"2024-09-11T01:27:07.523059+0000","last_clean":"2024-09-11T01:27:07.523059+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:27:07.523059+0000","last_undegraded":"2024-09-11T01:27:07.523059+0000","last_fullsized":"2024-09-11T01:27:07.523059+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20325,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528271+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:27:07.528271+0000","last_peered":"2024-09-11T01:27:07.528271+0000","last_clean":"2024-09-11T01:27:07.528271+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:27:07.528271+0000","last_undegraded":"2024-09-11T01:27:07.528271+0000","last_fullsized":"2024-09-11T01:27:07.528271+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20252,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.522347+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:27:07.522347+0000","last_peered":"2024-09-11T01:27:07.522347+0000","last_clean":"2024-09-11T01:27:07.522347+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:27:07.522347+0000","last_undegraded":"2024-09-11T01:27:07.522347+0000","last_fullsized":"2024-09-11T01:27:07.522347+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20185,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.522423+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:27:07.522423+0000","last_peered":"2024-09-11T01:27:07.522423+0000","last_clean":"2024-09-11T01:27:07.522423+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:27:07.522423+0000","last_undegraded":"2024-09-11T01:27:07.522423+0000","last_fullsized":"2024-09-11T01:27:07.522423+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20431,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.527154+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:27:07.527154+0000","last_peered":"2024-09-11T01:27:07.527154+0000","last_clean":"2024-09-11T01:27:07.527154+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:27:07.527154+0000","last_undegraded":"2024-09-11T01:27:07.527154+0000","last_fullsized":"2024-09-11T01:27:07.527154+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20085,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.526901+0000","last_change":"2024-09-11T01:26:43.604424+0000","last_active":"2024-09-11T01:27:07.526901+0000","last_peered":"2024-09-11T01:27:07.526901+0000","last_clean":"2024-09-11T01:27:07.526901+0000","last_became_active":"2024-09-11T01:26:43.498430+0000","last_became_peered":"2024-09-11T01:26:43.498430+0000","last_unstale":"2024-09-11T01:27:07.526901+0000","last_undegraded":"2024-09-11T01:27:07.526901+0000","last_fullsized":"2024-09-11T01:27:07.526901+0000","mapping_epoch":64,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:13.180371+0000","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3988,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20433,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.521558+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:27:07.521558+0000","last_peered":"2024-09-11T01:27:07.521558+0000","last_clean":"2024-09-11T01:27:07.521558+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:27:07.521558+0000","last_undegraded":"2024-09-11T01:27:07.521558+0000","last_fullsized":"2024-09-11T01:27:07.521558+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.526781+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:27:07.526781+0000","last_peered":"2024-09-11T01:27:07.526781+0000","last_clean":"2024-09-11T01:27:07.526781+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:27:07.526781+0000","last_undegraded":"2024-09-11T01:27:07.526781+0000","last_fullsized":"2024-09-11T01:27:07.526781+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28030,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.527075+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:27:07.527075+0000","last_peered":"2024-09-11T01:27:07.527075+0000","last_clean":"2024-09-11T01:27:07.527075+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:27:07.527075+0000","last_undegraded":"2024-09-11T01:27:07.527075+0000","last_fullsized":"2024-09-11T01:27:07.527075+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.520946+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:27:07.520946+0000","last_peered":"2024-09-11T01:27:07.520946+0000","last_clean":"2024-09-11T01:27:07.520946+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:27:07.520946+0000","last_undegraded":"2024-09-11T01:27:07.520946+0000","last_fullsized":"2024-09-11T01:27:07.520946+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":28025,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.625143+0000","last_change":"2024-09-11T01:27:07.625011+0000","last_active":"2024-09-11T01:27:07.625143+0000","last_peered":"2024-09-11T01:27:07.625143+0000","last_clean":"2024-09-11T01:27:07.625143+0000","last_became_active":"2024-09-11T01:27:07.529673+0000","last_became_peered":"2024-09-11T01:27:07.529673+0000","last_unstale":"2024-09-11T01:27:07.625143+0000","last_undegraded":"2024-09-11T01:27:07.625143+0000","last_fullsized":"2024-09-11T01:27:07.625143+0000","mapping_epoch":66,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:25:06.093676+0000","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3986,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":150,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.521775+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:27:07.521775+0000","last_peered":"2024-09-11T01:27:07.521775+0000","last_clean":"2024-09-11T01:27:07.521775+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:27:07.521775+0000","last_undegraded":"2024-09-11T01:27:07.521775+0000","last_fullsized":"2024-09-11T01:27:07.521775+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28435,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.521118+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:27:07.521118+0000","last_peered":"2024-09-11T01:27:07.521118+0000","last_clean":"2024-09-11T01:27:07.521118+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:27:07.521118+0000","last_undegraded":"2024-09-11T01:27:07.521118+0000","last_fullsized":"2024-09-11T01:27:07.521118+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":128,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.527066+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:27:07.527066+0000","last_peered":"2024-09-11T01:27:07.527066+0000","last_clean":"2024-09-11T01:27:07.527066+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:27:07.527066+0000","last_undegraded":"2024-09-11T01:27:07.527066+0000","last_fullsized":"2024-09-11T01:27:07.527066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27826,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.520647+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:27:07.520647+0000","last_peered":"2024-09-11T01:27:07.520647+0000","last_clean":"2024-09-11T01:27:07.520647+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:27:07.520647+0000","last_undegraded":"2024-09-11T01:27:07.520647+0000","last_fullsized":"2024-09-11T01:27:07.520647+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20435,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528488+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:27:07.528488+0000","last_peered":"2024-09-11T01:27:07.528488+0000","last_clean":"2024-09-11T01:27:07.528488+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:27:07.528488+0000","last_undegraded":"2024-09-11T01:27:07.528488+0000","last_fullsized":"2024-09-11T01:27:07.528488+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528226+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:27:07.528226+0000","last_peered":"2024-09-11T01:27:07.528226+0000","last_clean":"2024-09-11T01:27:07.528226+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:27:07.528226+0000","last_undegraded":"2024-09-11T01:27:07.528226+0000","last_fullsized":"2024-09-11T01:27:07.528226+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27964,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528314+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:27:07.528314+0000","last_peered":"2024-09-11T01:27:07.528314+0000","last_clean":"2024-09-11T01:27:07.528314+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:27:07.528314+0000","last_undegraded":"2024-09-11T01:27:07.528314+0000","last_fullsized":"2024-09-11T01:27:07.528314+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.522796+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:27:07.522796+0000","last_peered":"2024-09-11T01:27:07.522796+0000","last_clean":"2024-09-11T01:27:07.522796+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:27:07.522796+0000","last_undegraded":"2024-09-11T01:27:07.522796+0000","last_fullsized":"2024-09-11T01:27:07.522796+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24643,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528300+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:27:07.528300+0000","last_peered":"2024-09-11T01:27:07.528300+0000","last_clean":"2024-09-11T01:27:07.528300+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:27:07.528300+0000","last_undegraded":"2024-09-11T01:27:07.528300+0000","last_fullsized":"2024-09-11T01:27:07.528300+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528259+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:27:07.528259+0000","last_peered":"2024-09-11T01:27:07.528259+0000","last_clean":"2024-09-11T01:27:07.528259+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:27:07.528259+0000","last_undegraded":"2024-09-11T01:27:07.528259+0000","last_fullsized":"2024-09-11T01:27:07.528259+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":123,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528414+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:27:07.528414+0000","last_peered":"2024-09-11T01:27:07.528414+0000","last_clean":"2024-09-11T01:27:07.528414+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:27:07.528414+0000","last_undegraded":"2024-09-11T01:27:07.528414+0000","last_fullsized":"2024-09-11T01:27:07.528414+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20115,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.527118+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:27:07.527118+0000","last_peered":"2024-09-11T01:27:07.527118+0000","last_clean":"2024-09-11T01:27:07.527118+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:27:07.527118+0000","last_undegraded":"2024-09-11T01:27:07.527118+0000","last_fullsized":"2024-09-11T01:27:07.527118+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47639,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":104800256,"data_stored":104789819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182588,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517768,"kb_used_data":62712,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93854072,"statfs":{"total":96636764160,"available":96106569728,"internally_reserved":0,"allocated":64217088,"data_stored":63840229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986360,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":406136,"kb_used_data":41320,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93965704,"statfs":{"total":96636764160,"available":96220880896,"internally_reserved":0,"allocated":42311680,"data_stored":41938898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659304,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501432,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":500859,"kb_avail":93870408,"statfs":{"total":96636764160,"available":96123297792,"internally_reserved":0,"allocated":565248,"data_stored":188389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512880303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63164416,"data_stored":63160937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":41631744,"data_stored":41628265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:12.220 INFO:tasks.rep_lost_unfound_delete.ceph_manager:clean! 2024-09-11T01:27:12.220 INFO:tasks.rep_lost_unfound_delete.ceph_manager:waiting for clean 2024-09-11T01:27:12.220 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:12.498 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:27:12.498 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:27:12.514 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":544,"stamp":"2024-09-11T01:27:10.888793+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47640,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1425336,"kb_used_data":104584,"kb_used_omap":12,"kb_used_meta":1320691,"kb_avail":281690184,"statfs":{"total":289910292480,"available":288450748416,"internally_reserved":0,"allocated":107094016,"data_stored":105967516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1352387588},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1192,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002060"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28544,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.524154+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:27:07.524154+0000","last_peered":"2024-09-11T01:27:07.524154+0000","last_clean":"2024-09-11T01:27:07.524154+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:27:07.524154+0000","last_undegraded":"2024-09-11T01:27:07.524154+0000","last_fullsized":"2024-09-11T01:27:07.524154+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20502,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.523059+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:27:07.523059+0000","last_peered":"2024-09-11T01:27:07.523059+0000","last_clean":"2024-09-11T01:27:07.523059+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:27:07.523059+0000","last_undegraded":"2024-09-11T01:27:07.523059+0000","last_fullsized":"2024-09-11T01:27:07.523059+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20325,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528271+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:27:07.528271+0000","last_peered":"2024-09-11T01:27:07.528271+0000","last_clean":"2024-09-11T01:27:07.528271+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:27:07.528271+0000","last_undegraded":"2024-09-11T01:27:07.528271+0000","last_fullsized":"2024-09-11T01:27:07.528271+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20252,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.522347+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:27:07.522347+0000","last_peered":"2024-09-11T01:27:07.522347+0000","last_clean":"2024-09-11T01:27:07.522347+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:27:07.522347+0000","last_undegraded":"2024-09-11T01:27:07.522347+0000","last_fullsized":"2024-09-11T01:27:07.522347+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20185,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.522423+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:27:07.522423+0000","last_peered":"2024-09-11T01:27:07.522423+0000","last_clean":"2024-09-11T01:27:07.522423+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:27:07.522423+0000","last_undegraded":"2024-09-11T01:27:07.522423+0000","last_fullsized":"2024-09-11T01:27:07.522423+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20431,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.527154+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:27:07.527154+0000","last_peered":"2024-09-11T01:27:07.527154+0000","last_clean":"2024-09-11T01:27:07.527154+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:27:07.527154+0000","last_undegraded":"2024-09-11T01:27:07.527154+0000","last_fullsized":"2024-09-11T01:27:07.527154+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20085,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.526901+0000","last_change":"2024-09-11T01:26:43.604424+0000","last_active":"2024-09-11T01:27:07.526901+0000","last_peered":"2024-09-11T01:27:07.526901+0000","last_clean":"2024-09-11T01:27:07.526901+0000","last_became_active":"2024-09-11T01:26:43.498430+0000","last_became_peered":"2024-09-11T01:26:43.498430+0000","last_unstale":"2024-09-11T01:27:07.526901+0000","last_undegraded":"2024-09-11T01:27:07.526901+0000","last_fullsized":"2024-09-11T01:27:07.526901+0000","mapping_epoch":64,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:13.180371+0000","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3988,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20433,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.521558+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:27:07.521558+0000","last_peered":"2024-09-11T01:27:07.521558+0000","last_clean":"2024-09-11T01:27:07.521558+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:27:07.521558+0000","last_undegraded":"2024-09-11T01:27:07.521558+0000","last_fullsized":"2024-09-11T01:27:07.521558+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.526781+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:27:07.526781+0000","last_peered":"2024-09-11T01:27:07.526781+0000","last_clean":"2024-09-11T01:27:07.526781+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:27:07.526781+0000","last_undegraded":"2024-09-11T01:27:07.526781+0000","last_fullsized":"2024-09-11T01:27:07.526781+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28030,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.527075+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:27:07.527075+0000","last_peered":"2024-09-11T01:27:07.527075+0000","last_clean":"2024-09-11T01:27:07.527075+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:27:07.527075+0000","last_undegraded":"2024-09-11T01:27:07.527075+0000","last_fullsized":"2024-09-11T01:27:07.527075+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.520946+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:27:07.520946+0000","last_peered":"2024-09-11T01:27:07.520946+0000","last_clean":"2024-09-11T01:27:07.520946+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:27:07.520946+0000","last_undegraded":"2024-09-11T01:27:07.520946+0000","last_fullsized":"2024-09-11T01:27:07.520946+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":28025,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.625143+0000","last_change":"2024-09-11T01:27:07.625011+0000","last_active":"2024-09-11T01:27:07.625143+0000","last_peered":"2024-09-11T01:27:07.625143+0000","last_clean":"2024-09-11T01:27:07.625143+0000","last_became_active":"2024-09-11T01:27:07.529673+0000","last_became_peered":"2024-09-11T01:27:07.529673+0000","last_unstale":"2024-09-11T01:27:07.625143+0000","last_undegraded":"2024-09-11T01:27:07.625143+0000","last_fullsized":"2024-09-11T01:27:07.625143+0000","mapping_epoch":66,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:25:06.093676+0000","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3986,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":150,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.521775+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:27:07.521775+0000","last_peered":"2024-09-11T01:27:07.521775+0000","last_clean":"2024-09-11T01:27:07.521775+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:27:07.521775+0000","last_undegraded":"2024-09-11T01:27:07.521775+0000","last_fullsized":"2024-09-11T01:27:07.521775+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28435,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.521118+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:27:07.521118+0000","last_peered":"2024-09-11T01:27:07.521118+0000","last_clean":"2024-09-11T01:27:07.521118+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:27:07.521118+0000","last_undegraded":"2024-09-11T01:27:07.521118+0000","last_fullsized":"2024-09-11T01:27:07.521118+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":128,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.527066+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:27:07.527066+0000","last_peered":"2024-09-11T01:27:07.527066+0000","last_clean":"2024-09-11T01:27:07.527066+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:27:07.527066+0000","last_undegraded":"2024-09-11T01:27:07.527066+0000","last_fullsized":"2024-09-11T01:27:07.527066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27826,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.520647+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:27:07.520647+0000","last_peered":"2024-09-11T01:27:07.520647+0000","last_clean":"2024-09-11T01:27:07.520647+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:27:07.520647+0000","last_undegraded":"2024-09-11T01:27:07.520647+0000","last_fullsized":"2024-09-11T01:27:07.520647+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20435,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528488+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:27:07.528488+0000","last_peered":"2024-09-11T01:27:07.528488+0000","last_clean":"2024-09-11T01:27:07.528488+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:27:07.528488+0000","last_undegraded":"2024-09-11T01:27:07.528488+0000","last_fullsized":"2024-09-11T01:27:07.528488+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528226+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:27:07.528226+0000","last_peered":"2024-09-11T01:27:07.528226+0000","last_clean":"2024-09-11T01:27:07.528226+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:27:07.528226+0000","last_undegraded":"2024-09-11T01:27:07.528226+0000","last_fullsized":"2024-09-11T01:27:07.528226+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27964,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528314+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:27:07.528314+0000","last_peered":"2024-09-11T01:27:07.528314+0000","last_clean":"2024-09-11T01:27:07.528314+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:27:07.528314+0000","last_undegraded":"2024-09-11T01:27:07.528314+0000","last_fullsized":"2024-09-11T01:27:07.528314+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.522796+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:27:07.522796+0000","last_peered":"2024-09-11T01:27:07.522796+0000","last_clean":"2024-09-11T01:27:07.522796+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:27:07.522796+0000","last_undegraded":"2024-09-11T01:27:07.522796+0000","last_fullsized":"2024-09-11T01:27:07.522796+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24643,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528300+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:27:07.528300+0000","last_peered":"2024-09-11T01:27:07.528300+0000","last_clean":"2024-09-11T01:27:07.528300+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:27:07.528300+0000","last_undegraded":"2024-09-11T01:27:07.528300+0000","last_fullsized":"2024-09-11T01:27:07.528300+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528259+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:27:07.528259+0000","last_peered":"2024-09-11T01:27:07.528259+0000","last_clean":"2024-09-11T01:27:07.528259+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:27:07.528259+0000","last_undegraded":"2024-09-11T01:27:07.528259+0000","last_fullsized":"2024-09-11T01:27:07.528259+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":123,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528414+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:27:07.528414+0000","last_peered":"2024-09-11T01:27:07.528414+0000","last_clean":"2024-09-11T01:27:07.528414+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:27:07.528414+0000","last_undegraded":"2024-09-11T01:27:07.528414+0000","last_fullsized":"2024-09-11T01:27:07.528414+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20115,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.527118+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:27:07.527118+0000","last_peered":"2024-09-11T01:27:07.527118+0000","last_clean":"2024-09-11T01:27:07.527118+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:27:07.527118+0000","last_undegraded":"2024-09-11T01:27:07.527118+0000","last_fullsized":"2024-09-11T01:27:07.527118+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47639,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":104800256,"data_stored":104789819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182588,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517768,"kb_used_data":62712,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93854072,"statfs":{"total":96636764160,"available":96106569728,"internally_reserved":0,"allocated":64217088,"data_stored":63840229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986360,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":406136,"kb_used_data":41320,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93965704,"statfs":{"total":96636764160,"available":96220880896,"internally_reserved":0,"allocated":42311680,"data_stored":41938898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659304,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501432,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":500859,"kb_avail":93870408,"statfs":{"total":96636764160,"available":96123297792,"internally_reserved":0,"allocated":565248,"data_stored":188389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512880303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63164416,"data_stored":63160937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":41631744,"data_stored":41628265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:12.515 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:12.811 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:27:12.812 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:27:12.828 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":544,"stamp":"2024-09-11T01:27:10.888793+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47640,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1425336,"kb_used_data":104584,"kb_used_omap":12,"kb_used_meta":1320691,"kb_avail":281690184,"statfs":{"total":289910292480,"available":288450748416,"internally_reserved":0,"allocated":107094016,"data_stored":105967516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1352387588},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1192,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1192,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002060"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28544,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.524154+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:27:07.524154+0000","last_peered":"2024-09-11T01:27:07.524154+0000","last_clean":"2024-09-11T01:27:07.524154+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:27:07.524154+0000","last_undegraded":"2024-09-11T01:27:07.524154+0000","last_fullsized":"2024-09-11T01:27:07.524154+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20502,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.523059+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:27:07.523059+0000","last_peered":"2024-09-11T01:27:07.523059+0000","last_clean":"2024-09-11T01:27:07.523059+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:27:07.523059+0000","last_undegraded":"2024-09-11T01:27:07.523059+0000","last_fullsized":"2024-09-11T01:27:07.523059+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20325,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528271+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:27:07.528271+0000","last_peered":"2024-09-11T01:27:07.528271+0000","last_clean":"2024-09-11T01:27:07.528271+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:27:07.528271+0000","last_undegraded":"2024-09-11T01:27:07.528271+0000","last_fullsized":"2024-09-11T01:27:07.528271+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20252,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.522347+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:27:07.522347+0000","last_peered":"2024-09-11T01:27:07.522347+0000","last_clean":"2024-09-11T01:27:07.522347+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:27:07.522347+0000","last_undegraded":"2024-09-11T01:27:07.522347+0000","last_fullsized":"2024-09-11T01:27:07.522347+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20185,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.522423+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:27:07.522423+0000","last_peered":"2024-09-11T01:27:07.522423+0000","last_clean":"2024-09-11T01:27:07.522423+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:27:07.522423+0000","last_undegraded":"2024-09-11T01:27:07.522423+0000","last_fullsized":"2024-09-11T01:27:07.522423+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20431,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.527154+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:27:07.527154+0000","last_peered":"2024-09-11T01:27:07.527154+0000","last_clean":"2024-09-11T01:27:07.527154+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:27:07.527154+0000","last_undegraded":"2024-09-11T01:27:07.527154+0000","last_fullsized":"2024-09-11T01:27:07.527154+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20085,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.526901+0000","last_change":"2024-09-11T01:26:43.604424+0000","last_active":"2024-09-11T01:27:07.526901+0000","last_peered":"2024-09-11T01:27:07.526901+0000","last_clean":"2024-09-11T01:27:07.526901+0000","last_became_active":"2024-09-11T01:26:43.498430+0000","last_became_peered":"2024-09-11T01:26:43.498430+0000","last_unstale":"2024-09-11T01:27:07.526901+0000","last_undegraded":"2024-09-11T01:27:07.526901+0000","last_fullsized":"2024-09-11T01:27:07.526901+0000","mapping_epoch":64,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:13.180371+0000","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3988,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20433,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.521558+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:27:07.521558+0000","last_peered":"2024-09-11T01:27:07.521558+0000","last_clean":"2024-09-11T01:27:07.521558+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:27:07.521558+0000","last_undegraded":"2024-09-11T01:27:07.521558+0000","last_fullsized":"2024-09-11T01:27:07.521558+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.526781+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:27:07.526781+0000","last_peered":"2024-09-11T01:27:07.526781+0000","last_clean":"2024-09-11T01:27:07.526781+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:27:07.526781+0000","last_undegraded":"2024-09-11T01:27:07.526781+0000","last_fullsized":"2024-09-11T01:27:07.526781+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28030,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.527075+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:27:07.527075+0000","last_peered":"2024-09-11T01:27:07.527075+0000","last_clean":"2024-09-11T01:27:07.527075+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:27:07.527075+0000","last_undegraded":"2024-09-11T01:27:07.527075+0000","last_fullsized":"2024-09-11T01:27:07.527075+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.520946+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:27:07.520946+0000","last_peered":"2024-09-11T01:27:07.520946+0000","last_clean":"2024-09-11T01:27:07.520946+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:27:07.520946+0000","last_undegraded":"2024-09-11T01:27:07.520946+0000","last_fullsized":"2024-09-11T01:27:07.520946+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":28025,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.625143+0000","last_change":"2024-09-11T01:27:07.625011+0000","last_active":"2024-09-11T01:27:07.625143+0000","last_peered":"2024-09-11T01:27:07.625143+0000","last_clean":"2024-09-11T01:27:07.625143+0000","last_became_active":"2024-09-11T01:27:07.529673+0000","last_became_peered":"2024-09-11T01:27:07.529673+0000","last_unstale":"2024-09-11T01:27:07.625143+0000","last_undegraded":"2024-09-11T01:27:07.625143+0000","last_fullsized":"2024-09-11T01:27:07.625143+0000","mapping_epoch":66,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:25:06.093676+0000","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3986,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":150,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.521775+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:27:07.521775+0000","last_peered":"2024-09-11T01:27:07.521775+0000","last_clean":"2024-09-11T01:27:07.521775+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:27:07.521775+0000","last_undegraded":"2024-09-11T01:27:07.521775+0000","last_fullsized":"2024-09-11T01:27:07.521775+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28435,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.521118+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:27:07.521118+0000","last_peered":"2024-09-11T01:27:07.521118+0000","last_clean":"2024-09-11T01:27:07.521118+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:27:07.521118+0000","last_undegraded":"2024-09-11T01:27:07.521118+0000","last_fullsized":"2024-09-11T01:27:07.521118+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":128,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.527066+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:27:07.527066+0000","last_peered":"2024-09-11T01:27:07.527066+0000","last_clean":"2024-09-11T01:27:07.527066+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:27:07.527066+0000","last_undegraded":"2024-09-11T01:27:07.527066+0000","last_fullsized":"2024-09-11T01:27:07.527066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27826,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.520647+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:27:07.520647+0000","last_peered":"2024-09-11T01:27:07.520647+0000","last_clean":"2024-09-11T01:27:07.520647+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:27:07.520647+0000","last_undegraded":"2024-09-11T01:27:07.520647+0000","last_fullsized":"2024-09-11T01:27:07.520647+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20435,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528488+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:27:07.528488+0000","last_peered":"2024-09-11T01:27:07.528488+0000","last_clean":"2024-09-11T01:27:07.528488+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:27:07.528488+0000","last_undegraded":"2024-09-11T01:27:07.528488+0000","last_fullsized":"2024-09-11T01:27:07.528488+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528226+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:27:07.528226+0000","last_peered":"2024-09-11T01:27:07.528226+0000","last_clean":"2024-09-11T01:27:07.528226+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:27:07.528226+0000","last_undegraded":"2024-09-11T01:27:07.528226+0000","last_fullsized":"2024-09-11T01:27:07.528226+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27964,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528314+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:27:07.528314+0000","last_peered":"2024-09-11T01:27:07.528314+0000","last_clean":"2024-09-11T01:27:07.528314+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:27:07.528314+0000","last_undegraded":"2024-09-11T01:27:07.528314+0000","last_fullsized":"2024-09-11T01:27:07.528314+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.522796+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:27:07.522796+0000","last_peered":"2024-09-11T01:27:07.522796+0000","last_clean":"2024-09-11T01:27:07.522796+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:27:07.522796+0000","last_undegraded":"2024-09-11T01:27:07.522796+0000","last_fullsized":"2024-09-11T01:27:07.522796+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24643,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528300+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:27:07.528300+0000","last_peered":"2024-09-11T01:27:07.528300+0000","last_clean":"2024-09-11T01:27:07.528300+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:27:07.528300+0000","last_undegraded":"2024-09-11T01:27:07.528300+0000","last_fullsized":"2024-09-11T01:27:07.528300+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528259+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:27:07.528259+0000","last_peered":"2024-09-11T01:27:07.528259+0000","last_clean":"2024-09-11T01:27:07.528259+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:27:07.528259+0000","last_undegraded":"2024-09-11T01:27:07.528259+0000","last_fullsized":"2024-09-11T01:27:07.528259+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":123,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528414+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:27:07.528414+0000","last_peered":"2024-09-11T01:27:07.528414+0000","last_clean":"2024-09-11T01:27:07.528414+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:27:07.528414+0000","last_undegraded":"2024-09-11T01:27:07.528414+0000","last_fullsized":"2024-09-11T01:27:07.528414+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20115,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.527118+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:27:07.527118+0000","last_peered":"2024-09-11T01:27:07.527118+0000","last_clean":"2024-09-11T01:27:07.527118+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:27:07.527118+0000","last_undegraded":"2024-09-11T01:27:07.527118+0000","last_fullsized":"2024-09-11T01:27:07.527118+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47639,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":104800256,"data_stored":104789819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182588,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517768,"kb_used_data":62712,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93854072,"statfs":{"total":96636764160,"available":96106569728,"internally_reserved":0,"allocated":64217088,"data_stored":63840229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986360,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":406136,"kb_used_data":41320,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93965704,"statfs":{"total":96636764160,"available":96220880896,"internally_reserved":0,"allocated":42311680,"data_stored":41938898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659304,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501432,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":500859,"kb_avail":93870408,"statfs":{"total":96636764160,"available":96123297792,"internally_reserved":0,"allocated":565248,"data_stored":188389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512880303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63164416,"data_stored":63160937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":41631744,"data_stored":41628265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:12.829 INFO:tasks.rep_lost_unfound_delete.ceph_manager:clean! 2024-09-11T01:27:12.829 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-11T01:27:12.843 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-11T01:27:12.843 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:13.128 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:27:13.128 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:27:13.144 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":545,"stamp":"2024-09-11T01:27:12.889269+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47640,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1425336,"kb_used_data":104584,"kb_used_omap":12,"kb_used_meta":1320691,"kb_avail":281690184,"statfs":{"total":289910292480,"available":288450748416,"internally_reserved":0,"allocated":107094016,"data_stored":105967516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1352387588},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-440,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":440,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002025"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28544,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.524154+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:27:07.524154+0000","last_peered":"2024-09-11T01:27:07.524154+0000","last_clean":"2024-09-11T01:27:07.524154+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:27:07.524154+0000","last_undegraded":"2024-09-11T01:27:07.524154+0000","last_fullsized":"2024-09-11T01:27:07.524154+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20502,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.523059+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:27:07.523059+0000","last_peered":"2024-09-11T01:27:07.523059+0000","last_clean":"2024-09-11T01:27:07.523059+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:27:07.523059+0000","last_undegraded":"2024-09-11T01:27:07.523059+0000","last_fullsized":"2024-09-11T01:27:07.523059+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20325,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528271+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:27:07.528271+0000","last_peered":"2024-09-11T01:27:07.528271+0000","last_clean":"2024-09-11T01:27:07.528271+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:27:07.528271+0000","last_undegraded":"2024-09-11T01:27:07.528271+0000","last_fullsized":"2024-09-11T01:27:07.528271+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20252,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.522347+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:27:07.522347+0000","last_peered":"2024-09-11T01:27:07.522347+0000","last_clean":"2024-09-11T01:27:07.522347+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:27:07.522347+0000","last_undegraded":"2024-09-11T01:27:07.522347+0000","last_fullsized":"2024-09-11T01:27:07.522347+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20185,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.522423+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:27:07.522423+0000","last_peered":"2024-09-11T01:27:07.522423+0000","last_clean":"2024-09-11T01:27:07.522423+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:27:07.522423+0000","last_undegraded":"2024-09-11T01:27:07.522423+0000","last_fullsized":"2024-09-11T01:27:07.522423+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20431,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.527154+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:27:07.527154+0000","last_peered":"2024-09-11T01:27:07.527154+0000","last_clean":"2024-09-11T01:27:07.527154+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:27:07.527154+0000","last_undegraded":"2024-09-11T01:27:07.527154+0000","last_fullsized":"2024-09-11T01:27:07.527154+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20085,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.526901+0000","last_change":"2024-09-11T01:26:43.604424+0000","last_active":"2024-09-11T01:27:07.526901+0000","last_peered":"2024-09-11T01:27:07.526901+0000","last_clean":"2024-09-11T01:27:07.526901+0000","last_became_active":"2024-09-11T01:26:43.498430+0000","last_became_peered":"2024-09-11T01:26:43.498430+0000","last_unstale":"2024-09-11T01:27:07.526901+0000","last_undegraded":"2024-09-11T01:27:07.526901+0000","last_fullsized":"2024-09-11T01:27:07.526901+0000","mapping_epoch":64,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:13.180371+0000","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3988,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20433,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.521558+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:27:07.521558+0000","last_peered":"2024-09-11T01:27:07.521558+0000","last_clean":"2024-09-11T01:27:07.521558+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:27:07.521558+0000","last_undegraded":"2024-09-11T01:27:07.521558+0000","last_fullsized":"2024-09-11T01:27:07.521558+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.526781+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:27:07.526781+0000","last_peered":"2024-09-11T01:27:07.526781+0000","last_clean":"2024-09-11T01:27:07.526781+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:27:07.526781+0000","last_undegraded":"2024-09-11T01:27:07.526781+0000","last_fullsized":"2024-09-11T01:27:07.526781+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28030,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.527075+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:27:07.527075+0000","last_peered":"2024-09-11T01:27:07.527075+0000","last_clean":"2024-09-11T01:27:07.527075+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:27:07.527075+0000","last_undegraded":"2024-09-11T01:27:07.527075+0000","last_fullsized":"2024-09-11T01:27:07.527075+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.520946+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:27:07.520946+0000","last_peered":"2024-09-11T01:27:07.520946+0000","last_clean":"2024-09-11T01:27:07.520946+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:27:07.520946+0000","last_undegraded":"2024-09-11T01:27:07.520946+0000","last_fullsized":"2024-09-11T01:27:07.520946+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":28025,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.625143+0000","last_change":"2024-09-11T01:27:07.625011+0000","last_active":"2024-09-11T01:27:07.625143+0000","last_peered":"2024-09-11T01:27:07.625143+0000","last_clean":"2024-09-11T01:27:07.625143+0000","last_became_active":"2024-09-11T01:27:07.529673+0000","last_became_peered":"2024-09-11T01:27:07.529673+0000","last_unstale":"2024-09-11T01:27:07.625143+0000","last_undegraded":"2024-09-11T01:27:07.625143+0000","last_fullsized":"2024-09-11T01:27:07.625143+0000","mapping_epoch":66,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:25:06.093676+0000","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3986,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":150,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.521775+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:27:07.521775+0000","last_peered":"2024-09-11T01:27:07.521775+0000","last_clean":"2024-09-11T01:27:07.521775+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:27:07.521775+0000","last_undegraded":"2024-09-11T01:27:07.521775+0000","last_fullsized":"2024-09-11T01:27:07.521775+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28435,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.521118+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:27:07.521118+0000","last_peered":"2024-09-11T01:27:07.521118+0000","last_clean":"2024-09-11T01:27:07.521118+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:27:07.521118+0000","last_undegraded":"2024-09-11T01:27:07.521118+0000","last_fullsized":"2024-09-11T01:27:07.521118+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":128,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.527066+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:27:07.527066+0000","last_peered":"2024-09-11T01:27:07.527066+0000","last_clean":"2024-09-11T01:27:07.527066+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:27:07.527066+0000","last_undegraded":"2024-09-11T01:27:07.527066+0000","last_fullsized":"2024-09-11T01:27:07.527066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27826,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.520647+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:27:07.520647+0000","last_peered":"2024-09-11T01:27:07.520647+0000","last_clean":"2024-09-11T01:27:07.520647+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:27:07.520647+0000","last_undegraded":"2024-09-11T01:27:07.520647+0000","last_fullsized":"2024-09-11T01:27:07.520647+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20435,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528488+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:27:07.528488+0000","last_peered":"2024-09-11T01:27:07.528488+0000","last_clean":"2024-09-11T01:27:07.528488+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:27:07.528488+0000","last_undegraded":"2024-09-11T01:27:07.528488+0000","last_fullsized":"2024-09-11T01:27:07.528488+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528226+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:27:07.528226+0000","last_peered":"2024-09-11T01:27:07.528226+0000","last_clean":"2024-09-11T01:27:07.528226+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:27:07.528226+0000","last_undegraded":"2024-09-11T01:27:07.528226+0000","last_fullsized":"2024-09-11T01:27:07.528226+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27964,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528314+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:27:07.528314+0000","last_peered":"2024-09-11T01:27:07.528314+0000","last_clean":"2024-09-11T01:27:07.528314+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:27:07.528314+0000","last_undegraded":"2024-09-11T01:27:07.528314+0000","last_fullsized":"2024-09-11T01:27:07.528314+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.522796+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:27:07.522796+0000","last_peered":"2024-09-11T01:27:07.522796+0000","last_clean":"2024-09-11T01:27:07.522796+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:27:07.522796+0000","last_undegraded":"2024-09-11T01:27:07.522796+0000","last_fullsized":"2024-09-11T01:27:07.522796+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24643,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528300+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:27:07.528300+0000","last_peered":"2024-09-11T01:27:07.528300+0000","last_clean":"2024-09-11T01:27:07.528300+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:27:07.528300+0000","last_undegraded":"2024-09-11T01:27:07.528300+0000","last_fullsized":"2024-09-11T01:27:07.528300+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528259+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:27:07.528259+0000","last_peered":"2024-09-11T01:27:07.528259+0000","last_clean":"2024-09-11T01:27:07.528259+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:27:07.528259+0000","last_undegraded":"2024-09-11T01:27:07.528259+0000","last_fullsized":"2024-09-11T01:27:07.528259+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":123,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528414+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:27:07.528414+0000","last_peered":"2024-09-11T01:27:07.528414+0000","last_clean":"2024-09-11T01:27:07.528414+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:27:07.528414+0000","last_undegraded":"2024-09-11T01:27:07.528414+0000","last_fullsized":"2024-09-11T01:27:07.528414+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20115,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.527118+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:27:07.527118+0000","last_peered":"2024-09-11T01:27:07.527118+0000","last_clean":"2024-09-11T01:27:07.527118+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:27:07.527118+0000","last_undegraded":"2024-09-11T01:27:07.527118+0000","last_fullsized":"2024-09-11T01:27:07.527118+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47639,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":104800256,"data_stored":104789819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182588,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517768,"kb_used_data":62712,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93854072,"statfs":{"total":96636764160,"available":96106569728,"internally_reserved":0,"allocated":64217088,"data_stored":63840229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986360,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":406136,"kb_used_data":41320,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93965704,"statfs":{"total":96636764160,"available":96220880896,"internally_reserved":0,"allocated":42311680,"data_stored":41938898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659304,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501432,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":500859,"kb_avail":93870408,"statfs":{"total":96636764160,"available":96123297792,"internally_reserved":0,"allocated":565248,"data_stored":188389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512880303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63164416,"data_stored":63160937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":41631744,"data_stored":41628265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:13.146 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:13.426 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:27:13.426 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:27:13.442 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":545,"stamp":"2024-09-11T01:27:12.889269+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47640,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1425336,"kb_used_data":104584,"kb_used_omap":12,"kb_used_meta":1320691,"kb_avail":281690184,"statfs":{"total":289910292480,"available":288450748416,"internally_reserved":0,"allocated":107094016,"data_stored":105967516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1352387588},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-440,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":440,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002025"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28544,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.524154+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:27:07.524154+0000","last_peered":"2024-09-11T01:27:07.524154+0000","last_clean":"2024-09-11T01:27:07.524154+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:27:07.524154+0000","last_undegraded":"2024-09-11T01:27:07.524154+0000","last_fullsized":"2024-09-11T01:27:07.524154+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20502,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.523059+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:27:07.523059+0000","last_peered":"2024-09-11T01:27:07.523059+0000","last_clean":"2024-09-11T01:27:07.523059+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:27:07.523059+0000","last_undegraded":"2024-09-11T01:27:07.523059+0000","last_fullsized":"2024-09-11T01:27:07.523059+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20325,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528271+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:27:07.528271+0000","last_peered":"2024-09-11T01:27:07.528271+0000","last_clean":"2024-09-11T01:27:07.528271+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:27:07.528271+0000","last_undegraded":"2024-09-11T01:27:07.528271+0000","last_fullsized":"2024-09-11T01:27:07.528271+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20252,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.522347+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:27:07.522347+0000","last_peered":"2024-09-11T01:27:07.522347+0000","last_clean":"2024-09-11T01:27:07.522347+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:27:07.522347+0000","last_undegraded":"2024-09-11T01:27:07.522347+0000","last_fullsized":"2024-09-11T01:27:07.522347+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20185,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.522423+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:27:07.522423+0000","last_peered":"2024-09-11T01:27:07.522423+0000","last_clean":"2024-09-11T01:27:07.522423+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:27:07.522423+0000","last_undegraded":"2024-09-11T01:27:07.522423+0000","last_fullsized":"2024-09-11T01:27:07.522423+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20431,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.527154+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:27:07.527154+0000","last_peered":"2024-09-11T01:27:07.527154+0000","last_clean":"2024-09-11T01:27:07.527154+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:27:07.527154+0000","last_undegraded":"2024-09-11T01:27:07.527154+0000","last_fullsized":"2024-09-11T01:27:07.527154+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20085,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.526901+0000","last_change":"2024-09-11T01:26:43.604424+0000","last_active":"2024-09-11T01:27:07.526901+0000","last_peered":"2024-09-11T01:27:07.526901+0000","last_clean":"2024-09-11T01:27:07.526901+0000","last_became_active":"2024-09-11T01:26:43.498430+0000","last_became_peered":"2024-09-11T01:26:43.498430+0000","last_unstale":"2024-09-11T01:27:07.526901+0000","last_undegraded":"2024-09-11T01:27:07.526901+0000","last_fullsized":"2024-09-11T01:27:07.526901+0000","mapping_epoch":64,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:13.180371+0000","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3988,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20433,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.521558+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:27:07.521558+0000","last_peered":"2024-09-11T01:27:07.521558+0000","last_clean":"2024-09-11T01:27:07.521558+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:27:07.521558+0000","last_undegraded":"2024-09-11T01:27:07.521558+0000","last_fullsized":"2024-09-11T01:27:07.521558+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.526781+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:27:07.526781+0000","last_peered":"2024-09-11T01:27:07.526781+0000","last_clean":"2024-09-11T01:27:07.526781+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:27:07.526781+0000","last_undegraded":"2024-09-11T01:27:07.526781+0000","last_fullsized":"2024-09-11T01:27:07.526781+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28030,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.527075+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:27:07.527075+0000","last_peered":"2024-09-11T01:27:07.527075+0000","last_clean":"2024-09-11T01:27:07.527075+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:27:07.527075+0000","last_undegraded":"2024-09-11T01:27:07.527075+0000","last_fullsized":"2024-09-11T01:27:07.527075+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.520946+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:27:07.520946+0000","last_peered":"2024-09-11T01:27:07.520946+0000","last_clean":"2024-09-11T01:27:07.520946+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:27:07.520946+0000","last_undegraded":"2024-09-11T01:27:07.520946+0000","last_fullsized":"2024-09-11T01:27:07.520946+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":28025,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.625143+0000","last_change":"2024-09-11T01:27:07.625011+0000","last_active":"2024-09-11T01:27:07.625143+0000","last_peered":"2024-09-11T01:27:07.625143+0000","last_clean":"2024-09-11T01:27:07.625143+0000","last_became_active":"2024-09-11T01:27:07.529673+0000","last_became_peered":"2024-09-11T01:27:07.529673+0000","last_unstale":"2024-09-11T01:27:07.625143+0000","last_undegraded":"2024-09-11T01:27:07.625143+0000","last_fullsized":"2024-09-11T01:27:07.625143+0000","mapping_epoch":66,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:25:06.093676+0000","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3986,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":150,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.521775+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:27:07.521775+0000","last_peered":"2024-09-11T01:27:07.521775+0000","last_clean":"2024-09-11T01:27:07.521775+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:27:07.521775+0000","last_undegraded":"2024-09-11T01:27:07.521775+0000","last_fullsized":"2024-09-11T01:27:07.521775+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28435,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.521118+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:27:07.521118+0000","last_peered":"2024-09-11T01:27:07.521118+0000","last_clean":"2024-09-11T01:27:07.521118+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:27:07.521118+0000","last_undegraded":"2024-09-11T01:27:07.521118+0000","last_fullsized":"2024-09-11T01:27:07.521118+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":128,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.527066+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:27:07.527066+0000","last_peered":"2024-09-11T01:27:07.527066+0000","last_clean":"2024-09-11T01:27:07.527066+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:27:07.527066+0000","last_undegraded":"2024-09-11T01:27:07.527066+0000","last_fullsized":"2024-09-11T01:27:07.527066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27826,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.520647+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:27:07.520647+0000","last_peered":"2024-09-11T01:27:07.520647+0000","last_clean":"2024-09-11T01:27:07.520647+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:27:07.520647+0000","last_undegraded":"2024-09-11T01:27:07.520647+0000","last_fullsized":"2024-09-11T01:27:07.520647+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20435,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528488+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:27:07.528488+0000","last_peered":"2024-09-11T01:27:07.528488+0000","last_clean":"2024-09-11T01:27:07.528488+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:27:07.528488+0000","last_undegraded":"2024-09-11T01:27:07.528488+0000","last_fullsized":"2024-09-11T01:27:07.528488+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528226+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:27:07.528226+0000","last_peered":"2024-09-11T01:27:07.528226+0000","last_clean":"2024-09-11T01:27:07.528226+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:27:07.528226+0000","last_undegraded":"2024-09-11T01:27:07.528226+0000","last_fullsized":"2024-09-11T01:27:07.528226+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27964,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528314+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:27:07.528314+0000","last_peered":"2024-09-11T01:27:07.528314+0000","last_clean":"2024-09-11T01:27:07.528314+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:27:07.528314+0000","last_undegraded":"2024-09-11T01:27:07.528314+0000","last_fullsized":"2024-09-11T01:27:07.528314+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.522796+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:27:07.522796+0000","last_peered":"2024-09-11T01:27:07.522796+0000","last_clean":"2024-09-11T01:27:07.522796+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:27:07.522796+0000","last_undegraded":"2024-09-11T01:27:07.522796+0000","last_fullsized":"2024-09-11T01:27:07.522796+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24643,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528300+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:27:07.528300+0000","last_peered":"2024-09-11T01:27:07.528300+0000","last_clean":"2024-09-11T01:27:07.528300+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:27:07.528300+0000","last_undegraded":"2024-09-11T01:27:07.528300+0000","last_fullsized":"2024-09-11T01:27:07.528300+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528259+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:27:07.528259+0000","last_peered":"2024-09-11T01:27:07.528259+0000","last_clean":"2024-09-11T01:27:07.528259+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:27:07.528259+0000","last_undegraded":"2024-09-11T01:27:07.528259+0000","last_fullsized":"2024-09-11T01:27:07.528259+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":123,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528414+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:27:07.528414+0000","last_peered":"2024-09-11T01:27:07.528414+0000","last_clean":"2024-09-11T01:27:07.528414+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:27:07.528414+0000","last_undegraded":"2024-09-11T01:27:07.528414+0000","last_fullsized":"2024-09-11T01:27:07.528414+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20115,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.527118+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:27:07.527118+0000","last_peered":"2024-09-11T01:27:07.527118+0000","last_clean":"2024-09-11T01:27:07.527118+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:27:07.527118+0000","last_undegraded":"2024-09-11T01:27:07.527118+0000","last_fullsized":"2024-09-11T01:27:07.527118+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47639,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":104800256,"data_stored":104789819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182588,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517768,"kb_used_data":62712,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93854072,"statfs":{"total":96636764160,"available":96106569728,"internally_reserved":0,"allocated":64217088,"data_stored":63840229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986360,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":406136,"kb_used_data":41320,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93965704,"statfs":{"total":96636764160,"available":96220880896,"internally_reserved":0,"allocated":42311680,"data_stored":41938898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659304,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501432,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":500859,"kb_avail":93870408,"statfs":{"total":96636764160,"available":96123297792,"internally_reserved":0,"allocated":565248,"data_stored":188389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512880303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63164416,"data_stored":63160937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":41631744,"data_stored":41628265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:13.443 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-11T01:27:13.444 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:13.717 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:27:13.717 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:27:13.733 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":545,"stamp":"2024-09-11T01:27:12.889269+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47640,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1425336,"kb_used_data":104584,"kb_used_omap":12,"kb_used_meta":1320691,"kb_avail":281690184,"statfs":{"total":289910292480,"available":288450748416,"internally_reserved":0,"allocated":107094016,"data_stored":105967516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1352387588},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-440,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":440,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002025"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28544,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.524154+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:27:07.524154+0000","last_peered":"2024-09-11T01:27:07.524154+0000","last_clean":"2024-09-11T01:27:07.524154+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:27:07.524154+0000","last_undegraded":"2024-09-11T01:27:07.524154+0000","last_fullsized":"2024-09-11T01:27:07.524154+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20502,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.523059+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:27:07.523059+0000","last_peered":"2024-09-11T01:27:07.523059+0000","last_clean":"2024-09-11T01:27:07.523059+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:27:07.523059+0000","last_undegraded":"2024-09-11T01:27:07.523059+0000","last_fullsized":"2024-09-11T01:27:07.523059+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20325,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528271+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:27:07.528271+0000","last_peered":"2024-09-11T01:27:07.528271+0000","last_clean":"2024-09-11T01:27:07.528271+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:27:07.528271+0000","last_undegraded":"2024-09-11T01:27:07.528271+0000","last_fullsized":"2024-09-11T01:27:07.528271+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20252,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.522347+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:27:07.522347+0000","last_peered":"2024-09-11T01:27:07.522347+0000","last_clean":"2024-09-11T01:27:07.522347+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:27:07.522347+0000","last_undegraded":"2024-09-11T01:27:07.522347+0000","last_fullsized":"2024-09-11T01:27:07.522347+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20185,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.522423+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:27:07.522423+0000","last_peered":"2024-09-11T01:27:07.522423+0000","last_clean":"2024-09-11T01:27:07.522423+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:27:07.522423+0000","last_undegraded":"2024-09-11T01:27:07.522423+0000","last_fullsized":"2024-09-11T01:27:07.522423+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20431,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.527154+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:27:07.527154+0000","last_peered":"2024-09-11T01:27:07.527154+0000","last_clean":"2024-09-11T01:27:07.527154+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:27:07.527154+0000","last_undegraded":"2024-09-11T01:27:07.527154+0000","last_fullsized":"2024-09-11T01:27:07.527154+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20085,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.526901+0000","last_change":"2024-09-11T01:26:43.604424+0000","last_active":"2024-09-11T01:27:07.526901+0000","last_peered":"2024-09-11T01:27:07.526901+0000","last_clean":"2024-09-11T01:27:07.526901+0000","last_became_active":"2024-09-11T01:26:43.498430+0000","last_became_peered":"2024-09-11T01:26:43.498430+0000","last_unstale":"2024-09-11T01:27:07.526901+0000","last_undegraded":"2024-09-11T01:27:07.526901+0000","last_fullsized":"2024-09-11T01:27:07.526901+0000","mapping_epoch":64,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:13.180371+0000","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3988,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20433,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.521558+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:27:07.521558+0000","last_peered":"2024-09-11T01:27:07.521558+0000","last_clean":"2024-09-11T01:27:07.521558+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:27:07.521558+0000","last_undegraded":"2024-09-11T01:27:07.521558+0000","last_fullsized":"2024-09-11T01:27:07.521558+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.526781+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:27:07.526781+0000","last_peered":"2024-09-11T01:27:07.526781+0000","last_clean":"2024-09-11T01:27:07.526781+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:27:07.526781+0000","last_undegraded":"2024-09-11T01:27:07.526781+0000","last_fullsized":"2024-09-11T01:27:07.526781+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28030,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.527075+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:27:07.527075+0000","last_peered":"2024-09-11T01:27:07.527075+0000","last_clean":"2024-09-11T01:27:07.527075+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:27:07.527075+0000","last_undegraded":"2024-09-11T01:27:07.527075+0000","last_fullsized":"2024-09-11T01:27:07.527075+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.520946+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:27:07.520946+0000","last_peered":"2024-09-11T01:27:07.520946+0000","last_clean":"2024-09-11T01:27:07.520946+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:27:07.520946+0000","last_undegraded":"2024-09-11T01:27:07.520946+0000","last_fullsized":"2024-09-11T01:27:07.520946+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":28025,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.625143+0000","last_change":"2024-09-11T01:27:07.625011+0000","last_active":"2024-09-11T01:27:07.625143+0000","last_peered":"2024-09-11T01:27:07.625143+0000","last_clean":"2024-09-11T01:27:07.625143+0000","last_became_active":"2024-09-11T01:27:07.529673+0000","last_became_peered":"2024-09-11T01:27:07.529673+0000","last_unstale":"2024-09-11T01:27:07.625143+0000","last_undegraded":"2024-09-11T01:27:07.625143+0000","last_fullsized":"2024-09-11T01:27:07.625143+0000","mapping_epoch":66,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:25:06.093676+0000","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3986,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":150,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.521775+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:27:07.521775+0000","last_peered":"2024-09-11T01:27:07.521775+0000","last_clean":"2024-09-11T01:27:07.521775+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:27:07.521775+0000","last_undegraded":"2024-09-11T01:27:07.521775+0000","last_fullsized":"2024-09-11T01:27:07.521775+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28435,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.521118+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:27:07.521118+0000","last_peered":"2024-09-11T01:27:07.521118+0000","last_clean":"2024-09-11T01:27:07.521118+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:27:07.521118+0000","last_undegraded":"2024-09-11T01:27:07.521118+0000","last_fullsized":"2024-09-11T01:27:07.521118+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":128,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.527066+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:27:07.527066+0000","last_peered":"2024-09-11T01:27:07.527066+0000","last_clean":"2024-09-11T01:27:07.527066+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:27:07.527066+0000","last_undegraded":"2024-09-11T01:27:07.527066+0000","last_fullsized":"2024-09-11T01:27:07.527066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27826,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.520647+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:27:07.520647+0000","last_peered":"2024-09-11T01:27:07.520647+0000","last_clean":"2024-09-11T01:27:07.520647+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:27:07.520647+0000","last_undegraded":"2024-09-11T01:27:07.520647+0000","last_fullsized":"2024-09-11T01:27:07.520647+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20435,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528488+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:27:07.528488+0000","last_peered":"2024-09-11T01:27:07.528488+0000","last_clean":"2024-09-11T01:27:07.528488+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:27:07.528488+0000","last_undegraded":"2024-09-11T01:27:07.528488+0000","last_fullsized":"2024-09-11T01:27:07.528488+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528226+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:27:07.528226+0000","last_peered":"2024-09-11T01:27:07.528226+0000","last_clean":"2024-09-11T01:27:07.528226+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:27:07.528226+0000","last_undegraded":"2024-09-11T01:27:07.528226+0000","last_fullsized":"2024-09-11T01:27:07.528226+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27964,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528314+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:27:07.528314+0000","last_peered":"2024-09-11T01:27:07.528314+0000","last_clean":"2024-09-11T01:27:07.528314+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:27:07.528314+0000","last_undegraded":"2024-09-11T01:27:07.528314+0000","last_fullsized":"2024-09-11T01:27:07.528314+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.522796+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:27:07.522796+0000","last_peered":"2024-09-11T01:27:07.522796+0000","last_clean":"2024-09-11T01:27:07.522796+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:27:07.522796+0000","last_undegraded":"2024-09-11T01:27:07.522796+0000","last_fullsized":"2024-09-11T01:27:07.522796+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24643,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528300+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:27:07.528300+0000","last_peered":"2024-09-11T01:27:07.528300+0000","last_clean":"2024-09-11T01:27:07.528300+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:27:07.528300+0000","last_undegraded":"2024-09-11T01:27:07.528300+0000","last_fullsized":"2024-09-11T01:27:07.528300+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528259+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:27:07.528259+0000","last_peered":"2024-09-11T01:27:07.528259+0000","last_clean":"2024-09-11T01:27:07.528259+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:27:07.528259+0000","last_undegraded":"2024-09-11T01:27:07.528259+0000","last_fullsized":"2024-09-11T01:27:07.528259+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":123,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528414+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:27:07.528414+0000","last_peered":"2024-09-11T01:27:07.528414+0000","last_clean":"2024-09-11T01:27:07.528414+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:27:07.528414+0000","last_undegraded":"2024-09-11T01:27:07.528414+0000","last_fullsized":"2024-09-11T01:27:07.528414+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20115,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.527118+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:27:07.527118+0000","last_peered":"2024-09-11T01:27:07.527118+0000","last_clean":"2024-09-11T01:27:07.527118+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:27:07.527118+0000","last_undegraded":"2024-09-11T01:27:07.527118+0000","last_fullsized":"2024-09-11T01:27:07.527118+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47639,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":104800256,"data_stored":104789819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182588,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":517768,"kb_used_data":62712,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93854072,"statfs":{"total":96636764160,"available":96106569728,"internally_reserved":0,"allocated":64217088,"data_stored":63840229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986360,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":406136,"kb_used_data":41320,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93965704,"statfs":{"total":96636764160,"available":96220880896,"internally_reserved":0,"allocated":42311680,"data_stored":41938898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659304,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501432,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":500859,"kb_avail":93870408,"statfs":{"total":96636764160,"available":96123297792,"internally_reserved":0,"allocated":565248,"data_stored":188389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512880303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":63164416,"data_stored":63160937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":41631744,"data_stored":41628265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:13.734 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-11T01:27:14.016 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:27:14.016 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":67,"fsid":"6d328773-3914-4412-91b7-393379984de3","created":"2024-09-11T01:11:03.508628+0000","modified":"2024-09-11T01:27:07.505334+0000","last_up_change":"2024-09-11T01:13:07.625340+0000","last_in_change":"2024-09-11T01:12:59.834020+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":8,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-11T01:11:18.999872+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"unfounddel_pool","create_time":"2024-09-11T01:11:30.626364+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"e160603b-65bb-420f-9130-6d7c6506f2a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":21,"up_from":25,"up_thru":66,"down_at":22,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6800","nonce":2613596984}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6801","nonce":2613596984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6803","nonce":2613596984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6802","nonce":2613596984}]},"public_addr":"172.21.15.122:6800/2613596984","cluster_addr":"172.21.15.122:6801/2613596984","heartbeat_back_addr":"172.21.15.122:6803/2613596984","heartbeat_front_addr":"172.21.15.122:6802/2613596984","state":["exists","up"]},{"osd":1,"uuid":"6fbfe9e1-4255-4c44-b619-5fd57486e5fe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":26,"up_from":41,"up_thru":64,"down_at":27,"lost_at":27,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6808","nonce":3142337928}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6809","nonce":3142337928}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6811","nonce":3142337928}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6810","nonce":3142337928}]},"public_addr":"172.21.15.122:6808/3142337928","cluster_addr":"172.21.15.122:6809/3142337928","heartbeat_back_addr":"172.21.15.122:6811/3142337928","heartbeat_front_addr":"172.21.15.122:6810/3142337928","state":["exists","up"]},{"osd":2,"uuid":"e27771fd-ac9d-4a64-a01f-465a5acade91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":8,"last_clean_end":17,"up_from":31,"up_thru":56,"down_at":19,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6804","nonce":1506556606}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6805","nonce":1506556606}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6807","nonce":1506556606}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.122:6806","nonce":1506556606}]},"public_addr":"172.21.15.122:6804/1506556606","cluster_addr":"172.21.15.122:6805/1506556606","heartbeat_back_addr":"172.21.15.122:6807/1506556606","heartbeat_front_addr":"172.21.15.122:6806/1506556606","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:11:12.059865+0000","dead_epoch":22},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:11:11.519118+0000","dead_epoch":27},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-11T01:11:11.402743+0000","dead_epoch":19}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-11T01:27:15.032 INFO:tasks.ceph:Scrubbing osd.0 2024-09-11T01:27:15.032 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-11T01:27:15.184 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-11T01:27:15.184 INFO:teuthology.orchestra.run.smithi122.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-11T01:27:15.184 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-11T01:27:15.199 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-11T01:27:15.468 INFO:teuthology.orchestra.run.smithi122.stderr:instructed osd(s) 0 to deep-scrub 2024-09-11T01:27:15.485 INFO:tasks.ceph:Scrubbing osd.1 2024-09-11T01:27:15.485 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-11T01:27:15.628 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-11T01:27:15.629 INFO:teuthology.orchestra.run.smithi122.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-11T01:27:15.629 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-11T01:27:15.644 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-11T01:27:15.940 INFO:teuthology.orchestra.run.smithi122.stderr:instructed osd(s) 1 to deep-scrub 2024-09-11T01:27:15.961 INFO:tasks.ceph:Scrubbing osd.2 2024-09-11T01:27:15.961 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-11T01:27:16.104 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-11T01:27:16.104 INFO:teuthology.orchestra.run.smithi122.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-11T01:27:16.104 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-11T01:27:16.119 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-11T01:27:16.411 INFO:teuthology.orchestra.run.smithi122.stderr:instructed osd(s) 2 to deep-scrub 2024-09-11T01:27:16.430 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:16.705 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:27:16.706 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:27:16.721 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":546,"stamp":"2024-09-11T01:27:14.889593+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47640,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160839,"ondisk_log_size":160839,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1419936,"kb_used_data":99184,"kb_used_omap":12,"kb_used_meta":1320691,"kb_avail":281695584,"statfs":{"total":289910292480,"available":288456278016,"internally_reserved":0,"allocated":101564416,"data_stored":100437916,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1352387588},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-440,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":440,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.381303"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28544,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.524154+0000","last_change":"2024-09-11T01:23:45.031717+0000","last_active":"2024-09-11T01:27:07.524154+0000","last_peered":"2024-09-11T01:27:07.524154+0000","last_clean":"2024-09-11T01:27:07.524154+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:27:07.524154+0000","last_undegraded":"2024-09-11T01:27:07.524154+0000","last_fullsized":"2024-09-11T01:27:07.524154+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:41:44.990278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20502,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.523059+0000","last_change":"2024-09-11T01:12:36.757293+0000","last_active":"2024-09-11T01:27:07.523059+0000","last_peered":"2024-09-11T01:27:07.523059+0000","last_clean":"2024-09-11T01:27:07.523059+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:27:07.523059+0000","last_undegraded":"2024-09-11T01:27:07.523059+0000","last_fullsized":"2024-09-11T01:27:07.523059+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1801","last_scrub_stamp":"2024-09-11T01:12:36.757173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:36.757173+0000","objects_scrubbed":1755,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:50:11.113841+0000","scrub_duration":831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20325,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528271+0000","last_change":"2024-09-11T01:22:56.148638+0000","last_active":"2024-09-11T01:27:07.528271+0000","last_peered":"2024-09-11T01:27:07.528271+0000","last_clean":"2024-09-11T01:27:07.528271+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:27:07.528271+0000","last_undegraded":"2024-09-11T01:27:07.528271+0000","last_fullsized":"2024-09-11T01:27:07.528271+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'1723","last_scrub_stamp":"2024-09-11T01:12:37.612157+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:37.612157+0000","objects_scrubbed":1695,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:35:26.995657+0000","scrub_duration":675,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20252,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.522347+0000","last_change":"2024-09-11T01:12:16.122313+0000","last_active":"2024-09-11T01:27:07.522347+0000","last_peered":"2024-09-11T01:27:07.522347+0000","last_clean":"2024-09-11T01:27:07.522347+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:27:07.522347+0000","last_undegraded":"2024-09-11T01:27:07.522347+0000","last_fullsized":"2024-09-11T01:27:07.522347+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:11:30.697577+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:37:54.534279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20185,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.522423+0000","last_change":"2024-09-11T01:12:52.565242+0000","last_active":"2024-09-11T01:27:07.522423+0000","last_peered":"2024-09-11T01:27:07.522423+0000","last_clean":"2024-09-11T01:27:07.522423+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:27:07.522423+0000","last_undegraded":"2024-09-11T01:27:07.522423+0000","last_fullsized":"2024-09-11T01:27:07.522423+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3045","last_scrub_stamp":"2024-09-11T01:12:52.561686+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:52.561686+0000","objects_scrubbed":2988,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:14:19.550588+0000","scrub_duration":1499,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20431,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.527154+0000","last_change":"2024-09-11T01:25:23.194180+0000","last_active":"2024-09-11T01:27:07.527154+0000","last_peered":"2024-09-11T01:27:07.527154+0000","last_clean":"2024-09-11T01:27:07.527154+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:27:07.527154+0000","last_undegraded":"2024-09-11T01:27:07.527154+0000","last_fullsized":"2024-09-11T01:27:07.527154+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"39'3723","last_scrub_stamp":"2024-09-11T01:13:04.086089+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:13:04.086089+0000","objects_scrubbed":3654,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:56:12.658900+0000","scrub_duration":1831,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20085,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.526901+0000","last_change":"2024-09-11T01:26:43.604424+0000","last_active":"2024-09-11T01:27:07.526901+0000","last_peered":"2024-09-11T01:27:07.526901+0000","last_clean":"2024-09-11T01:27:07.526901+0000","last_became_active":"2024-09-11T01:26:43.498430+0000","last_became_peered":"2024-09-11T01:26:43.498430+0000","last_unstale":"2024-09-11T01:27:07.526901+0000","last_undegraded":"2024-09-11T01:27:07.526901+0000","last_fullsized":"2024-09-11T01:27:07.526901+0000","mapping_epoch":64,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3190","last_scrub_stamp":"2024-09-11T01:12:54.410173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:54.410173+0000","objects_scrubbed":3137,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:27:13.180371+0000","scrub_duration":1319,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3988,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20433,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.521558+0000","last_change":"2024-09-11T01:12:53.413176+0000","last_active":"2024-09-11T01:27:07.521558+0000","last_peered":"2024-09-11T01:27:07.521558+0000","last_clean":"2024-09-11T01:27:07.521558+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:27:07.521558+0000","last_undegraded":"2024-09-11T01:27:07.521558+0000","last_fullsized":"2024-09-11T01:27:07.521558+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3171","last_scrub_stamp":"2024-09-11T01:12:53.412983+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:53.412983+0000","objects_scrubbed":3122,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:43:02.783440+0000","scrub_duration":1327,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.526903+0000","last_change":"2024-09-11T01:13:09.654544+0000","last_active":"2024-09-11T01:27:07.526903+0000","last_peered":"2024-09-11T01:27:07.526903+0000","last_clean":"2024-09-11T01:27:07.526903+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:27:07.526903+0000","last_undegraded":"2024-09-11T01:27:07.526903+0000","last_fullsized":"2024-09-11T01:27:07.526903+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:49:30.671205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28030,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.527075+0000","last_change":"2024-09-11T01:20:22.087726+0000","last_active":"2024-09-11T01:27:07.527075+0000","last_peered":"2024-09-11T01:27:07.527075+0000","last_clean":"2024-09-11T01:27:07.527075+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:27:07.527075+0000","last_undegraded":"2024-09-11T01:27:07.527075+0000","last_fullsized":"2024-09-11T01:27:07.527075+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2071","last_scrub_stamp":"2024-09-11T01:12:40.856413+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:40.856413+0000","objects_scrubbed":2022,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:17:51.303906+0000","scrub_duration":959,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.520946+0000","last_change":"2024-09-11T01:12:16.121120+0000","last_active":"2024-09-11T01:27:07.520946+0000","last_peered":"2024-09-11T01:27:07.520946+0000","last_clean":"2024-09-11T01:27:07.520946+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:27:07.520946+0000","last_undegraded":"2024-09-11T01:27:07.520946+0000","last_fullsized":"2024-09-11T01:27:07.520946+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:25.764542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":28025,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.625143+0000","last_change":"2024-09-11T01:27:07.625011+0000","last_active":"2024-09-11T01:27:07.625143+0000","last_peered":"2024-09-11T01:27:07.625143+0000","last_clean":"2024-09-11T01:27:07.625143+0000","last_became_active":"2024-09-11T01:27:07.529673+0000","last_became_peered":"2024-09-11T01:27:07.529673+0000","last_unstale":"2024-09-11T01:27:07.625143+0000","last_undegraded":"2024-09-11T01:27:07.625143+0000","last_fullsized":"2024-09-11T01:27:07.625143+0000","mapping_epoch":66,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3497","last_scrub_stamp":"2024-09-11T01:12:59.058452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:59.058452+0000","objects_scrubbed":3405,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:25:06.093676+0000","scrub_duration":1919,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3986,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"14'2","reported_seq":150,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.521775+0000","last_change":"2024-09-11T01:13:09.837379+0000","last_active":"2024-09-11T01:27:07.521775+0000","last_peered":"2024-09-11T01:27:07.521775+0000","last_clean":"2024-09-11T01:27:07.521775+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:27:07.521775+0000","last_undegraded":"2024-09-11T01:27:07.521775+0000","last_fullsized":"2024-09-11T01:27:07.521775+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:00:13.821154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28435,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.521118+0000","last_change":"2024-09-11T01:24:25.265582+0000","last_active":"2024-09-11T01:27:07.521118+0000","last_peered":"2024-09-11T01:27:07.521118+0000","last_clean":"2024-09-11T01:27:07.521118+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:27:07.521118+0000","last_undegraded":"2024-09-11T01:27:07.521118+0000","last_fullsized":"2024-09-11T01:27:07.521118+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2223","last_scrub_stamp":"2024-09-11T01:12:41.868472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:41.868472+0000","objects_scrubbed":2178,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:15:02.453263+0000","scrub_duration":963,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"12'1","reported_seq":128,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.527066+0000","last_change":"2024-09-11T01:13:09.768655+0000","last_active":"2024-09-11T01:27:07.527066+0000","last_peered":"2024-09-11T01:27:07.527066+0000","last_clean":"2024-09-11T01:27:07.527066+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:27:07.527066+0000","last_undegraded":"2024-09-11T01:27:07.527066+0000","last_fullsized":"2024-09-11T01:27:07.527066+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.613586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27826,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.520647+0000","last_change":"2024-09-11T01:15:21.714804+0000","last_active":"2024-09-11T01:27:07.520647+0000","last_peered":"2024-09-11T01:27:07.520647+0000","last_clean":"2024-09-11T01:27:07.520647+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:27:07.520647+0000","last_undegraded":"2024-09-11T01:27:07.520647+0000","last_fullsized":"2024-09-11T01:27:07.520647+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2269","last_scrub_stamp":"2024-09-11T01:12:42.890920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:42.890920+0000","objects_scrubbed":2209,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T09:44:44.241620+0000","scrub_duration":999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20436,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528636+0000","last_change":"2024-09-11T01:25:03.115792+0000","last_active":"2024-09-11T01:27:07.528636+0000","last_peered":"2024-09-11T01:27:07.528636+0000","last_clean":"2024-09-11T01:27:07.528636+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:27:07.528636+0000","last_undegraded":"2024-09-11T01:27:07.528636+0000","last_fullsized":"2024-09-11T01:27:07.528636+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3338","last_scrub_stamp":"2024-09-11T01:12:56.185967+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.185967+0000","objects_scrubbed":3261,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":3,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:22:39.274895+0000","scrub_duration":2055,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528226+0000","last_change":"2024-09-11T01:13:09.902847+0000","last_active":"2024-09-11T01:27:07.528226+0000","last_peered":"2024-09-11T01:27:07.528226+0000","last_clean":"2024-09-11T01:27:07.528226+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:27:07.528226+0000","last_undegraded":"2024-09-11T01:27:07.528226+0000","last_fullsized":"2024-09-11T01:27:07.528226+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T07:55:42.611614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27964,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528314+0000","last_change":"2024-09-11T01:17:58.971568+0000","last_active":"2024-09-11T01:27:07.528314+0000","last_peered":"2024-09-11T01:27:07.528314+0000","last_clean":"2024-09-11T01:27:07.528314+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:27:07.528314+0000","last_undegraded":"2024-09-11T01:27:07.528314+0000","last_fullsized":"2024-09-11T01:27:07.528314+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3415","last_scrub_stamp":"2024-09-11T01:12:58.109581+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:58.109581+0000","objects_scrubbed":3337,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:07:35.860176+0000","scrub_duration":1975,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":119,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.522796+0000","last_change":"2024-09-11T01:12:16.123473+0000","last_active":"2024-09-11T01:27:07.522796+0000","last_peered":"2024-09-11T01:27:07.522796+0000","last_clean":"2024-09-11T01:27:07.522796+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:27:07.522796+0000","last_undegraded":"2024-09-11T01:27:07.522796+0000","last_fullsized":"2024-09-11T01:27:07.522796+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:24:22.115367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24643,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528300+0000","last_change":"2024-09-11T01:22:37.025140+0000","last_active":"2024-09-11T01:27:07.528300+0000","last_peered":"2024-09-11T01:27:07.528300+0000","last_clean":"2024-09-11T01:27:07.528300+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:27:07.528300+0000","last_undegraded":"2024-09-11T01:27:07.528300+0000","last_fullsized":"2024-09-11T01:27:07.528300+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'3438","last_scrub_stamp":"2024-09-11T01:12:56.824758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:56.824758+0000","objects_scrubbed":3387,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:30:31.055292+0000","scrub_duration":1663,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":118,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528259+0000","last_change":"2024-09-11T01:13:09.905370+0000","last_active":"2024-09-11T01:27:07.528259+0000","last_peered":"2024-09-11T01:27:07.528259+0000","last_clean":"2024-09-11T01:27:07.528259+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:27:07.528259+0000","last_undegraded":"2024-09-11T01:27:07.528259+0000","last_fullsized":"2024-09-11T01:27:07.528259+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:33:59.243460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":123,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.528414+0000","last_change":"2024-09-11T01:13:09.661009+0000","last_active":"2024-09-11T01:27:07.528414+0000","last_peered":"2024-09-11T01:27:07.528414+0000","last_clean":"2024-09-11T01:27:07.528414+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:27:07.528414+0000","last_undegraded":"2024-09-11T01:27:07.528414+0000","last_fullsized":"2024-09-11T01:27:07.528414+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:19.623673+0000","last_clean_scrub_stamp":"2024-09-11T01:11:19.623673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:03:25.964038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"49'19952","reported_seq":20115,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:07.527118+0000","last_change":"2024-09-11T01:26:03.078051+0000","last_active":"2024-09-11T01:27:07.527118+0000","last_peered":"2024-09-11T01:27:07.527118+0000","last_clean":"2024-09-11T01:27:07.527118+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:27:07.527118+0000","last_undegraded":"2024-09-11T01:27:07.527118+0000","last_fullsized":"2024-09-11T01:27:07.527118+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'2993","last_scrub_stamp":"2024-09-11T01:12:51.738891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:11:30.697577+0000","last_clean_scrub_stamp":"2024-09-11T01:12:51.738891+0000","objects_scrubbed":2919,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:28:57.939768+0000","scrub_duration":1627,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47639,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":99270656,"data_stored":99260219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160836,"ondisk_log_size":160836,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182589,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":515368,"kb_used_data":60312,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93856472,"statfs":{"total":96636764160,"available":96109027328,"internally_reserved":0,"allocated":61759488,"data_stored":61382629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986361,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":403136,"kb_used_data":38320,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93968704,"statfs":{"total":96636764160,"available":96223952896,"internally_reserved":0,"allocated":39239680,"data_stored":38866898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659305,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501432,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":500859,"kb_avail":93870408,"statfs":{"total":96636764160,"available":96123297792,"internally_reserved":0,"allocated":565248,"data_stored":188389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512880303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":60706816,"data_stored":60703337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":38559744,"data_stored":38556265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:16.726 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2024-09-11T01:11:30.697577+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=11, tm_sec=30, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=27, tm_sec=14, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:27:16.726 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2024-09-11T01:12:36.757173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=12, tm_sec=36, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=27, tm_sec=14, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:27:16.726 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2024-09-11T01:12:37.612157+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=12, tm_sec=37, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=27, tm_sec=14, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:27:16.726 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2024-09-11T01:11:30.697577+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=11, tm_sec=30, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=27, tm_sec=14, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:27:16.726 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2024-09-11T01:12:52.561686+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=12, tm_sec=52, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=27, tm_sec=14, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:27:16.727 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2024-09-11T01:13:04.086089+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=13, tm_sec=4, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=27, tm_sec=14, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:27:16.727 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2024-09-11T01:12:54.410173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=12, tm_sec=54, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=27, tm_sec=14, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:27:16.727 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2024-09-11T01:12:53.412983+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=12, tm_sec=53, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=27, tm_sec=14, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:27:16.727 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-11T01:11:19.623673+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=11, tm_sec=19, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=27, tm_sec=14, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:27:16.727 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-11T01:12:40.856413+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=12, tm_sec=40, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=27, tm_sec=14, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:27:16.727 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2024-09-11T01:11:19.623673+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=11, tm_sec=19, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=27, tm_sec=14, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:27:16.727 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-09-11T01:12:59.058452+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=12, tm_sec=59, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=27, tm_sec=14, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:27:16.727 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2024-09-11T01:11:19.623673+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=11, tm_sec=19, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=27, tm_sec=14, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:27:16.728 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-11T01:12:41.868472+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=12, tm_sec=41, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=27, tm_sec=14, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:27:16.728 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2024-09-11T01:11:19.623673+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=11, tm_sec=19, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=27, tm_sec=14, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:27:16.728 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-11T01:12:42.890920+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=12, tm_sec=42, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=27, tm_sec=14, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:27:16.728 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-11T01:12:56.185967+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=12, tm_sec=56, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=27, tm_sec=14, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:27:16.728 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2024-09-11T01:11:19.623673+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=11, tm_sec=19, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=27, tm_sec=14, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:27:16.728 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-11T01:12:58.109581+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=12, tm_sec=58, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=27, tm_sec=14, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:27:16.728 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2024-09-11T01:11:19.623673+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=11, tm_sec=19, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=27, tm_sec=14, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:27:16.729 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-11T01:12:56.824758+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=12, tm_sec=56, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=27, tm_sec=14, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:27:16.729 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2024-09-11T01:11:19.623673+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=11, tm_sec=19, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=27, tm_sec=14, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:27:16.729 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2024-09-11T01:11:19.623673+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=11, tm_sec=19, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=27, tm_sec=14, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:27:16.729 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-11T01:12:51.738891+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=12, tm_sec=51, tm_wday=2, tm_yday=255, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=11, tm_hour=1, tm_min=27, tm_sec=14, tm_wday=2, tm_yday=255, tm_isdst=0) 2024-09-11T01:27:16.729 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-11T01:27:36.730 DEBUG:teuthology.orchestra.run.smithi122:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-11T01:27:37.041 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-11T01:27:37.041 INFO:teuthology.orchestra.run.smithi122.stderr:dumped all 2024-09-11T01:27:37.058 INFO:teuthology.orchestra.run.smithi122.stdout:{"pg_ready":true,"pg_map":{"version":557,"stamp":"2024-09-11T01:27:36.895460+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":636,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282288,"num_write_kb":486435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47640,"num_bytes_recovered":36720659,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160842,"ondisk_log_size":160842,"up":48,"acting":48,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":56,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":1400532,"kb_used_data":79780,"kb_used_omap":12,"kb_used_meta":1320691,"kb_avail":281714988,"statfs":{"total":289910292480,"available":288476147712,"internally_reserved":0,"allocated":81694720,"data_stored":80571699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13308,"internal_metadata":1352387588},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003206"},"pg_stats":[{"pgid":"2.f","version":"49'20266","reported_seq":28552,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:20.197750+0000","last_change":"2024-09-11T01:27:20.197750+0000","last_active":"2024-09-11T01:27:20.197750+0000","last_peered":"2024-09-11T01:27:20.197750+0000","last_clean":"2024-09-11T01:27:20.197750+0000","last_became_active":"2024-09-11T01:23:44.971128+0000","last_became_peered":"2024-09-11T01:23:44.971128+0000","last_unstale":"2024-09-11T01:27:20.197750+0000","last_undegraded":"2024-09-11T01:27:20.197750+0000","last_fullsized":"2024-09-11T01:27:20.197750+0000","mapping_epoch":54,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":55,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'20266","last_scrub_stamp":"2024-09-11T01:27:20.197574+0000","last_deep_scrub":"49'20266","last_deep_scrub_stamp":"2024-09-11T01:27:20.197574+0000","last_clean_scrub_stamp":"2024-09-11T01:27:20.197574+0000","objects_scrubbed":0,"log_size":10066,"log_dups_size":0,"ondisk_log_size":10066,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:41:23.303104+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20266,"num_write_kb":40532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4072,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.e","version":"49'20352","reported_seq":20510,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:19.840089+0000","last_change":"2024-09-11T01:27:19.840089+0000","last_active":"2024-09-11T01:27:19.840089+0000","last_peered":"2024-09-11T01:27:19.840089+0000","last_clean":"2024-09-11T01:27:19.840089+0000","last_became_active":"2024-09-11T01:12:16.122624+0000","last_became_peered":"2024-09-11T01:12:16.122624+0000","last_unstale":"2024-09-11T01:27:19.840089+0000","last_undegraded":"2024-09-11T01:27:19.840089+0000","last_fullsized":"2024-09-11T01:27:19.840089+0000","mapping_epoch":35,"log_start":"45'10300","ondisk_log_start":"45'10300","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'20352","last_scrub_stamp":"2024-09-11T01:27:19.839218+0000","last_deep_scrub":"49'20352","last_deep_scrub_stamp":"2024-09-11T01:27:19.839218+0000","last_clean_scrub_stamp":"2024-09-11T01:27:19.839218+0000","objects_scrubbed":0,"log_size":10052,"log_dups_size":0,"ondisk_log_size":10052,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:19:19.673851+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18591,"num_write_kb":33672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.d","version":"49'20144","reported_seq":20334,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:28.098739+0000","last_change":"2024-09-11T01:27:28.098596+0000","last_active":"2024-09-11T01:27:28.098739+0000","last_peered":"2024-09-11T01:27:28.098739+0000","last_clean":"2024-09-11T01:27:28.098739+0000","last_became_active":"2024-09-11T01:22:56.007832+0000","last_became_peered":"2024-09-11T01:22:56.007832+0000","last_unstale":"2024-09-11T01:27:28.098739+0000","last_undegraded":"2024-09-11T01:27:28.098739+0000","last_fullsized":"2024-09-11T01:27:28.098739+0000","mapping_epoch":52,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'20144","last_scrub_stamp":"2024-09-11T01:27:28.098409+0000","last_deep_scrub":"49'20144","last_deep_scrub_stamp":"2024-09-11T01:27:28.098409+0000","last_clean_scrub_stamp":"2024-09-11T01:27:28.098409+0000","objects_scrubbed":0,"log_size":10044,"log_dups_size":0,"ondisk_log_size":10044,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:01:44.964579+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":18445,"num_write_kb":33497,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3898,"num_bytes_recovered":4096,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"49'20134","reported_seq":20260,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:20.833405+0000","last_change":"2024-09-11T01:27:20.833405+0000","last_active":"2024-09-11T01:27:20.833405+0000","last_peered":"2024-09-11T01:27:20.833405+0000","last_clean":"2024-09-11T01:27:20.833405+0000","last_became_active":"2024-09-11T01:12:16.122038+0000","last_became_peered":"2024-09-11T01:12:16.122038+0000","last_unstale":"2024-09-11T01:27:20.833405+0000","last_undegraded":"2024-09-11T01:27:20.833405+0000","last_fullsized":"2024-09-11T01:27:20.833405+0000","mapping_epoch":35,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'20134","last_scrub_stamp":"2024-09-11T01:27:20.833205+0000","last_deep_scrub":"49'20134","last_deep_scrub_stamp":"2024-09-11T01:27:20.833205+0000","last_clean_scrub_stamp":"2024-09-11T01:27:20.833205+0000","objects_scrubbed":0,"log_size":10034,"log_dups_size":0,"ondisk_log_size":10034,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:58:00.452434+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20134,"num_write_kb":40268,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.b","version":"49'20009","reported_seq":20194,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:19.243851+0000","last_change":"2024-09-11T01:27:19.243703+0000","last_active":"2024-09-11T01:27:19.243851+0000","last_peered":"2024-09-11T01:27:19.243851+0000","last_clean":"2024-09-11T01:27:19.243851+0000","last_became_active":"2024-09-11T01:12:15.134191+0000","last_became_peered":"2024-09-11T01:12:15.134191+0000","last_unstale":"2024-09-11T01:27:19.243851+0000","last_undegraded":"2024-09-11T01:27:19.243851+0000","last_fullsized":"2024-09-11T01:27:19.243851+0000","mapping_epoch":34,"log_start":"45'10000","ondisk_log_start":"45'10000","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'20009","last_scrub_stamp":"2024-09-11T01:27:19.243513+0000","last_deep_scrub":"49'20009","last_deep_scrub_stamp":"2024-09-11T01:27:19.243513+0000","last_clean_scrub_stamp":"2024-09-11T01:27:19.243513+0000","objects_scrubbed":0,"log_size":10009,"log_dups_size":0,"ondisk_log_size":10009,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:22:40.697980+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17014,"num_write_kb":28052,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.a","version":"49'20227","reported_seq":20440,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:26.075674+0000","last_change":"2024-09-11T01:27:26.075510+0000","last_active":"2024-09-11T01:27:26.075674+0000","last_peered":"2024-09-11T01:27:26.075674+0000","last_clean":"2024-09-11T01:27:26.075674+0000","last_became_active":"2024-09-11T01:25:23.088248+0000","last_became_peered":"2024-09-11T01:25:23.088248+0000","last_unstale":"2024-09-11T01:27:26.075674+0000","last_undegraded":"2024-09-11T01:27:26.075674+0000","last_fullsized":"2024-09-11T01:27:26.075674+0000","mapping_epoch":60,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'20227","last_scrub_stamp":"2024-09-11T01:27:26.075284+0000","last_deep_scrub":"49'20227","last_deep_scrub_stamp":"2024-09-11T01:27:26.075284+0000","last_clean_scrub_stamp":"2024-09-11T01:27:26.075284+0000","objects_scrubbed":0,"log_size":10027,"log_dups_size":0,"ondisk_log_size":10027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:50:49.827113+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16568,"num_write_kb":25828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3953,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"49'19930","reported_seq":20093,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:25.104951+0000","last_change":"2024-09-11T01:27:25.104951+0000","last_active":"2024-09-11T01:27:25.104951+0000","last_peered":"2024-09-11T01:27:25.104951+0000","last_clean":"2024-09-11T01:27:25.104951+0000","last_became_active":"2024-09-11T01:26:43.498430+0000","last_became_peered":"2024-09-11T01:26:43.498430+0000","last_unstale":"2024-09-11T01:27:25.104951+0000","last_undegraded":"2024-09-11T01:27:25.104951+0000","last_fullsized":"2024-09-11T01:27:25.104951+0000","mapping_epoch":64,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'19930","last_scrub_stamp":"2024-09-11T01:27:25.104764+0000","last_deep_scrub":"49'19930","last_deep_scrub_stamp":"2024-09-11T01:27:25.104764+0000","last_clean_scrub_stamp":"2024-09-11T01:27:25.104764+0000","objects_scrubbed":0,"log_size":10030,"log_dups_size":0,"ondisk_log_size":10030,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:48:34.799906+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16787,"num_write_kb":27300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3988,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.8","version":"49'20254","reported_seq":20441,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:18.229128+0000","last_change":"2024-09-11T01:27:18.229128+0000","last_active":"2024-09-11T01:27:18.229128+0000","last_peered":"2024-09-11T01:27:18.229128+0000","last_clean":"2024-09-11T01:27:18.229128+0000","last_became_active":"2024-09-11T01:12:15.133806+0000","last_became_peered":"2024-09-11T01:12:15.133806+0000","last_unstale":"2024-09-11T01:27:18.229128+0000","last_undegraded":"2024-09-11T01:27:18.229128+0000","last_fullsized":"2024-09-11T01:27:18.229128+0000","mapping_epoch":34,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'20254","last_scrub_stamp":"2024-09-11T01:27:18.228925+0000","last_deep_scrub":"49'20254","last_deep_scrub_stamp":"2024-09-11T01:27:18.228925+0000","last_clean_scrub_stamp":"2024-09-11T01:27:18.228925+0000","objects_scrubbed":0,"log_size":10054,"log_dups_size":0,"ondisk_log_size":10054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:29:31.869354+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17130,"num_write_kb":28016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":127,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:16.136018+0000","last_change":"2024-09-11T01:27:16.136018+0000","last_active":"2024-09-11T01:27:16.136018+0000","last_peered":"2024-09-11T01:27:16.136018+0000","last_clean":"2024-09-11T01:27:16.136018+0000","last_became_active":"2024-09-11T01:13:09.653961+0000","last_became_peered":"2024-09-11T01:13:09.653961+0000","last_unstale":"2024-09-11T01:27:16.136018+0000","last_undegraded":"2024-09-11T01:27:16.136018+0000","last_fullsized":"2024-09-11T01:27:16.136018+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:27:16.135851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:27:16.135851+0000","last_clean_scrub_stamp":"2024-09-11T01:27:16.135851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T10:05:14.098638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001253164,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"49'19973","reported_seq":28038,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:21.171769+0000","last_change":"2024-09-11T01:27:21.171769+0000","last_active":"2024-09-11T01:27:21.171769+0000","last_peered":"2024-09-11T01:27:21.171769+0000","last_clean":"2024-09-11T01:27:21.171769+0000","last_became_active":"2024-09-11T01:20:22.015498+0000","last_became_peered":"2024-09-11T01:20:22.015498+0000","last_unstale":"2024-09-11T01:27:21.171769+0000","last_undegraded":"2024-09-11T01:27:21.171769+0000","last_fullsized":"2024-09-11T01:27:21.171769+0000","mapping_epoch":48,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'19973","last_scrub_stamp":"2024-09-11T01:27:21.171581+0000","last_deep_scrub":"49'19973","last_deep_scrub_stamp":"2024-09-11T01:27:21.171581+0000","last_clean_scrub_stamp":"2024-09-11T01:27:21.171581+0000","objects_scrubbed":0,"log_size":10073,"log_dups_size":0,"ondisk_log_size":10073,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:35:03.102622+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17944,"num_write_kb":31844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3935,"num_bytes_recovered":4505600,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":127,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:17.844709+0000","last_change":"2024-09-11T01:27:17.844709+0000","last_active":"2024-09-11T01:27:17.844709+0000","last_peered":"2024-09-11T01:27:17.844709+0000","last_clean":"2024-09-11T01:27:17.844709+0000","last_became_active":"2024-09-11T01:12:16.120834+0000","last_became_peered":"2024-09-11T01:12:16.120834+0000","last_unstale":"2024-09-11T01:27:17.844709+0000","last_undegraded":"2024-09-11T01:27:17.844709+0000","last_fullsized":"2024-09-11T01:27:17.844709+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:27:17.844522+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:27:17.844522+0000","last_clean_scrub_stamp":"2024-09-11T01:27:17.844522+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:50:50.296545+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"49'19868","reported_seq":28033,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:17.272824+0000","last_change":"2024-09-11T01:27:17.272824+0000","last_active":"2024-09-11T01:27:17.272824+0000","last_peered":"2024-09-11T01:27:17.272824+0000","last_clean":"2024-09-11T01:27:17.272824+0000","last_became_active":"2024-09-11T01:27:07.529673+0000","last_became_peered":"2024-09-11T01:27:07.529673+0000","last_unstale":"2024-09-11T01:27:17.272824+0000","last_undegraded":"2024-09-11T01:27:17.272824+0000","last_fullsized":"2024-09-11T01:27:17.272824+0000","mapping_epoch":66,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'19868","last_scrub_stamp":"2024-09-11T01:27:17.272640+0000","last_deep_scrub":"49'19868","last_deep_scrub_stamp":"2024-09-11T01:27:17.272640+0000","last_clean_scrub_stamp":"2024-09-11T01:27:17.272640+0000","objects_scrubbed":0,"log_size":10068,"log_dups_size":0,"ondisk_log_size":10068,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:55:31.192413+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16455,"num_write_kb":26100,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3986,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.2","version":"67'3","reported_seq":159,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:16.265800+0000","last_change":"2024-09-11T01:27:16.265800+0000","last_active":"2024-09-11T01:27:16.265800+0000","last_peered":"2024-09-11T01:27:16.265800+0000","last_clean":"2024-09-11T01:27:16.265800+0000","last_became_active":"2024-09-11T01:13:09.836225+0000","last_became_peered":"2024-09-11T01:13:09.836225+0000","last_unstale":"2024-09-11T01:27:16.265800+0000","last_undegraded":"2024-09-11T01:27:16.265800+0000","last_fullsized":"2024-09-11T01:27:16.265800+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'3","last_scrub_stamp":"2024-09-11T01:27:16.265691+0000","last_deep_scrub":"67'3","last_deep_scrub_stamp":"2024-09-11T01:27:16.265691+0000","last_clean_scrub_stamp":"2024-09-11T01:27:16.265691+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:55:52.721455+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0019910449999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"49'20207","reported_seq":28443,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:18.852450+0000","last_change":"2024-09-11T01:27:18.852450+0000","last_active":"2024-09-11T01:27:18.852450+0000","last_peered":"2024-09-11T01:27:18.852450+0000","last_clean":"2024-09-11T01:27:18.852450+0000","last_became_active":"2024-09-11T01:24:25.143929+0000","last_became_peered":"2024-09-11T01:24:25.143929+0000","last_unstale":"2024-09-11T01:27:18.852450+0000","last_undegraded":"2024-09-11T01:27:18.852450+0000","last_fullsized":"2024-09-11T01:27:18.852450+0000","mapping_epoch":56,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":57,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'20207","last_scrub_stamp":"2024-09-11T01:27:18.852248+0000","last_deep_scrub":"49'20207","last_deep_scrub_stamp":"2024-09-11T01:27:18.852248+0000","last_clean_scrub_stamp":"2024-09-11T01:27:18.852248+0000","objects_scrubbed":0,"log_size":10007,"log_dups_size":0,"ondisk_log_size":10007,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:24:34.650768+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18024,"num_write_kb":31692,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4026,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.3","version":"67'2","reported_seq":137,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:19.135289+0000","last_change":"2024-09-11T01:27:19.135289+0000","last_active":"2024-09-11T01:27:19.135289+0000","last_peered":"2024-09-11T01:27:19.135289+0000","last_clean":"2024-09-11T01:27:19.135289+0000","last_became_active":"2024-09-11T01:13:09.768265+0000","last_became_peered":"2024-09-11T01:13:09.768265+0000","last_unstale":"2024-09-11T01:27:19.135289+0000","last_undegraded":"2024-09-11T01:27:19.135289+0000","last_fullsized":"2024-09-11T01:27:19.135289+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'2","last_scrub_stamp":"2024-09-11T01:27:19.135123+0000","last_deep_scrub":"67'2","last_deep_scrub_stamp":"2024-09-11T01:27:19.135123+0000","last_clean_scrub_stamp":"2024-09-11T01:27:19.135123+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T11:53:29.865192+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0029311039999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"49'19796","reported_seq":27834,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:21.832944+0000","last_change":"2024-09-11T01:27:21.832944+0000","last_active":"2024-09-11T01:27:21.832944+0000","last_peered":"2024-09-11T01:27:21.832944+0000","last_clean":"2024-09-11T01:27:21.832944+0000","last_became_active":"2024-09-11T01:15:21.701284+0000","last_became_peered":"2024-09-11T01:15:21.701284+0000","last_unstale":"2024-09-11T01:27:21.832944+0000","last_undegraded":"2024-09-11T01:27:21.832944+0000","last_fullsized":"2024-09-11T01:27:21.832944+0000","mapping_epoch":44,"log_start":"45'9700","ondisk_log_start":"45'9700","created":16,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'19796","last_scrub_stamp":"2024-09-11T01:27:21.832759+0000","last_deep_scrub":"49'19796","last_deep_scrub_stamp":"2024-09-11T01:27:21.832759+0000","last_clean_scrub_stamp":"2024-09-11T01:27:21.832759+0000","objects_scrubbed":0,"log_size":10096,"log_dups_size":0,"ondisk_log_size":10096,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:30:31.842248+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17581,"num_write_kb":30744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3927,"num_bytes_recovered":16076800,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"49'20264","reported_seq":20444,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:24.127468+0000","last_change":"2024-09-11T01:27:24.127468+0000","last_active":"2024-09-11T01:27:24.127468+0000","last_peered":"2024-09-11T01:27:24.127468+0000","last_clean":"2024-09-11T01:27:24.127468+0000","last_became_active":"2024-09-11T01:25:03.018163+0000","last_became_peered":"2024-09-11T01:25:03.018163+0000","last_unstale":"2024-09-11T01:27:24.127468+0000","last_undegraded":"2024-09-11T01:27:24.127468+0000","last_fullsized":"2024-09-11T01:27:24.127468+0000","mapping_epoch":58,"log_start":"45'10200","ondisk_log_start":"45'10200","created":16,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'20264","last_scrub_stamp":"2024-09-11T01:27:24.127292+0000","last_deep_scrub":"49'20264","last_deep_scrub_stamp":"2024-09-11T01:27:24.127292+0000","last_clean_scrub_stamp":"2024-09-11T01:27:24.127292+0000","objects_scrubbed":0,"log_size":10064,"log_dups_size":0,"ondisk_log_size":10064,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:47:06.645313+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17001,"num_write_kb":27480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3951,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":126,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:17.088075+0000","last_change":"2024-09-11T01:27:17.088075+0000","last_active":"2024-09-11T01:27:17.088075+0000","last_peered":"2024-09-11T01:27:17.088075+0000","last_clean":"2024-09-11T01:27:17.088075+0000","last_became_active":"2024-09-11T01:13:09.902514+0000","last_became_peered":"2024-09-11T01:13:09.902514+0000","last_unstale":"2024-09-11T01:27:17.088075+0000","last_undegraded":"2024-09-11T01:27:17.088075+0000","last_fullsized":"2024-09-11T01:27:17.088075+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:27:17.087879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:27:17.087879+0000","last_clean_scrub_stamp":"2024-09-11T01:27:17.087879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:38.435762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0023811589999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"49'19884","reported_seq":27972,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:27.057995+0000","last_change":"2024-09-11T01:27:27.057995+0000","last_active":"2024-09-11T01:27:27.057995+0000","last_peered":"2024-09-11T01:27:27.057995+0000","last_clean":"2024-09-11T01:27:27.057995+0000","last_became_active":"2024-09-11T01:17:58.936300+0000","last_became_peered":"2024-09-11T01:17:58.936300+0000","last_unstale":"2024-09-11T01:27:27.057995+0000","last_undegraded":"2024-09-11T01:27:27.057995+0000","last_fullsized":"2024-09-11T01:27:27.057995+0000","mapping_epoch":46,"log_start":"45'9800","ondisk_log_start":"45'9800","created":16,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'19884","last_scrub_stamp":"2024-09-11T01:27:27.057793+0000","last_deep_scrub":"49'19884","last_deep_scrub_stamp":"2024-09-11T01:27:27.057793+0000","last_clean_scrub_stamp":"2024-09-11T01:27:27.057793+0000","objects_scrubbed":0,"log_size":10084,"log_dups_size":0,"ondisk_log_size":10084,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:59:27.125032+0000","scrub_duration":23,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16539,"num_write_kb":26404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3942,"num_bytes_recovered":16134144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":127,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:16.822478+0000","last_change":"2024-09-11T01:27:16.822478+0000","last_active":"2024-09-11T01:27:16.822478+0000","last_peered":"2024-09-11T01:27:16.822478+0000","last_clean":"2024-09-11T01:27:16.822478+0000","last_became_active":"2024-09-11T01:12:16.122962+0000","last_became_peered":"2024-09-11T01:12:16.122962+0000","last_unstale":"2024-09-11T01:27:16.822478+0000","last_undegraded":"2024-09-11T01:27:16.822478+0000","last_fullsized":"2024-09-11T01:27:16.822478+0000","mapping_epoch":35,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:27:16.822315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:27:16.822315+0000","last_clean_scrub_stamp":"2024-09-11T01:27:16.822315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:57:38.349348+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"49'20176","reported_seq":24651,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:23.125816+0000","last_change":"2024-09-11T01:27:23.125816+0000","last_active":"2024-09-11T01:27:23.125816+0000","last_peered":"2024-09-11T01:27:23.125816+0000","last_clean":"2024-09-11T01:27:23.125816+0000","last_became_active":"2024-09-11T01:22:36.929087+0000","last_became_peered":"2024-09-11T01:22:36.929087+0000","last_unstale":"2024-09-11T01:27:23.125816+0000","last_undegraded":"2024-09-11T01:27:23.125816+0000","last_fullsized":"2024-09-11T01:27:23.125816+0000","mapping_epoch":50,"log_start":"45'10100","ondisk_log_start":"45'10100","created":16,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"49'20176","last_scrub_stamp":"2024-09-11T01:27:23.125645+0000","last_deep_scrub":"49'20176","last_deep_scrub_stamp":"2024-09-11T01:27:23.125645+0000","last_clean_scrub_stamp":"2024-09-11T01:27:23.125645+0000","objects_scrubbed":0,"log_size":10076,"log_dups_size":0,"ondisk_log_size":10076,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:57:14.387964+0000","scrub_duration":19,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16785,"num_write_kb":26796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4041,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":126,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:18.090435+0000","last_change":"2024-09-11T01:27:18.090435+0000","last_active":"2024-09-11T01:27:18.090435+0000","last_peered":"2024-09-11T01:27:18.090435+0000","last_clean":"2024-09-11T01:27:18.090435+0000","last_became_active":"2024-09-11T01:13:09.903189+0000","last_became_peered":"2024-09-11T01:13:09.903189+0000","last_unstale":"2024-09-11T01:27:18.090435+0000","last_undegraded":"2024-09-11T01:27:18.090435+0000","last_fullsized":"2024-09-11T01:27:18.090435+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:27:18.090132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:27:18.090132+0000","last_clean_scrub_stamp":"2024-09-11T01:27:18.090132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:46:00.686516+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0023584980000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":131,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:20.182725+0000","last_change":"2024-09-11T01:27:20.182725+0000","last_active":"2024-09-11T01:27:20.182725+0000","last_peered":"2024-09-11T01:27:20.182725+0000","last_clean":"2024-09-11T01:27:20.182725+0000","last_became_active":"2024-09-11T01:13:09.660220+0000","last_became_peered":"2024-09-11T01:13:09.660220+0000","last_unstale":"2024-09-11T01:27:20.182725+0000","last_undegraded":"2024-09-11T01:27:20.182725+0000","last_fullsized":"2024-09-11T01:27:20.182725+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-11T01:27:20.182542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-11T01:27:20.182542+0000","last_clean_scrub_stamp":"2024-09-11T01:27:20.182542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T12:42:37.045234+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.0014651060000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"67'19953","reported_seq":20124,"reported_epoch":67,"state":"active+clean","last_fresh":"2024-09-11T01:27:22.144622+0000","last_change":"2024-09-11T01:27:22.144622+0000","last_active":"2024-09-11T01:27:22.144622+0000","last_peered":"2024-09-11T01:27:22.144622+0000","last_clean":"2024-09-11T01:27:22.144622+0000","last_became_active":"2024-09-11T01:26:02.989982+0000","last_became_peered":"2024-09-11T01:26:02.989982+0000","last_unstale":"2024-09-11T01:27:22.144622+0000","last_undegraded":"2024-09-11T01:27:22.144622+0000","last_fullsized":"2024-09-11T01:27:22.144622+0000","mapping_epoch":62,"log_start":"45'9900","ondisk_log_start":"45'9900","created":16,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"67'19953","last_scrub_stamp":"2024-09-11T01:27:22.144408+0000","last_deep_scrub":"67'19953","last_deep_scrub_stamp":"2024-09-11T01:27:22.144408+0000","last_clean_scrub_stamp":"2024-09-11T01:27:22.144408+0000","objects_scrubbed":1,"log_size":10053,"log_dups_size":0,"ondisk_log_size":10053,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T08:13:18.742825+0000","scrub_duration":27,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":17022,"num_write_kb":28208,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3920,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":16,"stat_sum":{"num_bytes":617,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":282286,"num_write_kb":486433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":47639,"num_bytes_recovered":36720640,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":79400960,"data_stored":79394002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160837,"ondisk_log_size":160837,"up":32,"acting":32,"num_store_stats":3},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":0,"up_from":25,"seq":107374182593,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":505768,"kb_used_data":50712,"kb_used_omap":4,"kb_used_meta":455035,"kb_avail":93866072,"statfs":{"total":96636764160,"available":96118857728,"internally_reserved":0,"allocated":51929088,"data_stored":51552229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4437,"internal_metadata":465956523},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":31,"seq":133143986365,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":393332,"kb_used_data":28516,"kb_used_omap":4,"kb_used_meta":364795,"kb_avail":93978508,"statfs":{"total":96636764160,"available":96233992192,"internally_reserved":0,"allocated":29200384,"data_stored":28831081,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4438,"internal_metadata":373550762},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":41,"seq":176093659309,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":501432,"kb_used_data":552,"kb_used_omap":4,"kb_used_meta":500859,"kb_avail":93870408,"statfs":{"total":96636764160,"available":96123297792,"internally_reserved":0,"allocated":565248,"data_stored":188389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4433,"internal_metadata":512880303},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":50876416,"data_stored":50872937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":28520448,"data_stored":28520448,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-11T01:27:37.060 DEBUG:teuthology.orchestra.run.smithi122:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-11T01:27:37.392 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-11T01:27:37.393 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-11T01:27:37.393 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-11T01:27:37.393 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T01:27:42.669 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2024-09-11T01:27:43.494 INFO:tasks.ceph.osd.0:Stopped 2024-09-11T01:27:43.495 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-11T01:27:43.495 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T01:27:43.595 INFO:tasks.ceph.osd.1:Stopped 2024-09-11T01:27:43.596 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-11T01:27:43.596 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T01:27:48.072 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2024-09-11T01:27:48.073 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2024-09-11T01:27:49.697 INFO:tasks.ceph.osd.2:Stopped 2024-09-11T01:27:49.697 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-11T01:27:49.698 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-11T01:27:49.698 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T01:27:49.727 INFO:tasks.ceph.mgr.x:Stopped 2024-09-11T01:27:49.727 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-11T01:27:49.728 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-11T01:27:49.728 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T01:27:49.783 INFO:tasks.ceph.mon.a:Stopped 2024-09-11T01:27:49.783 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-11T01:27:49.784 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T01:27:49.839 INFO:tasks.ceph.mon.b:Stopped 2024-09-11T01:27:49.840 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-11T01:27:49.840 INFO:teuthology.orchestra.run:waiting for 300 2024-09-11T01:27:49.901 INFO:tasks.ceph.mon.c:Stopped 2024-09-11T01:27:49.901 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-11T01:27:49.901 DEBUG:teuthology.orchestra.run.smithi122:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v 'slow request' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-09-11T01:27:50.101 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi122.front.sepia.ceph.com 2024-09-11T01:27:50.101 DEBUG:teuthology.orchestra.run.smithi122:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-11T01:27:50.997 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi122.front.sepia.ceph.com 2024-09-11T01:27:50.997 DEBUG:teuthology.orchestra.run.smithi122:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-11T01:27:51.270 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi122.front.sepia.ceph.com 2024-09-11T01:27:51.270 DEBUG:teuthology.orchestra.run.smithi122:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-11T01:27:51.444 INFO:tasks.ceph:Archiving mon data... 2024-09-11T01:27:51.445 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/hyelloji-2024-09-10_17:16:14-rados-wip-hemanth-testing-2024-09-10-0723-distro-default-smithi/7899875/data/mon.a.tgz 2024-09-11T01:27:51.446 DEBUG:teuthology.orchestra.run.smithi122:> mktemp 2024-09-11T01:27:51.461 INFO:teuthology.orchestra.run.smithi122.stdout:/tmp/tmp.ROjsWgc4rG 2024-09-11T01:27:51.461 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.ROjsWgc4rG 2024-09-11T01:27:51.595 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0666 /tmp/tmp.ROjsWgc4rG 2024-09-11T01:27:51.686 DEBUG:teuthology.orchestra.remote:smithi122:/tmp/tmp.ROjsWgc4rG is 306KB 2024-09-11T01:27:51.748 DEBUG:teuthology.orchestra.run.smithi122:> rm -fr /tmp/tmp.ROjsWgc4rG 2024-09-11T01:27:51.764 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/hyelloji-2024-09-10_17:16:14-rados-wip-hemanth-testing-2024-09-10-0723-distro-default-smithi/7899875/data/mon.b.tgz 2024-09-11T01:27:51.764 DEBUG:teuthology.orchestra.run.smithi122:> mktemp 2024-09-11T01:27:51.811 INFO:teuthology.orchestra.run.smithi122.stdout:/tmp/tmp.BuqOKyAyXM 2024-09-11T01:27:51.811 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.BuqOKyAyXM 2024-09-11T01:27:51.925 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0666 /tmp/tmp.BuqOKyAyXM 2024-09-11T01:27:51.987 DEBUG:teuthology.orchestra.remote:smithi122:/tmp/tmp.BuqOKyAyXM is 312KB 2024-09-11T01:27:52.003 DEBUG:teuthology.orchestra.run.smithi122:> rm -fr /tmp/tmp.BuqOKyAyXM 2024-09-11T01:27:52.009 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/hyelloji-2024-09-10_17:16:14-rados-wip-hemanth-testing-2024-09-10-0723-distro-default-smithi/7899875/data/mon.c.tgz 2024-09-11T01:27:52.009 DEBUG:teuthology.orchestra.run.smithi122:> mktemp 2024-09-11T01:27:52.055 INFO:teuthology.orchestra.run.smithi122.stdout:/tmp/tmp.biJkDmsQTr 2024-09-11T01:27:52.055 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.biJkDmsQTr 2024-09-11T01:27:52.173 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 0666 /tmp/tmp.biJkDmsQTr 2024-09-11T01:27:52.239 DEBUG:teuthology.orchestra.remote:smithi122:/tmp/tmp.biJkDmsQTr is 312KB 2024-09-11T01:27:52.300 DEBUG:teuthology.orchestra.run.smithi122:> rm -fr /tmp/tmp.biJkDmsQTr 2024-09-11T01:27:52.306 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-11T01:27:52.306 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-11T01:27:52.433 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-11T01:27:52.433 INFO:tasks.ceph:Archiving crash dumps... 2024-09-11T01:27:52.435 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/lib/ceph/crash to /home/teuthworker/archive/hyelloji-2024-09-10_17:16:14-rados-wip-hemanth-testing-2024-09-10-0723-distro-default-smithi/7899875/remote/smithi122/crash 2024-09-11T01:27:52.436 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-11T01:27:52.487 INFO:tasks.ceph:Compressing logs... 2024-09-11T01:27:52.487 DEBUG:teuthology.orchestra.run.smithi122:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-11T01:27:52.554 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28968.log 2024-09-11T01:27:52.555 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48499.log 2024-09-11T01:27:52.555 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.28968.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28968.log.gz 2024-09-11T01:27:52.555 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39489.log 2024-09-11T01:27:52.556 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48499.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48499.log.gz 2024-09-11T01:27:52.556 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32391.log 2024-09-11T01:27:52.557 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38577.log 2024-09-11T01:27:52.557 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39489.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39489.log.gz 2024-09-11T01:27:52.557 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.32391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49490.log 0.0% 2024-09-11T01:27:52.557 INFO:teuthology.orchestra.run.smithi122.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32391.log.gz 2024-09-11T01:27:52.558 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39645.log 2024-09-11T01:27:52.558 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38577.log.gz 2024-09-11T01:27:52.558 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49490.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49490.log.gz 2024-09-11T01:27:52.558 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52171.log 2024-09-11T01:27:52.560 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39359.log 2024-09-11T01:27:52.560 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39645.log: /var/log/ceph/ceph-client.admin.52171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48265.log 2024-09-11T01:27:52.560 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39645.log.gz 2024-09-11T01:27:52.560 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52171.log.gz 2024-09-11T01:27:52.560 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34962.log 2024-09-11T01:27:52.560 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39359.log.gz 2024-09-11T01:27:52.560 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47745.log 2024-09-11T01:27:52.560 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48265.log.gz 2024-09-11T01:27:52.560 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43268.log 2024-09-11T01:27:52.560 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34962.log.gz 2024-09-11T01:27:52.564 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30604.log 2024-09-11T01:27:52.564 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47745.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47745.log.gz 2024-09-11T01:27:52.564 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30832.log 2024-09-11T01:27:52.564 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43268.log.gz 2024-09-11T01:27:52.564 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.30604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43502.log 2024-09-11T01:27:52.564 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30604.log.gz 2024-09-11T01:27:52.564 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50634.log 2024-09-11T01:27:52.564 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.30832.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30832.log.gz 2024-09-11T01:27:52.564 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35441.log 2024-09-11T01:27:52.565 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43502.log.gz 2024-09-11T01:27:52.565 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50946.log 2024-09-11T01:27:52.565 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50634.log.gz 2024-09-11T01:27:52.565 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46131.log 2024-09-11T01:27:52.565 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35441.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35441.log.gz 2024-09-11T01:27:52.565 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50946.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50946.log.gz 2024-09-11T01:27:52.565 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46964.log 2024-09-11T01:27:52.565 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40870.log 2024-09-11T01:27:52.565 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46131.log.gz 2024-09-11T01:27:52.565 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42930.log 2024-09-11T01:27:52.565 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46964.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46964.log.gz 2024-09-11T01:27:52.565 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30037.log 2024-09-11T01:27:52.565 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40870.log.gz 2024-09-11T01:27:52.572 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30920.log 2024-09-11T01:27:52.572 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42930.log.gz 2024-09-11T01:27:52.572 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38343.log 2024-09-11T01:27:52.572 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.30037.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30037.log.gz 2024-09-11T01:27:52.572 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.30920.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.30920.log.gz -- 2024-09-11T01:27:52.572 INFO:teuthology.orchestra.run.smithi122.stderr: /var/log/ceph/ceph-client.admin.31272.log 2024-09-11T01:27:52.572 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29547.log 2024-09-11T01:27:52.572 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38343.log.gz 2024-09-11T01:27:52.572 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34780.log 2024-09-11T01:27:52.572 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.31272.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31272.log.gz 2024-09-11T01:27:52.572 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29547.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.51102.log -- replaced with /var/log/ceph/ceph-client.admin.29547.log.gz 2024-09-11T01:27:52.573 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-11T01:27:52.573 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39801.log 2024-09-11T01:27:52.573 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34780.log.gz 2024-09-11T01:27:52.573 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48421.log 2024-09-11T01:27:52.573 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51102.log.gz 2024-09-11T01:27:52.573 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47198.log 2024-09-11T01:27:52.573 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39801.log.gz 2024-09-11T01:27:52.573 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42199.log 2024-09-11T01:27:52.573 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48421.log.gz 2024-09-11T01:27:52.573 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40272.log 2024-09-11T01:27:52.573 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47198.log.gz 2024-09-11T01:27:52.573 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51519.log 2024-09-11T01:27:52.573 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42199.log.gz 2024-09-11T01:27:52.573 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48317.log 2024-09-11T01:27:52.574 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40272.log.gz 2024-09-11T01:27:52.574 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44856.log 2024-09-11T01:27:52.574 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51519.log.gz 2024-09-11T01:27:52.574 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44882.log 2024-09-11T01:27:52.574 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48317.log.gz 2024-09-11T01:27:52.574 INFO:teuthology.orchestra.run.smithi122.stderr:gzip/var/log/ceph/ceph-client.admin.44856.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.34494.log 2024-09-11T01:27:52.574 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44856.log.gz 2024-09-11T01:27:52.574 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41419.log 2024-09-11T01:27:52.574 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44882.log.gz 2024-09-11T01:27:52.574 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29150.log 2024-09-11T01:27:52.574 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34494.log.gz 2024-09-11T01:27:52.574 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39333.log 2024-09-11T01:27:52.574 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41419.log.gz 2024-09-11T01:27:52.574 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34026.log 2024-09-11T01:27:52.574 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29150.log.gz 2024-09-11T01:27:52.575 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39333.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49828.log 2024-09-11T01:27:52.575 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39333.log.gz 2024-09-11T01:27:52.591 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31052.log 2024-09-11T01:27:52.591 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34026.log.gz 2024-09-11T01:27:52.591 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39124.log 2024-09-11T01:27:52.591 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49828.log.gz 2024-09-11T01:27:52.591 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.31052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46704.log 2024-09-11T01:27:52.591 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31052.log.gz 2024-09-11T01:27:52.591 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38759.log 2024-09-11T01:27:52.591 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39124.log.gz 2024-09-11T01:27:52.591 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46704.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38262.log 2024-09-11T01:27:52.591 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46704.log.gz 2024-09-11T01:27:52.591 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49698.log 2024-09-11T01:27:52.592 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38759.log.gz 2024-09-11T01:27:52.592 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46339.log 2024-09-11T01:27:52.592 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38262.log.gz 2024-09-11T01:27:52.592 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29696.log 2024-09-11T01:27:52.592 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49698.log.gz 2024-09-11T01:27:52.592 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42956.log 2024-09-11T01:27:52.592 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46339.log.gz 2024-09-11T01:27:52.592 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47537.log 2024-09-11T01:27:52.592 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29696.log.gz 2024-09-11T01:27:52.592 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37300.log 2024-09-11T01:27:52.592 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42956.log.gz 2024-09-11T01:27:52.592 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34182.log 2024-09-11T01:27:52.592 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47537.log.gz 2024-09-11T01:27:52.593 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45428.log 2024-09-11T01:27:52.593 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37300.log.gz 2024-09-11T01:27:52.593 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48996.log 2024-09-11T01:27:52.593 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34182.log.gz 2024-09-11T01:27:52.593 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44283.log 2024-09-11T01:27:52.593 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45428.log.gz 2024-09-11T01:27:52.593 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50972.log 2024-09-11T01:27:52.593 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48996.log.gz 2024-09-11T01:27:52.593 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42511.log 2024-09-11T01:27:52.593 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44283.log.gz 2024-09-11T01:27:52.593 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50972.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46313.log 2024-09-11T01:27:52.593 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50972.log.gz 2024-09-11T01:27:52.593 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34312.log 2024-09-11T01:27:52.593 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42511.log.gz 2024-09-11T01:27:52.594 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49022.log 2024-09-11T01:27:52.594 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46313.log.gz 2024-09-11T01:27:52.594 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49594.log 2024-09-11T01:27:52.594 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34312.log.gz 2024-09-11T01:27:52.594 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49776.log 2024-09-11T01:27:52.594 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49022.log.gz 2024-09-11T01:27:52.594 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37586.log 2024-09-11T01:27:52.594 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49594.log.gz 2024-09-11T01:27:52.594 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41549.log 2024-09-11T01:27:52.594 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49776.log.gz 2024-09-11T01:27:52.594 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45975.log 2024-09-11T01:27:52.594 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37586.log.gz 2024-09-11T01:27:52.594 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41549.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35199.log 2024-09-11T01:27:52.594 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41549.log.gz 2024-09-11T01:27:52.595 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48239.log 2024-09-11T01:27:52.595 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45975.log.gz 2024-09-11T01:27:52.595 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46730.log 2024-09-11T01:27:52.595 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35199.log.gz 2024-09-11T01:27:52.595 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41471.log 2024-09-11T01:27:52.595 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48239.log.gz 2024-09-11T01:27:52.595 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46730.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-11T01:27:52.595 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46730.log.gz 2024-09-11T01:27:52.595 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49958.log 2024-09-11T01:27:52.595 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41471.log.gz 2024-09-11T01:27:52.595 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36728.log 2024-09-11T01:27:52.595 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39879.log 2024-09-11T01:27:52.595 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49958.log.gz 2024-09-11T01:27:52.595 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35040.log 2024-09-11T01:27:52.596 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36728.log.gz 2024-09-11T01:27:52.596 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39879.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50244.log 2024-09-11T01:27:52.596 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39879.log.gz 2024-09-11T01:27:52.596 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.35040.log: /var/log/ceph/ceph-client.admin.48944.log 2024-09-11T01:27:52.596 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35040.log.gz 2024-09-11T01:27:52.596 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50244.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30111.log 2024-09-11T01:27:52.596 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50244.log.gz 2024-09-11T01:27:52.596 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30450.log 2024-09-11T01:27:52.596 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48944.log.gz 2024-09-11T01:27:52.596 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35551.log 2024-09-11T01:27:52.596 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.30111.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30111.log.gz 2024-09-11T01:27:52.596 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.30450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43632.log 2024-09-11T01:27:52.596 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30450.log.gz 2024-09-11T01:27:52.596 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46626.log 2024-09-11T01:27:52.597 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35551.log.gz 2024-09-11T01:27:52.597 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40246.log 2024-09-11T01:27:52.597 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43632.log.gz 2024-09-11T01:27:52.597 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49360.log 2024-09-11T01:27:52.597 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46626.log.gz 2024-09-11T01:27:52.597 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46548.log 2024-09-11T01:27:52.597 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40246.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40246.log.gz 2024-09-11T01:27:52.597 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37170.log 2024-09-11T01:27:52.597 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49360.log.gz 2024-09-11T01:27:52.597 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46548.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.46548.log.gz 2024-09-11T01:27:52.597 INFO:teuthology.orchestra.run.smithi122.stderr: -- /var/log/ceph/ceph-client.admin.44413.log 2024-09-11T01:27:52.597 INFO:teuthology.orchestra.run.smithi122.stderr:gzip/var/log/ceph/ceph-client.admin.37170.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.47094.log 2024-09-11T01:27:52.597 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37170.log.gz 2024-09-11T01:27:52.597 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46418.log 2024-09-11T01:27:52.598 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44413.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44413.log.gz 2024-09-11T01:27:52.598 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42277.log 2024-09-11T01:27:52.598 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47094.log.gz 2024-09-11T01:27:52.598 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37820.log 2024-09-11T01:27:52.598 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46418.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46418.log.gz 2024-09-11T01:27:52.598 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43736.log 2024-09-11T01:27:52.598 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42277.log.gz 2024-09-11T01:27:52.598 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37820.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37820.log.gz 2024-09-11T01:27:52.598 INFO:teuthology.orchestra.run.smithi122.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43346.log 2024-09-11T01:27:52.598 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50374.log 2024-09-11T01:27:52.598 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43736.log.gz 2024-09-11T01:27:52.598 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41783.log 2024-09-11T01:27:52.598 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43346.log.gz 2024-09-11T01:27:52.864 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30137.log 0.0% 2024-09-11T01:27:52.864 INFO:teuthology.orchestra.run.smithi122.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50374.log.gz 2024-09-11T01:27:52.864 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50816.log 2024-09-11T01:27:52.864 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41783.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41783.log.gz 2024-09-11T01:27:52.864 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.30137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34885.log 2024-09-11T01:27:52.864 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30137.log.gz 2024-09-11T01:27:52.864 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50816.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.50816.log.gz /var/log/ceph/ceph-client.admin.35485.log 2024-09-11T01:27:52.864 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-11T01:27:52.864 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41861.log 2024-09-11T01:27:52.864 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34885.log.gz 2024-09-11T01:27:52.864 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41053.log 2024-09-11T01:27:52.865 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35485.log.gz 2024-09-11T01:27:52.865 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41861.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41861.log.gz 2024-09-11T01:27:52.865 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45324.log 2024-09-11T01:27:52.865 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.41053.log: /var/log/ceph/ceph-client.admin.41627.log 2024-09-11T01:27:52.865 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41053.log.gz 2024-09-11T01:27:52.865 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50036.log 2024-09-11T01:27:52.865 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45324.log.gz 2024-09-11T01:27:52.865 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50764.log 2024-09-11T01:27:52.865 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41627.log.gz 2024-09-11T01:27:52.865 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38655.log 2024-09-11T01:27:52.865 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50036.log.gz 2024-09-11T01:27:52.865 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34676.log 2024-09-11T01:27:52.865 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50764.log.gz 2024-09-11T01:27:52.865 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48187.log 2024-09-11T01:27:52.866 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38655.log.gz 2024-09-11T01:27:52.866 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51206.log 2024-09-11T01:27:52.866 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34676.log.gz 2024-09-11T01:27:52.866 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35243.log 2024-09-11T01:27:52.866 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48187.log.gz 2024-09-11T01:27:52.866 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36544.log 2024-09-11T01:27:52.866 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51206.log.gz 2024-09-11T01:27:52.866 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29232.log 2024-09-11T01:27:52.866 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35243.log.gz 2024-09-11T01:27:52.866 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41653.log 2024-09-11T01:27:52.866 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36544.log.gz 2024-09-11T01:27:52.866 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38054.log 2024-09-11T01:27:52.866 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29232.log.gz 2024-09-11T01:27:52.867 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48369.log 2024-09-11T01:27:52.867 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41653.log.gz 2024-09-11T01:27:52.867 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44517.log 2024-09-11T01:27:52.867 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38054.log.gz 2024-09-11T01:27:52.867 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50322.log 2024-09-11T01:27:52.867 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48369.log.gz 2024-09-11T01:27:52.867 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36962.log 2024-09-11T01:27:52.867 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44517.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44517.log.gz 2024-09-11T01:27:52.867 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29548.log 2024-09-11T01:27:52.867 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50322.log.gz 2024-09-11T01:27:52.867 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46496.log 2024-09-11T01:27:52.867 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36962.log.gz 2024-09-11T01:27:52.867 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31008.log 2024-09-11T01:27:52.867 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29548.log.gz 2024-09-11T01:27:52.867 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30783.log 2024-09-11T01:27:52.868 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46496.log.gz 2024-09-11T01:27:52.868 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.31008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41887.log 2024-09-11T01:27:52.868 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31008.log.gz 2024-09-11T01:27:52.868 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.30783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32257.log 2024-09-11T01:27:52.868 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30783.log.gz 2024-09-11T01:27:52.868 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31294.log 2024-09-11T01:27:52.868 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41887.log.gz 2024-09-11T01:27:52.868 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.32257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51337.log 2024-09-11T01:27:52.868 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32257.log.gz 2024-09-11T01:27:52.868 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.31294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52015.log 2024-09-11T01:27:52.868 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31294.log.gz 2024-09-11T01:27:52.868 INFO:teuthology.orchestra.run.smithi122.stderr:gzip/var/log/ceph/ceph-client.admin.51337.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.47849.log 2024-09-11T01:27:52.868 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51337.log.gz 2024-09-11T01:27:52.868 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.52015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46574.log 2024-09-11T01:27:52.869 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52015.log.gz 2024-09-11T01:27:52.869 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47849.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35309.log 2024-09-11T01:27:52.869 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47849.log.gz 2024-09-11T01:27:52.869 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51911.log 2024-09-11T01:27:52.869 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46574.log.gz 2024-09-11T01:27:52.869 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40480.log 2024-09-11T01:27:52.869 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35309.log.gz 2024-09-11T01:27:52.869 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46183.log 2024-09-11T01:27:52.869 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51911.log.gz 2024-09-11T01:27:52.870 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34468.log/var/log/ceph/ceph-client.admin.40480.log: 2024-09-11T01:27:52.870 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40480.log.gz 2024-09-11T01:27:52.870 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33453.log 2024-09-11T01:27:52.870 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46183.log.gz 2024-09-11T01:27:52.870 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47406.log 2024-09-11T01:27:52.870 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34468.log.gz 2024-09-11T01:27:52.870 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43606.log 2024-09-11T01:27:52.870 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33453.log.gz 2024-09-11T01:27:52.870 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42017.log 2024-09-11T01:27:52.870 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47406.log.gz 2024-09-11T01:27:52.870 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42485.log 2024-09-11T01:27:52.870 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43606.log.gz 2024-09-11T01:27:52.871 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46235.log 2024-09-11T01:27:52.871 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42017.log.gz 2024-09-11T01:27:52.871 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42485.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42485.log.gz 2024-09-11T01:27:52.871 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47875.log 2024-09-11T01:27:52.871 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50790.log 2024-09-11T01:27:52.871 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46235.log.gz 2024-09-11T01:27:52.871 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40610.log 2024-09-11T01:27:52.871 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47875.log.gz 2024-09-11T01:27:52.871 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29228.log 2024-09-11T01:27:52.871 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50790.log.gz 2024-09-11T01:27:52.872 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42043.log 2024-09-11T01:27:52.872 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40610.log.gz 2024-09-11T01:27:52.872 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45116.log 2024-09-11T01:27:52.872 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29228.log.gz 2024-09-11T01:27:52.872 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40350.log 2024-09-11T01:27:52.872 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42043.log.gz 2024-09-11T01:27:52.872 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45116.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43996.log 2024-09-11T01:27:52.872 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45116.log.gz 2024-09-11T01:27:52.872 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49542.log 2024-09-11T01:27:52.872 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40350.log.gz 2024-09-11T01:27:52.872 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49386.log 2024-09-11T01:27:52.873 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43996.log.gz 2024-09-11T01:27:52.873 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49542.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44829.log 2024-09-11T01:27:52.873 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49542.log.gz 2024-09-11T01:27:52.873 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50894.log 2024-09-11T01:27:52.873 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49386.log.gz 2024-09-11T01:27:52.873 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44829.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43398.log 2024-09-11T01:27:52.873 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44829.log.gz 2024-09-11T01:27:52.873 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35463.log 2024-09-11T01:27:52.873 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50894.log.gz 2024-09-11T01:27:52.873 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25337.log 2024-09-11T01:27:52.873 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43398.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43398.log.gz 2024-09-11T01:27:52.874 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43788.log 2024-09-11T01:27:52.874 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35463.log.gz 2024-09-11T01:27:52.874 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.25337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50426.log 2024-09-11T01:27:52.874 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25337.log.gz 2024-09-11T01:27:52.874 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31162.log 2024-09-11T01:27:52.874 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43788.log.gz 2024-09-11T01:27:52.874 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50426.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41809.log 2024-09-11T01:27:52.874 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50426.log.gz 2024-09-11T01:27:52.874 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.31162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48681.log 2024-09-11T01:27:52.874 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31162.log.gz 2024-09-11T01:27:52.874 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47016.log 2024-09-11T01:27:52.875 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41809.log.gz 2024-09-11T01:27:52.875 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30318.log 2024-09-11T01:27:52.875 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48681.log.gz 2024-09-11T01:27:52.875 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40298.log 2024-09-11T01:27:52.875 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47016.log.gz 2024-09-11T01:27:52.875 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-11T01:27:52.875 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.30318.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30318.log.gz 2024-09-11T01:27:52.875 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49048.log 2024-09-11T01:27:52.875 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40298.log.gz 2024-09-11T01:27:52.875 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49750.log 2024-09-11T01:27:52.875 INFO:teuthology.orchestra.run.smithi122.stderr:gzip/var/log/ceph/ceph-client.admin.49048.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.45220.log 2024-09-11T01:27:52.875 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49048.log.gz 2024-09-11T01:27:52.876 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41679.log 2024-09-11T01:27:52.876 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49750.log.gz 2024-09-11T01:27:52.876 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44647.log 2024-09-11T01:27:52.876 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45220.log.gz 2024-09-11T01:27:52.876 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38525.log 2024-09-11T01:27:52.876 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41679.log.gz 2024-09-11T01:27:52.876 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38184.log 2024-09-11T01:27:52.876 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44647.log.gz 2024-09-11T01:27:52.876 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42303.log 0.0% 2024-09-11T01:27:52.876 INFO:teuthology.orchestra.run.smithi122.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38525.log.gz 2024-09-11T01:27:52.877 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44491.log 2024-09-11T01:27:52.877 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38184.log.gz 2024-09-11T01:27:52.877 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50140.log 2024-09-11T01:27:52.877 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42303.log.gz 2024-09-11T01:27:52.877 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44491.log: gzip -5 --verbose 0.0% -- /var/log/ceph/ceph-client.admin.49204.log -- replaced with /var/log/ceph/ceph-client.admin.44491.log.gz 2024-09-11T01:27:52.877 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-11T01:27:52.877 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41157.log 2024-09-11T01:27:52.877 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50140.log.gz 2024-09-11T01:27:52.877 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49204.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49204.log.gz 2024-09-11T01:27:52.877 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32491.log 2024-09-11T01:27:52.878 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46053.log 2024-09-11T01:27:52.878 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41157.log.gz 2024-09-11T01:27:52.878 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.32491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46782.log 2024-09-11T01:27:52.878 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32491.log.gz 2024-09-11T01:27:52.878 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42774.log 2024-09-11T01:27:52.878 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46053.log.gz 2024-09-11T01:27:52.878 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52119.log 2024-09-11T01:27:52.878 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46782.log.gz 2024-09-11T01:27:52.878 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47589.log 2024-09-11T01:27:52.878 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42774.log.gz 2024-09-11T01:27:52.878 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.52119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39749.log 2024-09-11T01:27:52.878 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52119.log.gz 2024-09-11T01:27:52.879 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36884.log 2024-09-11T01:27:52.879 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47589.log.gz 2024-09-11T01:27:52.879 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52093.log 2024-09-11T01:27:52.879 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39749.log.gz 2024-09-11T01:27:52.879 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38811.log 2024-09-11T01:27:52.879 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36884.log.gz 2024-09-11T01:27:52.879 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.52093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42407.log 2024-09-11T01:27:52.879 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52093.log.gz 2024-09-11T01:27:52.879 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30085.log 2024-09-11T01:27:52.879 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38811.log.gz 2024-09-11T01:27:52.879 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42407.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40584.log 2024-09-11T01:27:52.879 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42407.log.gz 2024-09-11T01:27:52.880 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.30085.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.30085.log.gz -- 2024-09-11T01:27:52.880 INFO:teuthology.orchestra.run.smithi122.stderr: /var/log/ceph/ceph-client.admin.39463.log 2024-09-11T01:27:52.880 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41001.log 2024-09-11T01:27:52.880 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40584.log.gz 2024-09-11T01:27:52.880 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39463.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39072.log 2024-09-11T01:27:52.880 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39463.log.gz 2024-09-11T01:27:52.880 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41001.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.41001.log.gz --verbose 2024-09-11T01:27:52.880 INFO:teuthology.orchestra.run.smithi122.stderr: -- /var/log/ceph/ceph-client.admin.30296.log 2024-09-11T01:27:52.880 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5/var/log/ceph/ceph-client.admin.39072.log: --verbose -- /var/log/ceph/ceph-client.admin.51702.log 2024-09-11T01:27:52.880 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39072.log.gz 2024-09-11T01:27:52.880 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.30296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41731.log 2024-09-11T01:27:52.881 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30296.log.gz 2024-09-11T01:27:52.881 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28994.log 2024-09-11T01:27:52.881 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51702.log.gz 2024-09-11T01:27:52.881 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37716.log 2024-09-11T01:27:52.881 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41731.log.gz 2024-09-11T01:27:52.881 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.28994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37768.log 2024-09-11T01:27:52.881 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28994.log.gz 2024-09-11T01:27:52.881 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44960.log 2024-09-11T01:27:52.881 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37716.log.gz 2024-09-11T01:27:52.881 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37768.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-11T01:27:52.881 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37768.log.gz 2024-09-11T01:27:52.881 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44960.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44960.log.gz 2024-09-11T01:27:52.882 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44231.log 2024-09-11T01:27:52.882 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39046.log 2024-09-11T01:27:52.882 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.44231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43372.log 2024-09-11T01:27:52.882 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44231.log.gz 2024-09-11T01:27:52.882 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38889.log 2024-09-11T01:27:52.882 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39046.log.gz 2024-09-11T01:27:52.882 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49620.log 2024-09-11T01:27:52.882 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43372.log.gz 2024-09-11T01:27:52.882 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38889.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38889.log.gz 2024-09-11T01:27:52.882 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37690.log 2024-09-11T01:27:52.882 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47979.log 2024-09-11T01:27:52.883 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49620.log.gz 2024-09-11T01:27:52.883 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46834.log 2024-09-11T01:27:52.883 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37690.log.gz 2024-09-11T01:27:52.883 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40087.log 2024-09-11T01:27:52.883 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47979.log.gz 2024-09-11T01:27:52.883 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46209.log 2024-09-11T01:27:52.883 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46834.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46834.log.gz 2024-09-11T01:27:52.883 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45662.log 2024-09-11T01:27:52.883 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40087.log.gz 2024-09-11T01:27:52.883 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46860.log 2024-09-11T01:27:52.883 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46209.log.gz 2024-09-11T01:27:52.883 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37274.log 2024-09-11T01:27:52.884 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45662.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45662.log.gz 2024-09-11T01:27:52.884 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40975.log 2024-09-11T01:27:52.884 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46860.log.gz 2024-09-11T01:27:52.884 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39541.log 2024-09-11T01:27:52.884 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37274.log.gz 2024-09-11T01:27:52.884 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48473.log 2024-09-11T01:27:52.884 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40975.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40975.log.gz 2024-09-11T01:27:52.884 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42329.log 2024-09-11T01:27:52.884 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39541.log.gz 2024-09-11T01:27:52.884 INFO:teuthology.orchestra.run.smithi122.stderr:gzip/var/log/ceph/ceph-client.admin.48473.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.50270.log 2024-09-11T01:27:52.885 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48473.log.gz 2024-09-11T01:27:52.885 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37950.log 2024-09-11T01:27:52.885 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42329.log.gz 2024-09-11T01:27:52.885 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50270.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50270.log.gz 2024-09-11T01:27:52.885 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34936.log 2024-09-11T01:27:52.885 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29046.log 2024-09-11T01:27:52.885 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37950.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37950.log.gz 2024-09-11T01:27:52.885 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39853.log 2024-09-11T01:27:52.885 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34936.log.gz 2024-09-11T01:27:52.885 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40922.log 2024-09-11T01:27:52.885 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29046.log.gz 2024-09-11T01:27:52.888 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50478.log 2024-09-11T01:27:52.888 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39853.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39853.log.gz 2024-09-11T01:27:52.888 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46027.log 2024-09-11T01:27:52.888 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40922.log.gz 2024-09-11T01:27:52.888 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50478.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50738.log 2024-09-11T01:27:52.888 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50478.log.gz 2024-09-11T01:27:52.889 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29777.log 2024-09-11T01:27:52.889 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46027.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46027.log.gz 2024-09-11T01:27:52.889 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52067.log 2024-09-11T01:27:52.889 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50738.log.gz 2024-09-11T01:27:52.889 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42563.log 2024-09-11T01:27:52.889 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38080.log 2024-09-11T01:27:52.889 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29777.log.gz 2024-09-11T01:27:52.889 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.52067.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52067.log.gz 2024-09-11T01:27:52.889 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35397.log 2024-09-11T01:27:52.889 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42563.log.gz 2024-09-11T01:27:52.889 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32338.log 2024-09-11T01:27:52.889 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38080.log.gz 2024-09-11T01:27:52.889 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35397.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35397.log.gz 2024-09-11T01:27:52.890 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40532.log 2024-09-11T01:27:52.890 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37144.log 2024-09-11T01:27:52.890 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.32338.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32338.log.gz 2024-09-11T01:27:52.890 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45610.log 2024-09-11T01:27:52.890 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40532.log.gz 2024-09-11T01:27:52.890 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37144.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37144.log.gz 2024-09-11T01:27:52.890 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30942.log 2024-09-11T01:27:52.890 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5/var/log/ceph/ceph-client.admin.45610.log: --verbose -- /var/log/ceph/ceph-client.admin.49308.log 2024-09-11T01:27:52.890 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45610.log.gz 2024-09-11T01:27:52.890 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.30942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29622.log 2024-09-11T01:27:52.890 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30942.log.gz 2024-09-11T01:27:52.890 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50920.log 2024-09-11T01:27:52.890 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49308.log.gz 2024-09-11T01:27:52.890 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35507.log 2024-09-11T01:27:52.891 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29622.log.gz 2024-09-11T01:27:52.891 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44257.log 2024-09-11T01:27:52.891 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50920.log.gz 2024-09-11T01:27:52.891 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35507.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35507.log.gzgzip 2024-09-11T01:27:52.891 INFO:teuthology.orchestra.run.smithi122.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.50114.log 2024-09-11T01:27:52.891 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51258.log 2024-09-11T01:27:52.891 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44257.log.gz 2024-09-11T01:27:52.891 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35529.log 2024-09-11T01:27:52.891 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50114.log.gz 2024-09-11T01:27:52.891 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41601.log 2024-09-11T01:27:52.891 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51258.log.gz 2024-09-11T01:27:52.891 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40168.log 2024-09-11T01:27:52.891 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35529.log.gz 2024-09-11T01:27:52.891 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38707.log 2024-09-11T01:27:52.892 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41601.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41601.log.gz 2024-09-11T01:27:52.892 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30274.log 2024-09-11T01:27:52.892 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40168.log.gz 2024-09-11T01:27:52.892 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.38707.log: /var/log/ceph/ceph-client.admin.45480.log 2024-09-11T01:27:52.892 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38707.log.gz 2024-09-11T01:27:52.892 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48161.log 2024-09-11T01:27:52.892 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.30274.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30274.log.gz 2024-09-11T01:27:52.892 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37560.log 2024-09-11T01:27:52.892 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45480.log.gz 2024-09-11T01:27:52.892 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30252.log 2024-09-11T01:27:52.892 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48161.log.gz 2024-09-11T01:27:52.892 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41341.log 2024-09-11T01:27:52.892 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37560.log.gz 2024-09-11T01:27:52.892 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.30252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41289.log 2024-09-11T01:27:52.893 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30252.log.gz 2024-09-11T01:27:52.893 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43164.log 2024-09-11T01:27:52.893 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41341.log.gz 2024-09-11T01:27:52.893 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39827.log 2024-09-11T01:27:52.893 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41289.log.gz 2024-09-11T01:27:52.893 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40844.log 2024-09-11T01:27:52.893 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43164.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43164.log.gz 2024-09-11T01:27:52.893 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39827.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45168.log 2024-09-11T01:27:52.893 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39827.log.gz 2024-09-11T01:27:52.893 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38785.log 2024-09-11T01:27:52.893 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40844.log.gz 2024-09-11T01:27:52.893 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45142.log 2024-09-11T01:27:52.893 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45168.log.gz 2024-09-11T01:27:52.894 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38785.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38785.log.gz 2024-09-11T01:27:52.894 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46287.log 2024-09-11T01:27:52.894 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45064.log 2024-09-11T01:27:52.894 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45142.log.gz 2024-09-11T01:27:52.894 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28948.log 2024-09-11T01:27:52.894 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46287.log.gz 2024-09-11T01:27:52.894 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45064.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32283.log 2024-09-11T01:27:52.894 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45064.log.gz 2024-09-11T01:27:52.894 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.28948.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49412.log 2024-09-11T01:27:52.894 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28948.log.gz 2024-09-11T01:27:52.894 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.32283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34728.log 2024-09-11T01:27:52.894 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32283.log.gz 2024-09-11T01:27:52.894 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44309.log 2024-09-11T01:27:52.895 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49412.log.gz 2024-09-11T01:27:52.895 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49646.log 2024-09-11T01:27:52.895 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34728.log.gz 2024-09-11T01:27:52.895 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46392.log 2024-09-11T01:27:52.895 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44309.log.gz 2024-09-11T01:27:52.895 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40454.log 2024-09-11T01:27:52.895 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49646.log.gz 2024-09-11T01:27:52.895 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37976.log 2024-09-11T01:27:52.895 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46392.log.gz 2024-09-11T01:27:52.895 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51180.log 2024-09-11T01:27:52.895 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40454.log.gz 2024-09-11T01:27:52.895 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38551.log 2024-09-11T01:27:52.896 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37976.log.gz 2024-09-11T01:27:52.896 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51415.log 2024-09-11T01:27:52.896 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51180.log.gz 2024-09-11T01:27:52.896 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48629.log 2024-09-11T01:27:52.896 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38551.log.gz 2024-09-11T01:27:52.896 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50712.log 2024-09-11T01:27:52.896 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51415.log.gz 2024-09-11T01:27:52.896 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33765.log 2024-09-11T01:27:52.896 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48629.log.gz 2024-09-11T01:27:52.896 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30986.log 2024-09-11T01:27:52.896 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50712.log.gz 2024-09-11T01:27:52.896 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37872.log 2024-09-11T01:27:52.896 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33765.log.gz 2024-09-11T01:27:52.896 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.30986.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37404.log 2024-09-11T01:27:52.897 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30986.log.gz 2024-09-11T01:27:52.897 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42225.log 2024-09-11T01:27:52.897 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37872.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37872.log.gz 2024-09-11T01:27:52.897 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36596.log 2024-09-11T01:27:52.897 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37404.log.gz 2024-09-11T01:27:52.897 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32543.log 2024-09-11T01:27:52.897 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42225.log.gz 2024-09-11T01:27:52.897 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36596.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43216.log 2024-09-11T01:27:52.897 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36596.log.gz 2024-09-11T01:27:52.897 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.32543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42696.log 2024-09-11T01:27:52.897 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32543.log.gz 2024-09-11T01:27:52.897 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43216.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48213.log 2024-09-11T01:27:52.897 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43216.log.gz 2024-09-11T01:27:52.898 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45194.log 2024-09-11T01:27:52.898 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42696.log.gz 2024-09-11T01:27:52.898 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48213.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39307.log 2024-09-11T01:27:52.898 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48213.log.gz 2024-09-11T01:27:52.898 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39098.log 2024-09-11T01:27:52.898 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45194.log.gz 2024-09-11T01:27:52.898 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39437.log 2024-09-11T01:27:52.898 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39307.log.gz 2024-09-11T01:27:52.898 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39098.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48759.log 2024-09-11T01:27:52.898 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39098.log.gz 2024-09-11T01:27:52.898 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51676.log 2024-09-11T01:27:52.898 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39437.log.gz 2024-09-11T01:27:52.898 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46444.log 2024-09-11T01:27:52.899 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48759.log.gz 2024-09-11T01:27:52.899 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37846.log 2024-09-11T01:27:52.899 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51676.log.gz 2024-09-11T01:27:52.899 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42878.log 2024-09-11T01:27:52.899 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46444.log.gz 2024-09-11T01:27:52.899 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37846.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44048.log 2024-09-11T01:27:52.899 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37846.log.gz 2024-09-11T01:27:52.899 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42878.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45402.log 2024-09-11T01:27:52.899 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42878.log.gz 2024-09-11T01:27:52.899 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30516.log 2024-09-11T01:27:52.899 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44048.log.gz 2024-09-11T01:27:52.899 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29436.log 2024-09-11T01:27:52.899 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45402.log.gz 2024-09-11T01:27:52.899 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.30516.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31030.log 2024-09-11T01:27:52.900 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30516.log.gz 2024-09-11T01:27:52.900 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42121.log 2024-09-11T01:27:52.900 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29436.log.gz 2024-09-11T01:27:52.900 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.31030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48866.log 2024-09-11T01:27:52.900 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31030.log.gz 2024-09-11T01:27:52.900 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35014.log 2024-09-11T01:27:52.900 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42121.log.gz 2024-09-11T01:27:52.900 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47146.log 2024-09-11T01:27:52.900 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48866.log.gz 2024-09-11T01:27:52.900 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38941.log 2024-09-11T01:27:52.900 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35014.log.gz 2024-09-11T01:27:52.900 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48083.log 2024-09-11T01:27:52.901 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47146.log.gz 2024-09-11T01:27:52.901 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40766.log 2024-09-11T01:27:52.901 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38941.log.gz 2024-09-11T01:27:52.901 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40402.log 2024-09-11T01:27:52.901 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48083.log.gz 2024-09-11T01:27:52.901 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51545.log 2024-09-11T01:27:52.901 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40766.log.gz 2024-09-11T01:27:52.901 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32387.log 2024-09-11T01:27:52.901 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40402.log.gz 2024-09-11T01:27:52.901 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38967.log 2024-09-11T01:27:52.901 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51545.log.gz 2024-09-11T01:27:52.901 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.32387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39775.log 2024-09-11T01:27:52.901 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32387.log.gz 2024-09-11T01:27:52.901 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38967.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45740.log 2024-09-11T01:27:52.902 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38967.log.gz 2024-09-11T01:27:52.902 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43112.log 2024-09-11T01:27:52.902 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39775.log.gz 2024-09-11T01:27:52.902 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43476.log 2024-09-11T01:27:52.902 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45740.log.gz 2024-09-11T01:27:52.902 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34364.log 2024-09-11T01:27:52.902 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43112.log.gz 2024-09-11T01:27:52.902 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49152.log 2024-09-11T01:27:52.902 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43476.log.gz 2024-09-11T01:27:52.902 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34364.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47511.log 2024-09-11T01:27:52.902 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34364.log.gz 2024-09-11T01:27:52.902 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46522.log 2024-09-11T01:27:52.902 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49152.log.gz 2024-09-11T01:27:52.902 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28484.log 2024-09-11T01:27:52.903 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47511.log.gz 2024-09-11T01:27:52.903 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43424.log 2024-09-11T01:27:52.903 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46522.log.gz 2024-09-11T01:27:52.903 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.28484.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39567.log 2024-09-11T01:27:52.903 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28484.log.gz 2024-09-11T01:27:52.903 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38317.log 2024-09-11T01:27:52.903 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43424.log: /var/log/ceph/ceph-client.admin.39567.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43424.log.gz 2024-09-11T01:27:52.903 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47042.log 2024-09-11T01:27:52.903 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39567.log.gz 2024-09-11T01:27:52.903 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28923.log 2024-09-11T01:27:52.903 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38317.log.gz 2024-09-11T01:27:52.903 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30033.log 2024-09-11T01:27:52.903 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47042.log.gz 2024-09-11T01:27:52.903 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.28923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50608.log 2024-09-11T01:27:52.904 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28923.log.gz 2024-09-11T01:27:52.904 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.30033.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30033.log.gz 2024-09-11T01:27:52.904 INFO:teuthology.orchestra.run.smithi122.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.31404.log 2024-09-11T01:27:52.904 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50608.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51807.log 2024-09-11T01:27:52.904 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50608.log.gz 2024-09-11T01:27:52.904 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.31404.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.29881.log -- replaced with /var/log/ceph/ceph-client.admin.31404.log.gz 2024-09-11T01:27:52.904 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-11T01:27:52.904 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51807.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49932.log 2024-09-11T01:27:52.904 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51807.log.gz 2024-09-11T01:27:52.904 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34988.log 2024-09-11T01:27:52.904 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29881.log.gz 2024-09-11T01:27:52.904 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36936.log 2024-09-11T01:27:52.905 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49932.log.gz 2024-09-11T01:27:52.905 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46105.log 2024-09-11T01:27:52.905 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34988.log.gz 2024-09-11T01:27:52.905 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44361.log 2024-09-11T01:27:52.905 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36936.log.gz 2024-09-11T01:27:52.905 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41105.log 2024-09-11T01:27:52.905 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46105.log.gz 2024-09-11T01:27:52.905 INFO:teuthology.orchestra.run.smithi122.stderr:gzip/var/log/ceph/ceph-client.admin.44361.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.38915.log 2024-09-11T01:27:52.905 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44361.log.gz 2024-09-11T01:27:52.905 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41105.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41105.log.gz 2024-09-11T01:27:52.905 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29829.log 2024-09-11T01:27:52.905 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47563.log 2024-09-11T01:27:52.905 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38915.log.gz 2024-09-11T01:27:52.905 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29829.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29829.log.gz 2024-09-11T01:27:52.906 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45923.log 2024-09-11T01:27:52.906 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39150.log 2024-09-11T01:27:52.906 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47563.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47563.log.gz 2024-09-11T01:27:52.906 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39593.log 2024-09-11T01:27:52.906 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45923.log.gz 2024-09-11T01:27:52.906 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34572.log 2024-09-11T01:27:52.906 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39150.log.gz 2024-09-11T01:27:52.906 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50868.log 2024-09-11T01:27:52.906 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39593.log.gz 2024-09-11T01:27:52.906 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34572.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51232.log 2024-09-11T01:27:52.906 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34572.log.gz 2024-09-11T01:27:52.906 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30494.log 2024-09-11T01:27:52.906 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50868.log.gz 2024-09-11T01:27:52.907 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33613.log 2024-09-11T01:27:52.907 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51232.log.gz 2024-09-11T01:27:52.907 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.30494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43580.log 2024-09-11T01:27:52.907 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30494.log.gz 2024-09-11T01:27:52.907 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33613.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.33613.log.gz /var/log/ceph/ceph-client.admin.36622.log 2024-09-11T01:27:52.907 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-11T01:27:52.907 INFO:teuthology.orchestra.run.smithi122.stderr:gzip/var/log/ceph/ceph-client.admin.43580.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.29020.log 2024-09-11T01:27:52.907 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43580.log.gz 2024-09-11T01:27:52.907 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39957.log 2024-09-11T01:27:52.907 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36622.log.gz 2024-09-11T01:27:52.907 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32439.log 2024-09-11T01:27:52.907 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29020.log.gz 2024-09-11T01:27:52.907 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48918.log 2024-09-11T01:27:52.907 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39957.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39957.log.gz 2024-09-11T01:27:52.908 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.32439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33665.log 2024-09-11T01:27:52.908 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32439.log.gz 2024-09-11T01:27:52.908 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48918.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48788.log 2024-09-11T01:27:52.908 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48918.log.gz 2024-09-11T01:27:52.908 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47250.log 2024-09-11T01:27:52.908 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33665.log.gz 2024-09-11T01:27:52.908 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38395.log 2024-09-11T01:27:52.908 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48788.log.gz 2024-09-11T01:27:52.908 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52195.log 2024-09-11T01:27:52.908 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47250.log.gz 2024-09-11T01:27:52.908 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41235.log 2024-09-11T01:27:52.908 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38395.log.gz 2024-09-11T01:27:52.908 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.52195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34104.log 2024-09-11T01:27:52.909 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52195.log.gz 2024-09-11T01:27:52.909 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41235.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34754.log 2024-09-11T01:27:52.909 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41235.log.gz 2024-09-11T01:27:52.909 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47068.log 2024-09-11T01:27:52.909 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34104.log.gz 2024-09-11T01:27:52.909 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48655.log 2024-09-11T01:27:52.909 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34754.log.gz 2024-09-11T01:27:52.909 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47068.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47068.log.gz 2024-09-11T01:27:52.909 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45819.log 2024-09-11T01:27:52.909 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45766.log 2024-09-11T01:27:52.909 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48655.log.gz 2024-09-11T01:27:52.909 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48733.log 2024-09-11T01:27:52.909 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45819.log.gz 2024-09-11T01:27:52.909 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38499.log 2024-09-11T01:27:52.910 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45766.log.gz 2024-09-11T01:27:52.910 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48733.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40714.log 2024-09-11T01:27:52.910 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48733.log.gz 2024-09-11T01:27:52.910 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35573.log 2024-09-11T01:27:52.910 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38499.log.gz 2024-09-11T01:27:52.910 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37092.log 2024-09-11T01:27:52.910 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40714.log.gz 2024-09-11T01:27:52.910 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30736.log 2024-09-11T01:27:52.910 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35573.log.gz 2024-09-11T01:27:52.910 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38132.log 2024-09-11T01:27:52.910 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37092.log.gz 2024-09-11T01:27:52.910 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.30736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48577.log 2024-09-11T01:27:52.910 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30736.log.gz 2024-09-11T01:27:52.910 INFO:teuthology.orchestra.run.smithi122.stderr:gzip/var/log/ceph/ceph-client.admin.38132.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44100.log 2024-09-11T01:27:52.911 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38132.log.gz 2024-09-11T01:27:52.911 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32443.log 2024-09-11T01:27:52.911 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48577.log.gz 2024-09-11T01:27:52.911 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46652.log 2024-09-11T01:27:52.911 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44100.log.gz 2024-09-11T01:27:52.911 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.32443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29468.log 2024-09-11T01:27:52.911 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32443.log.gz 2024-09-11T01:27:52.911 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46652.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31140.log 2024-09-11T01:27:52.911 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46652.log.gz 2024-09-11T01:27:52.911 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49802.log 2024-09-11T01:27:52.911 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29468.log.gz 2024-09-11T01:27:52.911 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.31140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39983.log 2024-09-11T01:27:52.911 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31140.log.gz 2024-09-11T01:27:52.912 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48551.log 2024-09-11T01:27:52.912 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49802.log.gz 2024-09-11T01:27:52.912 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39983.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39983.log.gz 2024-09-11T01:27:52.912 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29236.log 2024-09-11T01:27:52.912 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47485.log 2024-09-11T01:27:52.912 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48551.log.gz 2024-09-11T01:27:52.912 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51441.log 2024-09-11T01:27:52.912 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29236.log.gz 2024-09-11T01:27:52.912 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32569.log 2024-09-11T01:27:52.912 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47485.log.gz 2024-09-11T01:27:52.912 INFO:teuthology.orchestra.run.smithi122.stderr:gzip/var/log/ceph/ceph-client.admin.51441.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.29078.log 2024-09-11T01:27:52.912 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51441.log.gz 2024-09-11T01:27:52.912 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.32569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45792.log 2024-09-11T01:27:52.913 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32569.log.gz 2024-09-11T01:27:52.913 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31118.log 2024-09-11T01:27:52.913 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29078.log.gz 2024-09-11T01:27:52.913 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45792.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45792.log.gz 2024-09-11T01:27:52.913 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41757.log 2024-09-11T01:27:52.913 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49880.log 2024-09-11T01:27:52.913 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.31118.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31118.log.gz 2024-09-11T01:27:52.913 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34416.log 2024-09-11T01:27:52.913 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41757.log.gz 2024-09-11T01:27:52.913 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45090.log 2024-09-11T01:27:52.913 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49880.log.gz 2024-09-11T01:27:52.913 INFO:teuthology.orchestra.run.smithi122.stderr:gzip/var/log/ceph/ceph-client.admin.34416.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.45897.log 2024-09-11T01:27:52.913 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34416.log.gz 2024-09-11T01:27:52.914 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40428.log 2024-09-11T01:27:52.914 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45090.log.gz 2024-09-11T01:27:52.914 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37898.log 2024-09-11T01:27:52.914 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45897.log.gz 2024-09-11T01:27:52.914 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45845.log 2024-09-11T01:27:52.914 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40428.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40428.log.gz 2024-09-11T01:27:52.914 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47901.log 2024-09-11T01:27:52.914 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37898.log.gz 2024-09-11T01:27:52.914 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29384.log 2024-09-11T01:27:52.914 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45845.log.gz 2024-09-11T01:27:52.914 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41705.log 2024-09-11T01:27:52.914 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47901.log.gz 2024-09-11T01:27:52.914 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40194.log 2024-09-11T01:27:52.914 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29384.log.gz 2024-09-11T01:27:52.915 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41705.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-11T01:27:52.915 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41705.log.gz 2024-09-11T01:27:52.915 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5/var/log/ceph/ceph-client.admin.40194.log: --verbose -- /var/log/ceph/ceph-client.admin.36780.log 2024-09-11T01:27:52.915 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40194.log.gz 2024-09-11T01:27:52.915 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44751.log 2024-09-11T01:27:52.915 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44908.log 2024-09-11T01:27:52.915 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36780.log.gz 2024-09-11T01:27:52.915 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51024.log 2024-09-11T01:27:52.915 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44751.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44751.log.gz 2024-09-11T01:27:52.915 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29157.log 2024-09-11T01:27:52.915 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44908.log.gz 2024-09-11T01:27:52.915 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33872.log 2024-09-11T01:27:52.915 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51024.log.gz 2024-09-11T01:27:52.915 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48814.log 2024-09-11T01:27:52.916 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29157.log.gz 2024-09-11T01:27:52.916 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44543.log 2024-09-11T01:27:52.916 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33872.log.gz 2024-09-11T01:27:52.916 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44803.log 2024-09-11T01:27:52.916 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48814.log.gz 2024-09-11T01:27:52.916 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41939.log 2024-09-11T01:27:52.916 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44543.log.gz 2024-09-11T01:27:52.916 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50582.log 2024-09-11T01:27:52.916 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44803.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44803.log.gz 2024-09-11T01:27:52.916 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39385.log 2024-09-11T01:27:52.916 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41939.log.gz 2024-09-11T01:27:52.916 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33820.log 2024-09-11T01:27:52.916 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50582.log.gz 2024-09-11T01:27:52.917 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47328.log 2024-09-11T01:27:52.917 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39385.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39385.log.gz 2024-09-11T01:27:52.917 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31096.log 2024-09-11T01:27:52.917 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33820.log.gz 2024-09-11T01:27:52.917 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51963.log 2024-09-11T01:27:52.917 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47328.log.gz 2024-09-11T01:27:52.917 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.31096.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37638.log 2024-09-11T01:27:52.917 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31096.log.gz 2024-09-11T01:27:52.917 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34835.log 2024-09-11T01:27:52.917 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51963.log.gz 2024-09-11T01:27:52.917 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37638.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52145.log 2024-09-11T01:27:52.917 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37638.log.gz 2024-09-11T01:27:52.917 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34835.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34835.log.gz 2024-09-11T01:27:52.917 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36702.log 2024-09-11T01:27:52.918 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.52145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36806.log 2024-09-11T01:27:52.918 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52145.log.gz 2024-09-11T01:27:52.918 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5/var/log/ceph/ceph-client.admin.36702.log: --verbose -- /var/log/ceph/ceph-client.admin.51363.log 2024-09-11T01:27:52.918 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36702.log.gz 2024-09-11T01:27:52.918 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35639.log 2024-09-11T01:27:52.918 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36806.log.gz 2024-09-11T01:27:52.918 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51363.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.51363.log.gz -- 2024-09-11T01:27:52.918 INFO:teuthology.orchestra.run.smithi122.stderr: /var/log/ceph/ceph-client.admin.30208.log 2024-09-11T01:27:52.918 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35639.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30670.log 2024-09-11T01:27:52.918 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35639.log.gz 2024-09-11T01:27:52.918 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.30208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25436.log 2024-09-11T01:27:52.918 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30208.log.gz 2024-09-11T01:27:52.918 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.30670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30854.log 2024-09-11T01:27:52.919 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30670.log.gz 2024-09-11T01:27:52.919 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.25436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35089.log 2024-09-11T01:27:52.919 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25436.log.gz 2024-09-11T01:27:52.919 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.30854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45871.log 2024-09-11T01:27:52.919 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30854.log.gz 2024-09-11T01:27:52.919 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35089.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35089.log.gz 2024-09-11T01:27:52.919 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39254.log 2024-09-11T01:27:52.919 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.45871.log: /var/log/ceph/ceph-client.admin.37742.log 2024-09-11T01:27:52.919 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45871.log.gz 2024-09-11T01:27:52.919 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37612.log 2024-09-11T01:27:52.919 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39254.log.gz 2024-09-11T01:27:52.919 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37742.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37742.log.gz 2024-09-11T01:27:52.919 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51833.log 2024-09-11T01:27:52.919 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35111.log 2024-09-11T01:27:52.920 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37612.log.gz 2024-09-11T01:27:52.920 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44934.log 2024-09-11T01:27:52.920 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51833.log.gz 2024-09-11T01:27:52.920 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38210.log 2024-09-11T01:27:52.920 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35111.log.gz 2024-09-11T01:27:52.920 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5/var/log/ceph/ceph-client.admin.44934.log: --verbose -- /var/log/ceph/ceph-client.admin.40113.log 2024-09-11T01:27:52.920 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44934.log.gz 2024-09-11T01:27:52.920 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29722.log 2024-09-11T01:27:52.920 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38210.log.gz 2024-09-11T01:27:52.920 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40324.log 2024-09-11T01:27:52.920 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40113.log.gz 2024-09-11T01:27:52.920 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29618.log 2024-09-11T01:27:52.920 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29722.log.gz 2024-09-11T01:27:52.920 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42852.log 2024-09-11T01:27:52.921 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40324.log.gz 2024-09-11T01:27:52.921 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51650.log 2024-09-11T01:27:52.921 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29618.log.gz 2024-09-11T01:27:52.921 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50556.log 2024-09-11T01:27:52.921 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42852.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42852.log.gz 2024-09-11T01:27:52.921 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44986.log 2024-09-11T01:27:52.921 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51650.log.gz 2024-09-11T01:27:52.921 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42615.log 2024-09-11T01:27:52.921 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50556.log.gz 2024-09-11T01:27:52.921 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30648.log 2024-09-11T01:27:52.921 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44986.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44986.log.gz 2024-09-11T01:27:52.921 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42069.log 2024-09-11T01:27:52.921 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42615.log.gz 2024-09-11T01:27:52.921 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.30648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52041.log 2024-09-11T01:27:52.922 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30648.log.gz 2024-09-11T01:27:52.922 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37430.log 2024-09-11T01:27:52.922 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42069.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42069.log.gz 2024-09-11T01:27:52.922 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.52041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46600.log 2024-09-11T01:27:52.922 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52041.log.gz 2024-09-11T01:27:52.922 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37430.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49438.log 2024-09-11T01:27:52.922 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37430.log.gz 2024-09-11T01:27:52.922 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37352.log 2024-09-11T01:27:52.922 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46600.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46600.log.gz 2024-09-11T01:27:52.922 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43710.log 2024-09-11T01:27:52.922 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49438.log.gz 2024-09-11T01:27:52.922 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29332.log 2024-09-11T01:27:52.922 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37352.log.gz 2024-09-11T01:27:52.922 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40220.log/var/log/ceph/ceph-client.admin.43710.log: 2024-09-11T01:27:52.923 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43710.log.gz 2024-09-11T01:27:52.923 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44595.log 2024-09-11T01:27:52.923 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29332.log.gz 2024-09-11T01:27:52.923 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49568.log 2024-09-11T01:27:52.923 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40220.log.gz 2024-09-11T01:27:52.923 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50296.log 2024-09-11T01:27:52.923 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44595.log.gz 2024-09-11T01:27:52.923 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38733.log 2024-09-11T01:27:52.923 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49568.log.gz 2024-09-11T01:27:52.923 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51989.log 2024-09-11T01:27:52.923 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50296.log.gz 2024-09-11T01:27:52.923 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36648.log 2024-09-11T01:27:52.923 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38733.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38733.log.gz 2024-09-11T01:27:52.923 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51989.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51989.log.gz 2024-09-11T01:27:52.924 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45949.log 2024-09-11T01:27:52.924 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49516.log 2024-09-11T01:27:52.924 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36648.log.gz 2024-09-11T01:27:52.924 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38291.log 2024-09-11T01:27:52.924 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45949.log.gz 2024-09-11T01:27:52.924 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45714.log 2024-09-11T01:27:52.924 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49516.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49516.log.gz 2024-09-11T01:27:52.924 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29077.log 2024-09-11T01:27:52.924 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38291.log.gz 2024-09-11T01:27:52.924 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39020.log 2024-09-11T01:27:52.924 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45714.log.gz 2024-09-11T01:27:52.924 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51285.log 2024-09-11T01:27:52.924 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29077.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29077.log.gz 2024-09-11T01:27:52.924 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50348.log 2024-09-11T01:27:52.925 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39020.log.gz 2024-09-11T01:27:52.925 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34624.log 2024-09-11T01:27:52.925 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51285.log.gz 2024-09-11T01:27:52.925 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39515.log 2024-09-11T01:27:52.925 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50348.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50348.log.gz 2024-09-11T01:27:52.925 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34156.log 2024-09-11T01:27:52.925 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34624.log.gz 2024-09-11T01:27:52.925 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33791.log 2024-09-11T01:27:52.925 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39515.log.gz 2024-09-11T01:27:52.925 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37222.log 2024-09-11T01:27:52.925 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34156.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34156.log.gz 2024-09-11T01:27:52.925 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.28898.log 2024-09-11T01:27:52.925 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33791.log.gz 2024-09-11T01:27:52.926 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31360.log 2024-09-11T01:27:52.926 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37222.log.gz 2024-09-11T01:27:52.926 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38369.log 2024-09-11T01:27:52.926 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.28898.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.28898.log.gz 2024-09-11T01:27:52.926 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.31360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47667.log 2024-09-11T01:27:52.926 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31360.log.gz 2024-09-11T01:27:52.926 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31228.log 2024-09-11T01:27:52.926 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38369.log.gz 2024-09-11T01:27:52.926 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47667.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-11T01:27:52.926 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47667.log.gz 2024-09-11T01:27:52.926 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.31228.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38473.log 2024-09-11T01:27:52.926 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31228.log.gz 2024-09-11T01:27:52.927 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44335.log 2024-09-11T01:27:52.927 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38473.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51598.log 2024-09-11T01:27:52.927 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38473.log.gz 2024-09-11T01:27:52.927 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48892.log 2024-09-11T01:27:52.927 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44335.log.gz 2024-09-11T01:27:52.927 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34078.log 2024-09-11T01:27:52.927 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51598.log.gz 2024-09-11T01:27:52.927 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25089.log 2024-09-11T01:27:52.927 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48892.log.gz 2024-09-11T01:27:52.927 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34078.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34078.log.gz 2024-09-11T01:27:52.927 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41367.log 2024-09-11T01:27:52.927 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.25089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38421.log 2024-09-11T01:27:52.927 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25089.log.gz 2024-09-11T01:27:52.928 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41367.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47120.log 2024-09-11T01:27:52.928 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41367.log.gz 2024-09-11T01:27:52.928 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38421.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47927.log 2024-09-11T01:27:52.928 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38421.log.gz 2024-09-11T01:27:52.928 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38837.log 2024-09-11T01:27:52.928 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47120.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47120.log.gz 2024-09-11T01:27:52.928 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43918.log 2024-09-11T01:27:52.928 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47927.log.gz 2024-09-11T01:27:52.928 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41991.log 2024-09-11T01:27:52.928 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38837.log.gz 2024-09-11T01:27:52.928 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30163.log 2024-09-11T01:27:52.928 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43918.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43918.log.gz 2024-09-11T01:27:52.928 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41991.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-11T01:27:52.928 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41991.log.gz 2024-09-11T01:27:52.929 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.30163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50218.log 2024-09-11T01:27:52.929 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30163.log.gz 2024-09-11T01:27:52.929 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47641.log 2024-09-11T01:27:52.929 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50218.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44699.log 2024-09-11T01:27:52.929 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50218.log.gz 2024-09-11T01:27:52.929 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50010.log 2024-09-11T01:27:52.929 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47641.log.gz 2024-09-11T01:27:52.929 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35419.log 2024-09-11T01:27:52.929 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44699.log.gz 2024-09-11T01:27:52.929 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49230.log 2024-09-11T01:27:52.929 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50010.log.gz 2024-09-11T01:27:52.929 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47276.log 2024-09-11T01:27:52.929 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35419.log.gz 2024-09-11T01:27:52.929 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40662.log 2024-09-11T01:27:52.930 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49230.log.gz 2024-09-11T01:27:52.930 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38236.log 2024-09-11T01:27:52.930 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47276.log.gz 2024-09-11T01:27:52.930 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47953.log 2024-09-11T01:27:52.930 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40662.log.gz 2024-09-11T01:27:52.930 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31338.log 2024-09-11T01:27:52.930 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38236.log.gz 2024-09-11T01:27:52.930 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43528.log 2024-09-11T01:27:52.930 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47953.log.gz 2024-09-11T01:27:52.930 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.31338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34650.log 2024-09-11T01:27:52.930 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31338.log.gz 2024-09-11T01:27:52.930 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43658.log 2024-09-11T01:27:52.930 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43528.log.gz 2024-09-11T01:27:52.930 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37040.log 2024-09-11T01:27:52.931 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34650.log.gz 2024-09-11T01:27:52.931 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33661.log 2024-09-11T01:27:52.931 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43658.log.gz 2024-09-11T01:27:52.931 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32517.log 2024-09-11T01:27:52.931 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37040.log.gz 2024-09-11T01:27:52.931 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35617.log 2024-09-11T01:27:52.931 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33661.log.gz 2024-09-11T01:27:52.931 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.32517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44178.log 2024-09-11T01:27:52.931 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32517.log.gz 2024-09-11T01:27:52.931 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37456.log 2024-09-11T01:27:52.931 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35617.log.gz 2024-09-11T01:27:52.931 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44178.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45272.log 2024-09-11T01:27:52.932 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44178.log.gz 2024-09-11T01:27:52.932 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48395.log 2024-09-11T01:27:52.932 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37456.log.gz 2024-09-11T01:27:52.932 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40506.log 2024-09-11T01:27:52.932 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45272.log.gz 2024-09-11T01:27:52.932 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44621.log 2024-09-11T01:27:52.932 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48395.log.gz 2024-09-11T01:27:52.932 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41131.log 2024-09-11T01:27:52.932 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40506.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40506.log.gz 2024-09-11T01:27:52.932 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43086.log 2024-09-11T01:27:52.932 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44621.log.gz 2024-09-11T01:27:52.932 INFO:teuthology.orchestra.run.smithi122.stderr: 95.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-11T01:27:52.932 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33609.log 2024-09-11T01:27:52.933 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41131.log.gz 2024-09-11T01:27:52.933 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37924.log 2024-09-11T01:27:52.933 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43086.log.gz 2024-09-11T01:27:52.933 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34234.log 2024-09-11T01:27:52.933 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33609.log.gz 2024-09-11T01:27:52.933 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37924.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37924.log.gz 2024-09-11T01:27:52.933 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51311.log 2024-09-11T01:27:52.933 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43762.log 2024-09-11T01:27:52.933 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34234.log.gz 2024-09-11T01:27:52.933 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51311.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51311.log.gz 2024-09-11T01:27:52.933 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35066.log 2024-09-11T01:27:52.933 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43762.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31074.log 2024-09-11T01:27:52.933 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43762.log.gz 2024-09-11T01:27:52.934 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41079.log 2024-09-11T01:27:52.934 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35066.log.gz 2024-09-11T01:27:52.934 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.31074.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45636.log 2024-09-11T01:27:52.934 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31074.log.gz 2024-09-11T01:27:52.934 INFO:teuthology.orchestra.run.smithi122.stderr:gzip/var/log/ceph/ceph-client.admin.41079.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.49984.log 2024-09-11T01:27:52.934 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41079.log.gz 2024-09-11T01:27:52.934 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46678.log 2024-09-11T01:27:52.934 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45636.log.gz 2024-09-11T01:27:52.934 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41575.log 2024-09-11T01:27:52.934 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49984.log.gz 2024-09-11T01:27:52.934 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43320.log 2024-09-11T01:27:52.934 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46678.log.gz 2024-09-11T01:27:52.934 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43294.log 2024-09-11T01:27:52.934 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41575.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41575.log.gz 2024-09-11T01:27:52.935 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51467.log 2024-09-11T01:27:52.935 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43320.log.gz 2024-09-11T01:27:52.935 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34598.log 2024-09-11T01:27:52.935 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43294.log.gz 2024-09-11T01:27:52.935 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45454.log 2024-09-11T01:27:52.935 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51467.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51467.log.gz 2024-09-11T01:27:52.935 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49178.log 2024-09-11T01:27:52.935 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34598.log.gz 2024-09-11T01:27:52.935 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30186.log 2024-09-11T01:27:52.935 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45454.log.gz 2024-09-11T01:27:52.935 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44205.log 2024-09-11T01:27:52.935 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49178.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49178.log.gz 2024-09-11T01:27:52.935 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.30186.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-11T01:27:52.935 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30186.log.gz 2024-09-11T01:27:52.936 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31184.log 2024-09-11T01:27:52.936 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44205.log.gz 2024-09-11T01:27:52.936 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37794.log 2024-09-11T01:27:52.936 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph-client.admin.31184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36570.log 2024-09-11T01:27:52.936 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31184.log.gz 2024-09-11T01:27:52.936 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30384.log 2024-09-11T01:27:52.936 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37794.log.gz 2024-09-11T01:27:52.936 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51154.log 2024-09-11T01:27:52.936 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36570.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36570.log.gz 2024-09-11T01:27:52.936 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.30384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46886.log 2024-09-11T01:27:52.936 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30384.log.gz 2024-09-11T01:27:52.936 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35177.log 2024-09-11T01:27:52.936 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51154.log.gz 2024-09-11T01:27:52.937 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38447.log 2024-09-11T01:27:52.937 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46886.log.gz 2024-09-11T01:27:52.937 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29803.log 2024-09-11T01:27:52.937 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35177.log.gz 2024-09-11T01:27:52.937 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38447.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.33479.log 2024-09-11T01:27:52.937 INFO:teuthology.orchestra.run.smithi122.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38447.log.gz 2024-09-11T01:27:52.937 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41913.log 2024-09-11T01:27:52.937 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29803.log.gz 2024-09-11T01:27:52.937 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34130.log 2024-09-11T01:27:52.937 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33479.log.gz 2024-09-11T01:27:52.937 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41315.log 2024-09-11T01:27:52.937 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41913.log.gz 2024-09-11T01:27:52.937 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40376.log 2024-09-11T01:27:52.938 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34130.log.gz 2024-09-11T01:27:52.938 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33925.log/var/log/ceph/ceph-client.admin.41315.log: 2024-09-11T01:27:52.938 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41315.log.gz 2024-09-11T01:27:52.938 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49282.log 2024-09-11T01:27:52.938 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40376.log.gz 2024-09-11T01:27:52.938 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33925.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43450.log 2024-09-11T01:27:52.938 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33925.log.gz 2024-09-11T01:27:52.938 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51571.log 2024-09-11T01:27:52.938 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49282.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49282.log.gz 2024-09-11T01:27:52.938 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39176.log 2024-09-11T01:27:52.938 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43450.log.gz 2024-09-11T01:27:52.938 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51571.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40896.log 2024-09-11T01:27:52.938 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51571.log.gz 2024-09-11T01:27:52.938 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39176.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39176.log.gzgzip 2024-09-11T01:27:52.938 INFO:teuthology.orchestra.run.smithi122.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43008.log 2024-09-11T01:27:52.939 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40009.log 2024-09-11T01:27:52.939 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40896.log.gz 2024-09-11T01:27:52.939 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43008.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43554.log 2024-09-11T01:27:52.939 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43008.log.gz 2024-09-11T01:27:52.939 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47172.log 2024-09-11T01:27:52.939 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40009.log.gz 2024-09-11T01:27:52.939 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46365.log 2024-09-11T01:27:52.939 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43554.log.gz 2024-09-11T01:27:52.939 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47172.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44725.log 2024-09-11T01:27:52.939 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47172.log.gz 2024-09-11T01:27:52.939 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30714.log 2024-09-11T01:27:52.939 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46365.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46365.log.gz 2024-09-11T01:27:52.940 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34836.log 2024-09-11T01:27:52.940 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44725.log.gz 2024-09-11T01:27:52.940 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.30714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33509.log 2024-09-11T01:27:52.940 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30714.log.gz 2024-09-11T01:27:52.940 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40035.log 2024-09-11T01:27:52.940 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34836.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34836.log.gz 2024-09-11T01:27:52.940 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45246.log 2024-09-11T01:27:52.940 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33509.log.gz 2024-09-11T01:27:52.940 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43190.log 2024-09-11T01:27:52.940 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40035.log.gz 2024-09-11T01:27:52.940 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30964.log 2024-09-11T01:27:52.940 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45246.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45246.log.gz 2024-09-11T01:27:52.940 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48031.log 2024-09-11T01:27:52.940 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43190.log.gz 2024-09-11T01:27:52.941 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.30964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35331.log 2024-09-11T01:27:52.941 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30964.log.gz 2024-09-11T01:27:52.941 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34520.log 2024-09-11T01:27:52.941 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48031.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48031.log.gz 2024-09-11T01:27:52.941 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44074.log 2024-09-11T01:27:52.941 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35331.log.gz 2024-09-11T01:27:52.941 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34520.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33821.log 2024-09-11T01:27:52.941 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34520.log.gz 2024-09-11T01:27:52.941 INFO:teuthology.orchestra.run.smithi122.stderr:gzip/var/log/ceph/ceph-client.admin.44074.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.37014.log 2024-09-11T01:27:52.941 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44074.log.gz 2024-09-11T01:27:52.941 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29080.log 2024-09-11T01:27:52.941 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33821.log.gz 2024-09-11T01:27:52.942 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37014.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29306.log 2024-09-11T01:27:52.942 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37014.log.gz 2024-09-11T01:27:52.942 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29080.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35155.log 2024-09-11T01:27:52.942 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29080.log.gz 2024-09-11T01:27:52.942 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29626.log 2024-09-11T01:27:52.942 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29306.log.gz 2024-09-11T01:27:52.942 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30898.log 2024-09-11T01:27:52.942 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35155.log.gz 2024-09-11T01:27:52.942 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33973.log 2024-09-11T01:27:52.942 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29626.log.gz 2024-09-11T01:27:52.942 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.30898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49334.log 2024-09-11T01:27:52.942 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30898.log.gz 2024-09-11T01:27:52.942 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38681.log 2024-09-11T01:27:52.942 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33973.log.gz 2024-09-11T01:27:52.943 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48840.log 2024-09-11T01:27:52.943 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49334.log.gz 2024-09-11T01:27:52.943 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45532.log 2024-09-11T01:27:52.943 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38681.log.gz 2024-09-11T01:27:52.943 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43944.log 2024-09-11T01:27:52.943 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48840.log.gz 2024-09-11T01:27:52.943 INFO:teuthology.orchestra.run.smithi122.stderr:gzip/var/log/ceph/ceph-client.admin.45532.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.29410.log 2024-09-11T01:27:52.943 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45532.log.gz 2024-09-11T01:27:52.943 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49672.log 2024-09-11T01:27:52.943 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43944.log.gz 2024-09-11T01:27:52.943 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40061.log 2024-09-11T01:27:52.943 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29410.log.gz 2024-09-11T01:27:52.943 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49672.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49672.log.gz 2024-09-11T01:27:52.943 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46938.log 2024-09-11T01:27:52.944 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40818.log 2024-09-11T01:27:52.944 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40061.log.gz 2024-09-11T01:27:52.944 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45038.log 2024-09-11T01:27:52.944 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46938.log.gz 2024-09-11T01:27:52.944 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39202.log 2024-09-11T01:27:52.944 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40818.log.gz 2024-09-11T01:27:52.944 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48970.log 2024-09-11T01:27:52.944 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45038.log.gz 2024-09-11T01:27:52.944 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35133.log 2024-09-11T01:27:52.944 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39202.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39202.log.gz 2024-09-11T01:27:52.944 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49854.log 2024-09-11T01:27:52.944 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48970.log.gz 2024-09-11T01:27:52.944 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41523.log 2024-09-11T01:27:52.945 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35133.log.gz 2024-09-11T01:27:52.945 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47302.log 2024-09-11T01:27:52.945 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49854.log.gz 2024-09-11T01:27:52.945 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38603.log 2024-09-11T01:27:52.945 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41523.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41523.log.gz 2024-09-11T01:27:52.945 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47302.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47302.log.gz 2024-09-11T01:27:52.945 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29358.log 2024-09-11T01:27:52.945 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43034.log 2024-09-11T01:27:52.945 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38603.log.gz 2024-09-11T01:27:52.945 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40142.log 2024-09-11T01:27:52.945 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29358.log.gz 2024-09-11T01:27:52.945 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29540.log 2024-09-11T01:27:52.945 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43034.log.gz 2024-09-11T01:27:52.945 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35265.log 2024-09-11T01:27:52.946 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40142.log.gz 2024-09-11T01:27:52.946 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29540.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-11T01:27:52.946 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29540.log.gz 2024-09-11T01:27:52.946 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30758.log 2024-09-11T01:27:52.946 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35265.log.gz 2024-09-11T01:27:52.946 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34546.log 2024-09-11T01:27:52.946 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.30758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40740.log 2024-09-11T01:27:52.946 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30758.log.gz 2024-09-11T01:27:52.946 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30230.log 2024-09-11T01:27:52.946 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34546.log.gz 2024-09-11T01:27:52.946 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40740.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40740.log.gz 2024-09-11T01:27:52.946 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36754.log 2024-09-11T01:27:52.946 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.30230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44387.log 2024-09-11T01:27:52.947 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30230.log.gz 2024-09-11T01:27:52.947 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46808.log 2024-09-11T01:27:52.947 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36754.log.gz 2024-09-11T01:27:52.947 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44387.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38629.log 2024-09-11T01:27:52.947 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44387.log.gz 2024-09-11T01:27:52.947 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50660.log 2024-09-11T01:27:52.947 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46808.log.gz 2024-09-11T01:27:52.947 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50400.log 2024-09-11T01:27:52.947 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38629.log.gz 2024-09-11T01:27:52.947 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50660.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32309.log 2024-09-11T01:27:52.947 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50660.log.gz 2024-09-11T01:27:52.947 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46261.log 2024-09-11T01:27:52.947 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50400.log.gz 2024-09-11T01:27:52.948 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.32309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51859.log 2024-09-11T01:27:52.948 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32309.log.gz 2024-09-11T01:27:52.948 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46261.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46261.log.gz 2024-09-11T01:27:52.948 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34390.log 2024-09-11T01:27:52.948 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25313.log 2024-09-11T01:27:52.948 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51859.log.gz 2024-09-11T01:27:52.948 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30692.log 2024-09-11T01:27:52.948 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34390.log.gz 2024-09-11T01:27:52.948 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40688.log 2024-09-11T01:27:52.948 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.25313.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25313.log.gz 2024-09-11T01:27:52.948 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.30692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50998.log 2024-09-11T01:27:52.948 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30692.log.gz 2024-09-11T01:27:52.948 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41393.log 2024-09-11T01:27:52.949 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40688.log.gz 2024-09-11T01:27:52.949 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46912.log 2024-09-11T01:27:52.949 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50998.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50998.log.gz 2024-09-11T01:27:52.949 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41393.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30626.log 2024-09-11T01:27:52.949 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41393.log.gz 2024-09-11T01:27:52.949 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42537.log 2024-09-11T01:27:52.949 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46912.log.gz 2024-09-11T01:27:52.949 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33561.log 2024-09-11T01:27:52.949 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.30626.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30626.log.gz 2024-09-11T01:27:52.949 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48525.log 2024-09-11T01:27:52.949 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42537.log.gz 2024-09-11T01:27:52.949 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40792.log 2024-09-11T01:27:52.949 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33561.log.gz 2024-09-11T01:27:52.950 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33924.log 2024-09-11T01:27:52.950 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48525.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48525.log.gz 2024-09-11T01:27:52.950 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38002.log 2024-09-11T01:27:52.950 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40792.log.gz 2024-09-11T01:27:52.950 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47459.log 2024-09-11T01:27:52.950 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33924.log.gz 2024-09-11T01:27:52.950 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35353.log 2024-09-11T01:27:52.950 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38002.log.gz 2024-09-11T01:27:52.950 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42589.log 2024-09-11T01:27:52.950 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47459.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47459.log.gz 2024-09-11T01:27:52.950 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44673.log 2024-09-11T01:27:52.950 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35353.log.gz 2024-09-11T01:27:52.950 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34702.log 2024-09-11T01:27:52.950 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42589.log.gz 2024-09-11T01:27:52.951 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44673.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.21474.log 2024-09-11T01:27:52.951 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44673.log.gz 2024-09-11T01:27:52.951 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25289.log 2024-09-11T01:27:52.951 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34702.log.gz 2024-09-11T01:27:52.951 INFO:teuthology.orchestra.run.smithi122.stderr:gzip/var/log/ceph/ceph.tmp-client.admin.21474.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.31382.log 2024-09-11T01:27:52.951 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.21474.log.gz 2024-09-11T01:27:52.951 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.25289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34888.log 2024-09-11T01:27:52.951 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25289.log.gz 2024-09-11T01:27:52.951 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.31382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39671.log 2024-09-11T01:27:52.951 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31382.log.gz 2024-09-11T01:27:52.951 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50504.log 2024-09-11T01:27:52.951 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34888.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34888.log.gz 2024-09-11T01:27:52.951 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30472.log 2024-09-11T01:27:52.951 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39671.log.gz 2024-09-11T01:27:52.952 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35686.log 2024-09-11T01:27:52.952 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50504.log.gz 2024-09-11T01:27:52.952 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.30472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39931.log 2024-09-11T01:27:52.952 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30472.log.gz 2024-09-11T01:27:52.952 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40558.log 2024-09-11T01:27:52.952 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35686.log.gz 2024-09-11T01:27:52.952 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49100.log 2024-09-11T01:27:52.952 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39931.log.gz 2024-09-11T01:27:52.952 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40636.log 2024-09-11T01:27:52.952 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40558.log.gz 2024-09-11T01:27:52.952 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49100.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40948.log 2024-09-11T01:27:52.952 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49100.log.gz 2024-09-11T01:27:52.952 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35221.log 2024-09-11T01:27:52.952 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40636.log.gz 2024-09-11T01:27:52.953 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29933.log 2024-09-11T01:27:52.953 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.40948.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40948.log.gz 2024-09-11T01:27:52.953 INFO:teuthology.orchestra.run.smithi122.stderr:gzip/var/log/ceph/ceph-client.admin.35221.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.51076.log 2024-09-11T01:27:52.953 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35221.log.gz 2024-09-11T01:27:52.953 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29933.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29933.log.gz -5 2024-09-11T01:27:52.953 INFO:teuthology.orchestra.run.smithi122.stderr: --verbose -- /var/log/ceph/ceph-client.admin.42904.log 2024-09-11T01:27:52.953 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36676.log 2024-09-11T01:27:52.953 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51076.log.gz 2024-09-11T01:27:52.953 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43060.log 2024-09-11T01:27:52.953 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42904.log.gz 2024-09-11T01:27:52.953 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34806.log 2024-09-11T01:27:52.953 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36676.log.gz 2024-09-11T01:27:52.954 INFO:teuthology.orchestra.run.smithi122.stderr:gzip/var/log/ceph/ceph-client.admin.43060.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.34052.log 2024-09-11T01:27:52.954 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43060.log.gz 2024-09-11T01:27:52.954 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49724.log 2024-09-11T01:27:52.954 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34806.log.gz 2024-09-11T01:27:52.954 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34052.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39411.log 2024-09-11T01:27:52.954 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34052.log.gz 2024-09-11T01:27:52.954 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49724.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33508.log 2024-09-11T01:27:52.954 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49724.log.gz 2024-09-11T01:27:52.954 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51050.log 2024-09-11T01:27:52.954 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39411.log.gz 2024-09-11T01:27:52.954 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37196.log 2024-09-11T01:27:52.954 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33508.log.gz 2024-09-11T01:27:52.954 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51050.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30362.log 2024-09-11T01:27:52.954 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51050.log.gz 2024-09-11T01:27:52.955 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29158.log 2024-09-11T01:27:52.955 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37196.log.gz 2024-09-11T01:27:52.955 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.30362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30582.log 2024-09-11T01:27:52.955 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30362.log.gz 2024-09-11T01:27:52.955 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41027.log 2024-09-11T01:27:52.955 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29158.log.gz 2024-09-11T01:27:52.955 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.30582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48447.log 2024-09-11T01:27:52.955 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30582.log.gz 2024-09-11T01:27:52.955 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44569.log 2024-09-11T01:27:52.955 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41027.log.gz 2024-09-11T01:27:52.955 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46756.log 2024-09-11T01:27:52.955 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48447.log.gz 2024-09-11T01:27:52.955 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31250.log 2024-09-11T01:27:52.955 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44569.log.gz 2024-09-11T01:27:52.956 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47615.log 2024-09-11T01:27:52.956 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46756.log.gz 2024-09-11T01:27:52.956 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.31250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29929.log 2024-09-11T01:27:52.956 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31250.log.gz 2024-09-11T01:27:52.956 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37482.log 2024-09-11T01:27:52.956 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47615.log.gz 2024-09-11T01:27:52.956 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45506.log 2024-09-11T01:27:52.956 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29929.log.gz 2024-09-11T01:27:52.956 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45558.log 2024-09-11T01:27:52.956 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37482.log.gz 2024-09-11T01:27:52.956 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38106.log 2024-09-11T01:27:52.956 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45506.log.gz 2024-09-11T01:27:52.956 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51624.log 2024-09-11T01:27:52.957 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45558.log.gz 2024-09-11T01:27:52.957 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38106.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30406.log 2024-09-11T01:27:52.957 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38106.log.gz 2024-09-11T01:27:52.957 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42722.log 2024-09-11T01:27:52.957 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51624.log.gz 2024-09-11T01:27:52.957 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.30406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41209.log 2024-09-11T01:27:52.957 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30406.log.gz 2024-09-11T01:27:52.957 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30538.log 2024-09-11T01:27:52.957 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42722.log.gz 2024-09-11T01:27:52.957 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33873.log 2024-09-11T01:27:52.957 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41209.log.gz 2024-09-11T01:27:52.957 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.30538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45688.log 2024-09-11T01:27:52.957 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30538.log.gz 2024-09-11T01:27:52.957 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47693.log 2024-09-11T01:27:52.958 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33873.log.gz 2024-09-11T01:27:52.958 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32339.log 2024-09-11T01:27:52.958 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45688.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45688.log.gz 2024-09-11T01:27:52.958 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44126.log 2024-09-11T01:27:52.958 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47693.log.gz 2024-09-11T01:27:52.958 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.32339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33713.log 2024-09-11T01:27:52.958 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32339.log.gz 2024-09-11T01:27:52.958 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39280.log 2024-09-11T01:27:52.958 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44126.log.gz 2024-09-11T01:27:52.958 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35595.log 2024-09-11T01:27:52.958 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33713.log.gz 2024-09-11T01:27:52.959 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30428.log 2024-09-11T01:27:52.959 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39280.log.gz 2024-09-11T01:27:52.959 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50166.log 2024-09-11T01:27:52.959 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35595.log.gz 2024-09-11T01:27:52.959 INFO:teuthology.orchestra.run.smithi122.stderr:gzip/var/log/ceph/ceph-client.admin.30428.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.38028.log 2024-09-11T01:27:52.959 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30428.log.gz 2024-09-11T01:27:52.959 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50166.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37066.log 2024-09-11T01:27:52.959 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50166.log.gz 2024-09-11T01:27:52.959 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38028.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38028.log.gz 2024-09-11T01:27:52.959 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43814.log 2024-09-11T01:27:52.959 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31316.log 2024-09-11T01:27:52.960 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37066.log.gz 2024-09-11T01:27:52.960 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43814.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.43814.log.gz 2024-09-11T01:27:52.960 INFO:teuthology.orchestra.run.smithi122.stderr: /var/log/ceph/ceph-client.admin.49126.log 2024-09-11T01:27:52.960 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.31316.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39723.log 2024-09-11T01:27:52.960 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31316.log.gz 2024-09-11T01:27:52.960 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50686.log 2024-09-11T01:27:52.960 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49126.log.gz 2024-09-11T01:27:52.960 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39723.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39723.log.gz 2024-09-11T01:27:52.960 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25234.log 2024-09-11T01:27:52.960 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48109.log 2024-09-11T01:27:52.960 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50686.log.gz 2024-09-11T01:27:52.961 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.25234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41445.log 2024-09-11T01:27:52.961 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25234.log.gz 2024-09-11T01:27:52.961 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45350.log 2024-09-11T01:27:52.961 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48109.log.gz 2024-09-11T01:27:52.961 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41445.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34208.log 2024-09-11T01:27:52.961 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41445.log.gz 2024-09-11T01:27:52.961 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30876.log 2024-09-11T01:27:52.961 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45350.log.gz 2024-09-11T01:27:52.961 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34208.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38994.log 2024-09-11T01:27:52.961 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34208.log.gz 2024-09-11T01:27:52.961 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.30876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46990.log 2024-09-11T01:27:52.961 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30876.log.gz 2024-09-11T01:27:52.961 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44439.log 2024-09-11T01:27:52.962 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38994.log.gz 2024-09-11T01:27:52.962 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37664.log 2024-09-11T01:27:52.962 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46990.log.gz 2024-09-11T01:27:52.962 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33557.log 2024-09-11T01:27:52.962 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44439.log.gz 2024-09-11T01:27:52.962 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47433.log 2024-09-11T01:27:52.962 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37664.log.gz 2024-09-11T01:27:52.962 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33557.log: gzip 0.0% -5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.33557.log.gz -- 2024-09-11T01:27:52.962 INFO:teuthology.orchestra.run.smithi122.stderr: /var/log/ceph/ceph-client.admin.47224.log 2024-09-11T01:27:52.962 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48135.log 2024-09-11T01:27:52.962 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47433.log.gz 2024-09-11T01:27:52.962 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47797.log 2024-09-11T01:27:52.962 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47224.log.gz 2024-09-11T01:27:52.962 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34338.log 2024-09-11T01:27:52.963 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48135.log.gz 2024-09-11T01:27:52.963 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47797.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49464.log 2024-09-11T01:27:52.963 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47797.log.gz 2024-09-11T01:27:52.963 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34338.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-11T01:27:52.963 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34338.log.gz 2024-09-11T01:27:52.963 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49464.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31206.log 2024-09-11T01:27:52.963 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49464.log.gz 2024-09-11T01:27:52.963 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48343.log 2024-09-11T01:27:52.963 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.31206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43970.log 2024-09-11T01:27:52.963 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31206.log.gz 2024-09-11T01:27:52.963 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29985.log 2024-09-11T01:27:52.963 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48343.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48343.log.gz 2024-09-11T01:27:52.963 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43970.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37534.log 2024-09-11T01:27:52.963 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43970.log.gz 2024-09-11T01:27:52.964 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29985.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46157.log 2024-09-11T01:27:52.964 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29985.log.gz 2024-09-11T01:27:52.964 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48005.log 2024-09-11T01:27:52.964 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37534.log.gz 2024-09-11T01:27:52.964 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44152.log 2024-09-11T01:27:52.964 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46157.log.gz 2024-09-11T01:27:52.964 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48005.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42355.log 2024-09-11T01:27:52.964 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48005.log.gz 2024-09-11T01:27:52.964 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51885.log 2024-09-11T01:27:52.964 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44152.log.gz 2024-09-11T01:27:52.964 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43138.log 2024-09-11T01:27:52.964 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42355.log.gz 2024-09-11T01:27:52.964 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44777.log 2024-09-11T01:27:52.965 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51885.log.gz 2024-09-11T01:27:52.965 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33739.log 2024-09-11T01:27:52.965 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43138.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43138.log.gz 2024-09-11T01:27:52.965 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41261.log 2024-09-11T01:27:52.965 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44777.log.gz 2024-09-11T01:27:52.965 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33739.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33739.log.gz 2024-09-11T01:27:52.965 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42982.log 2024-09-11T01:27:52.965 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51781.log 2024-09-11T01:27:52.965 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41261.log.gz 2024-09-11T01:27:52.965 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42982.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42982.log.gz 2024-09-11T01:27:52.965 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36832.log 2024-09-11T01:27:52.965 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51389.log 2024-09-11T01:27:52.965 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51781.log.gz 2024-09-11T01:27:52.965 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37118.log 2024-09-11T01:27:52.966 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36832.log.gz 2024-09-11T01:27:52.966 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43242.log 2024-09-11T01:27:52.966 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51389.log.gz 2024-09-11T01:27:52.966 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46079.log 2024-09-11T01:27:52.966 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37118.log.gz 2024-09-11T01:27:52.966 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50530.log 2024-09-11T01:27:52.966 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43242.log.gz 2024-09-11T01:27:52.966 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45298.log 2024-09-11T01:27:52.966 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46079.log.gz 2024-09-11T01:27:52.966 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50530.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42147.log 2024-09-11T01:27:52.966 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50530.log.gz 2024-09-11T01:27:52.966 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45298.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47823.log 2024-09-11T01:27:52.966 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45298.log.gz 2024-09-11T01:27:52.966 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42147.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48291.log 2024-09-11T01:27:52.967 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42147.log.gz 2024-09-11T01:27:52.967 INFO:teuthology.orchestra.run.smithi122.stderr:gzip/var/log/ceph/ceph-client.admin.47823.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.42251.log 2024-09-11T01:27:52.967 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47823.log.gz 2024-09-11T01:27:52.967 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36988.log 2024-09-11T01:27:52.967 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48291.log.gz 2024-09-11T01:27:52.967 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44465.log 2024-09-11T01:27:52.967 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42251.log.gz 2024-09-11T01:27:52.967 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43892.log 2024-09-11T01:27:52.967 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36988.log.gz 2024-09-11T01:27:52.967 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49906.log 2024-09-11T01:27:52.967 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44465.log.gz 2024-09-11T01:27:52.967 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36858.log 2024-09-11T01:27:52.967 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43892.log.gz 2024-09-11T01:27:52.968 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50842.log 2024-09-11T01:27:52.968 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49906.log.gz 2024-09-11T01:27:52.968 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41183.log 2024-09-11T01:27:52.968 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36858.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36858.log.gz 2024-09-11T01:27:52.968 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48057.log 2024-09-11T01:27:52.968 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50842.log.gz 2024-09-11T01:27:52.968 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37378.log 2024-09-11T01:27:52.968 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41183.log.gz 2024-09-11T01:27:52.968 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41965.log 2024-09-11T01:27:52.968 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48057.log.gz 2024-09-11T01:27:52.968 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33997.log 2024-09-11T01:27:52.968 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37378.log.gz 2024-09-11T01:27:52.968 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46001.log 2024-09-11T01:27:52.969 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41965.log.gz 2024-09-11T01:27:52.969 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.33997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41835.log 2024-09-11T01:27:52.969 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33997.log.gz 2024-09-11T01:27:52.969 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29470.log 2024-09-11T01:27:52.969 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46001.log.gz 2024-09-11T01:27:52.969 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.41835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47354.log 2024-09-11T01:27:52.969 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41835.log.gz 2024-09-11T01:27:52.969 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50088.log 2024-09-11T01:27:52.969 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29470.log.gz 2024-09-11T01:27:52.969 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51937.log 2024-09-11T01:27:52.969 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47354.log.gz 2024-09-11T01:27:52.969 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47719.log 2024-09-11T01:27:52.969 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50088.log.gz 2024-09-11T01:27:52.969 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39228.log 2024-09-11T01:27:52.970 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51937.log.gz 2024-09-11T01:27:52.970 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42095.log 2024-09-11T01:27:52.970 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47719.log.gz 2024-09-11T01:27:52.970 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50192.log 2024-09-11T01:27:52.970 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39228.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39228.log.gz 2024-09-11T01:27:52.970 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47771.log 2024-09-11T01:27:52.970 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42095.log.gz 2024-09-11T01:27:52.970 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35661.log 2024-09-11T01:27:52.970 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50192.log.gz 2024-09-11T01:27:52.970 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47771.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37508.log 2024-09-11T01:27:52.970 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47771.log.gz 2024-09-11T01:27:52.970 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38863.log 2024-09-11T01:27:52.970 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35661.log.gz 2024-09-11T01:27:52.971 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25372.log 2024-09-11T01:27:52.971 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37508.log.gz 2024-09-11T01:27:52.971 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34286.log 2024-09-11T01:27:52.971 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38863.log.gz 2024-09-11T01:27:52.971 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.25372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43684.log 2024-09-11T01:27:52.971 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25372.log.gz 2024-09-11T01:27:52.971 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34286.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30809.log 2024-09-11T01:27:52.971 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34286.log.gz 2024-09-11T01:27:52.971 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42800.log 2024-09-11T01:27:52.971 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43684.log.gz 2024-09-11T01:27:52.971 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.30809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42381.log 2024-09-11T01:27:52.971 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30809.log.gz 2024-09-11T01:27:52.971 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42800.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46470.log 2024-09-11T01:27:52.972 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42800.log.gz 2024-09-11T01:27:52.972 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42433.log 2024-09-11T01:27:52.972 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42381.log.gz 2024-09-11T01:27:52.972 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.46470.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49074.log 2024-09-11T01:27:52.972 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46470.log.gz 2024-09-11T01:27:52.972 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48603.log 2024-09-11T01:27:52.972 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42433.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42433.log.gz 2024-09-11T01:27:52.972 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29852.log/var/log/ceph/ceph-client.admin.49074.log: 2024-09-11T01:27:52.972 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49074.log.gz 2024-09-11T01:27:52.972 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44022.log 2024-09-11T01:27:52.972 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48603.log.gz 2024-09-11T01:27:52.972 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42644.log 2024-09-11T01:27:52.972 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29852.log.gz 2024-09-11T01:27:52.972 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51128.log 2024-09-11T01:27:52.973 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.44022.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44022.log.gz 2024-09-11T01:27:52.973 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51755.log 2024-09-11T01:27:52.973 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42644.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42644.log.gz 2024-09-11T01:27:52.973 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51128.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30560.log 2024-09-11T01:27:52.973 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51128.log.gz 2024-09-11T01:27:52.973 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35375.log 2024-09-11T01:27:52.973 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51755.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51755.log.gz 2024-09-11T01:27:52.973 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.30560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51728.log 2024-09-11T01:27:52.973 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30560.log.gz 2024-09-11T01:27:52.973 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49256.log 2024-09-11T01:27:52.973 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35375.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35375.log.gz 2024-09-11T01:27:52.973 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35287.log 2024-09-11T01:27:52.973 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51728.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51728.log.gz 2024-09-11T01:27:52.973 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.49256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41497.log 2024-09-11T01:27:52.974 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49256.log.gz 2024-09-11T01:27:52.974 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.35287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50452.log 2024-09-11T01:27:52.974 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35287.log.gz 2024-09-11T01:27:52.974 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.41497.log: /var/log/ceph/ceph-client.admin.29880.log 2024-09-11T01:27:52.974 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41497.log.gz 2024-09-11T01:27:52.974 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45376.log 2024-09-11T01:27:52.974 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50452.log.gz 2024-09-11T01:27:52.974 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29748.log 2024-09-11T01:27:52.974 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29880.log.gz 2024-09-11T01:27:52.974 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45376.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29981.log 2024-09-11T01:27:52.974 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45376.log.gz 2024-09-11T01:27:52.974 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39619.log 2024-09-11T01:27:52.974 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29748.log.gz 2024-09-11T01:27:52.975 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34260.log 2024-09-11T01:27:52.975 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29981.log.gz 2024-09-11T01:27:52.975 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39619.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39619.log.gz 2024-09-11T01:27:52.975 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42670.log 2024-09-11T01:27:52.975 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42173.log 2024-09-11T01:27:52.975 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34260.log.gz 2024-09-11T01:27:52.975 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39905.log 2024-09-11T01:27:52.975 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42670.log.gz 2024-09-11T01:27:52.975 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39697.log 2024-09-11T01:27:52.975 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42173.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42173.log.gz 2024-09-11T01:27:52.975 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37248.log 2024-09-11T01:27:52.975 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39905.log.gz 2024-09-11T01:27:52.975 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.39697.log: gzip -5 --verbose 0.0% -- /var/log/ceph/ceph-client.admin.29469.log -- replaced with /var/log/ceph/ceph-client.admin.39697.log.gz 2024-09-11T01:27:52.976 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-11T01:27:52.976 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50062.log 2024-09-11T01:27:52.976 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37248.log.gz 2024-09-11T01:27:52.976 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.29469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45012.log 2024-09-11T01:27:52.976 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29469.log.gz 2024-09-11T01:27:52.976 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.50062.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38158.log 2024-09-11T01:27:52.976 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50062.log.gz 2024-09-11T01:27:52.976 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42826.log 2024-09-11T01:27:52.976 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45012.log.gz 2024-09-11T01:27:52.976 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.38158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37326.log 2024-09-11T01:27:52.976 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38158.log.gz 2024-09-11T01:27:52.976 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34442.log 2024-09-11T01:27:52.976 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42826.log.gz 2024-09-11T01:27:52.977 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42459.log 2024-09-11T01:27:52.977 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.37326.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37326.log.gz 2024-09-11T01:27:52.977 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47380.log 2024-09-11T01:27:52.977 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.34442.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34442.log.gz 2024-09-11T01:27:52.977 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32595.log 2024-09-11T01:27:52.977 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42459.log.gz 2024-09-11T01:27:52.977 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.47380.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43866.log 2024-09-11T01:27:52.977 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47380.log.gz 2024-09-11T01:27:52.977 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.32595.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32595.log.gz 2024-09-11T01:27:52.977 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48707.log 2024-09-11T01:27:52.977 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36910.log 2024-09-11T01:27:52.977 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43866.log.gz 2024-09-11T01:27:52.977 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.48707.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48707.log.gz 2024-09-11T01:27:52.977 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43840.log 2024-09-11T01:27:52.978 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.36910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45584.log 2024-09-11T01:27:52.978 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36910.log.gz 2024-09-11T01:27:52.978 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.43840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42748.log 2024-09-11T01:27:52.978 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43840.log.gz 2024-09-11T01:27:52.978 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.45584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30340.log 2024-09-11T01:27:52.978 INFO:teuthology.orchestra.run.smithi122.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.45584.log.gz 2024-09-11T01:27:52.978 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51493.log 2024-09-11T01:27:52.978 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.42748.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42748.log.gz 2024-09-11T01:27:52.978 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.30340.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30340.log.gz 2024-09-11T01:27:52.978 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/ceph-client.admin.51493.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51493.log.gz 2024-09-11T01:27:52.978 INFO:teuthology.orchestra.run.smithi122.stderr: 95.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-11T01:27:53.074 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2024-09-11T01:27:53.580 INFO:teuthology.orchestra.run.smithi122.stderr: 90.8% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-11T01:27:55.346 INFO:teuthology.orchestra.run.smithi122.stderr: 90.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-11T01:27:55.438 INFO:teuthology.orchestra.run.smithi122.stderr: 90.5% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-11T01:27:56.121 INFO:teuthology.orchestra.run.smithi122.stderr: 89.2% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-11T01:31:05.404 INFO:teuthology.orchestra.run.smithi122.stderr: 92.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-11T01:33:07.688 INFO:teuthology.orchestra.run.smithi122.stderr: 90.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-11T01:34:29.295 INFO:teuthology.orchestra.run.smithi122.stderr: 91.1% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-11T01:34:29.296 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-11T01:34:29.296 INFO:teuthology.orchestra.run.smithi122.stderr:real 6m36.762s 2024-09-11T01:34:29.296 INFO:teuthology.orchestra.run.smithi122.stderr:user 8m12.307s 2024-09-11T01:34:29.296 INFO:teuthology.orchestra.run.smithi122.stderr:sys 0m20.457s 2024-09-11T01:34:29.297 INFO:tasks.ceph:Archiving logs... 2024-09-11T01:34:29.298 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/log/ceph to /home/teuthworker/archive/hyelloji-2024-09-10_17:16:14-rados-wip-hemanth-testing-2024-09-10-0723-distro-default-smithi/7899875/remote/smithi122/log 2024-09-11T01:34:29.300 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-11T01:35:21.766 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-11T01:35:21.779 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-11T01:35:21.779 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-11T01:35:21.830 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-11T01:35:21.838 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-11T01:35:21.839 DEBUG:teuthology.orchestra.run.smithi122:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-11T01:35:21.903 INFO:teuthology.orchestra.run.smithi122.stdout: remote refid st t when poll reach delay offset jitter 2024-09-11T01:35:21.903 INFO:teuthology.orchestra.run.smithi122.stdout:============================================================================== 2024-09-11T01:35:21.903 INFO:teuthology.orchestra.run.smithi122.stdout:*hv01.front.sepi 67.205.162.81 3 u 10 64 377 0.101 -4.985 0.859 2024-09-11T01:35:21.903 INFO:teuthology.orchestra.run.smithi122.stdout:+hv02.front.sepi 96.245.170.99 3 u 9 64 377 0.060 -7.018 0.994 2024-09-11T01:35:21.904 INFO:teuthology.orchestra.run.smithi122.stdout:+hv03.front.sepi 74.6.168.72 3 u 9 64 377 0.103 -4.928 0.982 2024-09-11T01:35:21.904 INFO:teuthology.orchestra.run.smithi122.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 +0.000 0.000 2024-09-11T01:35:21.905 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-11T01:35:21.914 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-11T01:35:21.914 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-11T01:35:21.923 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-11T01:35:21.931 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-11T01:35:21.939 INFO:teuthology.task.internal:Duration was 2037.597506 seconds 2024-09-11T01:35:21.940 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-11T01:35:21.952 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-11T01:35:21.953 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-11T01:35:22.139 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-11T01:35:22.139 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi122.front.sepia.ceph.com 2024-09-11T01:35:22.139 DEBUG:teuthology.orchestra.run.smithi122:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-11T01:35:22.151 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-11T01:35:22.151 DEBUG:teuthology.orchestra.run.smithi122:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-11T01:35:22.234 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-11T01:35:22.235 DEBUG:teuthology.orchestra.run.smithi122:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-11T01:35:22.410 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-11T01:35:22.420 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-11T01:35:22.420 DEBUG:teuthology.orchestra.run.smithi122:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-11T01:35:22.441 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-11T01:35:22.451 DEBUG:teuthology.orchestra.run.smithi122:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-11T01:35:22.504 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern = core 2024-09-11T01:35:22.535 DEBUG:teuthology.orchestra.run.smithi122:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-11T01:35:22.583 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-11T01:35:22.584 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-11T01:35:22.593 INFO:teuthology.task.internal:Transferring archived files... 2024-09-11T01:35:22.594 DEBUG:teuthology.misc:Transferring archived files from smithi122:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/hyelloji-2024-09-10_17:16:14-rados-wip-hemanth-testing-2024-09-10-0723-distro-default-smithi/7899875/remote/smithi122 2024-09-11T01:35:22.594 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-11T01:35:22.648 INFO:teuthology.task.internal:Removing archive directory... 2024-09-11T01:35:22.649 DEBUG:teuthology.orchestra.run.smithi122:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-11T01:35:22.697 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-11T01:35:22.707 INFO:teuthology.task.internal:Not uploading archives. 2024-09-11T01:35:22.707 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-11T01:35:22.716 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-11T01:35:22.717 DEBUG:teuthology.orchestra.run.smithi122:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-11T01:35:22.744 INFO:teuthology.orchestra.run.smithi122.stdout: 658637 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 11 01:35 /home/ubuntu/cephtest 2024-09-11T01:35:22.745 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-11T01:35:22.757 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-11T01:35:22.779 INFO:teuthology.run:Summary data: description: rados/singleton/{all/lost-unfound-delete mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/{bluestore-options/write$/{write_v2} bluestore/bluestore-bitmap} rados supported-random-distro$/{ubuntu_latest}} duration: 2037.5975058078766 flavor: default owner: scheduled_hyelloji@teuthology success: true 2024-09-11T01:35:22.779 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-11T01:35:22.878 INFO:teuthology.run:pass